1<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 2<html> 3<!-- Created by GNU Texinfo 6.8, https://www.gnu.org/software/texinfo/ --> 4<head> 5<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 6<!-- Copyright (C) 1988-2022 Free Software Foundation, Inc. 7 8Permission is granted to copy, distribute and/or modify this document 9under the terms of the GNU Free Documentation License, Version 1.3 or 10any later version published by the Free Software Foundation; with no 11Invariant Sections, the Front-Cover texts being (a) (see below), and 12with the Back-Cover Texts being (b) (see below). A copy of the 13license is included in the section entitled "GNU 14Free Documentation License". 15 16(a) The FSF's Front-Cover Text is: 17 18A GNU Manual 19 20(b) The FSF's Back-Cover Text is: 21 22You have freedom to copy and modify this GNU Manual, like GNU 23 software. Copies published by the Free Software Foundation raise 24 funds for GNU development. --> 25<title>Installing GCC: Testing</title> 26 27<meta name="description" content="Installing GCC: Testing"> 28<meta name="keywords" content="Installing GCC: Testing"> 29<meta name="resource-type" content="document"> 30<meta name="distribution" content="global"> 31<meta name="Generator" content="makeinfo"> 32<meta name="viewport" content="width=device-width,initial-scale=1"> 33 34<style type="text/css"> 35<!-- 36a.copiable-anchor {visibility: hidden; text-decoration: none; line-height: 0em} 37a.summary-letter {text-decoration: none} 38blockquote.indentedblock {margin-right: 0em} 39div.display {margin-left: 3.2em} 40div.example {margin-left: 3.2em} 41kbd {font-style: oblique} 42pre.display {font-family: inherit} 43pre.format {font-family: inherit} 44pre.menu-comment {font-family: serif} 45pre.menu-preformatted {font-family: serif} 46span.nolinebreak {white-space: nowrap} 47span.roman {font-family: initial; font-weight: normal} 48span.sansserif {font-family: sans-serif; font-weight: normal} 49span:hover a.copiable-anchor {visibility: visible} 50ul.no-bullet {list-style: none} 51--> 52</style> 53 54 55</head> 56 57<body lang="en"> 58<h1 class="settitle" align="center">Installing GCC: Testing</h1> 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80<span id="index-Testing"></span> 81<span id="index-Installing-GCC_003a-Testing"></span> 82<span id="index-Testsuite"></span> 83 84<p>Before you install GCC, we encourage you to run the testsuites and to 85compare your results with results from a similar configuration that have 86been submitted to the 87<a href="https://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>. 88Some of these archived results are linked from the build status lists 89at <a href="https://gcc.gnu.org/buildstat.html">https://gcc.gnu.org/buildstat.html</a>, although not everyone who 90reports a successful build runs the testsuites and submits the results. 91This step is optional and may require you to download additional software, 92but it can give you confidence in your new GCC installation or point out 93problems before you install and start using your new GCC. 94</p> 95<p>First, you must have <a href="download.html">downloaded the testsuites</a>. 96These are part of the full distribution, but if you downloaded the 97“core” compiler plus any front ends, you must download the testsuites 98separately. 99</p> 100<p>Second, you must have the testing tools installed. This includes 101<a href="https://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect; 102the DejaGnu site has links to these. 103Some optional tests also require Python3 and pytest module. 104</p> 105<p>If the directories where <code>runtest</code> and <code>expect</code> were 106installed are not in the <code>PATH</code>, you may need to set the following 107environment variables appropriately, as in the following example (which 108assumes that DejaGnu has been installed under <samp>/usr/local</samp>): 109</p> 110<div class="example"> 111<pre class="example">TCL_LIBRARY = /usr/local/share/tcl8.0 112DEJAGNULIBS = /usr/local/share/dejagnu 113</pre></div> 114 115<p>(On systems such as Cygwin, these paths are required to be actual 116paths, not mounts or links; presumably this is due to some lack of 117portability in the DejaGnu code.) 118</p> 119 120<p>Finally, you can run the testsuite (which may take a long time): 121</p><div class="example"> 122<pre class="example">cd <var>objdir</var>; make -k check 123</pre></div> 124 125<p>This will test various components of GCC, such as compiler 126front ends and runtime libraries. While running the testsuite, DejaGnu 127might emit some harmless messages resembling 128‘<samp>WARNING: Couldn't find the global config file.</samp>’ or 129‘<samp>WARNING: Couldn't find tool init file</samp>’ that can be ignored. 130</p> 131<p>If you are testing a cross-compiler, you may want to run the testsuite 132on a simulator as described at <a href="https://gcc.gnu.org/simtest-howto.html">https://gcc.gnu.org/simtest-howto.html</a>. 133</p> 134<div class="section" id="How-can-you-run-the-testsuite-on-selected-tests_003f"> 135<h3 class="section">How can you run the testsuite on selected tests?</h3> 136 137<p>In order to run sets of tests selectively, there are targets 138‘<samp>make check-gcc</samp>’ and language specific ‘<samp>make check-c</samp>’, 139‘<samp>make check-c++</samp>’, ‘<samp>make check-d</samp>’ ‘<samp>make check-fortran</samp>’, 140‘<samp>make check-ada</samp>’, ‘<samp>make check-objc</samp>’, ‘<samp>make check-obj-c++</samp>’, 141‘<samp>make check-lto</samp>’ 142in the <samp>gcc</samp> subdirectory of the object directory. You can also 143just run ‘<samp>make check</samp>’ in a subdirectory of the object directory. 144</p> 145 146<p>A more selective way to just run all <code>gcc</code> execute tests in the 147testsuite is to use 148</p> 149<div class="example"> 150<pre class="example">make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>" 151</pre></div> 152 153<p>Likewise, in order to run only the <code>g++</code> “old-deja” tests in 154the testsuite with filenames matching ‘<samp>9805*</samp>’, you would use 155</p> 156<div class="example"> 157<pre class="example">make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>" 158</pre></div> 159 160<p>The file-matching expression following <var>filename</var><code>.exp=</code> is treated 161as a series of whitespace-delimited glob expressions so that multiple patterns 162may be passed, although any whitespace must either be escaped or surrounded by 163single quotes if multiple expressions are desired. For example, 164</p> 165<div class="example"> 166<pre class="example">make check-g++ RUNTESTFLAGS="old-deja.exp=9805*\ virtual2.c <var>other-options</var>" 167make check-g++ RUNTESTFLAGS="'old-deja.exp=9805* virtual2.c' <var>other-options</var>" 168</pre></div> 169 170<p>The <samp>*.exp</samp> files are located in the testsuite directories of the GCC 171source, the most important ones being <samp>compile.exp</samp>, 172<samp>execute.exp</samp>, <samp>dg.exp</samp> and <samp>old-deja.exp</samp>. 173To get a list of the possible <samp>*.exp</samp> files, pipe the 174output of ‘<samp>make check</samp>’ into a file and look at the 175‘<samp>Running … .exp</samp>’ lines. 176</p> 177</div> 178<div class="section" id="Passing-options-and-running-multiple-testsuites"> 179<h3 class="section">Passing options and running multiple testsuites</h3> 180 181<p>You can pass multiple options to the testsuite using the 182‘<samp>--target_board</samp>’ option of DejaGNU, either passed as part of 183‘<samp>RUNTESTFLAGS</samp>’, or directly to <code>runtest</code> if you prefer to 184work outside the makefiles. For example, 185</p> 186<div class="example"> 187<pre class="example">make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants" 188</pre></div> 189 190<p>will run the standard <code>g++</code> testsuites (“unix” is the target name 191for a standard native testsuite situation), passing 192‘<samp>-O3 -fmerge-constants</samp>’ to the compiler on every test, i.e., 193slashes separate options. 194</p> 195<p>You can run the testsuites multiple times using combinations of options 196with a syntax similar to the brace expansion of popular shells: 197</p> 198<div class="example"> 199<pre class="example">…"--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}" 200</pre></div> 201 202<p>(Note the empty option caused by the trailing comma in the final group.) 203The following will run each testsuite eight times using the ‘<samp>arm-sim</samp>’ 204target, as if you had specified all possible combinations yourself: 205</p> 206<div class="example"> 207<pre class="example">--target_board='arm-sim/-mhard-float/-O1 \ 208 arm-sim/-mhard-float/-O2 \ 209 arm-sim/-mhard-float/-O3 \ 210 arm-sim/-mhard-float \ 211 arm-sim/-msoft-float/-O1 \ 212 arm-sim/-msoft-float/-O2 \ 213 arm-sim/-msoft-float/-O3 \ 214 arm-sim/-msoft-float' 215</pre></div> 216 217<p>They can be combined as many times as you wish, in arbitrary ways. This 218list: 219</p> 220<div class="example"> 221<pre class="example">…"--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}" 222</pre></div> 223 224<p>will generate four combinations, all involving ‘<samp>-Wextra</samp>’. 225</p> 226<p>The disadvantage to this method is that the testsuites are run in serial, 227which is a waste on multiprocessor systems. For users with GNU Make and 228a shell which performs brace expansion, you can run the testsuites in 229parallel by having the shell perform the combinations and <code>make</code> 230do the parallel runs. Instead of using ‘<samp>--target_board</samp>’, use a 231special makefile target: 232</p> 233<div class="example"> 234<pre class="example">make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/… 235</pre></div> 236 237<p>For example, 238</p> 239<div class="example"> 240<pre class="example">make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} 241</pre></div> 242 243<p>will run three concurrent “make-gcc” testsuites, eventually testing all 244ten combinations as described above. Note that this is currently only 245supported in the <samp>gcc</samp> subdirectory. (To see how this works, try 246typing <code>echo</code> before the example given here.) 247</p> 248 249</div> 250<div class="section" id="How-to-interpret-test-results"> 251<h3 class="section">How to interpret test results</h3> 252 253<p>The result of running the testsuite are various <samp>*.sum</samp> and <samp>*.log</samp> 254files in the testsuite subdirectories. The <samp>*.log</samp> files contain a 255detailed log of the compiler invocations and the corresponding 256results, the <samp>*.sum</samp> files summarize the results. These summaries 257contain status codes for all tests: 258</p> 259<ul> 260<li> PASS: the test passed as expected 261</li><li> XPASS: the test unexpectedly passed 262</li><li> FAIL: the test unexpectedly failed 263</li><li> XFAIL: the test failed as expected 264</li><li> UNSUPPORTED: the test is not supported on this platform 265</li><li> ERROR: the testsuite detected an error 266</li><li> WARNING: the testsuite detected a possible problem 267</li></ul> 268 269<p>It is normal for some tests to report unexpected failures. At the 270current time the testing harness does not allow fine grained control 271over whether or not a test is expected to fail. This problem should 272be fixed in future releases. 273</p> 274 275</div> 276<div class="section" id="Submitting-test-results"> 277<h3 class="section">Submitting test results</h3> 278 279<p>If you want to report the results to the GCC project, use the 280<samp>contrib/test_summary</samp> shell script. Start it in the <var>objdir</var> with 281</p> 282<div class="example"> 283<pre class="example"><var>srcdir</var>/contrib/test_summary -p your_commentary.txt \ 284 -m gcc-testresults@gcc.gnu.org |sh 285</pre></div> 286 287<p>This script uses the <code>Mail</code> program to send the results, so 288make sure it is in your <code>PATH</code>. The file <samp>your_commentary.txt</samp> is 289prepended to the testsuite summary and should contain any special 290remarks you have on your results or your build environment. Please 291do not edit the testsuite result block or the subject line, as these 292messages may be automatically processed. 293</p> 294<hr /> 295<p> 296<p><a href="./index.html">Return to the GCC Installation page</a> 297</p> 298 299 300 301 302 303<hr></div> 304 305 306 307</body> 308</html> 309