1<html lang="en"> 2<head> 3<title>Installing GCC: Testing</title> 4<meta http-equiv="Content-Type" content="text/html"> 5<meta name="description" content="Installing GCC: Testing"> 6<meta name="generator" content="makeinfo 4.12"> 7<link title="Top" rel="top" href="#Top"> 8<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage"> 9<!-- 10Copyright (C) 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997, 111998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 122009, 2010, 2011 Free Software Foundation, Inc. 13 14 Permission is granted to copy, distribute and/or modify this document 15under the terms of the GNU Free Documentation License, Version 1.2 or 16any later version published by the Free Software Foundation; with no 17Invariant Sections, the Front-Cover texts being (a) (see below), and 18with the Back-Cover Texts being (b) (see below). A copy of the 19license is included in the section entitled "GNU Free Documentation License". 20 21(a) The FSF's Front-Cover Text is: 22 23 A GNU Manual 24 25(b) The FSF's Back-Cover Text is: 26 27 You have freedom to copy and modify this GNU Manual, like GNU 28 software. Copies published by the Free Software Foundation raise 29 funds for GNU development.--> 30<meta http-equiv="Content-Style-Type" content="text/css"> 31<style type="text/css"><!-- 32 pre.display { font-family:inherit } 33 pre.format { font-family:inherit } 34 pre.smalldisplay { font-family:inherit; font-size:smaller } 35 pre.smallformat { font-family:inherit; font-size:smaller } 36 pre.smallexample { font-size:smaller } 37 pre.smalllisp { font-size:smaller } 38 span.sc { font-variant:small-caps } 39 span.roman { font-family:serif; font-weight:normal; } 40 span.sansserif { font-family:sans-serif; font-weight:normal; } 41--></style> 42</head> 43<body> 44<h1 class="settitle">Installing GCC: Testing</h1> 45<a name="index-Testing-1"></a><a name="index-Installing-GCC_003a-Testing-2"></a><a name="index-Testsuite-3"></a> 46Before you install GCC, we encourage you to run the testsuites and to 47compare your results with results from a similar configuration that have 48been submitted to the 49<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>. 50Some of these archived results are linked from the build status lists 51at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who 52reports a successful build runs the testsuites and submits the results. 53This step is optional and may require you to download additional software, 54but it can give you confidence in your new GCC installation or point out 55problems before you install and start using your new GCC. 56 57 <p>First, you must have <a href="download.html">downloaded the testsuites</a>. 58These are part of the full distribution, but if you downloaded the 59“core” compiler plus any front ends, you must download the testsuites 60separately. 61 62 <p>Second, you must have the testing tools installed. This includes 63<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect; 64the DejaGnu site has links to these. 65 66 <p>If the directories where <samp><span class="command">runtest</span></samp> and <samp><span class="command">expect</span></samp> were 67installed are not in the <samp><span class="env">PATH</span></samp>, you may need to set the following 68environment variables appropriately, as in the following example (which 69assumes that DejaGnu has been installed under <samp><span class="file">/usr/local</span></samp>): 70 71<pre class="smallexample"> TCL_LIBRARY = /usr/local/share/tcl8.0 72 DEJAGNULIBS = /usr/local/share/dejagnu 73</pre> 74 <p>(On systems such as Cygwin, these paths are required to be actual 75paths, not mounts or links; presumably this is due to some lack of 76portability in the DejaGnu code.) 77 78 <p>Finally, you can run the testsuite (which may take a long time): 79<pre class="smallexample"> cd <var>objdir</var>; make -k check 80</pre> 81 <p>This will test various components of GCC, such as compiler 82front ends and runtime libraries. While running the testsuite, DejaGnu 83might emit some harmless messages resembling 84‘<samp><span class="samp">WARNING: Couldn't find the global config file.</span></samp>’ or 85‘<samp><span class="samp">WARNING: Couldn't find tool init file</span></samp>’ that can be ignored. 86 87 <p>If you are testing a cross-compiler, you may want to run the testsuite 88on a simulator as described at <a href="http://gcc.gnu.org/simtest-howto.html">http://gcc.gnu.org/simtest-howto.html</a>. 89 90<h3 class="section"><a name="TOC0"></a>How can you run the testsuite on selected tests?</h3> 91 92<p>In order to run sets of tests selectively, there are targets 93‘<samp><span class="samp">make check-gcc</span></samp>’ and ‘<samp><span class="samp">make check-g++</span></samp>’ 94in the <samp><span class="file">gcc</span></samp> subdirectory of the object directory. You can also 95just run ‘<samp><span class="samp">make check</span></samp>’ in a subdirectory of the object directory. 96 97 <p>A more selective way to just run all <samp><span class="command">gcc</span></samp> execute tests in the 98testsuite is to use 99 100<pre class="smallexample"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>" 101</pre> 102 <p>Likewise, in order to run only the <samp><span class="command">g++</span></samp> “old-deja” tests in 103the testsuite with filenames matching ‘<samp><span class="samp">9805*</span></samp>’, you would use 104 105<pre class="smallexample"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>" 106</pre> 107 <p>The <samp><span class="file">*.exp</span></samp> files are located in the testsuite directories of the GCC 108source, the most important ones being <samp><span class="file">compile.exp</span></samp>, 109<samp><span class="file">execute.exp</span></samp>, <samp><span class="file">dg.exp</span></samp> and <samp><span class="file">old-deja.exp</span></samp>. 110To get a list of the possible <samp><span class="file">*.exp</span></samp> files, pipe the 111output of ‘<samp><span class="samp">make check</span></samp>’ into a file and look at the 112‘<samp><span class="samp">Running ... .exp</span></samp>’ lines. 113 114<h3 class="section"><a name="TOC1"></a>Passing options and running multiple testsuites</h3> 115 116<p>You can pass multiple options to the testsuite using the 117‘<samp><span class="samp">--target_board</span></samp>’ option of DejaGNU, either passed as part of 118‘<samp><span class="samp">RUNTESTFLAGS</span></samp>’, or directly to <samp><span class="command">runtest</span></samp> if you prefer to 119work outside the makefiles. For example, 120 121<pre class="smallexample"> make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants" 122</pre> 123 <p>will run the standard <samp><span class="command">g++</span></samp> testsuites (“unix” is the target name 124for a standard native testsuite situation), passing 125‘<samp><span class="samp">-O3 -fmerge-constants</span></samp>’ to the compiler on every test, i.e., 126slashes separate options. 127 128 <p>You can run the testsuites multiple times using combinations of options 129with a syntax similar to the brace expansion of popular shells: 130 131<pre class="smallexample"> ..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}" 132</pre> 133 <p>(Note the empty option caused by the trailing comma in the final group.) 134The following will run each testsuite eight times using the ‘<samp><span class="samp">arm-sim</span></samp>’ 135target, as if you had specified all possible combinations yourself: 136 137<pre class="smallexample"> --target_board=arm-sim/-mhard-float/-O1 138 --target_board=arm-sim/-mhard-float/-O2 139 --target_board=arm-sim/-mhard-float/-O3 140 --target_board=arm-sim/-mhard-float 141 --target_board=arm-sim/-msoft-float/-O1 142 --target_board=arm-sim/-msoft-float/-O2 143 --target_board=arm-sim/-msoft-float/-O3 144 --target_board=arm-sim/-msoft-float 145</pre> 146 <p>They can be combined as many times as you wish, in arbitrary ways. This 147list: 148 149<pre class="smallexample"> ..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}" 150</pre> 151 <p>will generate four combinations, all involving ‘<samp><span class="samp">-Wextra</span></samp>’. 152 153 <p>The disadvantage to this method is that the testsuites are run in serial, 154which is a waste on multiprocessor systems. For users with GNU Make and 155a shell which performs brace expansion, you can run the testsuites in 156parallel by having the shell perform the combinations and <samp><span class="command">make</span></samp> 157do the parallel runs. Instead of using ‘<samp><span class="samp">--target_board</span></samp>’, use a 158special makefile target: 159 160<pre class="smallexample"> make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/... 161</pre> 162 <p>For example, 163 164<pre class="smallexample"> make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} 165</pre> 166 <p>will run three concurrent “make-gcc” testsuites, eventually testing all 167ten combinations as described above. Note that this is currently only 168supported in the <samp><span class="file">gcc</span></samp> subdirectory. (To see how this works, try 169typing <samp><span class="command">echo</span></samp> before the example given here.) 170 171<h3 class="section"><a name="TOC2"></a>Additional testing for Java Class Libraries</h3> 172 173<p>The Java runtime tests can be executed via ‘<samp><span class="samp">make check</span></samp>’ 174in the <samp><var>target</var><span class="file">/libjava/testsuite</span></samp> directory in 175the build tree. 176 177 <p>The <a href="http://sourceware.org/mauve/">Mauve Project</a> provides 178a suite of tests for the Java Class Libraries. This suite can be run 179as part of libgcj testing by placing the Mauve tree within the libjava 180testsuite at <samp><span class="file">libjava/testsuite/libjava.mauve/mauve</span></samp>, or by 181specifying the location of that tree when invoking ‘<samp><span class="samp">make</span></samp>’, as in 182‘<samp><span class="samp">make MAUVEDIR=~/mauve check</span></samp>’. 183 184<h3 class="section"><a name="TOC3"></a>How to interpret test results</h3> 185 186<p>The result of running the testsuite are various <samp><span class="file">*.sum</span></samp> and <samp><span class="file">*.log</span></samp> 187files in the testsuite subdirectories. The <samp><span class="file">*.log</span></samp> files contain a 188detailed log of the compiler invocations and the corresponding 189results, the <samp><span class="file">*.sum</span></samp> files summarize the results. These summaries 190contain status codes for all tests: 191 192 <ul> 193<li>PASS: the test passed as expected 194<li>XPASS: the test unexpectedly passed 195<li>FAIL: the test unexpectedly failed 196<li>XFAIL: the test failed as expected 197<li>UNSUPPORTED: the test is not supported on this platform 198<li>ERROR: the testsuite detected an error 199<li>WARNING: the testsuite detected a possible problem 200</ul> 201 202 <p>It is normal for some tests to report unexpected failures. At the 203current time the testing harness does not allow fine grained control 204over whether or not a test is expected to fail. This problem should 205be fixed in future releases. 206 207<h3 class="section"><a name="TOC4"></a>Submitting test results</h3> 208 209<p>If you want to report the results to the GCC project, use the 210<samp><span class="file">contrib/test_summary</span></samp> shell script. Start it in the <var>objdir</var> with 211 212<pre class="smallexample"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \ 213 -m gcc-testresults@gcc.gnu.org |sh 214</pre> 215 <p>This script uses the <samp><span class="command">Mail</span></samp> program to send the results, so 216make sure it is in your <samp><span class="env">PATH</span></samp>. The file <samp><span class="file">your_commentary.txt</span></samp> is 217prepended to the testsuite summary and should contain any special 218remarks you have on your results or your build environment. Please 219do not edit the testsuite result block or the subject line, as these 220messages may be automatically processed. 221 222 <p><hr /> 223<p><a href="./index.html">Return to the GCC Installation page</a> 224 225<!-- ***Final install*********************************************************** --> 226<!-- ***Binaries**************************************************************** --> 227<!-- ***Specific**************************************************************** --> 228<!-- ***Old documentation****************************************************** --> 229<!-- ***GFDL******************************************************************** --> 230<!-- *************************************************************************** --> 231<!-- Part 6 The End of the Document --> 232</body></html> 233 234