136ac495dSmrg<?xml version="1.0" encoding="UTF-8" standalone="no"?> 236ac495dSmrg<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Testing</title><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><meta name="keywords" content="ISO C++, test, testsuite, performance, conformance, ABI, exception safety" /><meta name="keywords" content="ISO C++, library" /><meta name="keywords" content="ISO C++, runtime, library" /><link rel="home" href="../index.html" title="The GNU C++ Library" /><link rel="up" href="appendix_porting.html" title="Appendix B. Porting and Maintenance" /><link rel="prev" href="internals.html" title="Porting to New Hardware or Operating Systems" /><link rel="next" href="abi.html" title="ABI Policy and Guidelines" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Testing</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><th width="60%" align="center">Appendix B. 336ac495dSmrg Porting and Maintenance 436ac495dSmrg 536ac495dSmrg</th><td width="20%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="manual.intro.setup.test"></a>Testing</h2></div></div></div><p> 636ac495dSmrgThe libstdc++ testsuite includes testing for standard conformance, 736ac495dSmrgregressions, ABI, and performance. 836ac495dSmrg</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"></a>Test Organization</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.layout"></a>Directory Layout</h4></div></div></div><p> 936ac495dSmrg The directory 1036ac495dSmrg <code class="filename"><em class="replaceable"><code>gccsrcdir</code></em>/libstdc++-v3/testsuite</code> 1136ac495dSmrg contains the individual test cases organized in sub-directories 1236ac495dSmrg corresponding to clauses of the C++ standard (detailed below), 1336ac495dSmrg the DejaGnu test harness support files, and sources to various 1436ac495dSmrg testsuite utilities that are packaged in a separate testing library. 1536ac495dSmrg</p><p> 1636ac495dSmrg All test cases for functionality required by the runtime components 1736ac495dSmrg of the C++ standard (ISO 14882) are files within the following 1836ac495dSmrg directories: 1936ac495dSmrg 2036ac495dSmrg </p><pre class="programlisting"> 2136ac495dSmrg 17_intro 2236ac495dSmrg 18_support 2336ac495dSmrg 19_diagnostics 2436ac495dSmrg 20_util 2536ac495dSmrg 21_strings 2636ac495dSmrg 22_locale 2736ac495dSmrg 23_containers 2836ac495dSmrg 24_iterators 2936ac495dSmrg 25_algorithms 3036ac495dSmrg 26_numerics 3136ac495dSmrg 27_io 3236ac495dSmrg 28_regex 3336ac495dSmrg 29_atomics 3436ac495dSmrg 30_threads 3536ac495dSmrg </pre><p> 3636ac495dSmrg</p><p> 3736ac495dSmrg In addition, the following directories include test files: 3836ac495dSmrg 3936ac495dSmrg</p><div class="variablelist"><dl class="variablelist compact"><dt><span class="term"><code class="filename">tr1</code></span></dt><dd>Tests for components as described by the Technical Report 4036ac495dSmrg on Standard Library Extensions (TR1). 4136ac495dSmrg </dd><dt><span class="term"><code class="filename">backward</code></span></dt><dd>Tests for backwards compatibility and deprecated features. 4236ac495dSmrg </dd><dt><span class="term"><code class="filename">demangle</code></span></dt><dd>Tests for <code class="function">__cxa_demangle</code>, the IA-64 C++ ABI 4336ac495dSmrg demangler. 4436ac495dSmrg </dd><dt><span class="term"><code class="filename">ext</code></span></dt><dd>Tests for extensions.</dd><dt><span class="term"><code class="filename">performance</code></span></dt><dd>Tests for performance analysis, and performance regressions. 4536ac495dSmrg </dd></dl></div><p> 4636ac495dSmrg </p><p> 4736ac495dSmrg Some directories don't have test files, but instead contain 4836ac495dSmrg auxiliary information: 4936ac495dSmrg 5036ac495dSmrg</p><div class="variablelist"><dl class="variablelist compact"><dt><span class="term"><code class="filename">config</code></span></dt><dd>Files for the DejaGnu test harness.</dd><dt><span class="term"><code class="filename">lib</code></span></dt><dd>Files for the DejaGnu test harness.</dd><dt><span class="term"><code class="filename">libstdc++*</code></span></dt><dd>Files for the DejaGnu test harness.</dd><dt><span class="term"><code class="filename">data</code></span></dt><dd>Sample text files for testing input and output.</dd><dt><span class="term"><code class="filename">util</code></span></dt><dd>Files for libtestc++, utilities and testing routines.</dd></dl></div><p> 5136ac495dSmrg </p><p> 5236ac495dSmrg Within a directory that includes test files, there may be 5336ac495dSmrg additional subdirectories, or files. Originally, test cases 5436ac495dSmrg were appended to one file that represented a particular section 5536ac495dSmrg of the chapter under test, and was named accordingly. For 5636ac495dSmrg instance, to test items related to <code class="code"> 21.3.6.1 - 5736ac495dSmrg <code class="function">basic_string::find</code> [lib.string::find]</code> 5836ac495dSmrg in the standard, the following was used: 5936ac495dSmrg</p><pre class="programlisting"> 21_strings/find.cc </pre><p> 6036ac495dSmrg However, that practice soon became a liability as the test cases 6136ac495dSmrg became huge and unwieldy, and testing new or extended 6236ac495dSmrg functionality (like wide characters or named locales) became 6336ac495dSmrg frustrating, leading to aggressive pruning of test cases on some 6436ac495dSmrg platforms that covered up implementation errors. Now, the test 6536ac495dSmrg suite has a policy of one file, one test case, which solves the 6636ac495dSmrg above issues and gives finer grained results and more manageable 6736ac495dSmrg error debugging. As an example, the test case quoted above 6836ac495dSmrg becomes: 6936ac495dSmrg</p><pre class="programlisting"> 21_strings/basic_string/find/char/1.cc 7036ac495dSmrg 21_strings/basic_string/find/char/2.cc 7136ac495dSmrg 21_strings/basic_string/find/char/3.cc 7236ac495dSmrg 21_strings/basic_string/find/wchar_t/1.cc 7336ac495dSmrg 21_strings/basic_string/find/wchar_t/2.cc 7436ac495dSmrg 21_strings/basic_string/find/wchar_t/3.cc</pre><p> 7536ac495dSmrg </p><p> 7636ac495dSmrg All new tests should be written with the policy of "one test 7736ac495dSmrg case, one file" in mind. 7836ac495dSmrg </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.naming"></a>Naming Conventions</h4></div></div></div><p> 7936ac495dSmrg In addition, there are some special names and suffixes that are 8036ac495dSmrg used within the testsuite to designate particular kinds of 8136ac495dSmrg tests. 8236ac495dSmrg </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="filename">_xin.cc</code></span></dt><dd> 8336ac495dSmrg This test case expects some kind of interactive input in order 8436ac495dSmrg to finish or pass. At the moment, the interactive tests are not 8536ac495dSmrg run by default. Instead, they are run by hand, like: 8636ac495dSmrg <pre class="programlisting"> 8736ac495dSmrgg++ 27_io/objects/char/3_xin.cc 8836ac495dSmrgcat 27_io/objects/char/3_xin.in | a.out</pre></dd><dt><span class="term"><code class="filename">.in</code></span></dt><dd> 8936ac495dSmrg This file contains the expected input for the corresponding <span class="emphasis"><em> 9036ac495dSmrg _xin.cc</em></span> test case. 9136ac495dSmrg </dd><dt><span class="term"><code class="filename">_neg.cc</code></span></dt><dd> 9236ac495dSmrg This test case is expected to fail: it's a negative test. At the 9336ac495dSmrg moment, these are almost always compile time errors. 9436ac495dSmrg </dd><dt><span class="term"><code class="filename">char</code></span></dt><dd> 9536ac495dSmrg This can either be a directory name or part of a longer file 9636ac495dSmrg name, and indicates that this file, or the files within this 9736ac495dSmrg directory are testing the <code class="code">char</code> instantiation of a 9836ac495dSmrg template. 9936ac495dSmrg </dd><dt><span class="term"><code class="filename">wchar_t</code></span></dt><dd> 10036ac495dSmrg This can either be a directory name or part of a longer file 10136ac495dSmrg name, and indicates that this file, or the files within this 10236ac495dSmrg directory are testing the <code class="code">wchar_t</code> instantiation of 10336ac495dSmrg a template. Some hosts do not support <code class="code">wchar_t</code> 10436ac495dSmrg functionality, so for these targets, all of these tests will not 10536ac495dSmrg be run. 10636ac495dSmrg </dd><dt><span class="term"><code class="filename">thread</code></span></dt><dd> 10736ac495dSmrg This can either be a directory name or part of a longer file 10836ac495dSmrg name, and indicates that this file, or the files within this 10936ac495dSmrg directory are testing situations where multiple threads are 11036ac495dSmrg being used. 11136ac495dSmrg </dd><dt><span class="term"><code class="filename">performance</code></span></dt><dd> 11236ac495dSmrg This can either be an enclosing directory name or part of a 11336ac495dSmrg specific file name. This indicates a test that is used to 11436ac495dSmrg analyze runtime performance, for performance regression testing, 11536ac495dSmrg or for other optimization related analysis. At the moment, these 11636ac495dSmrg test cases are not run by default. 11736ac495dSmrg </dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"></a>Running the Testsuite</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"></a>Basic</h4></div></div></div><p> 11836ac495dSmrg You can check the status of the build without installing it 11936ac495dSmrg using the DejaGnu harness, much like the rest of the gcc 12036ac495dSmrg tools, i.e. 12136ac495dSmrg <strong class="userinput"><code>make check</code></strong> 12236ac495dSmrg in the 12336ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em></code> 12436ac495dSmrg directory, or 12536ac495dSmrg <strong class="userinput"><code>make check-target-libstdc++-v3</code></strong> 12636ac495dSmrg in the 12736ac495dSmrg <code class="filename"><em class="replaceable"><code>gccbuilddir</code></em></code> 12836ac495dSmrg directory. 12936ac495dSmrg </p><p> 13036ac495dSmrg These commands are functionally equivalent and will create a 13136ac495dSmrg '<code class="filename">testsuite</code>' directory underneath 13236ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em></code> 13336ac495dSmrg containing the results of the 13436ac495dSmrg tests. Two results files will be generated: 13536ac495dSmrg <code class="filename">libstdc++.sum</code>, which is a PASS/FAIL summary 13636ac495dSmrg for each test, and 13736ac495dSmrg <code class="filename">libstdc++.log</code> which is a log of 13836ac495dSmrg the exact command-line passed to the compiler, the compiler 13936ac495dSmrg output, and the executable output (if any) for each test. 14036ac495dSmrg </p><p> 14136ac495dSmrg Archives of test results for various versions and platforms are 14236ac495dSmrg available on the GCC website in the <a class="link" href="http://gcc.gnu.org/gcc-4.3/buildstat.html" target="_top">build 14336ac495dSmrg status</a> section of each individual release, and are also 14436ac495dSmrg archived on a daily basis on the <a class="link" href="http://gcc.gnu.org/ml/gcc-testresults/current" target="_top">gcc-testresults</a> 14536ac495dSmrg mailing list. Please check either of these places for a similar 14636ac495dSmrg combination of source version, operating system, and host CPU. 14736ac495dSmrg </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.variations"></a>Variations</h4></div></div></div><p> 14836ac495dSmrg There are several options for running tests, including testing 14936ac495dSmrg the regression tests, testing a subset of the regression tests, 15036ac495dSmrg testing the performance tests, testing just compilation, testing 15136ac495dSmrg installed tools, etc. In addition, there is a special rule for 15236ac495dSmrg checking the exported symbols of the shared library. 15336ac495dSmrg </p><p> 15436ac495dSmrg To debug the DejaGnu test harness during runs, try invoking with a 15536ac495dSmrg specific argument to the variable <code class="varname">RUNTESTFLAGS</code>, 15636ac495dSmrg like so: 15736ac495dSmrg</p><pre class="programlisting"> 15836ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="-v" 15936ac495dSmrg</pre><p> 16036ac495dSmrg or 16136ac495dSmrg</p><pre class="programlisting"> 16236ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" 16336ac495dSmrg</pre><p> 16436ac495dSmrg </p><p> 16536ac495dSmrg To run a subset of the library tests, you can either generate the 16636ac495dSmrg <code class="filename">testsuite_files</code> file (described below) by running 16736ac495dSmrg <strong class="userinput"><code>make testsuite_files</code></strong> in the 16836ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em>/testsuite</code> 16936ac495dSmrg directory, then edit the 17036ac495dSmrg file to remove the tests you don't want and then run the testsuite as 17136ac495dSmrg normal, or you can specify a testsuite and a subset of tests in the 17236ac495dSmrg <code class="varname">RUNTESTFLAGS</code> variable. 17336ac495dSmrg </p><p> 17436ac495dSmrg For example, to run only the tests for containers you could use: 17536ac495dSmrg 17636ac495dSmrg</p><pre class="programlisting"> 17736ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="conformance.exp=23_containers/*" 17836ac495dSmrg</pre><p> 17936ac495dSmrg </p><p> 18036ac495dSmrg When combining this with other options in <code class="varname">RUNTESTFLAGS</code> 18136ac495dSmrg the <code class="option">testsuite.exp=testfiles</code> options must come first. 18236ac495dSmrg </p><p> 18336ac495dSmrg There are two ways to run on a simulator: set up <code class="envar">DEJAGNU</code> 18436ac495dSmrg to point to a specially crafted <code class="filename">site.exp</code>, 18536ac495dSmrg or pass down <code class="option">--target_board</code> flags. 18636ac495dSmrg </p><p> 18736ac495dSmrg Example flags to pass down for various embedded builds are as follows: 18836ac495dSmrg 18936ac495dSmrg</p><pre class="programlisting"> 19036ac495dSmrg --target=powerpc-eabisim <span class="emphasis"><em>(libgloss/sim)</em></span> 19136ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" 19236ac495dSmrg 19336ac495dSmrg --target=calmrisc32 <span class="emphasis"><em>(libgloss/sid)</em></span> 19436ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" 19536ac495dSmrg 19636ac495dSmrg --target=xscale-elf <span class="emphasis"><em>(newlib/sim)</em></span> 19736ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" 19836ac495dSmrg</pre><p> 19936ac495dSmrg </p><p> 20036ac495dSmrg Also, here is an example of how to run the libstdc++ testsuite 20136ac495dSmrg for a multilibed build directory with different ABI settings: 20236ac495dSmrg 20336ac495dSmrg </p><pre class="programlisting"> 20436ac495dSmrg make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' 20536ac495dSmrg</pre><p> 20636ac495dSmrg </p><p> 20736ac495dSmrg You can run the tests with a compiler and library that have 20836ac495dSmrg already been installed. Make sure that the compiler (e.g., 20936ac495dSmrg <span class="command"><strong>g++</strong></span>) is in your <code class="envar">PATH</code>. If you are 21036ac495dSmrg using shared libraries, then you must also ensure that the 21136ac495dSmrg directory containing the shared version of libstdc++ is in your 21236ac495dSmrg <code class="envar">LD_LIBRARY_PATH</code>, or 21336ac495dSmrg <a class="link" href="using_dynamic_or_shared.html#manual.intro.using.linkage.dynamic" title="Finding Dynamic or Shared Libraries">equivalent</a>. 21436ac495dSmrg If your GCC source tree is at 21536ac495dSmrg <code class="filename">/path/to/gcc</code>, 21636ac495dSmrg then you can run the tests as follows: 21736ac495dSmrg 21836ac495dSmrg</p><pre class="programlisting"> 21936ac495dSmrg runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite 22036ac495dSmrg</pre><p> 22136ac495dSmrg </p><p> 22236ac495dSmrg The testsuite will create a number of files in the directory in 22336ac495dSmrg which you run this command,. Some of those files might use the 22436ac495dSmrg same name as files created by other testsuites (like the ones 22536ac495dSmrg for GCC and G++), so you should not try to run all the 22636ac495dSmrg testsuites in parallel from the same directory. 22736ac495dSmrg </p><p> 22836ac495dSmrg In addition, there are some testing options that are mostly of 22936ac495dSmrg interest to library maintainers and system integrators. As such, 23036ac495dSmrg these tests may not work on all CPU and host combinations, and 23136ac495dSmrg may need to be executed in the 23236ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em>/testsuite</code> 23336ac495dSmrg directory. These 23436ac495dSmrg options include, but are not necessarily limited to, the 23536ac495dSmrg following: 23636ac495dSmrg </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><strong class="userinput"><code> 23736ac495dSmrg make testsuite_files 23836ac495dSmrg </code></strong></span></dt><dd><p> 23936ac495dSmrg Five files are generated that determine what test files 24036ac495dSmrg are run. These files are: 24136ac495dSmrg 24236ac495dSmrg </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"> <code class="filename">testsuite_files</code> </span></dt><dd> 24336ac495dSmrg This is a list of all the test cases that will be run. Each 24436ac495dSmrg test case is on a separate line, given with an absolute path 24536ac495dSmrg from the 24636ac495dSmrg <code class="filename"><em class="replaceable"><code>libsrcdir</code></em>/testsuite</code> 24736ac495dSmrg directory. 24836ac495dSmrg </dd><dt><span class="term"> <code class="filename">testsuite_files_interactive</code> </span></dt><dd> 24936ac495dSmrg This is a list of all the interactive test cases, using the 25036ac495dSmrg same format as the file list above. These tests are not run 25136ac495dSmrg by default. 25236ac495dSmrg </dd><dt><span class="term"> <code class="filename">testsuite_files_performance</code> </span></dt><dd> 25336ac495dSmrg This is a list of all the performance test cases, using the 25436ac495dSmrg same format as the file list above. These tests are not run 25536ac495dSmrg by default. 25636ac495dSmrg </dd><dt><span class="term"> <code class="filename">testsuite_thread</code> </span></dt><dd> 25736ac495dSmrg This file indicates that the host system can run tests which 25836ac495dSmrg involved multiple threads. 25936ac495dSmrg </dd><dt><span class="term"> <code class="filename">testsuite_wchar_t</code> </span></dt><dd> 26036ac495dSmrg This file indicates that the host system can run the 26136ac495dSmrg <code class="code">wchar_t</code> tests, and corresponds to the macro 26236ac495dSmrg definition <code class="literal">_GLIBCXX_USE_WCHAR_T</code> in the 26336ac495dSmrg file <code class="filename">c++config.h</code>. 26436ac495dSmrg </dd></dl></div><p> 26536ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 26636ac495dSmrg make check-abi 26736ac495dSmrg </code></strong></span></dt><dd><p> 26836ac495dSmrg The library ABI can be tested. This involves testing the shared 26936ac495dSmrg library against a baseline list of symbol exports that defines the 27036ac495dSmrg previous version of the ABI. The tests require that no exported 27136ac495dSmrg symbols are removed, no new symbols are added to the old symbol 27236ac495dSmrg versions, and any new symbols have the latest symbol version. 27336ac495dSmrg See <a class="link" href="abi.html#abi.versioning" title="Versioning">Versioning</a> for more details 27436ac495dSmrg of the ABI version history. 27536ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 27636ac495dSmrg make new-abi-baseline 27736ac495dSmrg </code></strong></span></dt><dd><p> 27836ac495dSmrg Generate a new baseline set of symbols exported from the library 27936ac495dSmrg (written to a file under 28036ac495dSmrg <code class="filename"><em class="replaceable"><code>libsrcdir</code></em>/config/abi/post/<em class="replaceable"><code>target</code></em>/</code>). 28136ac495dSmrg A different baseline symbols file is needed for each architecture and 28236ac495dSmrg is used by the <code class="literal">check-abi</code> target described above. 28336ac495dSmrg The files are usually re-generated by target maintainers for releases. 28436ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 28536ac495dSmrg make check-compile 28636ac495dSmrg </code></strong></span></dt><dd><p> 28736ac495dSmrg This rule compiles, but does not link or execute, the 28836ac495dSmrg <code class="filename">testsuite_files</code> test cases and displays the 28936ac495dSmrg output on stdout. 29036ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 29136ac495dSmrg make check-performance 29236ac495dSmrg </code></strong></span></dt><dd><p> 29336ac495dSmrg This rule runs through the 29436ac495dSmrg <code class="filename">testsuite_files_performance</code> test cases and 29536ac495dSmrg collects information for performance analysis and can be used to 29636ac495dSmrg spot performance regressions. Various timing information is 29736ac495dSmrg collected, as well as number of hard page faults, and memory 29836ac495dSmrg used. This is not run by default, and the implementation is in 29936ac495dSmrg flux. 30036ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 30136ac495dSmrg make check-debug 30236ac495dSmrg </code></strong></span></dt><dd><p> 30336ac495dSmrg This rule runs through the test suite under the 30436ac495dSmrg <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>. 30536ac495dSmrg </p></dd><dt><span class="term"><strong class="userinput"><code> 30636ac495dSmrg make check-parallel 30736ac495dSmrg </code></strong></span></dt><dd><p> 30836ac495dSmrg This rule runs through the test suite under the 30936ac495dSmrg <a class="link" href="parallel_mode.html" title="Chapter 18. Parallel Mode">parallel mode</a>. 31036ac495dSmrg </p></dd></dl></div><p> 31136ac495dSmrg We are interested in any strange failures of the testsuite; 31236ac495dSmrg please email the main libstdc++ mailing list if you see 31336ac495dSmrg something odd or have questions. 31436ac495dSmrg </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"></a>Permutations</h4></div></div></div><p> 31536ac495dSmrg The tests will be compiled with a set of default compiler flags defined 31636ac495dSmrg by the 31736ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em>/scripts/testsuite_flags</code> 31836ac495dSmrg file, as well as options specified in individual tests. You can run 31936ac495dSmrg the tests with different options by adding them to the output of 32036ac495dSmrg the <code class="option">--cxxflags</code> option of that script, or by setting 32136ac495dSmrg the <code class="varname">CXXFLAGS</code> variable when running 32236ac495dSmrg <span class="command"><strong>make</strong></span>, or via options for the DejaGnu test framework 32336ac495dSmrg (described below). The latter approach uses the 32436ac495dSmrg <code class="option">--target_board</code> option that was shown earlier, 32536ac495dSmrg but requires DejaGnu version 1.5.3 or newer to work reliably, so that the 32636ac495dSmrg <code class="literal">dg-options</code> in the test aren't overridden. 32736ac495dSmrg For example, to run the tests with 32836ac495dSmrg <code class="option">-O1 -D_GLIBCXX_ASSERTIONS</code> 32936ac495dSmrg you could use: 33036ac495dSmrg</p><pre class="programlisting"> make check RUNTESTFLAGS=--target_board=unix/-O1/-D_GLIBCXX_ASSERTIONS</pre><p> 33136ac495dSmrg </p><p> 33236ac495dSmrg The <code class="option">--target_board</code> option can also be used to run the 33336ac495dSmrg tests multiple times in different variations. For example, to run the 33436ac495dSmrg entire testsuite three times using <code class="option">-O3</code> but with 33536ac495dSmrg different <code class="option">-std</code> options: 33636ac495dSmrg</p><pre class="programlisting"> make check 'RUNTESTFLAGS=--target_board=unix/-O3\"{-std=gnu++98,-std=gnu++11,-std=gnu++14}\"'</pre><p> 33736ac495dSmrg N.B. that set of variations could also be written as 33836ac495dSmrg <code class="literal">unix/-O3\"{-std=gnu++98,-std=gnu++11,}\"</code> so that 33936ac495dSmrg the third variation would use the default for <code class="option">-std</code> 34036ac495dSmrg (which is <code class="option">-std=gnu++14</code> as of GCC 6). 34136ac495dSmrg </p><p> 34236ac495dSmrg To run the libstdc++ test suite under the 34336ac495dSmrg <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>, use 34436ac495dSmrg <strong class="userinput"><code>make check-debug</code></strong>. Alternatively, edit 34536ac495dSmrg <code class="filename"><em class="replaceable"><code>libbuilddir</code></em>/scripts/testsuite_flags</code> 34636ac495dSmrg to add the compile-time flag <code class="option">-D_GLIBCXX_DEBUG</code> to the 34736ac495dSmrg result printed by the <code class="option">--cxxflags</code> 34836ac495dSmrg option. Additionally, add the 34936ac495dSmrg <code class="option">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on 35036ac495dSmrg pedantic checking. The libstdc++ test suite should produce 35136ac495dSmrg the same results under debug mode that it does under release mode: 35236ac495dSmrg any deviation indicates an error in either the library or the test suite. 35336ac495dSmrg Note, however, that the number of tests that PASS may change, because 35436ac495dSmrg some test cases are skipped in normal mode, and some are skipped in 35536ac495dSmrg debug mode, as determined by the 35636ac495dSmrg <code class="literal">dg-require-<em class="replaceable"><code>support</code></em></code> 35736ac495dSmrg directives described below. 35836ac495dSmrg </p><p> 35936ac495dSmrg The <a class="link" href="parallel_mode.html" title="Chapter 18. Parallel Mode">parallel 36036ac495dSmrg mode</a> can be tested using 36136ac495dSmrg <strong class="userinput"><code>make check-parallel</code></strong>, or in much the same manner 36236ac495dSmrg as the debug mode, substituting 36336ac495dSmrg <code class="option">-D_GLIBCXX_PARALLEL</code> for 36436ac495dSmrg <code class="option">-D_GLIBCXX_DEBUG</code> in the previous paragraph. 36536ac495dSmrg </p><p> 36636ac495dSmrg Or, just run the testsuite 36736ac495dSmrg <code class="option">-D_GLIBCXX_DEBUG</code> or <code class="option">-D_GLIBCXX_PARALLEL</code> 36836ac495dSmrg in <code class="varname">CXXFLAGS</code> or <code class="varname">RUNTESTFLAGS</code>. 36936ac495dSmrg </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"></a>Writing a new test case</h3></div></div></div><p> 37036ac495dSmrg The first step in making a new test case is to choose the correct 37136ac495dSmrg directory and file name, given the organization as previously 37236ac495dSmrg described. 37336ac495dSmrg </p><p> 37436ac495dSmrg All files are copyright the FSF, and GPL'd: this is very 37536ac495dSmrg important. The first copyright year should correspond to the date 37636ac495dSmrg the file was checked in to version control. If a test is copied from 37736ac495dSmrg an existing file it should retain the copyright years from the 37836ac495dSmrg original file. 37936ac495dSmrg </p><p> 38036ac495dSmrg The DejaGnu instructions say to always return <code class="literal">0</code> 38136ac495dSmrg from <code class="function">main</code> to indicate success. Strictly speaking 38236ac495dSmrg this is redundant in C++, since returning from <code class="function">main</code> 38336ac495dSmrg is defined to return <code class="literal">0</code>. Most tests still have an 38436ac495dSmrg explicit return. 38536ac495dSmrg </p><p> 38636ac495dSmrg A bunch of utility functions and classes have already been 38736ac495dSmrg abstracted out into the testsuite utility library, <code class="code"> 38836ac495dSmrg libtestc++</code>. To use this functionality, just include the 38936ac495dSmrg appropriate header file: the library or specific object files will 39036ac495dSmrg automatically be linked in as part of the testsuite run. 39136ac495dSmrg </p><p> 39236ac495dSmrg Tests that need to perform runtime checks should use the 39336ac495dSmrg <code class="literal">VERIFY</code> macro, defined in the 39436ac495dSmrg <code class="filename"><testsuite_hooks.h></code> header. 395a2dc1f3fSmrg This expands to a custom assertion using 396a2dc1f3fSmrg <code class="function">__builtin_printf</code> and 397a2dc1f3fSmrg <code class="function">__builtin_abort</code> 398a2dc1f3fSmrg (to avoid using <code class="literal">assert</code> and being affected by 399a2dc1f3fSmrg <code class="literal">NDEBUG</code>). 400a2dc1f3fSmrg </p><p> 401a2dc1f3fSmrg Prior to GCC 7.1, <code class="literal">VERIFY</code> was defined differently. 402a2dc1f3fSmrg It usually expanded to the standard <code class="literal">assert</code> macro, but 403a2dc1f3fSmrg allowed targets to define it to something different. In order to support 404a2dc1f3fSmrg the alternative expansions of <code class="literal">VERIFY</code>, before any use 405a2dc1f3fSmrg of the macro there needed to be a variable called <code class="varname">test</code> 406a2dc1f3fSmrg in scope, which was usually defined like so (the attribute avoids 40736ac495dSmrg warnings about an unused variable): 40836ac495dSmrg </p><pre class="programlisting"> 40936ac495dSmrg bool test __attribute__((unused)) = true; 41036ac495dSmrg </pre><p> 411a2dc1f3fSmrg This is no longer needed, and should not be added to new tests. 41236ac495dSmrg </p><p> 41336ac495dSmrg The testsuite uses the DejaGnu framework to compile and run the tests. 41436ac495dSmrg Test cases are normal C++ files which contain special directives in 41536ac495dSmrg comments. These directives look like <code class="literal">{ dg-* ... }</code> 41636ac495dSmrg and tell DejaGnu what to do and what kinds of behavior are to be expected 41736ac495dSmrg for a test. The core DejaGnu directives are documented in the 41836ac495dSmrg <code class="filename">dg.exp</code> file installed by DejaGnu. 41936ac495dSmrg The GCC testsuites support additional directives 42036ac495dSmrg as described in the GCC internals documentation, see <a class="link" href="https://gcc.gnu.org/onlinedocs/gccint/Directives.html" target="_top">Syntax 42136ac495dSmrg and Descriptions of test directives</a>. GCC also defines many <a class="link" href="https://gcc.gnu.org/onlinedocs/gccint/Effective-Target-Keywords.html" target="_top"> 42236ac495dSmrg Keywords describing target attributes</a> (a.k.a effective targets) 42336ac495dSmrg which can be used where a target <em class="replaceable"><code>selector</code></em> can 42436ac495dSmrg appear. 42536ac495dSmrg </p><p> 42636ac495dSmrg Some directives commonly used in the libstdc++ testsuite are: 42736ac495dSmrg 42836ac495dSmrg</p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">{ dg-do <em class="replaceable"><code>do-what-keyword</code></em> [{ target/xfail <em class="replaceable"><code>selector</code></em> }] }</code></span></dt><dd>Where <em class="replaceable"><code>do-what-keyword</code></em> is usually 42936ac495dSmrg one of <code class="literal">run</code> (which is the default), 43036ac495dSmrg <code class="literal">compile</code>, or <code class="literal">link</code>, 43136ac495dSmrg and typical selectors are targets such as <code class="literal">*-*-gnu*</code> 43236ac495dSmrg or an effective target such as <code class="literal">c++11</code>. 43336ac495dSmrg </dd><dt><span class="term"><code class="literal">{ dg-require-<em class="replaceable"><code>support</code></em> args }</code></span></dt><dd>Skip the test if the target does not provide the required support. 43436ac495dSmrg See below for values of <em class="replaceable"><code>support</code></em>. 43536ac495dSmrg </dd><dt><span class="term"><code class="literal">{ dg-options <em class="replaceable"><code>options</code></em> [{ target <em class="replaceable"><code>selector</code></em> }] }</code></span></dt><dd></dd><dt><span class="term"><code class="literal">{ dg-error <em class="replaceable"><code>regexp</code></em> [ <em class="replaceable"><code>comment</code></em> [{ target/xfail <em class="replaceable"><code>selector</code></em> } [<em class="replaceable"><code>line</code></em>] ]] }</code></span></dt><dd></dd><dt><span class="term"><code class="literal">{ dg-excess-errors <em class="replaceable"><code>comment</code></em> [{ target/xfail <em class="replaceable"><code>selector</code></em> }] }</code></span></dt><dd></dd></dl></div><p> 43636ac495dSmrg For full details of these and other directives see the main GCC DejaGnu 43736ac495dSmrg documentation in the internals manual. 43836ac495dSmrg </p><p> 43936ac495dSmrg Test cases that use features of a particular C++ standard should specify 44036ac495dSmrg the minimum required standard as an effective target: 44136ac495dSmrg</p><pre class="programlisting"> // { dg-do run { target c++11 } }</pre><p> 44236ac495dSmrg or 44336ac495dSmrg</p><pre class="programlisting"> // { dg-require-effective-target c++11 }</pre><p> 44436ac495dSmrg Specifying the minimum required standard for a test allows it to be run 44536ac495dSmrg using later standards, so that we can verify that C++11 components still 44636ac495dSmrg work correctly when compiled as C++14 or later. Specifying a minimum also 44736ac495dSmrg means the test will be skipped if the test is compiled using 44836ac495dSmrg an older standard, e.g. using 44936ac495dSmrg <code class="option">RUNTESTFLAGS=--target_board=unix/-std=gnu++98</code>. 45036ac495dSmrg </p><p> 45136ac495dSmrg It is possible to indicate that a test should <span class="emphasis"><em>only</em></span> 45236ac495dSmrg be run for a specific standard (and not later standards) using an 45336ac495dSmrg effective target like <code class="literal">c++11_only</code>. However, this means 45436ac495dSmrg the test will be skipped by default (because the default mode is 45536ac495dSmrg <code class="literal">gnu++14</code>), and so will only run when 45636ac495dSmrg <code class="option">-std=gnu++11</code> or <code class="option">-std=c++11</code> is used 45736ac495dSmrg explicitly. For tests that require a specific standard it is better to 45836ac495dSmrg use a <code class="literal">dg-options</code> directive: 45936ac495dSmrg</p><pre class="programlisting"> // { dg-options "-std=gnu++11" }</pre><p> 46036ac495dSmrg This means the test will not get skipped by default, and will always use 46136ac495dSmrg the specific standard dialect that the test requires. This isn't needed 46236ac495dSmrg often, and most tests should use an effective target to specify a 46336ac495dSmrg minimum standard instead, to allow them to be tested for all 46436ac495dSmrg possible variations. 46536ac495dSmrg </p><p> 46636ac495dSmrg Similarly, tests which depend on a newer standard than the default 46736ac495dSmrg must use <code class="literal">dg-options</code> instead of (or in addition to) 46836ac495dSmrg an effective target, so that they are not skipped by default. 46936ac495dSmrg For example, tests for C++17 features should use 47036ac495dSmrg</p><pre class="programlisting"> // { dg-options "-std=gnu++17" }</pre><p> 47136ac495dSmrg before any <code class="literal">dg-do</code> such as: 47236ac495dSmrg</p><pre class="programlisting"> // { dg-do run "c++17" }</pre><p> 47336ac495dSmrg The <code class="literal">dg-options</code> directive must come first, so that 47436ac495dSmrg the <code class="literal">-std</code> flag has already been added to the options 47536ac495dSmrg before checking the <code class="literal">c++17</code> target. 47636ac495dSmrg </p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="tests.dg.examples"></a>Examples of Test Directives</h4></div></div></div><p> 47736ac495dSmrgExample 1: Testing compilation only: 47836ac495dSmrg</p><pre class="programlisting"> 47936ac495dSmrg// { dg-do compile } 48036ac495dSmrg</pre><p> 48136ac495dSmrg 48236ac495dSmrgExample 2: Testing for expected warnings on line 36, which all targets fail: 48336ac495dSmrg</p><pre class="programlisting"> 48436ac495dSmrg// { dg-warning "string literals" "" { xfail *-*-* } 36 } 48536ac495dSmrg</pre><p> 48636ac495dSmrg 48736ac495dSmrgExample 3: Testing for expected warnings on line 36: 48836ac495dSmrg</p><pre class="programlisting"> 48936ac495dSmrg// { dg-warning "string literals" "" { target *-*-* } 36 } 49036ac495dSmrg</pre><p> 49136ac495dSmrg 49236ac495dSmrgExample 4: Testing for compilation errors on line 41: 49336ac495dSmrg</p><pre class="programlisting"> 49436ac495dSmrg// { dg-do compile } 49536ac495dSmrg// { dg-error "no match for" "" { target *-*-* } 41 } 49636ac495dSmrg</pre><p> 49736ac495dSmrg 49836ac495dSmrgExample 5: Testing with special command line settings, or without the 49936ac495dSmrguse of pre-compiled headers, in particular the 50036ac495dSmrg<code class="filename">stdc++.h.gch</code> file. Any 50136ac495dSmrgoptions here will override the <code class="varname">DEFAULT_CXXFLAGS</code> and 50236ac495dSmrg<code class="varname">PCH_CXXFLAGS</code> set up in the <code class="filename">normal.exp</code> 50336ac495dSmrgfile: 50436ac495dSmrg</p><pre class="programlisting"> 50536ac495dSmrg// { dg-options "-O0" { target *-*-* } } 50636ac495dSmrg</pre><p> 50736ac495dSmrg 50836ac495dSmrgExample 6: Compiling and linking a test only for C++14 and later, and only 50936ac495dSmrgif Debug Mode is active: 51036ac495dSmrg</p><pre class="programlisting"> 51136ac495dSmrg// { dg-do link { target c++14 } } 51236ac495dSmrg// { dg-require-debug-mode "" } 51336ac495dSmrg</pre><p> 51436ac495dSmrg 51536ac495dSmrgExample 7: Running a test only on x86 targets, and only for C++11 and later, 51636ac495dSmrgwith specific options, and additional options for 32-bit x86: 51736ac495dSmrg</p><pre class="programlisting"> 51836ac495dSmrg// { dg-options "-fstrict-enums" } 51936ac495dSmrg// { dg-additional-options "-march=i486" { target ia32 } } 52036ac495dSmrg// { dg-do run { target { ia32 || x86_64-*-* } } } 52136ac495dSmrg// { dg-require-effective-target "c++11" } 52236ac495dSmrg</pre><p> 52336ac495dSmrg </p><p> 52436ac495dSmrg More examples can be found in the 52536ac495dSmrg <code class="filename">libstdc++-v3/testsuite/*/*.cc</code> files. 52636ac495dSmrg </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="tests.dg.directives"></a>Directives Specific to Libstdc++ Tests</h4></div></div></div><p> 52736ac495dSmrg In addition to the usual <a class="link" href="https://gcc.gnu.org/onlinedocs/gccint/Require-Support.html" target="_top">Variants 52836ac495dSmrg of <code class="literal">dg-require-<em class="replaceable"><code>support</code></em></code></a> 52936ac495dSmrg several more directives are available for use in libstdc++ tests, 53036ac495dSmrg including the following: 53136ac495dSmrg </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">dg-require-namedlocale</code> <em class="replaceable"><code>name</code></em></span></dt><dd><p>The named locale must be available. 53236ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-debug-mode ""</code></span></dt><dd><p>Skip the test if the Debug Mode is not active 53336ac495dSmrg (as determined by the <code class="literal">_GLIBCXX_DEBUG</code> macro). 53436ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-parallel-mode ""</code></span></dt><dd><p>Skip the test if the Parallel Mode is not active 53536ac495dSmrg (as determined by the <code class="literal">_GLIBCXX_PARALLEL</code> macro). 536*8feb0f0bSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-normal-mode ""</code></span></dt><dd><p>Skip the test if Debug or Parallel Mode is active. 53736ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-atomic-builtins ""</code></span></dt><dd><p>Skip the test if atomic operations on <span class="type">bool</span> 53836ac495dSmrg and <span class="type">int</span> are not lock-free. 53936ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-gthreads ""</code></span></dt><dd><p>Skip the test if the C++11 thread library is not 54036ac495dSmrg supported, as determined by the <code class="literal">_GLIBCXX_HAS_GTHREADS</code> 54136ac495dSmrg macro. 54236ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-gthreads-timed ""</code></span></dt><dd><p>Skip the test if C++11 timed mutexes are not supported, 54336ac495dSmrg as determined by the <code class="literal">_GLIBCXX_HAS_GTHREADS</code> and 54436ac495dSmrg <code class="literal">_GTHREAD_USE_MUTEX_TIMEDLOCK</code> macros. 54536ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-string-conversions ""</code></span></dt><dd><p>Skip the test if the C++11 <code class="function">to_string</code> 54636ac495dSmrg and <code class="function">stoi</code>, <code class="function">stod</code> etc. functions 54736ac495dSmrg are not fully supported (including wide character versions). 54836ac495dSmrg </p></dd><dt><span class="term"><code class="literal">dg-require-filesystem-ts ""</code></span></dt><dd><p>Skip the test if the Filesystem TS is not supported. 54936ac495dSmrg </p></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.harness"></a>Test Harness and Utilities</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.dejagnu"></a>DejaGnu Harness Details</h4></div></div></div><p> 55036ac495dSmrg Underlying details of testing for conformance and regressions are 55136ac495dSmrg abstracted via the GNU DejaGnu package. This is similar to the 55236ac495dSmrg rest of GCC. 55336ac495dSmrg </p><p>This is information for those looking at making changes to the testsuite 55436ac495dSmrgstructure, and/or needing to trace DejaGnu's actions with 55536ac495dSmrg<code class="option">--verbose</code>. 55636ac495dSmrgThis will not be useful to people who are "merely" adding new tests 55736ac495dSmrgto the existing structure. 55836ac495dSmrg</p><p>The first key point when working with DejaGnu is the idea of a "tool". 55936ac495dSmrgFiles, directories, and functions are all implicitly used when they are 56036ac495dSmrgnamed after the tool in use. Here, the tool will always be "libstdc++". 56136ac495dSmrg</p><p>The <code class="code">lib</code> subdir contains support routines. The 56236ac495dSmrg<code class="code">lib/libstdc++.exp</code> file ("support library") is loaded 56336ac495dSmrgautomagically, and must explicitly load the others. For example, files can 56436ac495dSmrgbe copied from the core compiler's support directory into <code class="code">lib</code>. 56536ac495dSmrg</p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are 56636ac495dSmrgour own. Callbacks must be prefixed with the name of the tool. To easily 56736ac495dSmrgdistinguish the others, by convention our own routines are named "v3-*". 56836ac495dSmrg</p><p>The next key point when working with DejaGnu is "test files". Any 56936ac495dSmrgdirectory whose name starts with the tool name will be searched for test files. 57036ac495dSmrg(We have only one.) In those directories, any <code class="code">.exp</code> file is 57136ac495dSmrgconsidered a test file, and will be run in turn. Our main test file is called 57236ac495dSmrg<code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the 57336ac495dSmrgcallbacks loaded from the support library. 57436ac495dSmrg</p><p>The <code class="code">config</code> directory is searched for any particular "target 57536ac495dSmrgboard" information unique to this library. This is currently unused and sets 57636ac495dSmrgonly default variables. 57736ac495dSmrg</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.utils"></a>Utilities</h4></div></div></div><p> 57836ac495dSmrg </p><p> 57936ac495dSmrg The testsuite directory also contains some files that implement 58036ac495dSmrg functionality that is intended to make writing test cases easier, 58136ac495dSmrg or to avoid duplication, or to provide error checking in a way that 58236ac495dSmrg is consistent across platforms and test harnesses. A stand-alone 58336ac495dSmrg executable, called <span class="emphasis"><em>abi_check</em></span>, and a static 58436ac495dSmrg library called <span class="emphasis"><em>libtestc++</em></span> are 58536ac495dSmrg constructed. Both of these items are not installed, and only used 58636ac495dSmrg during testing. 58736ac495dSmrg </p><p> 58836ac495dSmrg These files include the following functionality: 58936ac495dSmrg </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 59036ac495dSmrg <span class="emphasis"><em>testsuite_abi.h</em></span>, 59136ac495dSmrg <span class="emphasis"><em>testsuite_abi.cc</em></span>, 59236ac495dSmrg <span class="emphasis"><em>testsuite_abi_check.cc</em></span> 59336ac495dSmrg </p><p> 59436ac495dSmrg Creates the executable <span class="emphasis"><em>abi_check</em></span>. 59536ac495dSmrg Used to check correctness of symbol versioning, visibility of 59636ac495dSmrg exported symbols, and compatibility on symbols in the shared 59736ac495dSmrg library, for hosts that support this feature. More information 59836ac495dSmrg can be found in the ABI documentation <a class="link" href="abi.html" title="ABI Policy and Guidelines">here</a> 59936ac495dSmrg </p></li><li class="listitem"><p> 60036ac495dSmrg <span class="emphasis"><em>testsuite_allocator.h</em></span>, 60136ac495dSmrg <span class="emphasis"><em>testsuite_allocator.cc</em></span> 60236ac495dSmrg </p><p> 60336ac495dSmrg Contains specialized allocators that keep track of construction 60436ac495dSmrg and destruction. Also, support for overriding global new and 60536ac495dSmrg delete operators, including verification that new and delete 60636ac495dSmrg are called during execution, and that allocation over max_size 60736ac495dSmrg fails. 60836ac495dSmrg </p></li><li class="listitem"><p> 60936ac495dSmrg <span class="emphasis"><em>testsuite_character.h</em></span> 61036ac495dSmrg </p><p> 61136ac495dSmrg Contains <code class="code">std::char_traits</code> and 61236ac495dSmrg <code class="code">std::codecvt</code> specializations for a user-defined 61336ac495dSmrg POD. 61436ac495dSmrg </p></li><li class="listitem"><p> 61536ac495dSmrg <span class="emphasis"><em>testsuite_hooks.h</em></span>, 61636ac495dSmrg <span class="emphasis"><em>testsuite_hooks.cc</em></span> 61736ac495dSmrg </p><p> 61836ac495dSmrg A large number of utilities, including: 61936ac495dSmrg </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>VERIFY</p></li><li class="listitem"><p>set_memory_limits</p></li><li class="listitem"><p>verify_demangle</p></li><li class="listitem"><p>run_tests_wrapped_locale</p></li><li class="listitem"><p>run_tests_wrapped_env</p></li><li class="listitem"><p>try_named_locale</p></li><li class="listitem"><p>try_mkfifo</p></li><li class="listitem"><p>func_callback</p></li><li class="listitem"><p>counter</p></li><li class="listitem"><p>copy_tracker</p></li><li class="listitem"><p>copy_constructor</p></li><li class="listitem"><p>assignment_operator</p></li><li class="listitem"><p>destructor</p></li><li class="listitem"><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li class="listitem"><p> 62036ac495dSmrg <span class="emphasis"><em>testsuite_io.h</em></span> 62136ac495dSmrg </p><p> 62236ac495dSmrg Error, exception, and constraint checking for 62336ac495dSmrg <code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>. 62436ac495dSmrg </p></li><li class="listitem"><p> 62536ac495dSmrg <span class="emphasis"><em>testsuite_iterators.h</em></span> 62636ac495dSmrg </p><p> 62736ac495dSmrg Wrappers for various iterators. 62836ac495dSmrg </p></li><li class="listitem"><p> 62936ac495dSmrg <span class="emphasis"><em>testsuite_performance.h</em></span> 63036ac495dSmrg </p><p> 63136ac495dSmrg A number of class abstractions for performance counters, and 63236ac495dSmrg reporting functions including: 63336ac495dSmrg </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>time_counter</p></li><li class="listitem"><p>resource_counter</p></li><li class="listitem"><p>report_performance</p></li></ul></div></li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.special"></a>Special Topics</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.exception.safety"></a> 63436ac495dSmrg Qualifying Exception Safety Guarantees 63536ac495dSmrg <a id="id-1.3.6.3.5.7.2.1.1.1" class="indexterm"></a> 63636ac495dSmrg</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.overview"></a>Overview</h5></div></div></div><p> 63736ac495dSmrg Testing is composed of running a particular test sequence, 63836ac495dSmrg and looking at what happens to the surrounding code when 63936ac495dSmrg exceptions are thrown. Each test is composed of measuring 64036ac495dSmrg initial state, executing a particular sequence of code under 64136ac495dSmrg some instrumented conditions, measuring a final state, and 64236ac495dSmrg then examining the differences between the two states. 64336ac495dSmrg </p><p> 64436ac495dSmrg Test sequences are composed of constructed code sequences 64536ac495dSmrg that exercise a particular function or member function, and 64636ac495dSmrg either confirm no exceptions were generated, or confirm the 64736ac495dSmrg consistency/coherency of the test subject in the event of a 64836ac495dSmrg thrown exception. 64936ac495dSmrg </p><p> 65036ac495dSmrg Random code paths can be constructed using the basic test 65136ac495dSmrg sequences and instrumentation as above, only combined in a 65236ac495dSmrg random or pseudo-random way. 65336ac495dSmrg </p><p> To compute the code paths that throw, test instruments 65436ac495dSmrg are used that throw on allocation events 65536ac495dSmrg (<code class="classname">__gnu_cxx::throw_allocator_random</code> 65636ac495dSmrg and <code class="classname">__gnu_cxx::throw_allocator_limit</code>) 65736ac495dSmrg and copy, assignment, comparison, increment, swap, and 65836ac495dSmrg various operators 65936ac495dSmrg (<code class="classname">__gnu_cxx::throw_type_random</code> 66036ac495dSmrg and <code class="classname">__gnu_cxx::throw_type_limit</code>). Looping 66136ac495dSmrg through a given test sequence and conditionally throwing in 66236ac495dSmrg all instrumented places. Then, when the test sequence 66336ac495dSmrg completes without an exception being thrown, assume all 66436ac495dSmrg potential error paths have been exercised in a sequential 66536ac495dSmrg manner. 66636ac495dSmrg </p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.status"></a> 66736ac495dSmrg Existing tests 66836ac495dSmrg</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 66936ac495dSmrg Ad Hoc 67036ac495dSmrg </p><p> 67136ac495dSmrg For example, 67236ac495dSmrg <code class="filename">testsuite/23_containers/list/modifiers/3.cc</code>. 67336ac495dSmrg </p></li><li class="listitem"><p> 67436ac495dSmrg Policy Based Data Structures 67536ac495dSmrg </p><p> 67636ac495dSmrg For example, take the test 67736ac495dSmrg functor <code class="classname">rand_reg_test</code> in 67836ac495dSmrg in <code class="filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc</code>. This uses <code class="classname">container_rand_regression_test</code> in 67936ac495dSmrg<code class="filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h</code>. 68036ac495dSmrg 68136ac495dSmrg </p><p> 68236ac495dSmrg Which has several tests for container member functions, 68336ac495dSmrgIncludes control and test container objects. Configuration includes 68436ac495dSmrgrandom seed, iterations, number of distinct values, and the 68536ac495dSmrgprobability that an exception will be thrown. Assumes instantiating 68636ac495dSmrgcontainer uses an extension 68736ac495dSmrgallocator, <code class="classname">__gnu_cxx::throw_allocator_random</code>, 68836ac495dSmrgas the allocator type. 68936ac495dSmrg </p></li><li class="listitem"><p> 69036ac495dSmrg C++11 Container Requirements. 69136ac495dSmrg </p><p> 69236ac495dSmrg Coverage is currently limited to testing container 69336ac495dSmrg requirements for exception safety, 69436ac495dSmrg although <code class="classname">__gnu_cxx::throw_type</code> meets 69536ac495dSmrg the additional type requirements for testing numeric data 69636ac495dSmrg structures and instantiating algorithms. 69736ac495dSmrg </p><p> 69836ac495dSmrg Of particular interest is extending testing to algorithms and 69936ac495dSmrg then to parallel algorithms. Also io and locales. 70036ac495dSmrg </p><p> 70136ac495dSmrg The test instrumentation should also be extended to add 70236ac495dSmrg instrumentation to <code class="classname">iterator</code> 70336ac495dSmrg and <code class="classname">const_iterator</code> types that throw 70436ac495dSmrg conditionally on iterator operations. 70536ac495dSmrg </p></li></ul></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.containers"></a> 70636ac495dSmrgC++11 Requirements Test Sequence Descriptions 70736ac495dSmrg</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 70836ac495dSmrg Basic 70936ac495dSmrg </p><p> 71036ac495dSmrg Basic consistency on exception propagation tests. For 71136ac495dSmrg each container, an object of that container is constructed, 71236ac495dSmrg a specific member function is exercised in 71336ac495dSmrg a <code class="literal">try</code> block, and then any thrown 71436ac495dSmrg exceptions lead to error checking in the appropriate 71536ac495dSmrg <code class="literal">catch</code> block. The container's use of 71636ac495dSmrg resources is compared to the container's use prior to the 71736ac495dSmrg test block. Resource monitoring is limited to allocations 71836ac495dSmrg made through the container's <span class="type">allocator_type</span>, 71936ac495dSmrg which should be sufficient for container data 72036ac495dSmrg structures. Included in these tests are member functions 72136ac495dSmrg are <span class="type">iterator</span> and <span class="type">const_iterator</span> 72236ac495dSmrg operations, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, <code class="function">erase</code>, <code class="function">swap</code>, <code class="function">clear</code>, 72336ac495dSmrg and <code class="function">rehash</code>. The container in question is 72436ac495dSmrg instantiated with two instrumented template arguments, 72536ac495dSmrg with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 72636ac495dSmrg as the allocator type, and 72736ac495dSmrg with <code class="classname">__gnu_cxx::throw_type_limit</code> as 72836ac495dSmrg the value type. This allows the test to loop through 72936ac495dSmrg conditional throw points. 73036ac495dSmrg </p><p> 73136ac495dSmrg The general form is demonstrated in 73236ac495dSmrg <code class="filename">testsuite/23_containers/list/requirements/exception/basic.cc 73336ac495dSmrg </code>. The instantiating test object is <code class="classname">__gnu_test::basic_safety</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 73436ac495dSmrg </p></li><li class="listitem"><p> 73536ac495dSmrg Generation Prohibited 73636ac495dSmrg </p><p> 73736ac495dSmrg Exception generation tests. For each container, an object of 73836ac495dSmrg that container is constructed and all member functions 73936ac495dSmrg required to not throw exceptions are exercised. Included in 74036ac495dSmrg these tests are member functions 74136ac495dSmrg are <span class="type">iterator</span> and <span class="type">const_iterator</span> operations, <code class="function">erase</code>, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">swap</code>, 74236ac495dSmrg and <code class="function">clear</code>. The container in question is 74336ac495dSmrg instantiated with two instrumented template arguments, 74436ac495dSmrg with <code class="classname">__gnu_cxx::throw_allocator_random</code> 74536ac495dSmrg as the allocator type, and 74636ac495dSmrg with <code class="classname">__gnu_cxx::throw_type_random</code> as 74736ac495dSmrg the value type. This test does not loop, an instead is sudden 74836ac495dSmrg death: first error fails. 74936ac495dSmrg </p><p> 75036ac495dSmrg The general form is demonstrated in 75136ac495dSmrg <code class="filename">testsuite/23_containers/list/requirements/exception/generation_prohibited.cc 75236ac495dSmrg </code>. The instantiating test object is <code class="classname">__gnu_test::generation_prohibited</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 75336ac495dSmrg </p></li><li class="listitem"><p> 75436ac495dSmrg Propagation Consistent 75536ac495dSmrg </p><p> 75636ac495dSmrg Container rollback on exception propagation tests. For 75736ac495dSmrg each container, an object of that container is constructed, 75836ac495dSmrg a specific member function that requires rollback to a previous 75936ac495dSmrg known good state is exercised in 76036ac495dSmrg a <code class="literal">try</code> block, and then any thrown 76136ac495dSmrg exceptions lead to error checking in the appropriate 76236ac495dSmrg <code class="literal">catch</code> block. The container is compared to 76336ac495dSmrg the container's last known good state using such parameters 76436ac495dSmrg as size, contents, and iterator references. Included in these 76536ac495dSmrg tests are member functions 76636ac495dSmrg are <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, 76736ac495dSmrg and <code class="function">rehash</code>. The container in question is 76836ac495dSmrg instantiated with two instrumented template arguments, 76936ac495dSmrg with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 77036ac495dSmrg as the allocator type, and 77136ac495dSmrg with <code class="classname">__gnu_cxx::throw_type_limit</code> as 77236ac495dSmrg the value type. This allows the test to loop through 77336ac495dSmrg conditional throw points. 77436ac495dSmrg </p><p> 77536ac495dSmrg The general form demonstrated in 77636ac495dSmrg <code class="filename">testsuite/23_containers/list/requirements/exception/propagation_coherent.cc 77736ac495dSmrg </code>. The instantiating test object is <code class="classname">__gnu_test::propagation_coherent</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 77836ac495dSmrg </p></li></ul></div></div></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="appendix_porting.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Porting to New Hardware or Operating Systems </td><td width="20%" align="center"><a accesskey="h" href="../index.html">Home</a></td><td width="40%" align="right" valign="top"> ABI Policy and Guidelines</td></tr></table></div></body></html>