1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Test</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2" /><meta name="keywords" content=" ISO C++ , test , testsuite , performance , conformance , ABI , exception safety " /><meta name="keywords" content=" ISO C++ , library " /><link rel="home" href="../spine.html" title="The GNU C++ Library Documentation" /><link rel="up" href="appendix_porting.html" title="Appendix B. Porting and Maintenance" /><link rel="prev" href="internals.html" title="Porting to New Hardware or Operating Systems" /><link rel="next" href="abi.html" title="ABI Policy and Guidelines" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Test</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><th width="60%" align="center">Appendix B. 4 Porting and Maintenance 5 6</th><td width="20%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr></table><hr /></div><div class="sect1" title="Test"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="manual.intro.setup.test"></a>Test</h2></div></div></div><p> 7The libstdc++ testsuite includes testing for standard conformance, 8regressions, ABI, and performance. 9</p><div class="sect2" title="Organization"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"></a>Organization</h3></div></div></div><div class="sect3" title="Directory Layout"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.layout"></a>Directory Layout</h4></div></div></div><p> 10 The directory <span class="emphasis"><em>libsrcdir/testsuite</em></span> contains the 11 individual test cases organized in sub-directories corresponding to 12 chapters of the C++ standard (detailed below), the dejagnu test 13 harness support files, and sources to various testsuite utilities 14 that are packaged in a separate testing library. 15</p><p> 16 All test cases for functionality required by the runtime components 17 of the C++ standard (ISO 14882) are files within the following 18 directories. 19</p><pre class="programlisting"> 2017_intro 2118_support 2219_diagnostics 2320_util 2421_strings 2522_locale 2623_containers 2725_algorithms 2826_numerics 2927_io 3028_regex 3129_atomics 3230_threads 33 </pre><p> 34 In addition, the following directories include test files: 35 </p><pre class="programlisting"> 36tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1). 37backward Tests for backwards compatibility and deprecated features. 38demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler 39ext Tests for extensions. 40performance Tests for performance analysis, and performance regressions. 41 </pre><p> 42 Some directories don't have test files, but instead contain 43 auxiliary information: 44 </p><pre class="programlisting"> 45config Files for the dejagnu test harness. 46lib Files for the dejagnu test harness. 47libstdc++* Files for the dejagnu test harness. 48data Sample text files for testing input and output. 49util Files for libtestc++, utilities and testing routines. 50 </pre><p> 51 Within a directory that includes test files, there may be 52 additional subdirectories, or files. Originally, test cases 53 were appended to one file that represented a particular section 54 of the chapter under test, and was named accordingly. For 55 instance, to test items related to <code class="code"> 21.3.6.1 - 56 basic_string::find [lib.string::find]</code> in the standard, 57 the following was used: 58 </p><pre class="programlisting"> 5921_strings/find.cc 60 </pre><p> 61 However, that practice soon became a liability as the test cases 62 became huge and unwieldy, and testing new or extended 63 functionality (like wide characters or named locales) became 64 frustrating, leading to aggressive pruning of test cases on some 65 platforms that covered up implementation errors. Now, the test 66 suite has a policy of one file, one test case, which solves the 67 above issues and gives finer grained results and more manageable 68 error debugging. As an example, the test case quoted above 69 becomes: 70 </p><pre class="programlisting"> 7121_strings/basic_string/find/char/1.cc 7221_strings/basic_string/find/char/2.cc 7321_strings/basic_string/find/char/3.cc 7421_strings/basic_string/find/wchar_t/1.cc 7521_strings/basic_string/find/wchar_t/2.cc 7621_strings/basic_string/find/wchar_t/3.cc 77 </pre><p> 78 All new tests should be written with the policy of one test 79 case, one file in mind. 80 </p></div><div class="sect3" title="Naming Conventions"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.naming"></a>Naming Conventions</h4></div></div></div><p> 81 In addition, there are some special names and suffixes that are 82 used within the testsuite to designate particular kinds of 83 tests. 84 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p> 85 <span class="emphasis"><em>_xin.cc</em></span> 86 </p><p> 87 This test case expects some kind of interactive input in order 88 to finish or pass. At the moment, the interactive tests are not 89 run by default. Instead, they are run by hand, like: 90 </p><pre class="programlisting"> 91g++ 27_io/objects/char/3_xin.cc 92cat 27_io/objects/char/3_xin.in | a.out 93 </pre></li><li class="listitem"><p> 94 <span class="emphasis"><em>.in</em></span> 95 </p><p> 96 This file contains the expected input for the corresponding <span class="emphasis"><em> 97 _xin.cc</em></span> test case. 98 </p></li><li class="listitem"><p> 99 <span class="emphasis"><em>_neg.cc</em></span> 100 </p><p> 101 This test case is expected to fail: it's a negative test. At the 102 moment, these are almost always compile time errors. 103 </p></li><li class="listitem"><p> 104 <span class="emphasis"><em>char</em></span> 105 </p><p> 106 This can either be a directory name or part of a longer file 107 name, and indicates that this file, or the files within this 108 directory are testing the <code class="code">char</code> instantiation of a 109 template. 110 </p></li><li class="listitem"><p> 111 <span class="emphasis"><em>wchar_t</em></span> 112 </p><p> 113 This can either be a directory name or part of a longer file 114 name, and indicates that this file, or the files within this 115 directory are testing the <code class="code">wchar_t</code> instantiation of 116 a template. Some hosts do not support <code class="code">wchar_t</code> 117 functionality, so for these targets, all of these tests will not 118 be run. 119 </p></li><li class="listitem"><p> 120 <span class="emphasis"><em>thread</em></span> 121 </p><p> 122 This can either be a directory name or part of a longer file 123 name, and indicates that this file, or the files within this 124 directory are testing situations where multiple threads are 125 being used. 126 </p></li><li class="listitem"><p> 127 <span class="emphasis"><em>performance</em></span> 128 </p><p> 129 This can either be an enclosing directory name or part of a 130 specific file name. This indicates a test that is used to 131 analyze runtime performance, for performance regression testing, 132 or for other optimization related analysis. At the moment, these 133 test cases are not run by default. 134 </p></li></ul></div></div></div><div class="sect2" title="Running the Testsuite"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"></a>Running the Testsuite</h3></div></div></div><div class="sect3" title="Basic"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"></a>Basic</h4></div></div></div><p> 135 You can check the status of the build without installing it 136 using the dejagnu harness, much like the rest of the gcc 137 tools.</p><pre class="programlisting"> make check</pre><p>in the <span class="emphasis"><em>libbuilddir</em></span> directory.</p><p>or</p><pre class="programlisting"> make check-target-libstdc++-v3</pre><p>in the <span class="emphasis"><em>gccbuilddir</em></span> directory. 138 </p><p> 139 These commands are functionally equivalent and will create a 140 'testsuite' directory underneath 141 <span class="emphasis"><em>libbuilddir</em></span> containing the results of the 142 tests. Two results files will be generated: <span class="emphasis"><em> 143 libstdc++.sum</em></span>, which is a PASS/FAIL summary for each 144 test, and <span class="emphasis"><em>libstdc++.log</em></span> which is a log of 145 the exact command line passed to the compiler, the compiler 146 output, and the executable output (if any). 147 </p><p> 148 Archives of test results for various versions and platforms are 149 available on the GCC website in the <a class="ulink" href="http://gcc.gnu.org/gcc-4.3/buildstat.html" target="_top">build 150 status</a> section of each individual release, and are also 151 archived on a daily basis on the <a class="ulink" href="http://gcc.gnu.org/ml/gcc-testresults/current" target="_top">gcc-testresults</a> 152 mailing list. Please check either of these places for a similar 153 combination of source version, operating system, and host CPU. 154 </p></div><div class="sect3" title="Variations"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.variations"></a>Variations</h4></div></div></div><p> 155 There are several options for running tests, including testing 156 the regression tests, testing a subset of the regression tests, 157 testing the performance tests, testing just compilation, testing 158 installed tools, etc. In addition, there is a special rule for 159 checking the exported symbols of the shared library. 160 </p><p> 161 To debug the dejagnu test harness during runs, try invoking with a 162 specific argument to the variable RUNTESTFLAGS, as below. 163 </p><pre class="programlisting"> 164make check-target-libstdc++-v3 RUNTESTFLAGS="-v" 165</pre><p> 166 or 167 </p><pre class="programlisting"> 168make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" 169</pre><p> 170 To run a subset of the library tests, you will need to generate 171 the <span class="emphasis"><em>testsuite_files</em></span> file by running 172 <span class="command"><strong>make testsuite_files</strong></span> in the 173 <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory, described 174 below. Edit the file to remove the tests you don't want and 175 then run the testsuite as normal. 176 </p><p> 177 There are two ways to run on a simulator: set up DEJAGNU to point to a 178 specially crafted site.exp, or pass down --target_board flags. 179 </p><p> 180 Example flags to pass down for various embedded builds are as follows: 181 </p><pre class="programlisting"> 182 --target=powerpc-eabism (libgloss/sim) 183make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" 184 185--target=calmrisc32 (libgloss/sid) 186make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" 187 188--target=xscale-elf (newlib/sim) 189make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" 190</pre><p> 191 Also, here is an example of how to run the libstdc++ testsuite 192 for a multilibed build directory with different ABI settings: 193 </p><pre class="programlisting"> 194make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' 195</pre><p> 196 You can run the tests with a compiler and library that have 197 already been installed. Make sure that the compiler (e.g., 198 <code class="code">g++</code>) is in your <code class="code">PATH</code>. If you are 199 using shared libraries, then you must also ensure that the 200 directory containing the shared version of libstdc++ is in your 201 <code class="code">LD_LIBRARY_PATH</code>, or equivalent. If your GCC source 202 tree is at <code class="code">/path/to/gcc</code>, then you can run the tests 203 as follows: 204 </p><pre class="programlisting"> 205runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite 206</pre><p> 207 The testsuite will create a number of files in the directory in 208 which you run this command,. Some of those files might use the 209 same name as files created by other testsuites (like the ones 210 for GCC and G++), so you should not try to run all the 211 testsuites in parallel from the same directory. 212 </p><p> 213 In addition, there are some testing options that are mostly of 214 interest to library maintainers and system integrators. As such, 215 these tests may not work on all cpu and host combinations, and 216 may need to be executed in the 217 <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory. These 218 options include, but are not necessarily limited to, the 219 following: 220 </p><pre class="programlisting"> 221 make testsuite_files 222 </pre><p> 223 Five files are generated that determine what test files 224 are run. These files are: 225 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p> 226 <span class="emphasis"><em>testsuite_files</em></span> 227 </p><p> 228 This is a list of all the test cases that will be run. Each 229 test case is on a separate line, given with an absolute path 230 from the <span class="emphasis"><em>libsrcdir/testsuite</em></span> directory. 231 </p></li><li class="listitem"><p> 232 <span class="emphasis"><em>testsuite_files_interactive</em></span> 233 </p><p> 234 This is a list of all the interactive test cases, using the 235 same format as the file list above. These tests are not run 236 by default. 237 </p></li><li class="listitem"><p> 238 <span class="emphasis"><em>testsuite_files_performance</em></span> 239 </p><p> 240 This is a list of all the performance test cases, using the 241 same format as the file list above. These tests are not run 242 by default. 243 </p></li><li class="listitem"><p> 244 <span class="emphasis"><em>testsuite_thread</em></span> 245 </p><p> 246 This file indicates that the host system can run tests which 247 involved multiple threads. 248 </p></li><li class="listitem"><p> 249 <span class="emphasis"><em>testsuite_wchar_t</em></span> 250 </p><p> 251 This file indicates that the host system can run the wchar_t 252 tests, and corresponds to the macro definition <code class="code"> 253 _GLIBCXX_USE_WCHAR_T</code> in the file c++config.h. 254 </p></li></ul></div><pre class="programlisting"> 255 make check-abi 256 </pre><p> 257 The library ABI can be tested. This involves testing the shared 258 library against an ABI-defining previous version of symbol 259 exports. 260 </p><pre class="programlisting"> 261 make check-compile 262 </pre><p> 263 This rule compiles, but does not link or execute, the 264 <span class="emphasis"><em>testsuite_files</em></span> test cases and displays the 265 output on stdout. 266 </p><pre class="programlisting"> 267 make check-performance 268 </pre><p> 269 This rule runs through the 270 <span class="emphasis"><em>testsuite_files_performance</em></span> test cases and 271 collects information for performance analysis and can be used to 272 spot performance regressions. Various timing information is 273 collected, as well as number of hard page faults, and memory 274 used. This is not run by default, and the implementation is in 275 flux. 276 </p><p> 277 We are interested in any strange failures of the testsuite; 278 please email the main libstdc++ mailing list if you see 279 something odd or have questions. 280 </p></div><div class="sect3" title="Permutations"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"></a>Permutations</h4></div></div></div><p> 281 To run the libstdc++ test suite under the <a class="link" href="debug_mode.html" title="Chapter 30. Debug Mode">debug mode</a>, edit 282 <code class="filename">libstdc++-v3/scripts/testsuite_flags</code> to add the 283 compile-time flag <code class="constant">-D_GLIBCXX_DEBUG</code> to the 284 result printed by the <code class="literal">--build-cxx</code> 285 option. Additionally, add the 286 <code class="constant">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on 287 pedantic checking. The libstdc++ test suite should produce 288 precisely the same results under debug mode that it does under 289 release mode: any deviation indicates an error in either the 290 library or the test suite. 291 </p><p> 292 The <a class="link" href="parallel_mode.html" title="Chapter 31. Parallel Mode">parallel 293 mode</a> can be tested in much the same manner, substituting 294 <code class="constant">-D_GLIBCXX_PARALLEL</code> for 295 <code class="constant">-D_GLIBCXX_DEBUG</code> in the previous paragraph. 296 </p><p> 297 Or, just run the testsuites with <code class="constant">CXXFLAGS</code> 298 set to <code class="constant">-D_GLIBCXX_DEBUG</code> or 299 <code class="constant">-D_GLIBCXX_PARALLEL</code>. 300 </p></div></div><div class="sect2" title="Writing a new test case"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"></a>Writing a new test case</h3></div></div></div><p> 301 The first step in making a new test case is to choose the correct 302 directory and file name, given the organization as previously 303 described. 304 </p><p> 305 All files are copyright the FSF, and GPL'd: this is very 306 important. The first copyright year should correspond to the date 307 the file was checked in to SVN. 308 </p><p> 309 As per the dejagnu instructions, always return 0 from main to 310 indicate success. 311 </p><p> 312 A bunch of utility functions and classes have already been 313 abstracted out into the testsuite utility library, <code class="code"> 314 libtestc++</code>. To use this functionality, just include the 315 appropriate header file: the library or specific object files will 316 automatically be linked in as part of the testsuite run. 317 </p><p> 318 For a test that needs to take advantage of the dejagnu test 319 harness, what follows below is a list of special keyword that 320 harness uses. Basically, a test case contains dg-keywords (see 321 dg.exp) indicating what to do and what kinds of behavior are to be 322 expected. New test cases should be written with the new style 323 DejaGnu framework in mind. 324 </p><p> 325 To ease transition, here is the list of dg-keyword documentation 326 lifted from dg.exp. 327 </p><pre class="programlisting"> 328# The currently supported options are: 329# 330# dg-prms-id N 331# set prms_id to N 332# 333# dg-options "options ..." [{ target selector }] 334# specify special options to pass to the tool (eg: compiler) 335# 336# dg-do do-what-keyword [{ target/xfail selector }] 337# `do-what-keyword' is tool specific and is passed unchanged to 338# ${tool}-dg-test. An example is gcc where `keyword' can be any of: 339# preprocess|compile|assemble|link|run 340# and will do one of: produce a .i, produce a .s, produce a .o, 341# produce an a.out, or produce an a.out and run it (the default is 342# compile). 343# 344# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]] 345# indicate an error message <regexp> is expected on this line 346# (the test fails if it doesn't occur) 347# Linenum=0 for general tool messages (eg: -V arg missing). 348# "." means the current line. 349# 350# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]] 351# indicate a warning message <regexp> is expected on this line 352# (the test fails if it doesn't occur) 353# 354# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]] 355# indicate a bogus error message <regexp> use to occur here 356# (the test fails if it does occur) 357# 358# dg-build regexp comment [{ target/xfail selector }] 359# indicate the build use to fail for some reason 360# (errors covered here include bad assembler generated, tool crashes, 361# and link failures) 362# (the test fails if it does occur) 363# 364# dg-excess-errors comment [{ target/xfail selector }] 365# indicate excess errors are expected (any line) 366# (this should only be used sparingly and temporarily) 367# 368# dg-output regexp [{ target selector }] 369# indicate the expected output of the program is <regexp> 370# (there may be multiple occurrences of this, they are concatenated) 371# 372# dg-final { tcl code } 373# add some tcl code to be run at the end 374# (there may be multiple occurrences of this, they are concatenated) 375# (unbalanced braces must be \-escaped) 376# 377# "{ target selector }" is a list of expressions that determine whether the 378# test succeeds or fails for a particular target, or in some cases whether the 379# option applies for a particular target. If the case of `dg-do' it specifies 380# whether the test case is even attempted on the specified target. 381# 382# The target selector is always optional. The format is one of: 383# 384# { xfail *-*-* ... } - the test is expected to fail for the given targets 385# { target *-*-* ... } - the option only applies to the given targets 386# 387# At least one target must be specified, use *-*-* for "all targets". 388# At present it is not possible to specify both `xfail' and `target'. 389# "native" may be used in place of "*-*-*". 390 391Example 1: Testing compilation only 392// { dg-do compile } 393 394Example 2: Testing for expected warnings on line 36, which all targets fail 395// { dg-warning "string literals" "" { xfail *-*-* } 36 396 397Example 3: Testing for expected warnings on line 36 398// { dg-warning "string literals" "" { target *-*-* } 36 399 400Example 4: Testing for compilation errors on line 41 401// { dg-do compile } 402// { dg-error "no match for" "" { target *-*-* } 41 } 403 404Example 5: Testing with special command line settings, or without the 405use of pre-compiled headers, in particular the stdc++.h.gch file. Any 406options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set 407up in the normal.exp file. 408// { dg-options "-O0" { target *-*-* } } 409</pre><p> 410 More examples can be found in the libstdc++-v3/testsuite/*/*.cc files. 411 </p></div><div class="sect2" title="Test Harness and Utilities"><div class="titlepage"><div><div><h3 class="title"><a id="test.harness"></a>Test Harness and Utilities</h3></div></div></div><div class="sect3" title="Dejagnu Harness Details"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.dejagnu"></a>Dejagnu Harness Details</h4></div></div></div><p> 412 Underlying details of testing for conformance and regressions are 413 abstracted via the GNU Dejagnu package. This is similar to the 414 rest of GCC. 415 </p><p>This is information for those looking at making changes to the testsuite 416structure, and/or needing to trace dejagnu's actions with --verbose. This 417will not be useful to people who are "merely" adding new tests to the existing 418structure. 419</p><p>The first key point when working with dejagnu is the idea of a "tool". 420Files, directories, and functions are all implicitly used when they are 421named after the tool in use. Here, the tool will always be "libstdc++". 422</p><p>The <code class="code">lib</code> subdir contains support routines. The 423<code class="code">lib/libstdc++.exp</code> file ("support library") is loaded 424automagically, and must explicitly load the others. For example, files can 425be copied from the core compiler's support directory into <code class="code">lib</code>. 426</p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are 427our own. Callbacks must be prefixed with the name of the tool. To easily 428distinguish the others, by convention our own routines are named "v3-*". 429</p><p>The next key point when working with dejagnu is "test files". Any 430directory whose name starts with the tool name will be searched for test files. 431(We have only one.) In those directories, any <code class="code">.exp</code> file is 432considered a test file, and will be run in turn. Our main test file is called 433<code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the 434callbacks loaded from the support library. 435</p><p>The <code class="code">config</code> directory is searched for any particular "target 436board" information unique to this library. This is currently unused and sets 437only default variables. 438</p></div><div class="sect3" title="Utilities"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.utils"></a>Utilities</h4></div></div></div><p> 439 </p><p> 440 The testsuite directory also contains some files that implement 441 functionality that is intended to make writing test cases easier, 442 or to avoid duplication, or to provide error checking in a way that 443 is consistent across platforms and test harnesses. A stand-alone 444 executable, called <span class="emphasis"><em>abi_check</em></span>, and a static 445 library called <span class="emphasis"><em>libtestc++</em></span> are 446 constructed. Both of these items are not installed, and only used 447 during testing. 448 </p><p> 449 These files include the following functionality: 450 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p> 451 <span class="emphasis"><em>testsuite_abi.h</em></span>, 452 <span class="emphasis"><em>testsuite_abi.cc</em></span>, 453 <span class="emphasis"><em>testsuite_abi_check.cc</em></span> 454 </p><p> 455 Creates the executable <span class="emphasis"><em>abi_check</em></span>. 456 Used to check correctness of symbol versioning, visibility of 457 exported symbols, and compatibility on symbols in the shared 458 library, for hosts that support this feature. More information 459 can be found in the ABI documentation <a class="link" href="abi.html" title="ABI Policy and Guidelines">here</a> 460 </p></li><li class="listitem"><p> 461 <span class="emphasis"><em>testsuite_allocator.h</em></span>, 462 <span class="emphasis"><em>testsuite_allocator.cc</em></span> 463 </p><p> 464 Contains specialized allocators that keep track of construction 465 and destruction. Also, support for overriding global new and 466 delete operators, including verification that new and delete 467 are called during execution, and that allocation over max_size 468 fails. 469 </p></li><li class="listitem"><p> 470 <span class="emphasis"><em>testsuite_character.h</em></span> 471 </p><p> 472 Contains <code class="code">std::char_traits</code> and 473 <code class="code">std::codecvt</code> specializations for a user-defined 474 POD. 475 </p></li><li class="listitem"><p> 476 <span class="emphasis"><em>testsuite_hooks.h</em></span>, 477 <span class="emphasis"><em>testsuite_hooks.cc</em></span> 478 </p><p> 479 A large number of utilities, including: 480 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>VERIFY</p></li><li class="listitem"><p>set_memory_limits</p></li><li class="listitem"><p>verify_demangle</p></li><li class="listitem"><p>run_tests_wrapped_locale</p></li><li class="listitem"><p>run_tests_wrapped_env</p></li><li class="listitem"><p>try_named_locale</p></li><li class="listitem"><p>try_mkfifo</p></li><li class="listitem"><p>func_callback</p></li><li class="listitem"><p>counter</p></li><li class="listitem"><p>copy_tracker</p></li><li class="listitem"><p>copy_constructor</p></li><li class="listitem"><p>assignment_operator</p></li><li class="listitem"><p>destructor</p></li><li class="listitem"><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li class="listitem"><p> 481 <span class="emphasis"><em>testsuite_io.h</em></span> 482 </p><p> 483 Error, exception, and constraint checking for 484 <code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>. 485 </p></li><li class="listitem"><p> 486 <span class="emphasis"><em>testsuite_iterators.h</em></span> 487 </p><p> 488 Wrappers for various iterators. 489 </p></li><li class="listitem"><p> 490 <span class="emphasis"><em>testsuite_performance.h</em></span> 491 </p><p> 492 A number of class abstractions for performance counters, and 493 reporting functions including: 494 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>time_counter</p></li><li class="listitem"><p>resource_counter</p></li><li class="listitem"><p>report_performance</p></li></ul></div></li></ul></div></div></div><div class="sect2" title="Special Topics"><div class="titlepage"><div><div><h3 class="title"><a id="test.special"></a>Special Topics</h3></div></div></div><div class="sect3" title="Qualifying Exception Safety Guarantees"><div class="titlepage"><div><div><h4 class="title"><a id="test.exception.safety"></a> 495 Qualifying Exception Safety Guarantees 496 <a id="id650235" class="indexterm"></a> 497</h4></div></div></div><div class="sect4" title="Overview"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.overview"></a>Overview</h5></div></div></div><p> 498 Testing is composed of running a particular test sequence, 499 and looking at what happens to the surrounding code when 500 exceptions are thrown. Each test is composed of measuring 501 initial state, executing a particular sequence of code under 502 some instrumented conditions, measuring a final state, and 503 then examining the differences between the two states. 504 </p><p> 505 Test sequences are composed of constructed code sequences 506 that exercise a particular function or member function, and 507 either confirm no exceptions were generated, or confirm the 508 consistency/coherency of the test subject in the event of a 509 thrown exception. 510 </p><p> 511 Random code paths can be constructed using the the basic test 512 sequences and instrumentation as above, only combined in a 513 random or pseudo-random way. 514 </p><p> To compute the code paths that throw, test instruments 515 are used that throw on allocation events 516 (<code class="classname">__gnu_cxx::throw_allocator_random</code> 517 and <code class="classname">__gnu_cxx::throw_allocator_limit</code>) 518 and copy, assignment, comparison, increment, swap, and 519 various operators 520 (<code class="classname">__gnu_cxx::throw_type_random</code> 521 and <code class="classname">__gnu_cxx::throw_type_limit</code>). Looping 522 through a given test sequence and conditionally throwing in 523 all instrumented places. Then, when the test sequence 524 completes without an exception being thrown, assume all 525 potential error paths have been exercised in a sequential 526 manner. 527 </p></div><div class="sect4" title="Existing tests"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.status"></a> 528 Existing tests 529</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p> 530 Ad Hoc 531 </p><p> 532 For example, 533 <code class="filename">testsuite/23_containers/list/modifiers/3.cc</code>. 534 </p></li><li class="listitem"><p> 535 Policy Based Data Structures 536 </p><p> 537 For example, take the test 538 functor <code class="classname">rand_reg_test</code> in 539 in <code class="filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc</code>. This uses <code class="classname">container_rand_regression_test</code> in 540<code class="filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h</code>. 541 542 </p><p> 543 Which has several tests for container member functions, 544Includes control and test container objects. Configuration includes 545random seed, iterations, number of distinct values, and the 546probability that and exception will be thrown. Assumes instantiating 547container uses an extension 548allocator, <code class="classname">__gnu_cxx::throw_allocator_random</code>, 549as the allocator type. 550 </p></li><li class="listitem"><p> 551 C++0x Container Requirements. 552 </p><p> 553 Coverage is currently limited to testing container 554 requirements for exception safety, 555 although <code class="classname">__gnu_cxx::throw_type</code> meets 556 the additional type requirements for testing numeric data 557 structures and instantiating algorithms. 558 </p><p> 559 Of particular interest is extending testing to algorithms and 560 then to parallel algorithms. Also io and locales. 561 </p><p> 562 The test instrumentation should also be extended to add 563 instrumentation to <code class="classname">iterator</code> 564 and <code class="classname">const_iterator</code> types that throw 565 conditionally on iterator operations. 566 </p></li></ul></div></div><div class="sect4" title="C++0x Requirements Test Sequence Descriptions"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.containers"></a> 567C++0x Requirements Test Sequence Descriptions 568</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p> 569 Basic 570 </p><p> 571 Basic consistency on exception propagation tests. For 572 each container, an object of that container is constructed, 573 a specific member function is exercised in 574 a <code class="literal">try</code> block, and then any thrown 575 exceptions lead to error checking in the appropriate 576 <code class="literal">catch</code> block. The container's use of 577 resources is compared to the container's use prior to the 578 test block. Resource monitoring is limited to allocations 579 made through the container's <span class="type">allocator_type</span>, 580 which should be sufficient for container data 581 structures. Included in these tests are member functions 582 are <span class="type">iterator</span> and <span class="type">const_iterator</span> 583 operations, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, <code class="function">erase</code>, <code class="function">swap</code>, <code class="function">clear</code>, 584 and <code class="function">rehash</code>. The container in question is 585 instantiated with two instrumented template arguments, 586 with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 587 as the allocator type, and 588 with <code class="classname">__gnu_cxx::throw_type_limit</code> as 589 the value type. This allows the test to loop through 590 conditional throw points. 591 </p><p> 592 The general form is demonstrated in 593 <code class="filename">testsuite/23_containers/list/requirements/exception/basic.cc 594 </code>. The instantiating test object is <code class="classname">__gnu_test::basic_safety</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 595 </p></li><li class="listitem"><p> 596 Generation Prohibited 597 </p><p> 598 Exception generation tests. For each container, an object of 599 that container is constructed and all member functions 600 required to not throw exceptions are exercised. Included in 601 these tests are member functions 602 are <span class="type">iterator</span> and <span class="type">const_iterator</span> operations, <code class="function">erase</code>, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">swap</code>, 603 and <code class="function">clear</code>. The container in question is 604 instantiated with two instrumented template arguments, 605 with <code class="classname">__gnu_cxx::throw_allocator_random</code> 606 as the allocator type, and 607 with <code class="classname">__gnu_cxx::throw_type_random</code> as 608 the value type. This test does not loop, an instead is sudden 609 death: first error fails. 610 </p><p> 611 The general form is demonstrated in 612 <code class="filename">testsuite/23_containers/list/requirements/exception/generation_prohibited.cc 613 </code>. The instantiating test object is <code class="classname">__gnu_test::generation_prohibited</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 614 </p></li><li class="listitem"><p> 615 Propagation Consistent 616 </p><p> 617 Container rollback on exception propagation tests. For 618 each container, an object of that container is constructed, 619 a specific member function that requires rollback to a previous 620 known good state is exercised in 621 a <code class="literal">try</code> block, and then any thrown 622 exceptions lead to error checking in the appropriate 623 <code class="literal">catch</code> block. The container is compared to 624 the container's last known good state using such parameters 625 as size, contents, and iterator references. Included in these 626 tests are member functions 627 are <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, 628 and <code class="function">rehash</code>. The container in question is 629 instantiated with two instrumented template arguments, 630 with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 631 as the allocator type, and 632 with <code class="classname">__gnu_cxx::throw_type_limit</code> as 633 the value type. This allows the test to loop through 634 conditional throw points. 635 </p><p> 636 The general form demonstrated in 637 <code class="filename">testsuite/23_containers/list/requirements/exception/propagation_coherent.cc 638 </code>. The instantiating test object is <code class="classname">__gnu_test::propagation_coherent</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 639 </p></li></ul></div></div></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="appendix_porting.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Porting to New Hardware or Operating Systems </td><td width="20%" align="center"><a accesskey="h" href="../spine.html">Home</a></td><td width="40%" align="right" valign="top"> ABI Policy and Guidelines</td></tr></table></div></body></html> 640