1<?xml version="1.0" encoding="UTF-8" standalone="no"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Test</title><meta name="generator" content="DocBook XSL-NS Stylesheets V1.78.1" /><meta name="keywords" content="ISO C++, test, testsuite, performance, conformance, ABI, exception safety" /><meta name="keywords" content="ISO C++, library" /><meta name="keywords" content="ISO C++, runtime, library" /><link rel="home" href="../index.html" title="The GNU C++ Library" /><link rel="up" href="appendix_porting.html" title="Appendix B. Porting and Maintenance" /><link rel="prev" href="internals.html" title="Porting to New Hardware or Operating Systems" /><link rel="next" href="abi.html" title="ABI Policy and Guidelines" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Test</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><th width="60%" align="center">Appendix B. 3 Porting and Maintenance 4 5</th><td width="20%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="manual.intro.setup.test"></a>Test</h2></div></div></div><p> 6The libstdc++ testsuite includes testing for standard conformance, 7regressions, ABI, and performance. 8</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"></a>Organization</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.layout"></a>Directory Layout</h4></div></div></div><p> 9 The directory <span class="emphasis"><em>libsrcdir/testsuite</em></span> contains the 10 individual test cases organized in sub-directories corresponding to 11 clauses of the C++ standard (detailed below), the dejagnu test 12 harness support files, and sources to various testsuite utilities 13 that are packaged in a separate testing library. 14</p><p> 15 All test cases for functionality required by the runtime components 16 of the C++ standard (ISO 14882) are files within the following 17 directories. 18</p><pre class="programlisting"> 1917_intro 2018_support 2119_diagnostics 2220_util 2321_strings 2422_locale 2523_containers 2625_algorithms 2726_numerics 2827_io 2928_regex 3029_atomics 3130_threads 32 </pre><p> 33 In addition, the following directories include test files: 34 </p><pre class="programlisting"> 35tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1). 36backward Tests for backwards compatibility and deprecated features. 37demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler 38ext Tests for extensions. 39performance Tests for performance analysis, and performance regressions. 40 </pre><p> 41 Some directories don't have test files, but instead contain 42 auxiliary information: 43 </p><pre class="programlisting"> 44config Files for the dejagnu test harness. 45lib Files for the dejagnu test harness. 46libstdc++* Files for the dejagnu test harness. 47data Sample text files for testing input and output. 48util Files for libtestc++, utilities and testing routines. 49 </pre><p> 50 Within a directory that includes test files, there may be 51 additional subdirectories, or files. Originally, test cases 52 were appended to one file that represented a particular section 53 of the chapter under test, and was named accordingly. For 54 instance, to test items related to <code class="code"> 21.3.6.1 - 55 basic_string::find [lib.string::find]</code> in the standard, 56 the following was used: 57 </p><pre class="programlisting"> 5821_strings/find.cc 59 </pre><p> 60 However, that practice soon became a liability as the test cases 61 became huge and unwieldy, and testing new or extended 62 functionality (like wide characters or named locales) became 63 frustrating, leading to aggressive pruning of test cases on some 64 platforms that covered up implementation errors. Now, the test 65 suite has a policy of one file, one test case, which solves the 66 above issues and gives finer grained results and more manageable 67 error debugging. As an example, the test case quoted above 68 becomes: 69 </p><pre class="programlisting"> 7021_strings/basic_string/find/char/1.cc 7121_strings/basic_string/find/char/2.cc 7221_strings/basic_string/find/char/3.cc 7321_strings/basic_string/find/wchar_t/1.cc 7421_strings/basic_string/find/wchar_t/2.cc 7521_strings/basic_string/find/wchar_t/3.cc 76 </pre><p> 77 All new tests should be written with the policy of one test 78 case, one file in mind. 79 </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.naming"></a>Naming Conventions</h4></div></div></div><p> 80 In addition, there are some special names and suffixes that are 81 used within the testsuite to designate particular kinds of 82 tests. 83 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 84 <span class="emphasis"><em>_xin.cc</em></span> 85 </p><p> 86 This test case expects some kind of interactive input in order 87 to finish or pass. At the moment, the interactive tests are not 88 run by default. Instead, they are run by hand, like: 89 </p><pre class="programlisting"> 90g++ 27_io/objects/char/3_xin.cc 91cat 27_io/objects/char/3_xin.in | a.out 92 </pre></li><li class="listitem"><p> 93 <span class="emphasis"><em>.in</em></span> 94 </p><p> 95 This file contains the expected input for the corresponding <span class="emphasis"><em> 96 _xin.cc</em></span> test case. 97 </p></li><li class="listitem"><p> 98 <span class="emphasis"><em>_neg.cc</em></span> 99 </p><p> 100 This test case is expected to fail: it's a negative test. At the 101 moment, these are almost always compile time errors. 102 </p></li><li class="listitem"><p> 103 <span class="emphasis"><em>char</em></span> 104 </p><p> 105 This can either be a directory name or part of a longer file 106 name, and indicates that this file, or the files within this 107 directory are testing the <code class="code">char</code> instantiation of a 108 template. 109 </p></li><li class="listitem"><p> 110 <span class="emphasis"><em>wchar_t</em></span> 111 </p><p> 112 This can either be a directory name or part of a longer file 113 name, and indicates that this file, or the files within this 114 directory are testing the <code class="code">wchar_t</code> instantiation of 115 a template. Some hosts do not support <code class="code">wchar_t</code> 116 functionality, so for these targets, all of these tests will not 117 be run. 118 </p></li><li class="listitem"><p> 119 <span class="emphasis"><em>thread</em></span> 120 </p><p> 121 This can either be a directory name or part of a longer file 122 name, and indicates that this file, or the files within this 123 directory are testing situations where multiple threads are 124 being used. 125 </p></li><li class="listitem"><p> 126 <span class="emphasis"><em>performance</em></span> 127 </p><p> 128 This can either be an enclosing directory name or part of a 129 specific file name. This indicates a test that is used to 130 analyze runtime performance, for performance regression testing, 131 or for other optimization related analysis. At the moment, these 132 test cases are not run by default. 133 </p></li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"></a>Running the Testsuite</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"></a>Basic</h4></div></div></div><p> 134 You can check the status of the build without installing it 135 using the dejagnu harness, much like the rest of the gcc 136 tools.</p><pre class="programlisting"> make check</pre><p>in the <span class="emphasis"><em>libbuilddir</em></span> directory.</p><p>or</p><pre class="programlisting"> make check-target-libstdc++-v3</pre><p>in the <span class="emphasis"><em>gccbuilddir</em></span> directory. 137 </p><p> 138 These commands are functionally equivalent and will create a 139 'testsuite' directory underneath 140 <span class="emphasis"><em>libbuilddir</em></span> containing the results of the 141 tests. Two results files will be generated: <span class="emphasis"><em> 142 libstdc++.sum</em></span>, which is a PASS/FAIL summary for each 143 test, and <span class="emphasis"><em>libstdc++.log</em></span> which is a log of 144 the exact command line passed to the compiler, the compiler 145 output, and the executable output (if any). 146 </p><p> 147 Archives of test results for various versions and platforms are 148 available on the GCC website in the <a class="link" href="http://gcc.gnu.org/gcc-4.3/buildstat.html" target="_top">build 149 status</a> section of each individual release, and are also 150 archived on a daily basis on the <a class="link" href="http://gcc.gnu.org/ml/gcc-testresults/current" target="_top">gcc-testresults</a> 151 mailing list. Please check either of these places for a similar 152 combination of source version, operating system, and host CPU. 153 </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.variations"></a>Variations</h4></div></div></div><p> 154 There are several options for running tests, including testing 155 the regression tests, testing a subset of the regression tests, 156 testing the performance tests, testing just compilation, testing 157 installed tools, etc. In addition, there is a special rule for 158 checking the exported symbols of the shared library. 159 </p><p> 160 To debug the dejagnu test harness during runs, try invoking with a 161 specific argument to the variable RUNTESTFLAGS, as below. 162 </p><pre class="programlisting"> 163make check-target-libstdc++-v3 RUNTESTFLAGS="-v" 164</pre><p> 165 or 166 </p><pre class="programlisting"> 167make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" 168</pre><p> 169 To run a subset of the library tests, you can either generate the 170 <span class="emphasis"><em>testsuite_files</em></span> file (described below) by running 171 <span class="command"><strong>make testsuite_files</strong></span> in the 172 <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory, then edit the 173 file to remove the tests you don't want and then run the testsuite as 174 normal, or you can specify a testsuite and a subset of tests in the 175 RUNTESTFLAGS variable. 176 </p><p> 177 For example, to run only the tests for containers you could use: 178 </p><pre class="programlisting"> 179make check-target-libstdc++-v3 RUNTESTFLAGS="conformance.exp=23_containers/*" 180</pre><p> 181 When combining this with other options in RUNTESTFLAGS the 182 <span class="emphasis"><em>testsuite.exp=testfiles</em></span> options must come first. 183 </p><p> 184 There are two ways to run on a simulator: set up DEJAGNU to point to a 185 specially crafted site.exp, or pass down --target_board flags. 186 </p><p> 187 Example flags to pass down for various embedded builds are as follows: 188 </p><pre class="programlisting"> 189 --target=powerpc-eabism (libgloss/sim) 190make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" 191 192--target=calmrisc32 (libgloss/sid) 193make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" 194 195--target=xscale-elf (newlib/sim) 196make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" 197</pre><p> 198 Also, here is an example of how to run the libstdc++ testsuite 199 for a multilibed build directory with different ABI settings: 200 </p><pre class="programlisting"> 201make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' 202</pre><p> 203 You can run the tests with a compiler and library that have 204 already been installed. Make sure that the compiler (e.g., 205 <code class="code">g++</code>) is in your <code class="code">PATH</code>. If you are 206 using shared libraries, then you must also ensure that the 207 directory containing the shared version of libstdc++ is in your 208 <code class="code">LD_LIBRARY_PATH</code>, or equivalent. If your GCC source 209 tree is at <code class="code">/path/to/gcc</code>, then you can run the tests 210 as follows: 211 </p><pre class="programlisting"> 212runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite 213</pre><p> 214 The testsuite will create a number of files in the directory in 215 which you run this command,. Some of those files might use the 216 same name as files created by other testsuites (like the ones 217 for GCC and G++), so you should not try to run all the 218 testsuites in parallel from the same directory. 219 </p><p> 220 In addition, there are some testing options that are mostly of 221 interest to library maintainers and system integrators. As such, 222 these tests may not work on all cpu and host combinations, and 223 may need to be executed in the 224 <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory. These 225 options include, but are not necessarily limited to, the 226 following: 227 </p><pre class="programlisting"> 228 make testsuite_files 229 </pre><p> 230 Five files are generated that determine what test files 231 are run. These files are: 232 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 233 <span class="emphasis"><em>testsuite_files</em></span> 234 </p><p> 235 This is a list of all the test cases that will be run. Each 236 test case is on a separate line, given with an absolute path 237 from the <span class="emphasis"><em>libsrcdir/testsuite</em></span> directory. 238 </p></li><li class="listitem"><p> 239 <span class="emphasis"><em>testsuite_files_interactive</em></span> 240 </p><p> 241 This is a list of all the interactive test cases, using the 242 same format as the file list above. These tests are not run 243 by default. 244 </p></li><li class="listitem"><p> 245 <span class="emphasis"><em>testsuite_files_performance</em></span> 246 </p><p> 247 This is a list of all the performance test cases, using the 248 same format as the file list above. These tests are not run 249 by default. 250 </p></li><li class="listitem"><p> 251 <span class="emphasis"><em>testsuite_thread</em></span> 252 </p><p> 253 This file indicates that the host system can run tests which 254 involved multiple threads. 255 </p></li><li class="listitem"><p> 256 <span class="emphasis"><em>testsuite_wchar_t</em></span> 257 </p><p> 258 This file indicates that the host system can run the wchar_t 259 tests, and corresponds to the macro definition <code class="code"> 260 _GLIBCXX_USE_WCHAR_T</code> in the file c++config.h. 261 </p></li></ul></div><pre class="programlisting"> 262 make check-abi 263 </pre><p> 264 The library ABI can be tested. This involves testing the shared 265 library against an ABI-defining previous version of symbol 266 exports. 267 </p><pre class="programlisting"> 268 make check-compile 269 </pre><p> 270 This rule compiles, but does not link or execute, the 271 <span class="emphasis"><em>testsuite_files</em></span> test cases and displays the 272 output on stdout. 273 </p><pre class="programlisting"> 274 make check-performance 275 </pre><p> 276 This rule runs through the 277 <span class="emphasis"><em>testsuite_files_performance</em></span> test cases and 278 collects information for performance analysis and can be used to 279 spot performance regressions. Various timing information is 280 collected, as well as number of hard page faults, and memory 281 used. This is not run by default, and the implementation is in 282 flux. 283 </p><p> 284 We are interested in any strange failures of the testsuite; 285 please email the main libstdc++ mailing list if you see 286 something odd or have questions. 287 </p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"></a>Permutations</h4></div></div></div><p> 288 To run the libstdc++ test suite under the <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>, edit 289 <code class="filename">libstdc++-v3/scripts/testsuite_flags</code> to add the 290 compile-time flag <code class="constant">-D_GLIBCXX_DEBUG</code> to the 291 result printed by the <code class="literal">--build-cxx</code> 292 option. Additionally, add the 293 <code class="constant">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on 294 pedantic checking. The libstdc++ test suite should produce 295 precisely the same results under debug mode that it does under 296 release mode: any deviation indicates an error in either the 297 library or the test suite. 298 </p><p> 299 The <a class="link" href="parallel_mode.html" title="Chapter 18. Parallel Mode">parallel 300 mode</a> can be tested in much the same manner, substituting 301 <code class="constant">-D_GLIBCXX_PARALLEL</code> for 302 <code class="constant">-D_GLIBCXX_DEBUG</code> in the previous paragraph. 303 </p><p> 304 Or, just run the testsuites with <code class="constant">CXXFLAGS</code> 305 set to <code class="constant">-D_GLIBCXX_DEBUG</code> or 306 <code class="constant">-D_GLIBCXX_PARALLEL</code>. 307 </p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"></a>Writing a new test case</h3></div></div></div><p> 308 The first step in making a new test case is to choose the correct 309 directory and file name, given the organization as previously 310 described. 311 </p><p> 312 All files are copyright the FSF, and GPL'd: this is very 313 important. The first copyright year should correspond to the date 314 the file was checked in to SVN. 315 </p><p> 316 As per the dejagnu instructions, always return 0 from main to 317 indicate success. 318 </p><p> 319 A bunch of utility functions and classes have already been 320 abstracted out into the testsuite utility library, <code class="code"> 321 libtestc++</code>. To use this functionality, just include the 322 appropriate header file: the library or specific object files will 323 automatically be linked in as part of the testsuite run. 324 </p><p> 325 For a test that needs to take advantage of the dejagnu test 326 harness, what follows below is a list of special keyword that 327 harness uses. Basically, a test case contains dg-keywords (see 328 dg.exp) indicating what to do and what kinds of behavior are to be 329 expected. New test cases should be written with the new style 330 DejaGnu framework in mind. 331 </p><p> 332 To ease transition, here is the list of dg-keyword documentation 333 lifted from dg.exp. 334 </p><pre class="programlisting"> 335# The currently supported options are: 336# 337# dg-prms-id N 338# set prms_id to N 339# 340# dg-options "options ..." [{ target selector }] 341# specify special options to pass to the tool (eg: compiler) 342# 343# dg-do do-what-keyword [{ target/xfail selector }] 344# `do-what-keyword' is tool specific and is passed unchanged to 345# ${tool}-dg-test. An example is gcc where `keyword' can be any of: 346# preprocess|compile|assemble|link|run 347# and will do one of: produce a .i, produce a .s, produce a .o, 348# produce an a.out, or produce an a.out and run it (the default is 349# compile). 350# 351# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]] 352# indicate an error message <regexp> is expected on this line 353# (the test fails if it doesn't occur) 354# Linenum=0 for general tool messages (eg: -V arg missing). 355# "." means the current line. 356# 357# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]] 358# indicate a warning message <regexp> is expected on this line 359# (the test fails if it doesn't occur) 360# 361# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]] 362# indicate a bogus error message <regexp> use to occur here 363# (the test fails if it does occur) 364# 365# dg-build regexp comment [{ target/xfail selector }] 366# indicate the build use to fail for some reason 367# (errors covered here include bad assembler generated, tool crashes, 368# and link failures) 369# (the test fails if it does occur) 370# 371# dg-excess-errors comment [{ target/xfail selector }] 372# indicate excess errors are expected (any line) 373# (this should only be used sparingly and temporarily) 374# 375# dg-output regexp [{ target selector }] 376# indicate the expected output of the program is <regexp> 377# (there may be multiple occurrences of this, they are concatenated) 378# 379# dg-final { tcl code } 380# add some tcl code to be run at the end 381# (there may be multiple occurrences of this, they are concatenated) 382# (unbalanced braces must be \-escaped) 383# 384# "{ target selector }" is a list of expressions that determine whether the 385# test succeeds or fails for a particular target, or in some cases whether the 386# option applies for a particular target. If the case of `dg-do' it specifies 387# whether the test case is even attempted on the specified target. 388# 389# The target selector is always optional. The format is one of: 390# 391# { xfail *-*-* ... } - the test is expected to fail for the given targets 392# { target *-*-* ... } - the option only applies to the given targets 393# 394# At least one target must be specified, use *-*-* for "all targets". 395# At present it is not possible to specify both `xfail' and `target'. 396# "native" may be used in place of "*-*-*". 397 398Example 1: Testing compilation only 399// { dg-do compile } 400 401Example 2: Testing for expected warnings on line 36, which all targets fail 402// { dg-warning "string literals" "" { xfail *-*-* } 36 } 403 404Example 3: Testing for expected warnings on line 36 405// { dg-warning "string literals" "" { target *-*-* } 36 } 406 407Example 4: Testing for compilation errors on line 41 408// { dg-do compile } 409// { dg-error "no match for" "" { target *-*-* } 41 } 410 411Example 5: Testing with special command line settings, or without the 412use of pre-compiled headers, in particular the stdc++.h.gch file. Any 413options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set 414up in the normal.exp file. 415// { dg-options "-O0" { target *-*-* } } 416</pre><p> 417 More examples can be found in the libstdc++-v3/testsuite/*/*.cc files. 418 </p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.harness"></a>Test Harness and Utilities</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.dejagnu"></a>Dejagnu Harness Details</h4></div></div></div><p> 419 Underlying details of testing for conformance and regressions are 420 abstracted via the GNU Dejagnu package. This is similar to the 421 rest of GCC. 422 </p><p>This is information for those looking at making changes to the testsuite 423structure, and/or needing to trace dejagnu's actions with --verbose. This 424will not be useful to people who are "merely" adding new tests to the existing 425structure. 426</p><p>The first key point when working with dejagnu is the idea of a "tool". 427Files, directories, and functions are all implicitly used when they are 428named after the tool in use. Here, the tool will always be "libstdc++". 429</p><p>The <code class="code">lib</code> subdir contains support routines. The 430<code class="code">lib/libstdc++.exp</code> file ("support library") is loaded 431automagically, and must explicitly load the others. For example, files can 432be copied from the core compiler's support directory into <code class="code">lib</code>. 433</p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are 434our own. Callbacks must be prefixed with the name of the tool. To easily 435distinguish the others, by convention our own routines are named "v3-*". 436</p><p>The next key point when working with dejagnu is "test files". Any 437directory whose name starts with the tool name will be searched for test files. 438(We have only one.) In those directories, any <code class="code">.exp</code> file is 439considered a test file, and will be run in turn. Our main test file is called 440<code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the 441callbacks loaded from the support library. 442</p><p>The <code class="code">config</code> directory is searched for any particular "target 443board" information unique to this library. This is currently unused and sets 444only default variables. 445</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.utils"></a>Utilities</h4></div></div></div><p> 446 </p><p> 447 The testsuite directory also contains some files that implement 448 functionality that is intended to make writing test cases easier, 449 or to avoid duplication, or to provide error checking in a way that 450 is consistent across platforms and test harnesses. A stand-alone 451 executable, called <span class="emphasis"><em>abi_check</em></span>, and a static 452 library called <span class="emphasis"><em>libtestc++</em></span> are 453 constructed. Both of these items are not installed, and only used 454 during testing. 455 </p><p> 456 These files include the following functionality: 457 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 458 <span class="emphasis"><em>testsuite_abi.h</em></span>, 459 <span class="emphasis"><em>testsuite_abi.cc</em></span>, 460 <span class="emphasis"><em>testsuite_abi_check.cc</em></span> 461 </p><p> 462 Creates the executable <span class="emphasis"><em>abi_check</em></span>. 463 Used to check correctness of symbol versioning, visibility of 464 exported symbols, and compatibility on symbols in the shared 465 library, for hosts that support this feature. More information 466 can be found in the ABI documentation <a class="link" href="abi.html" title="ABI Policy and Guidelines">here</a> 467 </p></li><li class="listitem"><p> 468 <span class="emphasis"><em>testsuite_allocator.h</em></span>, 469 <span class="emphasis"><em>testsuite_allocator.cc</em></span> 470 </p><p> 471 Contains specialized allocators that keep track of construction 472 and destruction. Also, support for overriding global new and 473 delete operators, including verification that new and delete 474 are called during execution, and that allocation over max_size 475 fails. 476 </p></li><li class="listitem"><p> 477 <span class="emphasis"><em>testsuite_character.h</em></span> 478 </p><p> 479 Contains <code class="code">std::char_traits</code> and 480 <code class="code">std::codecvt</code> specializations for a user-defined 481 POD. 482 </p></li><li class="listitem"><p> 483 <span class="emphasis"><em>testsuite_hooks.h</em></span>, 484 <span class="emphasis"><em>testsuite_hooks.cc</em></span> 485 </p><p> 486 A large number of utilities, including: 487 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>VERIFY</p></li><li class="listitem"><p>set_memory_limits</p></li><li class="listitem"><p>verify_demangle</p></li><li class="listitem"><p>run_tests_wrapped_locale</p></li><li class="listitem"><p>run_tests_wrapped_env</p></li><li class="listitem"><p>try_named_locale</p></li><li class="listitem"><p>try_mkfifo</p></li><li class="listitem"><p>func_callback</p></li><li class="listitem"><p>counter</p></li><li class="listitem"><p>copy_tracker</p></li><li class="listitem"><p>copy_constructor</p></li><li class="listitem"><p>assignment_operator</p></li><li class="listitem"><p>destructor</p></li><li class="listitem"><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li class="listitem"><p> 488 <span class="emphasis"><em>testsuite_io.h</em></span> 489 </p><p> 490 Error, exception, and constraint checking for 491 <code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>. 492 </p></li><li class="listitem"><p> 493 <span class="emphasis"><em>testsuite_iterators.h</em></span> 494 </p><p> 495 Wrappers for various iterators. 496 </p></li><li class="listitem"><p> 497 <span class="emphasis"><em>testsuite_performance.h</em></span> 498 </p><p> 499 A number of class abstractions for performance counters, and 500 reporting functions including: 501 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; "><li class="listitem"><p>time_counter</p></li><li class="listitem"><p>resource_counter</p></li><li class="listitem"><p>report_performance</p></li></ul></div></li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="test.special"></a>Special Topics</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="test.exception.safety"></a> 502 Qualifying Exception Safety Guarantees 503 <a id="id-1.3.6.3.5.7.2.1.1.1" class="indexterm"></a> 504</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.overview"></a>Overview</h5></div></div></div><p> 505 Testing is composed of running a particular test sequence, 506 and looking at what happens to the surrounding code when 507 exceptions are thrown. Each test is composed of measuring 508 initial state, executing a particular sequence of code under 509 some instrumented conditions, measuring a final state, and 510 then examining the differences between the two states. 511 </p><p> 512 Test sequences are composed of constructed code sequences 513 that exercise a particular function or member function, and 514 either confirm no exceptions were generated, or confirm the 515 consistency/coherency of the test subject in the event of a 516 thrown exception. 517 </p><p> 518 Random code paths can be constructed using the basic test 519 sequences and instrumentation as above, only combined in a 520 random or pseudo-random way. 521 </p><p> To compute the code paths that throw, test instruments 522 are used that throw on allocation events 523 (<code class="classname">__gnu_cxx::throw_allocator_random</code> 524 and <code class="classname">__gnu_cxx::throw_allocator_limit</code>) 525 and copy, assignment, comparison, increment, swap, and 526 various operators 527 (<code class="classname">__gnu_cxx::throw_type_random</code> 528 and <code class="classname">__gnu_cxx::throw_type_limit</code>). Looping 529 through a given test sequence and conditionally throwing in 530 all instrumented places. Then, when the test sequence 531 completes without an exception being thrown, assume all 532 potential error paths have been exercised in a sequential 533 manner. 534 </p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.status"></a> 535 Existing tests 536</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 537 Ad Hoc 538 </p><p> 539 For example, 540 <code class="filename">testsuite/23_containers/list/modifiers/3.cc</code>. 541 </p></li><li class="listitem"><p> 542 Policy Based Data Structures 543 </p><p> 544 For example, take the test 545 functor <code class="classname">rand_reg_test</code> in 546 in <code class="filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc</code>. This uses <code class="classname">container_rand_regression_test</code> in 547<code class="filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h</code>. 548 549 </p><p> 550 Which has several tests for container member functions, 551Includes control and test container objects. Configuration includes 552random seed, iterations, number of distinct values, and the 553probability that an exception will be thrown. Assumes instantiating 554container uses an extension 555allocator, <code class="classname">__gnu_cxx::throw_allocator_random</code>, 556as the allocator type. 557 </p></li><li class="listitem"><p> 558 C++11 Container Requirements. 559 </p><p> 560 Coverage is currently limited to testing container 561 requirements for exception safety, 562 although <code class="classname">__gnu_cxx::throw_type</code> meets 563 the additional type requirements for testing numeric data 564 structures and instantiating algorithms. 565 </p><p> 566 Of particular interest is extending testing to algorithms and 567 then to parallel algorithms. Also io and locales. 568 </p><p> 569 The test instrumentation should also be extended to add 570 instrumentation to <code class="classname">iterator</code> 571 and <code class="classname">const_iterator</code> types that throw 572 conditionally on iterator operations. 573 </p></li></ul></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.containers"></a> 574C++11 Requirements Test Sequence Descriptions 575</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p> 576 Basic 577 </p><p> 578 Basic consistency on exception propagation tests. For 579 each container, an object of that container is constructed, 580 a specific member function is exercised in 581 a <code class="literal">try</code> block, and then any thrown 582 exceptions lead to error checking in the appropriate 583 <code class="literal">catch</code> block. The container's use of 584 resources is compared to the container's use prior to the 585 test block. Resource monitoring is limited to allocations 586 made through the container's <span class="type">allocator_type</span>, 587 which should be sufficient for container data 588 structures. Included in these tests are member functions 589 are <span class="type">iterator</span> and <span class="type">const_iterator</span> 590 operations, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, <code class="function">erase</code>, <code class="function">swap</code>, <code class="function">clear</code>, 591 and <code class="function">rehash</code>. The container in question is 592 instantiated with two instrumented template arguments, 593 with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 594 as the allocator type, and 595 with <code class="classname">__gnu_cxx::throw_type_limit</code> as 596 the value type. This allows the test to loop through 597 conditional throw points. 598 </p><p> 599 The general form is demonstrated in 600 <code class="filename">testsuite/23_containers/list/requirements/exception/basic.cc 601 </code>. The instantiating test object is <code class="classname">__gnu_test::basic_safety</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 602 </p></li><li class="listitem"><p> 603 Generation Prohibited 604 </p><p> 605 Exception generation tests. For each container, an object of 606 that container is constructed and all member functions 607 required to not throw exceptions are exercised. Included in 608 these tests are member functions 609 are <span class="type">iterator</span> and <span class="type">const_iterator</span> operations, <code class="function">erase</code>, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">swap</code>, 610 and <code class="function">clear</code>. The container in question is 611 instantiated with two instrumented template arguments, 612 with <code class="classname">__gnu_cxx::throw_allocator_random</code> 613 as the allocator type, and 614 with <code class="classname">__gnu_cxx::throw_type_random</code> as 615 the value type. This test does not loop, an instead is sudden 616 death: first error fails. 617 </p><p> 618 The general form is demonstrated in 619 <code class="filename">testsuite/23_containers/list/requirements/exception/generation_prohibited.cc 620 </code>. The instantiating test object is <code class="classname">__gnu_test::generation_prohibited</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 621 </p></li><li class="listitem"><p> 622 Propagation Consistent 623 </p><p> 624 Container rollback on exception propagation tests. For 625 each container, an object of that container is constructed, 626 a specific member function that requires rollback to a previous 627 known good state is exercised in 628 a <code class="literal">try</code> block, and then any thrown 629 exceptions lead to error checking in the appropriate 630 <code class="literal">catch</code> block. The container is compared to 631 the container's last known good state using such parameters 632 as size, contents, and iterator references. Included in these 633 tests are member functions 634 are <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, 635 and <code class="function">rehash</code>. The container in question is 636 instantiated with two instrumented template arguments, 637 with <code class="classname">__gnu_cxx::throw_allocator_limit</code> 638 as the allocator type, and 639 with <code class="classname">__gnu_cxx::throw_type_limit</code> as 640 the value type. This allows the test to loop through 641 conditional throw points. 642 </p><p> 643 The general form demonstrated in 644 <code class="filename">testsuite/23_containers/list/requirements/exception/propagation_coherent.cc 645 </code>. The instantiating test object is <code class="classname">__gnu_test::propagation_coherent</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>. 646 </p></li></ul></div></div></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="internals.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="appendix_porting.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Porting to New Hardware or Operating Systems </td><td width="20%" align="center"><a accesskey="h" href="../index.html">Home</a></td><td width="40%" align="right" valign="top"> ABI Policy and Guidelines</td></tr></table></div></body></html>