Lines Matching full:tests
9 :program:`lit` [*options*] [*tests*]
19 :program:`lit` should be run with one or more *tests* to run specified on the
20 command line. Tests can be either individual test files or directories to
21 search for tests (see :ref:`test-discovery`).
24 tests have been run :program:`lit` will print summary information on the number
25 of tests which passed or failed (see :ref:`test-status-results`). The
26 :program:`lit` program will execute with a non-zero exit code if any tests
33 :program:`lit` also includes a number of options for controlling how tests are
64 Run ``N`` tests in parallel. By default, this is automatically chosen to
89 Show less output, for example don't show information on tests that pass.
110 Show more information about all tests, for example the entire test
119 Show the names of unsupported tests.
123 Show the names of tests that were expected to fail.
132 Specify an additional ``PATH`` to use when searching for executables in tests.
136 Run individual tests under valgrind (using the memcheck tool). The
142 failure in) certain tests.
154 certain tests.
156 .. option:: --time-tests
158 Track the wall time individual tests take to execute and includes the results
159 in the summary output. This is useful for determining which tests in a test
164 Exit with status zero even if some tests fail.
176 By default, `lit` will run failing tests first, then run tests in descending
186 Run the tests in a random order, not failing/slowest first. Deprecated,
195 .. option:: --max-tests=N
197 Run at most ``N`` tests and then terminate.
201 Spend at most ``N`` seconds (approximately) running tests and then terminate.
207 Divide the set of selected tests into ``M`` equal-sized subsets or
217 Define the order in which tests are run. The supported values are:
219 - lexical - tests will be run in lexical order according to the test file
222 - random - tests will be run in random order.
224 - smart - tests that failed previously will be run first, then the remaining
225 tests, all in descending execution time order. This is the default as it
243 Run only those tests whose name matches the regular expression specified in
250 Filter out those tests whose name matches the regular expression specified in
257 Treat those tests whose name is in the semicolon separated list ``LIST`` as
270 In this case, all of the following tests are treated as ``XFAIL``:
289 Do not treat the specified tests as ``XFAIL``. The environment variable
309 .. option:: --show-tests
311 List all of the discovered tests and exit.
326 The inputs passed to :program:`lit` can be either individual tests, or entire
327 directories or hierarchies of tests to run. When :program:`lit` starts up, the
328 first thing it does is convert the inputs into a complete list of tests to run
336 how to find and run the tests inside the test suite.
339 list of inputs adding tests for individual files and recursively searching for
340 tests in directories.
342 This behavior makes it easy to specify a subset of tests to run, while still
343 allowing the test suite configuration to control exactly how tests are
344 interpreted. In addition, :program:`lit` always identifies tests by the test
363 tests containing an ``ALLOW_RETRIES:`` annotation.
373 The test succeeded, but it was expected to fail. This is used for tests which
389 which can report unsupported tests.
396 Depending on the test format tests may produce additional information about
409 arbitrary tests, and to expose a single convenient interface to these
410 tests. :program:`lit` itself doesn't know how to run tests, rather this logic is
416 As described in :ref:`test-discovery`, tests are always located inside a *test
417 suite*. Test suites serve to define the format of the tests they contain, the
418 logic for finding those tests, and any additional information to run the tests.
447 discover and run tests in the test suite. Generally this will be a builtin test
451 builds this is the directory that will be scanned for tests.
454 the object directory. This is where tests will be run and temporary output files
458 tests in the suite.
460 **standalone_tests** When true, mark a directory with tests expected to be run
465 **suffixes** For **lit** test formats which scan directories for tests, this
471 **unsupported** Mark an unsupported directory, all tests within it will be
491 directory (following *test_source_root*) looking for tests. When :program:`lit`
497 Tests are identified by the test suite they are contained within, and the
500 "virtual tests" which have a path that contains both the path to the actual
514 be used to define subdirectories of optional tests, or to change other
607 The following is an example of a test run output which consists of four tests A,
620 LIT EXAMPLE TESTS