Lines Matching full:test

33 .Nd Description of the ATF test program interface
35 The interface of ATF test programs is the interface of the test
43 The ATF interface can be understood as the mechanisms used by test programs
44 to communicate with the runtime engine as well as the assumptions that test
45 programs and test cases can make while running.
47 A test case is the most basic part of a test suite.  A test case is
49 item under test was a function, the test case would provide a single set of
51 test was a binary, the test case would provide a single set of arguments to
53 .Ss Test case parts
54 Test cases have three parts:
62 The actual test case which performs any desired testing and reports a
68 provided in cases where the test modifies system-wide state not known by
76 The following test case metadata properties must be exported in the test
77 case list for every test case:
80 Single-word string.  The name of the test case.  Must be unique within the
81 test program.
84 The following test case metadata properties may be exported in the
85 test case list for every test case:
88 Multi-word string.  A textual description for the test case.  Usually,
92 Boolean.  Whether the test case defines a cleanup routine or not.
94 Whitespace separated list of the architectures required by the test case.
95 If defined, the test case is skipped unless the host architecture matches
99 configuration variables that must be defined.  The test is skipped if any
103 these files is not found, the test case is skipped.
105 Whitespace separated list of the machine types required by the test case.
106 If defined, the test case is skipped unless the host machine type matches
110 names).  If any of these programs is not found, the test case is skipped.
118 the test case must be run as the superuser or otherwise it is skipped.  If
120 the test case must be run as an unprivileged user or else it is skipped.
122 Integer.  The amount of seconds the test case can run for before it is
127 propagated to the test cases:
136 A test case must always report a result by creating the results file
139 flag.  For convenience when running test cases without the runtime engine,
147 Aside from creating the results file, the process in which the test case
148 runs must terminate in particular ways for the test result to be considered
151 If the test case fails to create the test result, if the test result is
153 process does not match the requirements of the test result, the runtime
154 engine marks the test case as
158 state is decided by the runtime engine; a test case cannot report itself as
228 The runtime engine attempts to isolate test cases from other test cases in
229 the same test program and from the rest of the system by performing what is
231 .Em test case isolation .
233 Whenever the user runs a test program binary by hand (i.e. not through
235 the test program will print a warning message stating that test case
243 to tell the test programs that they are being run with isolation enabled.
245 The test case isolation performs the following:
248 Each test case body and cleanup routines are executed in independent
249 processes.  Corollary: the test case can do whatever it wants to the
253 The test case body and cleanup are executed in their own process groups.
256 process subtree once the test case finishes (or if the test case hangs).
258 The test case body and its cleanup (if any) are executed in a temporary
260 directory is shared among the body and cleanup parts of a single test case
262 Corollary: the test case body and cleanup routines can write to their
265 temporary directories after the execution of a test case.  Any file systems
290 This is a simple, best-effort attempt at allowing test cases to dump core
293 .Ss Test programs
294 A test program is, simply put, a collection of related test cases.  The
295 test program can be seen as a command-line dispatcher for the test cases.
296 A test program must provide one or more test cases.  If it does not contain
297 any test case, the runtime system will report it as invalid.
299 Test programs expose their list of test cases in a machine parseable
300 format.  The runtime engine obtains the list of test cases to know what
301 tests to run and to know how to set up the environment of each test prior
302 execution.  The test program must not do any test when asked to dump its
303 test case list.
305 The generic syntax to obtain the list of test cases included in a test
311 The list of test cases follows the following format:
345 The syntax to run a test case body part is:
347 <test-program> [-r resfile] [-s srcdir] [-v var=value]* <test-case>[:body]
350 This must run the test case body
357 which must be created by the test case; and always passes an absolute path
360 flag pointing to the directory containing the test program executable.
365 flag, and these can be later inspected by the test case at will.
367 A note to users: if you run the test case by hand (not through
371 isolation section apply.  This means that the test case can (and probably
373 that the test case is executed without e.g. clearing the environment, the
374 results of the test case may differ from those obtained when running the
375 test case inside the runtime engine.
377 (i.e. to run the test case code under GDB).
379 The syntax to run a test case cleanup part is:
381 <test-program> [-s srcdir] [-v var=value]* <test-case>:cleanup
384 This can only be performed if and only if the test case sets the
396 a separate process space.  The only way for test cases to transfer state
404 .Xr kyua-test 1 ,