1.\" Copyright 2012 Google Inc. 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions are 6.\" met: 7.\" 8.\" * Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" * Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" * Neither the name of Google Inc. nor the names of its contributors 14.\" may be used to endorse or promote products derived from this software 15.\" without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 18.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 19.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 20.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 21.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 22.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 23.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.Dd December 26, 2012 29.Dt KYUA-TESTER 1 30.Os 31.Sh NAME 32.Nm kyua-tester 33.Nd Scriptable interface to interact with test programs 34.Sh SYNOPSIS 35.Nm 36.Op Fl g Ar gid 37.Op Fl t Ar timeout 38.Op Fl u Ar uid 39.Ar list 40.Ar test_program 41.Nm 42.Op Fl g Ar gid 43.Op Fl t Ar timeout 44.Op Fl u Ar uid 45.Ar test 46.Op Fl v Ar var=value 47.Ar test_program 48.Ar test_case 49.Ar result_file 50.Sh DESCRIPTION 51This generic manual page does 52.Em not 53correspond to any specific binary. It describes the generic command-line 54interface provided by all Kyua testers shipped by the 55.Sq __PACKAGE__ 56package, which all have names of the form 57.Nm kyua-INTERFACE-NAME-tester . 58.Pp 59The main goal of a 60.Nm 61is to provide a scriptable interface to run 62.Em a single test case 63of 64.Em a single test program 65in a generic manner. The specific implementation of how to do this is up to the 66tester, as each 67.Nm 68binary implements a specific test interface. The following binaries are shipped 69with the 70.Sq __PACKAGE__ 71package: 72.Bl -tag -width XXXX 73.It Xr kyua-atf-tester 1 74An implementation of 75.Nm 76that wraps tests that follow the 77.Xr kyua-atf-interface 7 78interface. 79.It Xr kyua-plain-tester 1 80An implementation of 81.Nm 82that wraps tests that follow the 83.Xr kyua-plain-interface 7 84interface. 85.El 86.Pp 87.Xr kyua 1 88uses the various 89.Nm 90binaries to execute tests that implement diverse interfaces, without having to 91know about any of its specifics. It is possible for other packages to provide 92new testers that implement other interfaces as long as the tester's command line 93complies to what is described in this manual page. 94.Pp 95A 96.Nm 97implements a typical command-line interface based on subcommands. The tool 98takes a set of generic options first, a subcommand name, a set of any options 99accepted by such subcommand, and a set of arguments to the subcommand. 100.Pp 101The following options apply to all subcommands: 102.Bl -tag -width XtXtimeoutXX 103.It Fl g Ar gid 104GID of the user to switch to before running the test case. Defaults to the 105current GID. 106.It Fl t Ar timeout 107Seconds to allow the test program to run before killing it. Defaults to 60. 108.It Fl u Ar uid 109UID of the user to switch to before running the test case. Defaults to the 110current UID. 111.El 112.Pp 113Killing the tester should always be a safe operation, resulting in the 114subsequent termination of the test program being run and the cleanup of any used 115resources. 116.Ss Workflow 117The way 118.Xr kyua 1 119uses the testers to interact with individual test cases is the following: 120.Bl -enum 121.It 122For every test program listed in a 123.Xr kyuafile 5 , 124execute the 125.Sq list 126subcommand of the 127.Nm 128that matches the interface name in the test program description. 129.It 130.Xr kyua 1 131collects the test cases returned in the previous step and checks if they are 132runnable according to their metadata. 133.It 134For every test case collected from the previous steps, execute the 135.Sq test 136subcommand of the same 137.Nm 138used in the test case listing. 139.It 140Reinterpret the generic test result and record it as necessary. 141.El 142.Ss The list subcommand 143The 144.Sq list 145subcommand extracts the list of test cases that are part of the given test 146program in a machine-parseable format. The test program should be provided as 147an absolute pathname. 148.Pp 149The format of the output is described in 150.Xr kyua-tester-list 5 . 151.Ss The test subcommand 152The 153.Sq test 154subcommand executes a single test case of a test program in a controlled manner, 155which means that it is placed in its own subprocess and its own work directory 156so that side-effects during execution are minimized. 157.Pp 158If the test is executed successfully, the 159.Ar result_file 160is created following the format described in 161.Xr kyua-tester-result 5 . 162.Pp 163If the test crashes halfway through and dumps core, 164.Nm 165will attempt to gather a stacktrace and print it as part of the test's output. 166.Pp 167The following options are specific to the test command: 168.Bl -tag -width XvXvar=nameXX 169.It Fl v Ar var=name 170Passes a configuration variable to the test case. 171.El 172.Pp 173The following configuration variables have special meaning: 174.Bl -tag -width unprivilegedXuserXX 175.It unprivileged-user 176The name (not UID) of an unprivileged user that the test can use for its own 177purposes. 178.El 179.Sh ENVIRONMENT 180The following variables are recognized and can be freely tuned by the end user: 181.Bl -tag -width TMPDIRXX 182.It Va TMPDIR 183Path to the system-wide temporary directory. 184.Nm 185uses this location to place the work directory of test cases. 186.Pp 187The default value is 188.Pa __TMPDIR__ . 189.El 190.Sh EXIT STATUS 191.Nm 192returns the following exit codes: 193.Bl -tag -compact -width 0XX 194.It 0 195The tester ran correctly and the test passed. 196.It 1 197The tester ran correctly, but the test failed. 198.It 2 199The tester failed. The result of the test itself is irrelevant. 200.It 3 201The user invoked the tester incorrectly. 202.El 203.Sh SEE ALSO 204.Xr kyua-atf-tester 1 , 205.Xr kyua-plain-tester 1 206