1.\" Copyright 2012 Google Inc. 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions are 6.\" met: 7.\" 8.\" * Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" * Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" * Neither the name of Google Inc. nor the names of its contributors 14.\" may be used to endorse or promote products derived from this software 15.\" without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 18.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 19.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 20.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 21.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 22.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 23.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.Dd December 26, 2012 29.Dt KYUA-ATF-TESTER 1 30.Os 31.Sh NAME 32.Nm kyua-atf-tester 33.Nd Scriptable interface to interact with ATF test programs 34.Sh SYNOPSIS 35.Nm 36.Op Ar generic_flags 37.Ar subcommand 38.Op Ar subcommand_flags 39.Op Ar subcommand_args 40.Sh DESCRIPTION 41.Nm 42provides a scriptable interface to interact with test programs that implement 43the 44.Xr kyua-atf-interface 7 45test interface. 46This manual page describes the specific features that apply to this specific 47tester. For the generic manual page, which describes the command-line interface 48in detail, please see 49.Xr kyua-tester 1 . 50.Ss Results conversion 51The result format outputted by 52.Nm 53(described in 54.Xr kyua-tester-result 5 ) 55is more restricted than what the ATF test case itself can output. The following 56list details maps ATF test results to their generic results: 57.Bl -tag -width expectedXfailureXX 58.It expected_death 59Maps to 60.Sq expected_failure 61if the death happens; 62.Sq failed 63otherwise. 64.It expected_exit 65Maps to 66.Sq expected_failure 67if the test exits cleanly; 68.Sq failed 69otherwise. 70.It expected_failure 71Maps to 72.Sq expected_failure 73if there is any failure during the execution; 74.Sq failed 75otherwise. 76.It expected_signal 77Maps to 78.Sq expected_failure 79if the test terminates due to a signal; 80.Sq failed 81otherwise. 82.It expected_timeout 83Maps to 84.Sq expected_failure 85if the test times out; 86.Sq failed 87otherwise. 88.It failed 89Maps to 90.Sq failed . 91.It passed 92Maps to 93.Sq passed . 94.It skipped 95Maps to 96.Sq skipped . 97.El 98.Pp 99Any of the previous test results 100.Em can also map 101to 102.Sq broken 103in those cases where there is an unexpected problem in the communcation between 104the test and 105.Nm . 106For example, this will happen if result file created by the ATF test case does 107not comply to what 108.Xr kyua-atf-interface 7 109describes. 110.Ss Cleanup routines 111Because 112.Nm 113cannot keep any contextual information between the listing of test cases and 114their execution, the tester cannot easily (aka cheaply) know whether a test has 115or has not a cleanup routine when it is run. For this reason, the 116.Sq test 117subcommand of 118.Nm 119will assume, by default, that a test case 120.Em does have 121a cleanup routine. This postive assumption ensures that, if the tester is 122incorrectly executed, the test will be properly cleaned up. 123.Pp 124However, as an optimization, the caller can pass 125.Sq -v has_cleanup=false 126to the 127.Sq test 128subcommand to tell 129.Nm 130that the test being run does not have a cleanup routine. 131.Sh SEE ALSO 132.Xr kyua-tester 1 , 133.Xr kyua-atf-interface 7 134