1.\" 2.\" Automated Testing Framework (atf) 3.\" 4.\" Copyright (c) 2007 The NetBSD Foundation, Inc. 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND 17.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, 18.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 19.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 20.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY 21.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE 23.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 24.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 25.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR 26.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN 27.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.\" 29.Dd April 10, 2021 30.Dt ATF-RUN 1 31.Os 32.Sh NAME 33.Nm atf-run 34.Nd executes a collection of tests 35.Sh SYNOPSIS 36.Nm 37.Op Fl v Ar var1=value1 Op .. Fl v Ar varN=valueN 38.Op Ar test1 Op Ar .. testN 39.Nm 40.Fl h 41.Sh DESCRIPTION 42.Nm 43executes a collection of test cases, test programs, or a complete test suite. 44The results of each test are collected by the tool, and are then 45multiplexed into a single machine-parseable report; see 46.Xr atf-formats 5 47for more details. 48This report can later be transformed into many different and saner formats 49using the 50.Nm atf-report 51tool. 52.Pp 53The list of test programs to execute is read from an 54.Pa Atffile 55present in the current directory. 56This file describes the test suite stored in the directory it lives in, 57which aside from the list of test programs also includes meta-data and 58configuration variables. 59.Pp 60.Nm 61is also in charge of reading the configuration files that tune the behavior 62of each test program and passing down the necessary variables to them. 63More details on how this is done are given in the 64.Sx Configuration 65section. 66.Pp 67In the first synopsis form, 68.Nm 69parses the 70.Pa Atffile 71in the current directory and runs all the test programs specified in it. 72If any 73.Ar test 74arguments are given as part of the command line, those tests are 75executed instead of the complete list. Each 76.Ar test 77argument can be either the name of a test program, or a string of the form 78.Ar test_program:test_case 79to execute a single test case. 80.Pp 81In the second synopsis form, 82.Nm 83will print information about all supported options and their purpose. 84.Pp 85The following options are available: 86.Bl -tag -width XvXvarXvalueXX 87.It Fl h 88Shows a short summary of all available options and their purpose. 89.It Fl v Ar var=value 90Sets the configuration variable 91.Ar var 92to the given value 93.Ar value . 94.El 95.Ss Configuration 96.Nm 97reads configuration data from multiple places. 98After all of these places have been analyzed, a list of variable-value 99pairs are passed to the test programs to be run. 100.Pp 101The following locations are scanned for configuration data, in order. 102Items down the list override values defined above them: 103.Bl -enum 104.It 105Configuration variables defined in the 106.Pa Atffile . 107.It 108Configuration variables defined in the system-wide configuration file 109shared among all test suites. 110This lives in 111.Pa ${ATF_CONFDIR}/common.conf . 112.It 113Configuration variables defined in the system-wide test-suite-specific 114configuration file. 115This lives in 116.Pa ${ATF_CONFDIR}/<test-suite>.conf . 117.It 118Configuration variables defined in the user-specific configuration file 119shared among all test suites. 120This lives in 121.Pa ${HOME}/.atf/common.conf . 122.It 123Configuration variables defined in the user-specific test-suite-specific 124configuration file. 125This lives in 126.Pa ${HOME}/.atf/<test-suite>.conf . 127.It 128Configuration variables provided as part of the command line through the 129.Fl v 130option. 131.El 132.Pp 133The value of 134.Va ATF_CONFDIR 135in the above list determined as detailed in 136.Xr atf-config 1 . 137.Pp 138The following configuration variables are globally recognized: 139.Bl -tag -width XunprivilegedXuserXX 140.It Va unprivileged-user 141The name of the system user that atf-run will drop root privileges into 142for test cases defining 143.Sq require.user=unprivileged . 144Note that this is 145.Em not provided for security purposes ; 146this feature is only for the convenience of the user. 147.El 148.Ss Hooks 149.Nm Ns 's 150internal behavior can be customized by the system administrator and the 151user by means of hooks. 152These hooks are written in the shell script language for simplicity and 153are stored in the following files, which are read in the order provided 154below: 155.Bl -enum 156.It 157${ATF_CONFDIR}/atf-run.hooks 158.It 159${HOME}/.atf/atf-run.hooks 160.El 161.Pp 162The following hooks are supported: 163.Bl -tag -width infoXstartXhookXX 164.It info_start_hook 165Called before 166.Nm 167executes any test program. 168The purpose of this hook is to write additional 169.Sq info 170stanzas to the top of the output report; these are defined by the 171.Sq application/X-atf-tps format 172described in 173.Xr atf-formats 5 . 174Always use the 175.Sq atf_tps_writer_info 176function to print these. 177.Pp 178This takes no parameters. 179.It info_end_hook 180Similar to 181.Sq info_start_hook 182but executed after all test programs have been run so that additional 183.Sq info 184stanzas can be added to the bottom of the output report. 185.Pp 186This takes no parameters. 187.El 188.Pp 189All hooks are accompanied by a function named 190.Sq default_<hook_name> 191that can be executed by them to invoke the default behavior built into 192.Nm . 193For example, in order to extend the default 194.Sq info_start_hook 195hook, we could write the following function: 196.Bd -literal -offset indent 197info_start_hook() 198{ 199 default_info_start_hook "${@}" 200 201 atf_tps_writer_info "uptime" "$(uptime)" 202} 203.Ed 204.Sh SEE ALSO 205.Xr atf-report 1 , 206.Xr atf-test-program 1 , 207.Xr atf 7 208