xref: /minix3/external/bsd/kyua-atf-compat/dist/atf-run.1 (revision 11be35a165022172ed3cea20f2b5df0307540b0e)

.\" Copyright 2012 Google Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\"   notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\"   notice, this list of conditions and the following disclaimer in the
.\"   documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\"   may be used to endorse or promote products derived from this software
.\"   without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd June 16, 2012
.Dt ATF-RUN 1
.Os
.Sh NAME
.Nm atf-run
.Nd Kyua-based implementation of the deprecated atf-run
.Sh SYNOPSIS
.Nm
.Op Fl v Ar var=value Op .. Fl v Ar varN=valueN
.Op Ar test_program1 Op Ar .. test_programN
.Sh DESCRIPTION
.Em DISCLAIMER :
This version of
.Nm
provides a reimplementation of ATF's execution engine built on top of Kyua.
This is a transitional script and should only be considered a helper to
migrate away from ATF.
Therefore, this version of
.Nm
mimics the
.Em behavior
of the original
.Nm
but
.Em NOT
the specific output of the command.
The output of this implementation is significantly different as that of the
original one.
Please migrate to using
.Sq kyua test
as soon as feasible.
See the
.Sx Migration path
section below for further details.
.Pp
.Nm
executes a collection of test programs or, in other words, a complete
test suite.
The output generated by this tool is minimalistic: such output is only intended
to keep track of what
.Nm
is doing, as the execution of a test suite can take a long time.
In order to obtain detailed reports, please see the
.Xr atf-report 1
utility instead.
.Pp
The list of test programs to execute is read from a
.Pa Kyuafile
or an
.Pa Atffile
file present in the current directory, in this order.
These files describe the test suite stored in the directory they live in,
which aside from the list of test programs also includes meta-data and
configuration variables.
In those cases where only an Atffile exists, the
.Xr atf2kyua 1
utility is used to convert the old Atffiles to new Kyuafiles in a temporary
directory.
.Pp
.Nm
is also in charge of reading the configuration files that tune the behavior
of each test program and passing down the necessary variables to them.
More details on how this is done are given in the
.Sx Configuration
section.
.Pp
In the first synopsis form,
.Nm
parses the
.Pa Atffile
in the current directory and runs all the test programs specified in it.
If any test program names are given as part of the command line, those are
the ones executed instead of the complete list.
.Pp
The following options are available:
.Bl -tag -width XvXvarXvalueXX
.It Fl v Ar var=value
Sets the configuration variable
.Ar var
to the given value
.Ar value .
.El
.Ss Configuration
.Nm
reads configuration data from multiple places.
After all of these places have been analyzed, a list of variable-value
pairs are passed to the test programs to be run.
.Pp
The following locations are scanned for configuration data, in order.
Items down the list override values defined above them:
.Bl -enum
.It
Configuration variables defined in the system-wide configuration file
shared among all test suites.
This lives in
.Pa ${ATF_CONFDIR}/common.conf .
.It
Configuration variables defined in the system-wide test-suite-specific
configuration file.
This lives in
.Pa ${ATF_CONFDIR}/<test-suite>.conf .
.It
Configuration variables defined in the user-specific configuration file
shared among all test suites.
This lives in
.Pa ${HOME}/.atf/common.conf .
.It
Configuration variables defined in the user-specific test-suite-specific
configuration file.
This lives in
.Pa ${HOME}/.atf/<test-suite>.conf .
.It
Configuration variables provided as part of the command line through the
.Fl v
option.
.El
.Pp
The value of
.Va ATF_CONFDIR
in the above list is determined by the build settings of the
.Nm
script, but its value can be overriden by setting such variable in the
environment.
.Pp
The following configuration variables are globally recognized:
.Bl -tag -width XunprivilegedXuserXX
.It Va unprivileged-user
The name of the system user that atf-run will drop root privileges into
for test cases defining
.Sq require.user=unprivileged .
Note that this is
.Em not provided for security purposes ;
this feature is only for the convenience of the user.
.El
.Ss Migration path
Moving from
.Nm
to
.Sq kyua test
is rather simple.
.Pp
The first step is converting all of your Atffiles to Kyuafiles.
The
.Xr atf2kyua 1
script will help you in performing this initial conversion.
.Pp
Once all the Kyuafiles are in place, the following examples show how to
convert some common
.Nm
invocations to their corresponding
.Sq kyua test
ones.
.Bd -literal -offset indent
### Run all tests.
$ atf-run
$ kyua test

### Run specific test programs.
$ atf-run dir1/foo_test bar_test
$ kyua test dir1/foo_test bar_test

### Set configuration variables; foo is the name of your test suite.
$ atf-run -v unprivileged-user=nobody -v iterations=10
$ kyua test -v unprivileged-user=nobody -v test_suite.foo.iterations=10
.Ed
.Pp
That said,
.Xr kyua 1
is way more flexible than
.Nm
ever was.
Please see Kyua's documentation or built-in command-line help to explore
all the additional features included.
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr atf-report 1