xref: /netbsd-src/external/bsd/kyua-cli/dist/doc/kyua.1.in (revision 6b3a42af15b5e090c339512c790dd68f3d11a9d8)

.\" Copyright 2011 Google Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\"   notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\"   notice, this list of conditions and the following disclaimer in the
.\"   documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\"   may be used to endorse or promote products derived from this software
.\"   without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd February 7, 2013
.Dt KYUA 1
.Os
.Sh NAME
.Nm kyua
.Nd Command-line interface to Kyua quality assurance toolkit
.Sh SYNOPSIS
.Nm
.Op Fl -config Ar file
.Op Fl -logfile Ar file
.Op Fl -loglevel Ar level
.Op Fl -variable Ar name=value
.Ar command
.Op Ar command_options
.Op Ar command_arguments
.Sh DESCRIPTION
Kyua is a quality assurance toolkit: a package that provides libraries to
aid in the creation of test cases, and tools to execute such test cases and
generate reports of their results.
.Pp
This manual page describes
.Nm
(note the lowercase), which is the command-line interface to the Kyua
package.
.Pp
If you are here because all you want to know is how to run the tests in
.Pa __TESTSDIR__ ,
you most likely want to read the
.Xr tests 7
manual page instead if your system provides one.
.Pp
As can be observed in the synopsis, the interface of
.Nm
implements a common subcommand-based interface.  The arguments to the tool
specify, in this order: a set of common options that all the commands
accept, a required
.Ar command
name that specifies what
.Nm
should do, and
a set of possibly-optional
.Ar command_options
and
.Ar command_arguments
that are specific to the chosen command.
.Ss Options supported by all commands
The following options are recognized by all the commands.
Keep in mind that these must always be specified before the command name.
.Bl -tag -width XX
.It Fl -config Ar path , Fl c Ar path
Specifies the configuration file to process, which must be in the format
described in
.Xr kyua.conf 5 .
The special value
.Sq none
explicitly disables the loading of any configuration file.
.Pp
Defaults to
.Pa ~/.kyua/kyua.conf
if it exists, otherwise to
.Pa __CONFDIR__/kyua.conf
if it exists,
or else to
.Sq none .
.It Fl -logfile Ar path
Specifies the location of the file to which
.Nm
will log run time events useful for postmortem debugging.
.Pp
The default depends on different environment variables as described in
.Sx Logging ,
but typically the file will be stored within the user's home directory.
.It Fl -loglevel Ar level
Specifies the maximum logging level to record in the log file.
See
.Sx Logging
for more details.
.Pp
The default is
.Sq info .
.It Fl -variable Ar name=value , Fl v Ar name=value
Sets the
.Ar name
configuration variable to
.Ar value .
The values set through this option have preference over the values set in the
configuration file.
.Pp
The specified variable can either be a builtin variable or a test-suite
specific variable.  See
.Xr kyua.conf 5
for more details.
.El
.Ss Available commands
The following commands are generic and do not have any relation to the execution
of tests or the inspection of their results:
.Bl -tag -width reportXhtmlXX -offset indent
.It Ar about
Shows general program information.
See
.Xr kyua-about 1 .
.It Ar config
Inspects the values of the configuration variables.
See
.Xr kyua-config 1 .
.It Ar db-exec
Executes an arbitrary SQL statement in the store database and prints the
resulting table.
See
.Xr kyua-db-exec 1 .
.It Ar help
Shows usage information.
See
.Xr kyua-help 1 .
.El
.Pp
The following commands are used to generate reports based on the data previously
stored in the database:
.Bl -tag -width reportXhtmlXX -offset indent
.It Ar report
Generates a plain-text report.
See
.Xr kyua-report 1 .
.It Ar report-html
Generates an HTML report.
See
.Xr kyua-report-html 1 .
.El
.Pp
The following commands are used to interact with a test suite:
.Bl -tag -width reportXhtmlXX -offset indent
.It Ar debug
Executes a single test case in a controlled environment for debugging purposes.
See
.Xr kyua-debug 1 .
.It Ar list
Lists test cases and their metadata.
See
.Xr kyua-list 1 .
.It Ar test
Runs tests.
See
.Xr kyua-test 1 .
.El
.Ss Logging
.Nm
has a logging facility that collects all kinds of events at run time.
These events are always logged to a file so that the log is available when
it is most needed: right after a non-reproducible problem happens.
The only way to disable logging is by sending the log to
.Pa /dev/null .
.Pp
The location of the log file can be manually specified with the
.Fl -logfile
option, which applies to all commands.  If no file is explicitly specified, the
location of the log files is chosen in this order:
.Bl -enum -offset indent
.It
.Pa ${HOME}/.kyua/logs/
if
.Va HOME
is defined.
.It
.Pa ${TMPDIR}/
if
.Va TMPDIR
is defined.
.It
.Pa /tmp/ .
.El
.Pp
And the default naming scheme of the log files is:
.Sq <progname>.<timestamp>.log .
.Pp
The messages stored in the log file have a level (or severity) attached to
them.  These are:
.Bl -tag -width warningXX -offset indent
.It error
Fatal error messages.  The program generally terminates after these, either
in a clean manner or by crashing.
.It warning
Non-fatal error messages.  These generally report a condition that must be
addressed but the application can continue to run.
.It info
Informational messages.  These tell the user what the program was doing at
a general level of operation.
.It debug
Detailed informational messages.  These are often useful when debugging
problems in the application, as they contain lots of internal details.
.El
.Pp
The default log level is
.Sq info
unless expicitly overriden with
.Fl -loglevel .
.Pp
The log file is a plain text file containing one line per log record.  The
format of each line is as follows:
.Bd -literal -offset indent
timestamp entry_type pid file:line: message
.Ed
.Pp
.Ar entry_type
can be one of:
.Sq E
for an error,
.Sq W
for a warning,
.Sq I
for an informational message and
.Sq D
for a debug message.
.Ss Bug reporting
If you think you have encountered a bug in
.Nm ,
please take the time to let the developers know about it.  This will ensure
that the bug is addressed and potentially fixed in the next Kyua release.
.Pp
The first step in reporting a bug is to check if there already is a similar
bug in the database.  You can check what issues are currently in the
database by going to:
.Bd -literal -offset indent
http://code.google.com/p/kyua/issues/list
.Ed
.Pp
If there is no existing issue that describes an issue similar to the
one you are experiencing, you can open a new one by visiting:
.Bd -literal -offset indent
http://code.google.com/p/kyua/issues/entry
.Ed
.Pp
When doing so, please include as much detail as possible.  Among other
things, explain what operating system and platform you are running
.Nm
on, what were you trying to do, what exact messages you saw on the screen,
how did you expect the program to behave, and any other details that you
may find relevant.
.Pp
Also, please include a copy of the log file corresponding to the problem
you are experiencing.  Unless you have changed the location of the log
files, you can most likely find them in
.Pa ~/.kyua/logs/ .
If the problem is reproducible, it is good idea to regenerate the log file
with an increased log level so as to provide more information.  For
example:
.Bd -literal -offset indent
$ kyua --logfile=problem.log --loglevel=debug \\
    [rest of the command line]
.Ed
.Sh ENVIRONMENT
The following variables are recognized and can be freely tuned by the end user:
.Bl -tag -width COLUMNSXX
.It Va COLUMNS
The width of the screen, in number of characters.
.Nm
uses this to wrap long lines.
If not present, the width of the screen is determined from the terminal
stdout is connected to, and, if the guessing fails, this defaults to infinity.
.It Va HOME
Path to the user's home directory.
.Nm
uses this location to determine paths to configuration files and default log
files.
.It Va TMPDIR
Path to the system-wide temporary directory.
.Nm
uses this location to place the work directory of test cases, among other
things.
.Pp
The default value of this variable depends on the operating system.  In
general, it is
.Pa /tmp .
.El
.Pp
The following variables are also recognized, but you should not need to set them
during normal operation.
They are only provided to override the value of built-in values, which is useful
when testing
.Nm
itself:
.Bl -tag -width KYUAXCONFDIRXX
.It Va KYUA_CONFDIR
Path to the system-wide configuration files for
.Nm .
.Pp
Defaults to
.Pa __CONFDIR__ .
.It Va KYUA_DOCDIR
Path to the location of installed documentation.
.Pp
Defaults to
.Pa __DOCDIR__ .
.It Va KYUA_MISCDIR
Path to the location of the installed miscellaneous scripts and data
files provided by
.Nm .
.Pp
Defaults to
.Pa __MISCDIR__ .
.It Va KYUA_STOREDIR
Path to the location of the installed store support files; e.g. the
directory containing the SQL database schema.
.Pp
Defaults to
.Pa __STOREDIR__ .
.It Va KYUA_TESTERSDIR
Path to the location of the installed testers; e.g. the directory containing
programs like
.Pa plain_tester .
Defaults to
.Pa __TESTERSDIR__ .
.El
.Sh FILES
.Bl -tag -width XXXX
.It Pa ~/.kyua/kyua.conf
User-specific configuration file.
.It Pa ~/.kyua/logs/
Default location for the collected log files.
.It Pa __CONFDIR__/kyua.conf
System-wide configuration file.
.El
.Sh EXIT STATUS
.Nm
returns 0 on success, 1 on a controlled error condition in the given
subcommand, 2 on a general unexpected error and 3 on a usage error.
.Pp
The documentation of the subcommands in the corresponding manual pages only
details the difference between a successful exit (0) and the detection of a
controlled error (1).  Even though when those manual pages do not describe
any other exit statuses, codes above 1 can be returned.
.Sh SEE ALSO
.Xr atf 7 ,
.Xr tests 7
.Sh AUTHORS
For more details on the people that made
.Nm
possible and the license terms, run:
.Bd -literal -offset indent
$ kyua about
.Ed