1.\" Copyright 2011 Google Inc. 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions are 6.\" met: 7.\" 8.\" * Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" * Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" * Neither the name of Google Inc. nor the names of its contributors 14.\" may be used to endorse or promote products derived from this software 15.\" without specific prior written permission. 16.\" 17.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 18.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 19.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 20.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 21.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 22.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 23.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28.Dd February 7, 2013 29.Dt KYUA 1 30.Os 31.Sh NAME 32.Nm kyua 33.Nd Command-line interface to Kyua quality assurance toolkit 34.Sh SYNOPSIS 35.Nm 36.Op Fl -config Ar file 37.Op Fl -logfile Ar file 38.Op Fl -loglevel Ar level 39.Op Fl -variable Ar name=value 40.Ar command 41.Op Ar command_options 42.Op Ar command_arguments 43.Sh DESCRIPTION 44Kyua is a quality assurance toolkit: a package that provides libraries to 45aid in the creation of test cases, and tools to execute such test cases and 46generate reports of their results. 47.Pp 48This manual page describes 49.Nm 50(note the lowercase), which is the command-line interface to the Kyua 51package. 52.Pp 53If you are here because all you want to know is how to run the tests in 54.Pa __TESTSDIR__ , 55you most likely want to read the 56.Xr tests 7 57manual page instead if your system provides one. 58.Pp 59As can be observed in the synopsis, the interface of 60.Nm 61implements a common subcommand-based interface. The arguments to the tool 62specify, in this order: a set of common options that all the commands 63accept, a required 64.Ar command 65name that specifies what 66.Nm 67should do, and 68a set of possibly-optional 69.Ar command_options 70and 71.Ar command_arguments 72that are specific to the chosen command. 73.Ss Options supported by all commands 74The following options are recognized by all the commands. 75Keep in mind that these must always be specified before the command name. 76.Bl -tag -width XX 77.It Fl -config Ar path , Fl c Ar path 78Specifies the configuration file to process, which must be in the format 79described in 80.Xr kyua.conf 5 . 81The special value 82.Sq none 83explicitly disables the loading of any configuration file. 84.Pp 85Defaults to 86.Pa ~/.kyua/kyua.conf 87if it exists, otherwise to 88.Pa __CONFDIR__/kyua.conf 89if it exists, 90or else to 91.Sq none . 92.It Fl -logfile Ar path 93Specifies the location of the file to which 94.Nm 95will log run time events useful for postmortem debugging. 96.Pp 97The default depends on different environment variables as described in 98.Sx Logging , 99but typically the file will be stored within the user's home directory. 100.It Fl -loglevel Ar level 101Specifies the maximum logging level to record in the log file. 102See 103.Sx Logging 104for more details. 105.Pp 106The default is 107.Sq info . 108.It Fl -variable Ar name=value , Fl v Ar name=value 109Sets the 110.Ar name 111configuration variable to 112.Ar value . 113The values set through this option have preference over the values set in the 114configuration file. 115.Pp 116The specified variable can either be a builtin variable or a test-suite 117specific variable. See 118.Xr kyua.conf 5 119for more details. 120.El 121.Ss Available commands 122The following commands are generic and do not have any relation to the execution 123of tests or the inspection of their results: 124.Bl -tag -width reportXhtmlXX -offset indent 125.It Ar about 126Shows general program information. 127See 128.Xr kyua-about 1 . 129.It Ar config 130Inspects the values of the configuration variables. 131See 132.Xr kyua-config 1 . 133.It Ar db-exec 134Executes an arbitrary SQL statement in the store database and prints the 135resulting table. 136See 137.Xr kyua-db-exec 1 . 138.It Ar help 139Shows usage information. 140See 141.Xr kyua-help 1 . 142.El 143.Pp 144The following commands are used to generate reports based on the data previously 145stored in the database: 146.Bl -tag -width reportXhtmlXX -offset indent 147.It Ar report 148Generates a plain-text report. 149See 150.Xr kyua-report 1 . 151.It Ar report-html 152Generates an HTML report. 153See 154.Xr kyua-report-html 1 . 155.El 156.Pp 157The following commands are used to interact with a test suite: 158.Bl -tag -width reportXhtmlXX -offset indent 159.It Ar debug 160Executes a single test case in a controlled environment for debugging purposes. 161See 162.Xr kyua-debug 1 . 163.It Ar list 164Lists test cases and their metadata. 165See 166.Xr kyua-list 1 . 167.It Ar test 168Runs tests. 169See 170.Xr kyua-test 1 . 171.El 172.Ss Logging 173.Nm 174has a logging facility that collects all kinds of events at run time. 175These events are always logged to a file so that the log is available when 176it is most needed: right after a non-reproducible problem happens. 177The only way to disable logging is by sending the log to 178.Pa /dev/null . 179.Pp 180The location of the log file can be manually specified with the 181.Fl -logfile 182option, which applies to all commands. If no file is explicitly specified, the 183location of the log files is chosen in this order: 184.Bl -enum -offset indent 185.It 186.Pa ${HOME}/.kyua/logs/ 187if 188.Va HOME 189is defined. 190.It 191.Pa ${TMPDIR}/ 192if 193.Va TMPDIR 194is defined. 195.It 196.Pa /tmp/ . 197.El 198.Pp 199And the default naming scheme of the log files is: 200.Sq <progname>.<timestamp>.log . 201.Pp 202The messages stored in the log file have a level (or severity) attached to 203them. These are: 204.Bl -tag -width warningXX -offset indent 205.It error 206Fatal error messages. The program generally terminates after these, either 207in a clean manner or by crashing. 208.It warning 209Non-fatal error messages. These generally report a condition that must be 210addressed but the application can continue to run. 211.It info 212Informational messages. These tell the user what the program was doing at 213a general level of operation. 214.It debug 215Detailed informational messages. These are often useful when debugging 216problems in the application, as they contain lots of internal details. 217.El 218.Pp 219The default log level is 220.Sq info 221unless expicitly overriden with 222.Fl -loglevel . 223.Pp 224The log file is a plain text file containing one line per log record. The 225format of each line is as follows: 226.Bd -literal -offset indent 227timestamp entry_type pid file:line: message 228.Ed 229.Pp 230.Ar entry_type 231can be one of: 232.Sq E 233for an error, 234.Sq W 235for a warning, 236.Sq I 237for an informational message and 238.Sq D 239for a debug message. 240.Ss Bug reporting 241If you think you have encountered a bug in 242.Nm , 243please take the time to let the developers know about it. This will ensure 244that the bug is addressed and potentially fixed in the next Kyua release. 245.Pp 246The first step in reporting a bug is to check if there already is a similar 247bug in the database. You can check what issues are currently in the 248database by going to: 249.Bd -literal -offset indent 250http://code.google.com/p/kyua/issues/list 251.Ed 252.Pp 253If there is no existing issue that describes an issue similar to the 254one you are experiencing, you can open a new one by visiting: 255.Bd -literal -offset indent 256http://code.google.com/p/kyua/issues/entry 257.Ed 258.Pp 259When doing so, please include as much detail as possible. Among other 260things, explain what operating system and platform you are running 261.Nm 262on, what were you trying to do, what exact messages you saw on the screen, 263how did you expect the program to behave, and any other details that you 264may find relevant. 265.Pp 266Also, please include a copy of the log file corresponding to the problem 267you are experiencing. Unless you have changed the location of the log 268files, you can most likely find them in 269.Pa ~/.kyua/logs/ . 270If the problem is reproducible, it is good idea to regenerate the log file 271with an increased log level so as to provide more information. For 272example: 273.Bd -literal -offset indent 274$ kyua --logfile=problem.log --loglevel=debug \\ 275 [rest of the command line] 276.Ed 277.Sh ENVIRONMENT 278The following variables are recognized and can be freely tuned by the end user: 279.Bl -tag -width COLUMNSXX 280.It Va COLUMNS 281The width of the screen, in number of characters. 282.Nm 283uses this to wrap long lines. 284If not present, the width of the screen is determined from the terminal 285stdout is connected to, and, if the guessing fails, this defaults to infinity. 286.It Va HOME 287Path to the user's home directory. 288.Nm 289uses this location to determine paths to configuration files and default log 290files. 291.It Va TMPDIR 292Path to the system-wide temporary directory. 293.Nm 294uses this location to place the work directory of test cases, among other 295things. 296.Pp 297The default value of this variable depends on the operating system. In 298general, it is 299.Pa /tmp . 300.El 301.Pp 302The following variables are also recognized, but you should not need to set them 303during normal operation. 304They are only provided to override the value of built-in values, which is useful 305when testing 306.Nm 307itself: 308.Bl -tag -width KYUAXCONFDIRXX 309.It Va KYUA_CONFDIR 310Path to the system-wide configuration files for 311.Nm . 312.Pp 313Defaults to 314.Pa __CONFDIR__ . 315.It Va KYUA_DOCDIR 316Path to the location of installed documentation. 317.Pp 318Defaults to 319.Pa __DOCDIR__ . 320.It Va KYUA_MISCDIR 321Path to the location of the installed miscellaneous scripts and data 322files provided by 323.Nm . 324.Pp 325Defaults to 326.Pa __MISCDIR__ . 327.It Va KYUA_STOREDIR 328Path to the location of the installed store support files; e.g. the 329directory containing the SQL database schema. 330.Pp 331Defaults to 332.Pa __STOREDIR__ . 333.It Va KYUA_TESTERSDIR 334Path to the location of the installed testers; e.g. the directory containing 335programs like 336.Pa plain_tester . 337Defaults to 338.Pa __TESTERSDIR__ . 339.El 340.Sh FILES 341.Bl -tag -width XXXX 342.It Pa ~/.kyua/kyua.conf 343User-specific configuration file. 344.It Pa ~/.kyua/logs/ 345Default location for the collected log files. 346.It Pa __CONFDIR__/kyua.conf 347System-wide configuration file. 348.El 349.Sh EXIT STATUS 350.Nm 351returns 0 on success, 1 on a controlled error condition in the given 352subcommand, 2 on a general unexpected error and 3 on a usage error. 353.Pp 354The documentation of the subcommands in the corresponding manual pages only 355details the difference between a successful exit (0) and the detection of a 356controlled error (1). Even though when those manual pages do not describe 357any other exit statuses, codes above 1 can be returned. 358.Sh SEE ALSO 359.Xr atf 7 , 360.Xr tests 7 361.Sh AUTHORS 362For more details on the people that made 363.Nm 364possible and the license terms, run: 365.Bd -literal -offset indent 366$ kyua about 367.Ed 368