xref: /freebsd-src/contrib/kyua/doc/kyua-report.1.in (revision b0d29bc47dba79f6f38e67eabadfb4b32ffd9390)

.\" Copyright 2012 The Kyua Authors.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\"   notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\"   notice, this list of conditions and the following disclaimer in the
.\"   documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\"   may be used to endorse or promote products derived from this software
.\"   without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd October 13, 2014
.Dt KYUA-REPORT 1
.Os
.Sh NAME
.Nm "kyua report"
.Nd Generates reports with the results of a test suite run
.Sh SYNOPSIS
.Nm
.Op Fl -output Ar path
.Op Fl -results-file Ar file
.Op Fl -results-filter Ar types
.Op Fl -verbose
.Op Ar test_filter1 .. test_filterN
.Sh DESCRIPTION
The
.Nm
command parses a results file and generates a user-friendly, plaintext
report for user consumption on the terminal.
By default, these reports only display a summary of the execution of the full
test suite to highlight where problems may lie.
.Pp
The output of
.Nm
can be customized to display full details on all executed test cases.
Additionally, the optional arguments to
.Nm
are used to select which test programs or test cases to display.
These are filters and are described below in
.Sx Test filters .
.Pp
Reports generated by
.Nm
are
.Em not intended to be machine-parseable .
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -output Ar path
Specifies the path to which the report should be written to.
The special values
.Pa /dev/stdout
and
.Pa /dev/stderr
can be used to specify the standard output and the standard error,
respectively.
.It Fl -results-file Ar path , Fl s Ar path
__include__ results-file-flag-read.mdoc
.It Fl -results-filter Ar types
Comma-separated list of the test result types to include in the report.
The ordering of the values is respected so that you can determine how you
want the list of tests to be shown.
.Pp
The valid values are:
.Sq broken ,
.Sq failed ,
.Sq passed ,
.Sq skipped
and
.Sq xfail .
If the parameter supplied to the option is empty, filtering is suppressed
and all result types are shown in the report.
.Pp
The default value for this flag includes all the test results except the
passed tests.
Showing the passed tests by default clutters the report with too much
information, so only abnormal conditions are included.
.It Fl -verbose
Prints a detailed report of the execution.
In addition to all the information printed by default, verbose reports
include the runtime context of the test suite run, the metadata of each
test case, and the verbatim output of the test cases.
.El
.Ss Results files
__include__ results-files.mdoc
.Ss Test filters
__include__ test-filters.mdoc
.Sh EXIT STATUS
The
.Nm
command returns 0 if no filters were specified or if all filters match one
or more test cases.
If any filter fails to match any test case, the command returns 1.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh EXAMPLES
__include__ results-files-report-example.mdoc REPORT_COMMAND=report
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report-html 1 ,
.Xr kyua-report-junit 1