Merge atf-0.19.

Format string annotations and fixes for resulting fallout.

Fix merge conflicts after import of atf 0.17.

Resolve conflicts after the import of atf 0.15.

Post-import merge of atf-0.14.

Help merge of atf-0.12.

Resolve import conflicts.

Pull up revision 3becdd597a7f5f996eeceebf06ac7f77aaa7c30c from upstream:

Catch and report unhandled exceptions before they propagate to C land

The C++ interface to run test cases goes like this:

Pull up revision 3becdd597a7f5f996eeceebf06ac7f77aaa7c30c from upstream:

Catch and report unhandled exceptions before they propagate to C land

The C++ interface to run test cases goes like this:

1) C++ run function -> 2) C run function -> 3) C++ wrapper for
test case -> 4) test case head/body/cleanup

The previous code caught and reported unhandled exceptions in 1).
However, such approach does not seem to work everywhere. It fails,
for example, in NetBSD/i386 but works in NetBSD/amd64. I am not sure
which platform implementation is correct nor if there even _is_ a
defined behavior. No matter what, it feels wrong and clunky.

Move the last-resort exception catching to happen in 3) so that
exceptions don't propagate back to C.

Fixes the test-programs/result_test:result_exception test case in
NetBSD/i386 5.99.34.

show more ...

Fix merge conflicts for atf 0.10.

Introduce expected failures to atf. They can be used to flag tests
which are known to fail, e.g.:

atf_tc_set_md_var(tc, "xfail", "PR kern/43456");

Expected failures do not count towards th

Introduce expected failures to atf. They can be used to flag tests
which are known to fail, e.g.:

atf_tc_set_md_var(tc, "xfail", "PR kern/43456");

Expected failures do not count towards the ultimate pass/fail result
from the test run:

pain-rustique:39:~/<2>src/tests/fs/ptyfs> atf-run t_nullpts | atf-report
Tests root: /home/pooka/src/wholesrc2/src/tests/fs/ptyfs

t_nullpts (1/1): 1 test cases
nullrevoke: Expected failure: PR kern/43456

Summary for 1 test programs:
0 passed test cases.
0 failed test cases.
1 expected failures.
0 skipped test cases.
pain-rustique:40:~/<2>src/tests/fs/ptyfs> echo $?
0

However, an xfail test which passes will count as a failure, i.e.
xfail inverts test case success/fail. This way we can get a better
sense from the ultimate verdict of the NetBSD atf run by seeing if
there were any unexpected failures, i.e. new regressions.

This feature will be present in the upcoming atf 0.10 release,
possibly with finer grained control.

patch reviewed by jmmv

show more ...

Import atf 0.9:

* Added atf-sh, an interpreter to process test programs written using
the shell API. This is not really a shell interpreter by itself
though: it is just a wrapper around the syst

Import atf 0.9:

* Added atf-sh, an interpreter to process test programs written using
the shell API. This is not really a shell interpreter by itself
though: it is just a wrapper around the system shell that eases the
loading of the necessary ATF libraries.

* Removed atf-compile in favour of atf-sh.

* Added the use.fs metadata property to test case, which is used to
specify which test cases require file system access. This is to
highlight dependencies on external resources more clearly and to speed
up the execution of test suites by skipping the creation of many
unnecessary work directories.

* Fixed test programs to get a sane default value for their source
directory. This means that it should not be necessary any more to pass
-s when running test programs that do not live in the current
directory.

* Defining test case headers became optional. This is trivial to achieve
in shell-based tests but a bit ugly in C and C++. In C, use the new
ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use
ATF_TEST_CASE_WITHOUT_HEAD.

show more ...

Import atf 0.8. Changes in this release:

* Test programs no longer run several test cases in a row. The execution
of a test program now requires a test case name, and that single test
case is e

Import atf 0.8. Changes in this release:

* Test programs no longer run several test cases in a row. The execution
of a test program now requires a test case name, and that single test
case is executed. To execute several test cases, use the atf-run
utility as usual.

* Test programs no longer fork a subprocess to isolate the execution of
test cases. They run the test case code in-process, and a crash of the
test case will result in a crash of the test program. This is to ease
debugging of faulty test cases.

* Test programs no longer isolate their test cases. This means that they
will not create temporary directories nor sanitize the environment any
more. Yes: running a test case that depends on system state by hand
will most likely yield different results depending on where (machine,
directory, user environment, etc.) it is run. Isolation has been moved
to atf-run.

* Test programs no longer print a cryptic format (application/X-atf-tcs)
on a special file channel. They can now print whatever they want on
the screen. Because test programs can now only run one test case every
time, providing controlled output is not necessary any more.

* Test programs no longer write their status into a special file
descriptor. Instead, they create a file with the results, which is
later parsed by atf-run. This changes the semantics of the -r flag.

* atf-run has been adjusted to perform the test case isolation. As a
result, there is now a single canonical place that implements the
isolation of test caes. In previous releases, the three language
bindings (C, C++ and shell) had to be kept in sync with each other
(read: not a nice thing to do at all). As a side effect of this
change, writing bindings for other languages will be much, much easier
from now on.

* atf-run forks test programs on a test case basis, instead of on a test
program basis as it did before. This is to provide the test case
isolation that was before implemented by the test programs themselves.

* Removed the atf-exec tool. This was used to implement test case
isolation in atf-sh, but it is now unnecessary.

* It is now optional to define the descr meta-data property. It has been
proven to be mostly useless, because test cases often carry a
descriptive name of their own.

show more ...

Import atf 0.7. Changes in this release:

* Added build-time checks to atf-c and atf-c++. A binding for atf-sh
will come later.

* Migrated all build-time checks for header files to proper ATF te

Import atf 0.7. Changes in this release:

* Added build-time checks to atf-c and atf-c++. A binding for atf-sh
will come later.

* Migrated all build-time checks for header files to proper ATF tests.
This demonstrates the use of the new feature described above.

* Added an internal API for child process management.

* Converted all plain-text distribution documents to a Docbook canonical
version, and include pre-generated plain text and HTML copies in the
distribution file.

* Simplified the contents of the Makefile.am by regenerating it from a
canonical Makefile.am.m4 source. As a side-effect, some dependency
specifications were fixed.

* Migrated all checks from the check target to installcheck, as these
require ATF to be installed.

* Fixed sign comparison mismatches triggered by the now-enabled
-Wsign-compare.

* Fixed many memory and object leaks.

show more ...

Import ATF 0.6. Replaces the old ATF 0.5 that was in dist/atf.

Changes in this release:

* Make atf-exec be able to kill its child process after a certain period of
time; this is controlled throu

Import ATF 0.6. Replaces the old ATF 0.5 that was in dist/atf.

Changes in this release:

* Make atf-exec be able to kill its child process after a certain period of
time; this is controlled through the new -t option.

* Change atf-sh to use atf-exec's -t option to control the test case's
timeouts, instead of doing it internally. Same behavior as before, but
noticeably faster.

* atf-exec's -g option and atf-killpg are gone due to the previous change.

* Added the atf-check(1) tool, a program that executes a given command and
checks its exit code against a known value and allows the management of
stdout and stderr in multiple ways. This replaces the previous atf_check
function in the atf-sh library and exposes this functionality to both
atf-c and atf-c++.

* Added the ATF_REQUIRE family of macros to the C interface. These help
in checking for fatal test conditions. The old ATF_CHECK macros now
perform non-fatal checks only. I.e. by using ATF_CHECK, the test case
can now continue its execution and the failures will not be reported
until the end of the whole run.

* Extended the amount of ATF_CHECK_* C macros with new ones to provide more
features to the developer. These also have their corresponding
counterparts in the ATF_REQUIRE_* family. The new macros (listing the
suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and
_STREQ_MSG.

show more ...