atf_pytest_wrapper: fix use with pytest-7.4

As of pytest 7.4 it no longer walks all the way to the root directory
of the file system to find conftest files. As a result we don't find /
usr/tests/con

atf_pytest_wrapper: fix use with pytest-7.4

As of pytest 7.4 it no longer walks all the way to the root directory
of the file system to find conftest files. As a result we don't find /
usr/tests/conftest.py, and don't load atf_python. That in turn causes
atf_python tests to fail.

Explicitly set the confcutdir, as advised by the pytest changelog.

See also: https://github.com/pytest-dev/pytest/pull/11043
MFC after: 3 weeks
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D41064

show more ...

atf_pytest_wrapper: fix pytest output truncation

Pass `-vv` to pytest in order to always get the full output.
While here, enforce the modeline.

Differential Revision: https://reviews.freebsd.org/D3

atf_pytest_wrapper: fix pytest output truncation

Pass `-vv` to pytest in order to always get the full output.
While here, enforce the modeline.

Differential Revision: https://reviews.freebsd.org/D37894
MFC after: 2 weeks

show more ...

testing: provide meaningful error when pytest is not available

atf format does not provide any way of signalling any error message
back to the atf runner when listing tests. Work this around by
re

testing: provide meaningful error when pytest is not available

atf format does not provide any way of signalling any error message
back to the atf runner when listing tests. Work this around by
reporting "__test_cases_list_pytest_binary_not_found__" test instead.

Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D35721

show more ...

testing: pass ATF vars to pytest via env instead of arguments.

This change is a continuation of 9c42645a1e4d workaround.
Apparently pytest argument parser is not happy when parsing values
with spac

testing: pass ATF vars to pytest via env instead of arguments.

This change is a continuation of 9c42645a1e4d workaround.
Apparently pytest argument parser is not happy when parsing values
with spaces or just more than one --atf-var argument.
Switch wrapper to send these kv pairs as env variables. Specifically,
use _ATF_VAR_key=value format to distinguish from the other vars.

Add the `atf_vars` fixture returning all passed kv pairs as a dict.

Reviewed by: lwhsu
Differential Revision: https://reviews.freebsd.org/D35625
MFC after: 2 weeks

show more ...

testing: workaround pytest parser bug in pytest-atf-wrapper.

Reviewed by: lwhsu, kp
Differential Revision: https://reviews.freebsd.org/D35614
MFC after: 2 weeks

testing: move atf-pytest-wrapper to /usr/libexec

Move pytest wrapper to the collection of the other atf wrappers
in libexec. It solves the problem of combining bits & pieces from
bsd.test.mk and b

testing: move atf-pytest-wrapper to /usr/libexec

Move pytest wrapper to the collection of the other atf wrappers
in libexec. It solves the problem of combining bits & pieces from
bsd.test.mk and bgs.prog.mk to address "test binary, but not the
suite binary".

Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D35604
MFC after: 2 weeks

show more ...