dts: add test suite specification and discovery

Currently there is a lack of a definition which identifies all the test
suites available to test. This change intends to simplify the process to
disco

dts: add test suite specification and discovery

Currently there is a lack of a definition which identifies all the test
suites available to test. This change intends to simplify the process to
discover all the test suites and identify them.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add topology capability

Add support for marking test cases as requiring a certain topology. The
default topology is a two link topology and the other supported
topologies are one link and no li

dts: add topology capability

Add support for marking test cases as requiring a certain topology. The
default topology is a two link topology and the other supported
topologies are one link and no link topologies.

The TestProtocol of test suites and cases is extended with the topology
type each test suite or case requires. Each test case starts out as
requiring a two link topology and can be marked as requiring as
topology directly (by decorating the test case) or through its test
suite. If a test suite is decorated as requiring a certain topology, all
its test cases are marked as such. If both test suite and a test case
are decorated as requiring a topology, the test case cannot require a
more complex topology than the whole suite (but it can require a less
complex one). If a test suite is not decorated, this has no effect on
required test case topology.

Since the default topology is defined as a reference to one of the
actual topologies, the NoAliasEnum from the aenum package is utilized,
which removes the aliasing of Enums so that TopologyType.two_links and
TopologyType.default are distinct. This is needed to distinguish between
a user passed value and the default value being used (which is used when
a test suite is or isn't decorated).

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add NIC capabilities from Rx queue info

Add parsing for the show rxq info <port_id> <queue_id> tespmd command
and add support for the Scattered Rx capability.

Signed-off-by: Juraj Linkeš <jura

dts: add NIC capabilities from Rx queue info

Add parsing for the show rxq info <port_id> <queue_id> tespmd command
and add support for the Scattered Rx capability.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add NIC capability support

Some test cases or suites may be testing a NIC feature that is not
supported on all NICs, so add support for marking test cases or suites
as requiring NIC capabilitie

dts: add NIC capability support

Some test cases or suites may be testing a NIC feature that is not
supported on all NICs, so add support for marking test cases or suites
as requiring NIC capabilities.

The marking is done with a decorator, which populates the internal
required_capabilities attribute of TestProtocol. The NIC capability
itself is a wrapper around the NicCapability defined in testpmd_shell.
The reason is Enums cannot be extended and the class implements the
methods of its abstract base superclass.

The decorator API is designed to be simple to use. The arguments passed
to it are all from the testpmd shell. Everything else (even the actual
capability object creation) is done internally.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add basic capability support

A test case or suite may require certain capabilities to be present in
the tested environment. Add the basic infrastructure for checking the
support status of capab

dts: add basic capability support

A test case or suite may require certain capabilities to be present in
the tested environment. Add the basic infrastructure for checking the
support status of capabilities:
* The Capability ABC defining the common capability API
* Extension of the TestProtocol with required capabilities (each test
suite or case stores the capabilities it requires)
* Integration with the runner which calls the new APIs to get which
capabilities are supported.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add mechanism to skip test cases or suites

If a test case is not relevant to the testing environment (such as when
a NIC doesn't support a tested feature), the framework should skip it.
The mec

dts: add mechanism to skip test cases or suites

If a test case is not relevant to the testing environment (such as when
a NIC doesn't support a tested feature), the framework should skip it.
The mechanism is a skeleton without actual logic that would set a test
case or suite to be skipped.

The mechanism uses a protocol to extend test suites and test cases with
additional attributes that track whether the test case or suite should
be skipped the reason for skipping it.

Also update the results module with the new SKIP result.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...