dts: add blocklist test suite

This test suite tests the port blocklisting functionality built in
testpmd.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szc

dts: add blocklist test suite

This test suite tests the port blocklisting functionality built in
testpmd.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

show more ...

dts: use Pydantic in the configuration

This change brings in pydantic in place of warlock. Pydantic offers
a built-in model validation system in the classes, which allows for
a more resilient and si

dts: use Pydantic in the configuration

This change brings in pydantic in place of warlock. Pydantic offers
a built-in model validation system in the classes, which allows for
a more resilient and simpler code. As a consequence of this change:

- most validation is now built-in
- further validation is added to verify:
- cross referencing of node names and ports
- test suite and test cases names
- dictionaries representing the config schema are removed
- the config schema is no longer used and therefore dropped
- the TrafficGeneratorType enum has been changed from inheriting
StrEnum to the native str and Enum. This change was necessary to
enable the discriminator for object unions
- the structure of the classes has been slightly changed to perfectly
match the structure of the configuration files
- the test suite argument catches the ValidationError that
TestSuiteConfig can now raise
- the DPDK location has been wrapped under another configuration
mapping `dpdk_location`
- the DPDK locations are now structured and enforced by classes,
further simplifying the validation and handling thanks to
pattern matching

Bugzilla ID: 1508

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: support externally compiled DPDK

Enable the user to use either a DPDK source tree directory or a
tarball, with and without a pre-built build directory. These can be
stored on either SUT node or

dts: support externally compiled DPDK

Enable the user to use either a DPDK source tree directory or a
tarball, with and without a pre-built build directory. These can be
stored on either SUT node or the DTS host. The DPDK build setup or the
pre-built binaries can be specified through the configuration file,
the command line arguments or environment variables.

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: enforce one DPDK build per test run

Given a pre-built DPDK repository can be supplied to DTS, there is no
need to define multiple build targets. To simplify the process this
change makes each t

dts: enforce one DPDK build per test run

Given a pre-built DPDK repository can be supplied to DTS, there is no
need to define multiple build targets. To simplify the process this
change makes each test run use only one DPDK build whether it's
pre-built or it needs to be built by DTS.

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: rename build target to DPDK build

Since the DPDK may already be built, some more general name
is needed that includes both the DPDK location and the build
config (if we are going to build).

Si

dts: rename build target to DPDK build

Since the DPDK may already be built, some more general name
is needed that includes both the DPDK location and the build
config (if we are going to build).

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add VLAN test suite

Adds a test suite for verifying VLAN filtering, stripping, and insertion
functionality on a Poll Mode Driver.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jere

dts: add VLAN test suite

Adds a test suite for verifying VLAN filtering, stripping, and insertion
functionality on a Poll Mode Driver.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add random generation seed setting

When introducing pseudo-random generation in the test runs we need to
ensure that these can be reproduced by setting a pre-defined seed.
This commits adds the

dts: add random generation seed setting

When introducing pseudo-random generation in the test runs we need to
ensure that these can be reproduced by setting a pre-defined seed.
This commits adds the ability to set one or allow for one to be
generated and reported back to the user.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Alex Chapman <alex.chapman@arm.com>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...

dts: rename execution to test run

The configuration containing the combination of:
1. what testbed to use,
2. which tests to run,
3. and what build targets to test

is called an execution. This is c

dts: rename execution to test run

The configuration containing the combination of:
1. what testbed to use,
2. which tests to run,
3. and what build targets to test

is called an execution. This is confusing since we're using the exact
same term to describe other things and execution does not really capture
that well the three items listed above.

A new term is thus needed to describe the configuration. Test run is
much less confusing and better captures what the configuration contains.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: rename hugepage config parameter

The term 'amount' is used for uncountable nouns. Since total hugepages
is a discrete value (i.e. countable), the declaration of the 'amount'
key value pair shou

dts: rename hugepage config parameter

The term 'amount' is used for uncountable nouns. Since total hugepages
is a discrete value (i.e. countable), the declaration of the 'amount'
key value pair should be changes to a different term in both the config
and the rest of the code.

Bugzilla ID: 1370

Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: use exclusively 2MB hugepage in config

The previous implementation configures and allocates hugepage sizes
based on a system default. This can lead to two problems: overallocation of
hugepages

dts: use exclusively 2MB hugepage in config

The previous implementation configures and allocates hugepage sizes
based on a system default. This can lead to two problems: overallocation of
hugepages (which may crash the remote host), and configuration of hugepage
sizes that are not recommended during runtime. This new implementation
allows only 2MB hugepage allocation during runtime; any other unique
hugepage size must be configured by the end-user for initializing DTS.

If the amount of 2MB hugepages requested exceeds the amount of 2MB
hugepages already configured on the system, then the system will remount
hugepages to cover the difference. If the amount of hugepages requested is
either less than or equal to the amount already configured on the system,
then nothing is done.

Bugzilla ID: 1370

Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: add buffer scatter test suite

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered mu

dts: add buffer scatter test suite

This test suite provides testing of the support of scattered packets by
Poll Mode Drivers using testpmd, verifying the ability to receive and
transmit scattered multi-segment packets made up of multiple
non-contiguous memory buffers. This is tested through 5 different cases
in which the length of the packets sent are less than the mbuf size,
equal to the mbuf size, and 1, 4, and 5 bytes greater than the mbuf size
in order to show both the CRC and the packet data are capable of
existing in the first, second, or both buffers.

Naturally, if the PMD is capable of forwarding scattered packets which
it receives as input, this shows it is capable of both receiving and
transmitting scattered packets.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: filter test suites in executions

We're currently filtering which test cases to run after some setup
steps, such as DPDK build, have already been taken. This prohibits us to
mark the test suites

dts: filter test suites in executions

We're currently filtering which test cases to run after some setup
steps, such as DPDK build, have already been taken. This prohibits us to
mark the test suites and cases that were supposed to be run as blocked
when an earlier setup fails, as that information is not available at
that time.

To remedy this, move the filtering to the beginning of each execution.
This is the first action taken in each execution and if we can't filter
the test cases, such as due to invalid inputs, we abort the whole
execution. No test suites nor cases will be marked as blocked as we
don't know which were supposed to be run.

On top of that, the filtering takes place in the TestSuite class, which
should only concern itself with test suite and test case logic, not the
processing behind the scenes. The logic has been moved to DTSRunner
which should do all the processing needed to run test suites.

The filtering itself introduces a few changes/assumptions which are more
sensible than before:
1. Assumption: There is just one TestSuite child class in each test
suite module. This was an implicit assumption before as we couldn't
specify the TestSuite classes in the test run configuration, just the
modules. The name of the TestSuite child class starts with "Test" and
then corresponds to the name of the module with CamelCase naming.
2. Unknown test cases specified both in the test run configuration and
the environment variable/command line argument are no longer silently
ignored. This is a quality of life improvement for users, as they
could easily be not aware of the silent ignoration.

Also, a change in the code results in pycodestyle warning and error:
[E] E203 whitespace before ':'
[W] W503 line break before binary operator

These two are not PEP8 compliant, so they're disabled.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

show more ...

dts: add basic UDP test case

The test cases showcases the scapy traffic generator code.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Acked-by: Jeremy Spewock <jspewock@iol.unh.edu>

dts: add traffic generator config

Node configuration - where to connect, what ports to use and what TG to
use.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

dts: add smoke tests

Adds a new test suite for running smoke tests that verify general
configuration aspects of the system under test. If any of these tests
fail, the DTS execution terminates as par

dts: add smoke tests

Adds a new test suite for running smoke tests that verify general
configuration aspects of the system under test. If any of these tests
fail, the DTS execution terminates as part of a "fail-fast" model.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...

dts: add test suite config and runner

The config allows users to specify which test suites and test cases
within test suites to run.
Also add test suite running capabilities to dts runner.

Signed-o

dts: add test suite config and runner

The config allows users to specify which test suites and test cases
within test suites to run.
Also add test suite running capabilities to dts runner.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add test suite module

The module implements the base class that all test suites inherit from.
It implements methods common to all test suites.
The derived test suites implement test cases and a

dts: add test suite module

The module implements the base class that all test suites inherit from.
It implements methods common to all test suites.
The derived test suites implement test cases and any particular setup
needed for the suite or tests.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add node memory setup

Setup hugepages on nodes. This is useful not only on SUT nodes, but
also on TG nodes which use TGs that utilize hugepages.

The setup is opt-in, i.e. users need to supply

dts: add node memory setup

Setup hugepages on nodes. This is useful not only on SUT nodes, but
also on TG nodes which use TGs that utilize hugepages.

The setup is opt-in, i.e. users need to supply hugepage configuration to
instruct DTS to configure them. It not configured, hugepage
configuration will be skipped. This is helpful if users don't want DTS
to tamper with hugepages on their system.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add DPDK execution handling

Add methods for setting up and shutting down DPDK apps and for
constructing EAL parameters.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruc

dts: add DPDK execution handling

Add methods for setting up and shutting down DPDK apps and for
constructing EAL parameters.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add node and OS abstractions

The abstraction model in DTS is as follows:
Node, defining and implementing methods common to and the base of SUT
(system under test) Node and TG (traffic generator

dts: add node and OS abstractions

The abstraction model in DTS is as follows:
Node, defining and implementing methods common to and the base of SUT
(system under test) Node and TG (traffic generator) Node.
Remote Session, defining and implementing methods common to any remote
session implementation, such as SSH Session.
OSSession, defining and implementing methods common to any operating
system/distribution, such as Linux.

OSSession uses a derived Remote Session and Node in turn uses a derived
OSSession. This split delegates OS-specific and connection-specific code
to specialized classes designed to handle the differences.

The base classes implement the methods or parts of methods that are
common to all implementations and defines abstract methods that must be
implemented by derived classes.

Part of the abstractions is the DTS test execution skeleton:
execution setup, build setup and then test execution.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add config parser

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

The format of the configuration is YAML

dts: add config parser

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

The format of the configuration is YAML. It is validated according to a
json schema which also server as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.

This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.

There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...