dts: support externally compiled DPDK

Enable the user to use either a DPDK source tree directory or a
tarball, with and without a pre-built build directory. These can be
stored on either SUT node or

dts: support externally compiled DPDK

Enable the user to use either a DPDK source tree directory or a
tarball, with and without a pre-built build directory. These can be
stored on either SUT node or the DTS host. The DPDK build setup or the
pre-built binaries can be specified through the configuration file,
the command line arguments or environment variables.

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: improve output gathering in interactive shells

The current implementation of consuming output from interactive shells
relies on being able to find an expected prompt somewhere within the
output

dts: improve output gathering in interactive shells

The current implementation of consuming output from interactive shells
relies on being able to find an expected prompt somewhere within the
output buffer after sending the command. This is useful in situations
where the prompt does not appear in the output itself, but in some
practical cases (such as the starting of an XML-RPC server for scapy)
the prompt exists in one of the commands sent to the shell and this can
cause the command to exit early and creates a race condition between the
server starting and the first command being sent to the server.

This patch addresses this problem by searching for a line that strictly
ends with the provided prompt, rather than one that simply contains it,
so that the detection that a command is finished is more consistent. It
also adds a catch to detect when a command times out before finding the
prompt or the underlying SSH session dies so that the exception can be
wrapped into a more explicit one and be more consistent with the
non-interactive shells.

Bugzilla ID: 1359
Fixes: 88489c0501af ("dts: add smoke tests")

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>

show more ...

dts: add parsing utility module

Adds parsing text into a custom dataclass. It provides a new
`TextParser` dataclass to be inherited. This implements the `parse`
method, which combined with the parse

dts: add parsing utility module

Adds parsing text into a custom dataclass. It provides a new
`TextParser` dataclass to be inherited. This implements the `parse`
method, which combined with the parser functions, it can automatically
parse the value for each field.

This new utility will facilitate and simplify the parsing of complex
command outputs, while ensuring that the codebase does not get bloated
and stays flexible.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>

show more ...

dts: store stderr of remote command

Store the stderr of an executed command in RemoteCommandExecutionError.
Consequently, when the exception is logged the error message includes
the stderr.

Signed-

dts: store stderr of remote command

Store the stderr of an executed command in RemoteCommandExecutionError.
Consequently, when the exception is logged the error message includes
the stderr.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

show more ...

dts: add start/stop/mode to testpmd shell

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that d

dts: add start/stop/mode to testpmd shell

Added commonly used methods in testpmd such as starting and stopping
packet forwarding, changing forward modes, and verifying link status of
ports so that developers can configure testpmd and start forwarding
through the provided class rather than sending commands to the testpmd
session directly.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: update docstrings

Format according to the Google format and PEP257, with slight
deviations.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

dts: adjust code for doc generation

The standard Python tool for generating API documentation, Sphinx,
imports modules one-by-one when generating the documentation. This
requires code changes:
* pro

dts: adjust code for doc generation

The standard Python tool for generating API documentation, Sphinx,
imports modules one-by-one when generating the documentation. This
requires code changes:
* properly guarding argument parsing in the if __name__ == '__main__'
block,
* the logger used by DTS runner underwent the same treatment so that it
doesn't create log files outside of a DTS run,
* however, DTS uses the arguments to construct an object holding global
variables. The defaults for the global variables needed to be moved
from argument parsing elsewhere,
* importing the remote_session module from framework resulted in
circular imports because of one module trying to import another
module. This is fixed by reorganizing the code,
* some code reorganization was done because the resulting structure
makes more sense, improving documentation clarity.

The are some other changes which are documentation related:
* added missing type annotation so they appear in the generated docs,
* reordered arguments in some methods,
* removed superfluous arguments and attributes,
* change private functions/methods/attributes to private and vice-versa.

The above all appear in the generated documentation and the with them,
the documentation is improved.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...

dts: reformat to 100 line length

Reformat to 100 from the previous 88 to unify with C recommendations.

The C recommendation is the maximum with the ideal being 80.
The Python tools are not suitable

dts: reformat to 100 line length

Reformat to 100 from the previous 88 to unify with C recommendations.

The C recommendation is the maximum with the ideal being 80.
The Python tools are not suitable for this flexibility.

We require all patches with DTS code to be validated with the
devtools/dts-check-format.sh script, part of which is the black
formatting tool. We've set up black to format all of the codebase and
the reformat is needed so that future submitters are not affected.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Acked-by: Jeremy Spewock <jspewock@iol.unh.edu>

show more ...

dts: add smoke tests

Adds a new test suite for running smoke tests that verify general
configuration aspects of the system under test. If any of these tests
fail, the DTS execution terminates as par

dts: add smoke tests

Adds a new test suite for running smoke tests that verify general
configuration aspects of the system under test. If any of these tests
fail, the DTS execution terminates as part of a "fail-fast" model.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...

dts: replace pexpect with fabric

Pexpect is not a dedicated SSH connection library while Fabric is. With
Fabric, all SSH-related logic is provided and we can just focus on
what's DTS specific.

Sign

dts: replace pexpect with fabric

Pexpect is not a dedicated SSH connection library while Fabric is. With
Fabric, all SSH-related logic is provided and we can just focus on
what's DTS specific.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Acked-by: Jeremy Spewock <jspewock@iol.unh.edu>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add test suite module

The module implements the base class that all test suites inherit from.
It implements methods common to all test suites.
The derived test suites implement test cases and a

dts: add test suite module

The module implements the base class that all test suites inherit from.
It implements methods common to all test suites.
The derived test suites implement test cases and any particular setup
needed for the suite or tests.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add DPDK build on SUT

Add the ability to build DPDK and apps on the SUT, using a configured
target.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.

dts: add DPDK build on SUT

Add the ability to build DPDK and apps on the SUT, using a configured
target.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add SSH command verification

This is a basic capability needed to check whether the command execution
was successful or not. If not, raise a RemoteCommandExecutionError. When
a failure is expec

dts: add SSH command verification

This is a basic capability needed to check whether the command execution
was successful or not. If not, raise a RemoteCommandExecutionError. When
a failure is expected, the caller is supposed to catch the exception.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add node and OS abstractions

The abstraction model in DTS is as follows:
Node, defining and implementing methods common to and the base of SUT
(system under test) Node and TG (traffic generator

dts: add node and OS abstractions

The abstraction model in DTS is as follows:
Node, defining and implementing methods common to and the base of SUT
(system under test) Node and TG (traffic generator) Node.
Remote Session, defining and implementing methods common to any remote
session implementation, such as SSH Session.
OSSession, defining and implementing methods common to any operating
system/distribution, such as Linux.

OSSession uses a derived Remote Session and Node in turn uses a derived
OSSession. This split delegates OS-specific and connection-specific code
to specialized classes designed to handle the differences.

The base classes implement the methods or parts of methods that are
common to all implementations and defines abstract methods that must be
implemented by derived classes.

Part of the abstractions is the DTS test execution skeleton:
execution setup, build setup and then test execution.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Chenyu Huang <chenyux.huang@intel.com>
Tested-by: Patrick Robb <probb@iol.unh.edu>

show more ...

dts: add SSH session

The module uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be match

dts: add SSH session

The module uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...