dts: improve starting and stopping interactive shells

The InteractiveShell class currently relies on being cleaned up and
shutdown at the time of garbage collection, but this cleanup of the class
do

dts: improve starting and stopping interactive shells

The InteractiveShell class currently relies on being cleaned up and
shutdown at the time of garbage collection, but this cleanup of the class
does no verification that the session is still running prior to cleanup.
So, if a user were to call this method themselves prior to garbage
collection, it would be called twice and throw an exception when the
desired behavior is to do nothing since the session is already cleaned
up. This is solved by using a weakref and a finalize class which
achieves the same result of calling the method at garbage collection,
but also ensures that it is called exactly once.

Additionally, this fixes issues regarding starting a primary DPDK
application while another is still cleaning up via a retry when starting
interactive shells. It also adds catch for attempting to send a command
to an interactive shell that is not running to create a more descriptive
error message.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Tested-by: Dean Marx <dmarx@iol.unh.edu>

show more ...

dts: add context manager for interactive shells

Interactive shells are managed in a way currently where they are closed
and cleaned up at the time of garbage collection. Due to there being no
guaran

dts: add context manager for interactive shells

Interactive shells are managed in a way currently where they are closed
and cleaned up at the time of garbage collection. Due to there being no
guarantee of when this garbage collection happens in Python, there is no
way to consistently know when an application will be closed without
manually closing the application yourself when you are done with it.
This doesn't cause a problem in cases where you can start another
instance of the same application multiple times on a server, but this
isn't the case for primary applications in DPDK. The introduction of
primary applications, such as testpmd, adds a need for knowing previous
instances of the application have been stopped and cleaned up before
starting a new one, which the garbage collector does not provide.

To solve this problem, a new class is added which acts as a base class
for interactive shells that enforces that instances of the
application be managed using a context manager. Using a context manager
guarantees that once you leave the scope of the block where the
application is being used for any reason, the application will be closed
immediately. This avoids the possibility of the shell not being closed
due to an exception being raised or user error. The interactive shell
class then becomes shells that can be started/stopped manually or at the
time of garbage collection rather than through a context manager.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>

show more ...

dts: rework interactive shells

The way nodes and interactive shells interact makes it difficult to
develop for static type checking and hinting. The current system relies
on a top-down approach, att

dts: rework interactive shells

The way nodes and interactive shells interact makes it difficult to
develop for static type checking and hinting. The current system relies
on a top-down approach, attempting to give a generic interface to the
test developer, hiding the interaction of concrete shell classes as much
as possible. When working with strong typing this approach is not ideal,
as Python's implementation of generics is still rudimentary.

This rework reverses the tests interaction to a bottom-up approach,
allowing the test developer to call concrete shell classes directly,
and let them ingest nodes independently. While also re-enforcing type
checking and making the code easier to read.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...

dts: use params module for interactive shells

Make it so that interactive shells accept an implementation of `Params`
for app arguments. Convert EalParameters to use `Params` instead.

String comman

dts: use params module for interactive shells

Make it so that interactive shells accept an implementation of `Params`
for app arguments. Convert EalParameters to use `Params` instead.

String command line parameters can still be supplied by using the
`Params.from_str()` method.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>

show more ...

dts: skip first line of send command output

The first line of the InteractiveShell send_command method is generally
the command input field. This sometimes is unwanted, therefore this
commit enables

dts: skip first line of send command output

The first line of the InteractiveShell send_command method is generally
the command input field. This sometimes is unwanted, therefore this
commit enables the possibility of omitting the first line from the
returned output.

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

show more ...

dts: fix interactive shell prompt filtering

When sending a command using an instance of InteractiveShell the output
should filter out the trailing shell prompt when returning it. After
every command

dts: fix interactive shell prompt filtering

When sending a command using an instance of InteractiveShell the output
should filter out the trailing shell prompt when returning it. After
every command two shell prompts are summoned. One is consumed as it is
used as a delimiter for the command output. The second one is not
consumed and left for the next command to be sent.

Given that the consumed prompt is merely a delimiter, this should not be
added to the returned output, as it may be mistakenly be interpreted as
the command's own output.

Bugzilla ID: 1411
Fixes: 88489c0501af ("dts: add smoke tests")

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>

show more ...

dts: update mypy static checker

Update the mypy static checker to the latest version and fix all the
reported errors.

Bump up supported Poetry shell version.

Bugzilla ID: 1433

Signed-off-by: Luca

dts: update mypy static checker

Update the mypy static checker to the latest version and fix all the
reported errors.

Bump up supported Poetry shell version.

Bugzilla ID: 1433

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>

show more ...

dts: refactor logging configuration

Remove unused parts of the code and add useful features:
1. Add DTS execution stages such as execution and test suite to better
identify where in the DTS lifec

dts: refactor logging configuration

Remove unused parts of the code and add useful features:
1. Add DTS execution stages such as execution and test suite to better
identify where in the DTS lifecycle we are when investigating logs,
2. Logging to separate files in specific stages, which is mainly useful
for having test suite logs in additional separate files.
3. Remove the dependence on the settings module which enhances the
usefulness of the logger module, as it can now be imported in more
modules.

The execution stages and the files to log to are the same for all DTS
loggers. To achieve this, we have one DTS root logger which should be
used for handling stage switching and all other loggers are children of
this DTS root logger. The DTS root logger is the one where we change the
behavior of all loggers (the stage and which files to log to) and the
child loggers just log messages under a different name.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

show more ...

dts: update docstrings

Format according to the Google format and PEP257, with slight
deviations.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

dts: adjust code for doc generation

The standard Python tool for generating API documentation, Sphinx,
imports modules one-by-one when generating the documentation. This
requires code changes:
* pro

dts: adjust code for doc generation

The standard Python tool for generating API documentation, Sphinx,
imports modules one-by-one when generating the documentation. This
requires code changes:
* properly guarding argument parsing in the if __name__ == '__main__'
block,
* the logger used by DTS runner underwent the same treatment so that it
doesn't create log files outside of a DTS run,
* however, DTS uses the arguments to construct an object holding global
variables. The defaults for the global variables needed to be moved
from argument parsing elsewhere,
* importing the remote_session module from framework resulted in
circular imports because of one module trying to import another
module. This is fixed by reorganizing the code,
* some code reorganization was done because the resulting structure
makes more sense, improving documentation clarity.

The are some other changes which are documentation related:
* added missing type annotation so they appear in the generated docs,
* reordered arguments in some methods,
* removed superfluous arguments and attributes,
* change private functions/methods/attributes to private and vice-versa.

The above all appear in the generated documentation and the with them,
the documentation is improved.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

show more ...