usertools/pmdinfo: remove unneeded whitespace

Fix the warning
$ flake8 --max-line-length=100 dpdk-pmdinfo.py
dpdk-pmdinfo.py:217:40: E203 whitespace before ':'

Signed-off-by: Stephen Hemminger <ste

usertools/pmdinfo: remove unneeded whitespace

Fix the warning
$ flake8 --max-line-length=100 dpdk-pmdinfo.py
dpdk-pmdinfo.py:217:40: E203 whitespace before ':'

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Robin Jarry <rjarry@redhat.com>

show more ...

usertools/pmdinfo: fix usage typos

The docs and script usages are not consistent.

There is no .devices, .vendor_id nor .device_id fields. Fix usage to the
correct field names.

Some drivers do not

usertools/pmdinfo: fix usage typos

The docs and script usages are not consistent.

There is no .devices, .vendor_id nor .device_id fields. Fix usage to the
correct field names.

Some drivers do not expose any pci_ids, show how to use the []? jq
operator to avoid spurious errors.

Fixes: 0ce3cf4afd04 ("usertools/pmdinfo: rewrite simpler script")
Cc: stable@dpdk.org

Signed-off-by: Robin Jarry <rjarry@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

usertools/pmdinfo: rewrite simpler script

dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
merely prints multiple independent JSON lines which cannot be fed
directly to any J

usertools/pmdinfo: rewrite simpler script

dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag
merely prints multiple independent JSON lines which cannot be fed
directly to any JSON parser. Moreover, the script complexity is rather
high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF
sections. Rewrite it so that it can produce valid JSON.

Remove the PCI database parsing for PCI-ID to Vendor-Device names
conversion. This should be done by external scripts (if really needed).

The script passes flake8, black, isort and pylint checks.

I have tested this with a matrix of python/pyelftools versions:

pyelftools
0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29
3.6 ok ok ok ok ok ok ok ok
3.7 ok ok ok ok ok ok ok ok
Python 3.8 ok ok ok ok ok ok ok ok
3.9 ok ok ok ok ok *ok ok ok
3.10 fail fail fail fail ok ok ok ok

* Also tested on FreeBSD

All failures with python 3.10 are related to the same issue:

File "elftools/construct/lib/container.py", line 5, in <module>
from collections import MutableMapping
ImportError: cannot import name 'MutableMapping' from 'collections'

Python 3.10 support is only available since pyelftools 0.26. The script
will only work with Python 3.6 and later.

Update the minimal system requirements, docs and release notes.

Signed-off-by: Robin Jarry <rjarry@redhat.com>
Tested-by: Ferruh Yigit <ferruh.yigit@amd.com>
Tested-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

usertools/pmdinfo: fix plugin auto scan

Migration to argparse was incomplete.

$ dpdk-pmdinfo.py -p $(which dpdk-testpmd)
Traceback (most recent call last):
File "/usr/bin/dpdk-pmdinfo.py", line 6

usertools/pmdinfo: fix plugin auto scan

Migration to argparse was incomplete.

$ dpdk-pmdinfo.py -p $(which dpdk-testpmd)
Traceback (most recent call last):
File "/usr/bin/dpdk-pmdinfo.py", line 626, in <module>
main()
File "/usr/bin/dpdk-pmdinfo.py", line 596, in main
exit(scan_for_autoload_pmds(args[0]))
TypeError: 'Namespace' object does not support indexing

Fixes: 81255f27c65c ("usertools: replace optparse with argparse")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Robin Jarry <robin.jarry@6wind.com>

show more ...

usertools: fix pmdinfo parsing

In the display_pmd_info_strings function, the script parses the section
until to find a byte between 32 and 127, and get all data
until a byte equals to 0.
After, it s

usertools: fix pmdinfo parsing

In the display_pmd_info_strings function, the script parses the section
until to find a byte between 32 and 127, and get all data
until a byte equals to 0.
After, it searches "PMD_INFO_STRING" in the data and passes the whole
string in the parse_pmd_info_string function, which split the string
with "=" and convert it in python dict with json.loads().

But the string may contain a "=" before "PMD_INFO_STRING",
so it is not correctly split and will lead to an error
(json.decoder.JSONDecodeError).

Example of a string encountered that leads to an error:

"Ag%=C£°ÐÊ+Ë®{0´wË-£0òjB·;¾¬úPMD_INFO_STRING= {"name" :
"net_octeontx", "params" : "nr_port=<int> ", "pci_ids" : []}"

Fixes: c67c9a5c646a ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org

Signed-off-by: Julien Massonneau <julien.massonneau@6wind.com>

show more ...

usertools: do not test empty with function len

Python lint warns about using len(SEQUENCE) to determine if sequence is empty.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by:

usertools: do not test empty with function len

Python lint warns about using len(SEQUENCE) to determine if sequence is empty.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

usertools: replace optparse with argparse

The optparse module is deprecated and replaced with new argparse.
Using the python standard argument parser instead of C library
style getopt gives a number

usertools: replace optparse with argparse

The optparse module is deprecated and replaced with new argparse.
Using the python standard argument parser instead of C library
style getopt gives a number of advantages such as checking
for conflicting arguments, restricting choices, and automatically
generating help messages.

Some of the help messages are now less wordy.

The code now enforces the rule that only one of the pmdinfo formats
can be specified: raw or json.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

usertools: fix indentation

Python lint complains about indentation and missing spaces around commas.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

Signed-off-by: Stephen Hemminger

usertools: fix indentation

Python lint complains about indentation and missing spaces around commas.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

usertools: replace explicit boolean checks

Code reads better if unnecessary comparison with False and True
is not used.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

usertools: remove unnecessary parens and else

Python lint complains:
Unnecessary parens after 'if' keyword
Unnecessary parens after 'not' keyword
Unnecessary "else" after "return"

Signed-off-by: St

usertools: remove unnecessary parens and else

Python lint complains:
Unnecessary parens after 'if' keyword
Unnecessary parens after 'not' keyword
Unnecessary "else" after "return"

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

usertools: replace io.open

The builtin open() is the recommended approach in python3.
io.open was for compatibility with older versions.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

usertools: replace string.split

In python3 the standard way to split strings is to use the
split() on the string object itself. The old way is broken
and would cause a traceback.

Signed-off-by: Ste

usertools: replace string.split

In python3 the standard way to split strings is to use the
split() on the string object itself. The old way is broken
and would cause a traceback.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

usertools: fix pmdinfo parsing

This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a fi

usertools: fix pmdinfo parsing

This script inspects an ELF file (binary or shared library) and its
linked dependencies by following DT_NEEDED tags.
So far a simple librte_pmd prefix was used as a filter to only parse
DPDK drivers dependencies.
While the reason is not clear from the commitlog of the patch that
introduced this filter, it was probably added for performance reasons,
since going through all dependencies can be quite long.
Testing with a DPDK built before the driver name changes:
- running the script takes ~0.3s with the filter,
- running the script takes ~9s without the filter,

Now that we changed the driver library names, it becomes more difficult
to identify only DPDK drivers, but we can just filter on the librte_
prefix to identify DPDK libraries: the script later checks for the
PMD_INFO_STRING string in .rodata and it is enough to differentiate the
DPDK drivers from the other DPDK libraries.

Running the script with this patch takes ~0.5s.

A debug message was logged for each inspected file, it gives no useful
information and is removed.

Fixes: a20b2c01a7a1 ("build: standardize component names and defines")

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

support python 3 only

Changed scripts to explicitly use Python 3 only, to avoid
maintaining Python 2.
Removed deprecation notices.

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Signe

support python 3 only

Changed scripts to explicitly use Python 3 only, to avoid
maintaining Python 2.
Removed deprecation notices.

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ray Kinsella <mdr@ashroe.eu>

show more ...

add python2 deprecation notice

Prepare for python2 removal in 20.11.

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Nicolas Cha

add python2 deprecation notice

Prepare for python2 removal in 20.11.

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Nicolas Chautru <nicolas.chautru@intel.com>

show more ...

usertools: check for pci.ids in /usr/share/misc

Debian and Ubuntu switched years ago from /usr/share/hwdata to
/usr/share/misc, and the former is just a compat symlink now.
We are starting to get bu

usertools: check for pci.ids in /usr/share/misc

Debian and Ubuntu switched years ago from /usr/share/hwdata to
/usr/share/misc, and the former is just a compat symlink now.
We are starting to get bug reports to nudge us into changing.
So check the new path first, and the old one as a fallback.

Cc: stable@dpdk.org

Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

usertools: fix syntax warning in python 3.8

Silent the following warning when running script with python 3.8:

> /usr/bin/dpdk-pmdinfo:542: SyntaxWarning: "is" with a literal.
> Did you mean "=="?
>

usertools: fix syntax warning in python 3.8

Silent the following warning when running script with python 3.8:

> /usr/bin/dpdk-pmdinfo:542: SyntaxWarning: "is" with a literal.
> Did you mean "=="?
> if (autoload_path is None or autoload_path is ""):

As autoload_path can only be None or a string, directly check its bool
value.

Fixes: c67c9a5c646a ("tools: query binaries for HW and other support information")
Cc: stable@dpdk.org

Signed-off-by: Thomas Faivre <thomas.faivre@6wind.com>

show more ...

usertools: fix pmdinfo with python 3 and pyelftools>=0.24

Running dpdk-pmdinfo.py on Ubuntu 18.04 (bionic) with python 3 and
pyelftools installed produces no output but no error is reported
neither:

usertools: fix pmdinfo with python 3 and pyelftools>=0.24

Running dpdk-pmdinfo.py on Ubuntu 18.04 (bionic) with python 3 and
pyelftools installed produces no output but no error is reported
neither:

~$ python3 usertools/dpdk-pmdinfo.py -r build/app/testpmd
~$ echo $?
0

While with python 2, it works:

~# python2 usertools/dpdk-pmdinfo.py -r build/app/testpmd
{"pci_ids": [], "name": "dpio"}
{"pci_ids": [], "name": "dpbp"}
{"pci_ids": [], "name": "dpaa2_qdma"}
.....

On Ubuntu 18.04, pyelftools is version 0.24. The change log of
pyelftools v0.24 says:

- Symbol/section names are strings internally now, not bytestrings
(this may affect API usage in Python 3) (#76).

We cannot guess which version of pyelftools is actually being used. The
elftools.__version__ symbol is not consistent with each distro's package
version. For example, on Ubuntu 16.04 (xenial), the .deb package version
is '0.23-2' but elftools.__version__ contains '0.25'. This is certainly
due to partial backports.

To have a more consistent behaviour of this script across all versions
of python, add the unicode_literals future import so that literal
strings are now always "unicode".

Add 2 utility functions to force a string into bytes or bytes into an
unicode string.

Force pyelftools return values to unicode strings (will do nothing with
recent version of pyelftools).

If elffile.get_section_by_name returns None with a unicode section name,
try with the same one encoded as bytes.

Also, replace all open() calls by io.open() which behaves like the
builtin open in python 3. The only non-binary opened file is
/usr/share/hwdata/pci.ids which is UTF-8 encoded text. Explicitly
specify that encoding.

Link: https://github.com/eliben/pyelftools/blob/v0.24/CHANGES#L7
Link: https://github.com/eliben/pyelftools/commit/108eaea9e75a8b5a

Fixes: 54ca545dce4b ("make python scripts python2/3 compliant")
Cc: stable@dpdk.org

Signed-off-by: Robin Jarry <robin.jarry@6wind.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>

show more ...

usertools: add missing SPDX identifier

Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>

tools: move to usertools

Rename tools/ into usertools/ to differentiate from buildtools/
and devtools/ while making clear these scripts are part of
DPDK runtime.

Signed-off-by: Thomas Monjalon <tho

tools: move to usertools

Rename tools/ into usertools/ to differentiate from buildtools/
and devtools/ while making clear these scripts are part of
DPDK runtime.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>

show more ...