common/mlx5: fix error CQE handling for 128 bytes CQE

The completion queue element size can be independently configured
to report either 64 or 128 bytes CQEs by programming cqe_sz parameter
at CQ cr

common/mlx5: fix error CQE handling for 128 bytes CQE

The completion queue element size can be independently configured
to report either 64 or 128 bytes CQEs by programming cqe_sz parameter
at CQ creation. This parameter depends on the cache line size and
affects both regular CQEs and error CQEs. But the error handling
assumes that an error CQE is 64 bytes and doesn't take the padding
into consideration on platforms with 128-byte cache lines.
Fix the error CQE size in all error handling routines in mlx5.

Fixes: 957e45fb7bcb ("net/mlx5: handle Tx completion with error")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

compress/mlx5: add VF device ID

This adds the Virtual Function device ID to the list of
supported NVIDIA devices that run the MLX5 compress PMD.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
A

compress/mlx5: add VF device ID

This adds the Virtual Function device ID to the list of
supported NVIDIA devices that run the MLX5 compress PMD.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: support LZ4 algorithm

Add support for decompress LZ4 algorithm for mlx5 PMD.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

compress/mlx5: add xform validate function

Separate the xform validations from create function into new function.
The new function checks caps directly from HCA attr structure, so all
capabilities i

compress/mlx5: add xform validate function

Separate the xform validations from create function into new function.
The new function checks caps directly from HCA attr structure, so all
capabilities in priv structure were removed.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: remove unused field

The priv structure has variable named "min_block_size" coming from HCA
capabilities.

This field isn't used and copied into the priv structure for free.

This patc

compress/mlx5: remove unused field

The priv structure has variable named "min_block_size" coming from HCA
capabilities.

This field isn't used and copied into the priv structure for free.

This patch removes this field.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: support metadata layout added in BlueField-3

Commit [1] add support in BlueField-3, but BlueField-3 has different
GGA opaque structure than BlueField-2.

This patch updates the PRM st

compress/mlx5: support metadata layout added in BlueField-3

Commit [1] add support in BlueField-3, but BlueField-3 has different
GGA opaque structure than BlueField-2.

This patch updates the PRM structure to include both versions, and
calculate the relevant offset for each version in control path.

[1] Commit 559014f232b4 ("compress/mlx5: add Bluefield-3 device ID")

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: fix queue setup for partial transformations

The mlx5_compress_qp_setup() function creates QP for compress,
decompress and DMA. Thus, the MMO flag is turned on only when all
operations

compress/mlx5: fix queue setup for partial transformations

The mlx5_compress_qp_setup() function creates QP for compress,
decompress and DMA. Thus, the MMO flag is turned on only when all
operations are supported.

However, since partial transformations have been allowed, it should be
turn on for part of them.

This patch removes the compress MMO support requirement.

Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: fix output Adler-32 checksum offset

After de/compress dequeue, the output checksum is copied into the op
structure. The "output_checksum" field in op structure is "uint64_t"
type, and

compress/mlx5: fix output Adler-32 checksum offset

After de/compress dequeue, the output checksum is copied into the op
structure. The "output_checksum" field in op structure is "uint64_t"
type, and the 32-bit checksums (CRC32, Adler-32) are copied into the
lower 32 bits.

When both CRC32 and Adler-32 are configured, CRC32 is copied into the
lower 32 bits and Adler-32 into the upper 32 bits.
However, in mlx5 PMD Adler-32 without CRC, is mistakenly copied into the
upper 32 bits.

This patch updates Adler-32 output checksum to be copied into the
lower 32 bits.

Fixes: f8c97babc9f4 ("compress/mlx5: add data-path functions")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: fix decompress xform validation

In xform creation, it first validate the xform according the
capabilities.

One of validations verifies that given "hash_algo" is
"RTE_COMP_HASH_ALGO_N

compress/mlx5: fix decompress xform validation

In xform creation, it first validate the xform according the
capabilities.

One of validations verifies that given "hash_algo" is
"RTE_COMP_HASH_ALGO_NONE" for both compress and decompress xform
objects.
However, the validation for decompress checks it again for compress
xform object.

This patch changes it to verify decompress xform object.

Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

drivers: rename NVIDIA BlueField device IDs

This updates the PCI device names for all NVIDIA BlueField
family.

Signed-off-by: Raslan Darawsheh <rasland@nvidia.com>

compressdev: fix end of driver list

The "rte_compressdev_info_get()" function retrieves the contextual
information of a device.
The output structure "dev_info" contains a list of devices supported
c

compressdev: fix end of driver list

The "rte_compressdev_info_get()" function retrieves the contextual
information of a device.
The output structure "dev_info" contains a list of devices supported
capabilities for each supported algorithm.

In this function description, it says the element after the last valid
element has op field set to "RTE_COMP_ALGO_LIST_END".
On the other hand, when this function used by
"rte_compressdev_capability_get()" function, it uses
"RTE_COMP_ALGO_UNSPECIFIED" as end of list as same as the
"RTE_COMP_END_OF_CAPABILITIES_LIST()".

The mlx5 and qat PMDs use "RTE_COMP_ALGO_LIST_END" as the end of
capabilities list. When "rte_compressdev_capability_get()" function is
called with unsupported algorithm, it might read memory out of bound.

This patch change the "rte_compressdev_info_get()" function description
to say using "RTE_COMP_ALGO_UNSPECIFIED" as the end of capabilities
list.
In addition, it moves both mlx5 and qat PMDs to use
"RTE_COMP_ALGO_UNSPECIFIED" through
"RTE_COMP_END_OF_CAPABILITIES_LIST()" macro.

Fixes: 5d432f364078 ("compressdev: add device capabilities")
Fixes: 2d148597ce76 ("compress/qat: add gen-specific implementation")
Fixes: 384bac8d6555 ("compress/mlx5: add supported capabilities")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

compress/mlx5: add Bluefield-3 device ID

This adds the Bluefield-3 device ID to the list of
supported NVIDIA devices that run the MLX5 compress PMD.
The device is still in development stage.

Signed

compress/mlx5: add Bluefield-3 device ID

This adds the Bluefield-3 device ID to the list of
supported NVIDIA devices that run the MLX5 compress PMD.
The device is still in development stage.

Signed-off-by: Raslan Darawsheh <rasland@nvidia.com>

show more ...

bus/pci: make driver-only headers private

The pci bus interface is for drivers only.
Mark as internal and move the header in the driver headers list.

While at it, cleanup the code:
- fix indentatio

bus/pci: make driver-only headers private

The pci bus interface is for drivers only.
Mark as internal and move the header in the driver headers list.

While at it, cleanup the code:
- fix indentation,
- remove unneeded reference to bus specific singleton object,
- remove unneeded list head structure type,
- reorder the definitions and macro manipulating the bus singleton object,
- remove inclusion of rte_bus.h and fix the code that relied on implicit
inclusion,

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>

show more ...

compress/mlx5: support out-of-space status

When trying to dequeue, an OP may fail due to insufficient
space for the OP output, the compressdev API defines out-of-space
for OP status. The driver can

compress/mlx5: support out-of-space status

When trying to dequeue, an OP may fail due to insufficient
space for the OP output, the compressdev API defines out-of-space
for OP status. The driver can detect out-of-space errors and
report them to the user. Check if hw_error_syndrome specifies
out-of-space and set the OP status accordingly.
Also added an error message for a case of missing B-final flag.

Fixes: f8c97babc9f4 ("compress/mlx5: add data-path functions")
Cc: stable@dpdk.org

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: refactor devargs management

Improve the devargs handling in two aspects:
- Parse the devargs string only once.
- Return error and report for unknown keys.

The common driver parses on

common/mlx5: refactor devargs management

Improve the devargs handling in two aspects:
- Parse the devargs string only once.
- Return error and report for unknown keys.

The common driver parses once the devargs string into a dictionary, then
provides it to all the drivers' probe. Each driver updates within it
which keys it has used, then common driver receives the updated
dictionary and reports about unknown devargs.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free functions
found by nullfree.cocci

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

compress/mlx5: fix double close of device context

The context of the device opens once in the common probe and closes with
its removal.

If the probe of one of the drivers fails, it releases its res

compress/mlx5: fix double close of device context

The context of the device opens once in the common probe and closes with
its removal.

If the probe of one of the drivers fails, it releases its resources and
then the common closes the context.
But mistakenly in the compress probe, if there isn't enough capabilities
to support compress operations, it closes the device and then common
probe closes it again.

Remove the redundant closing from compress probe.

Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: fix redundant field in MR control structure

Inside the MR control structure there is a pointer to the common device.
This pointer enables access to the global cache as well as hardware

common/mlx5: fix redundant field in MR control structure

Inside the MR control structure there is a pointer to the common device.
This pointer enables access to the global cache as well as hardware
objects that may be required in case a new MR needs to be created.

The purpose of adding this pointer into the MR control structure was to
avoid its transfer as a parameter to all the functions of searching MR
in the caches.
However, adding it to this structure increased the Rx and Tx data-path
structures, all the fields that followed it were slightly moved away
which caused to a reduction in performance.

This patch removes the pointer from the structure. It can be accessed
through the "dev_gen_ptr" existing field using the "container_of"
operator.

Fixes: 334ed198ab4d ("common/mlx5: remove redundant parameter in MR search")

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: fix queue size in DevX queue pair creation

The number of WQEBBs was provided to QP create, and QP size was calculated
by multiplying the number of WQEBBs by 64, which is the send WQE si

common/mlx5: fix queue size in DevX queue pair creation

The number of WQEBBs was provided to QP create, and QP size was calculated
by multiplying the number of WQEBBs by 64, which is the send WQE size.
When creating RQ in the QP (i.e., vdpa driver), the queue size was bigger
because the receive WQE size is 16.
Provide queue size to QP create instead of the number of WQEBBs.

Fixes: f9213ab12cf9 ("common/mlx5: share DevX queue pair operations")

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

crypto/mlx5: fix queue size configuration

The DevX interface for QP creation expects the number of WQEBBs.
Wrongly, the number of descriptors was provided to the QP creation.
In addition, the QP siz

crypto/mlx5: fix queue size configuration

The DevX interface for QP creation expects the number of WQEBBs.
Wrongly, the number of descriptors was provided to the QP creation.
In addition, the QP size must be a power of 2 what was not guaranteed.
Provide the number of WQEBBs to the QP creation API.
Round up the SQ size to a power of 2.
Rename (sq/rq)_size to num_of_(send/receive)_wqes.

Fixes: 6152534e211e ("crypto/mlx5: support queue pairs operations")
Cc: stable@dpdk.org

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>

show more ...

common/mlx5: fix post doorbell barrier

The rdma-core library can map doorbell register in two ways, depending
on the environment variable "MLX5_SHUT_UP_BF":

- as regular cached memory, the variab

common/mlx5: fix post doorbell barrier

The rdma-core library can map doorbell register in two ways, depending
on the environment variable "MLX5_SHUT_UP_BF":

- as regular cached memory, the variable is either missing or set to
zero. This type of mapping may cause the significant doorbell
register writing latency and requires an explicit memory write
barrier to mitigate this issue and prevent write combining.

- as non-cached memory, the variable is present and set to not "0"
value. This type of mapping may cause performance impact under
heavy loading conditions but the explicit write memory barrier is
not required and it may improve core performance.

The UAR creation function maps a doorbell in one of the above ways
according to the system. In run time, it always adds an explicit memory
barrier after writing to.
In cases where the doorbell was mapped as non-cached memory, the
explicit memory barrier is unnecessary and may impair performance.

The commit [1] solved this problem for a Tx queue. In run time, it
checks the mapping type and provides the memory barrier after writing to
a Tx doorbell register if it is needed. The mapping type is extracted
directly from the uar_mmap_offset field in the queue properties.

This patch shares this code between the drivers and extends the above
solution for each of them.

[1] commit 8409a28573d3
("net/mlx5: control transmit doorbell register mapping")

Fixes: f8c97babc9f4 ("compress/mlx5: add data-path functions")
Fixes: 8e196c08ab53 ("crypto/mlx5: support enqueue/dequeue operations")
Fixes: 4d4e245ad637 ("regex/mlx5: support enqueue")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: fix doorbell mapping configuration

UAR mapping type can be affected by the devarg tx_db_nc, which can cause
setting the environment variable MLX5_SHUT_UP_BF.
So, the MLX5_SHUT_UP_BF val

common/mlx5: fix doorbell mapping configuration

UAR mapping type can be affected by the devarg tx_db_nc, which can cause
setting the environment variable MLX5_SHUT_UP_BF.
So, the MLX5_SHUT_UP_BF value and the UAR mapping parameter affect the
UAR cache mode.

Wrongly, the devarg was considered for the MLX5_SHUT_UP_BF but not for
the UAR mapping parameter in all the drivers except the net.

Take the tx_db_nc devarg into account for all the drivers.

Fixes: ca1418ce3910 ("common/mlx5: share device context object")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: make multi-process MR management port-agnostic

In the multi-process mechanism, there are things that the secondary
process does not perform itself but asks the primary process to perfor

common/mlx5: make multi-process MR management port-agnostic

In the multi-process mechanism, there are things that the secondary
process does not perform itself but asks the primary process to perform
for it.
There is a special API for communication between the processes that
receives parameters necessary for the specific action required as well
as a special structure called mp_id that contains the port number of the
processes through which the initial process finds the relevant ETH
device for the processes.

One of the operations performed through this mechanism is the creation
of a memory region, where the secondary process sends the virtual
address as a parameter and the mp_id structure with the port number
inside it.
However, once the memory area management is shared between the drivers
and either port number or ETH device is no longer relevant to them, it
seems unnecessary to continue communicating between the processes
through the mp_id variable.

In this patch we will remove the use of the above structure for all MR
management, and add to the specific parameter of operations a pointer to
the common device that contains everything needed to create/register MR.

Fixes: 9f1d636f3ef08 ("common/mlx5: share MR management")

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Reviewed-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: remove redundant parameter in MR search

Memory region management has recently been shared between drivers,
including the search for caches in the data plane.
The initial search in the l

common/mlx5: remove redundant parameter in MR search

Memory region management has recently been shared between drivers,
including the search for caches in the data plane.
The initial search in the local linear cache of the queue, usually
yields a result and one should not continue searching in the next level
caches.

The function that searches in the local cache gets the pointer to a
device as a parameter, that is not necessary for its operation
but for subsequent searches (which, as mentioned, usually do not
happen).
Transferring the device to a function and maintaining it, takes some
time and causes some impact on performance.

Add the pointer to the device as a field of the mr_ctrl structure. The
field will be updated during control path and will be used only when
needed in the search.

Fixes: fc59a1ec556b ("common/mlx5: share MR mempool registration")

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Reviewed-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

compress/mlx5: add block size option

Currently, the compression block size is 15 by default, which
is the maximum.

Add "log-block-size" devarg to select compression block size manually.
The value p

compress/mlx5: add block size option

Currently, the compression block size is 15 by default, which
is the maximum.

Add "log-block-size" devarg to select compression block size manually.
The value provided should be between 4 to 15.
Any out-of-range value will be defaulted to 15.

Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...