net/mlx5: fix shared Rx queue control release

Correct the reference counting and condition checking for shared Rx
queue control structures. This fix ensures proper memory management
during port stop

net/mlx5: fix shared Rx queue control release

Correct the reference counting and condition checking for shared Rx
queue control structures. This fix ensures proper memory management
during port stop and device close stages.

The changes move the control structure reference count decrease
outside the owners list empty condition, and adjust the reference
count check to subtract first, then evaluate.

This prevents potential crashes during port restart by
ensuring shared Rx queues' control structures are properly freed.

Fixes: 3c9a82fa6edc ("net/mlx5: fix Rx queue control management")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: fix Rx queue reference count in flushing flows

Some indirect table and hrxq is created in the rule creation with
QUEUE or RSS action. When stopping a port, the 'dev_started' is set
to 0 in

net/mlx5: fix Rx queue reference count in flushing flows

Some indirect table and hrxq is created in the rule creation with
QUEUE or RSS action. When stopping a port, the 'dev_started' is set
to 0 in the beginning. The mlx5_ind_table_obj_release() should still
do the dereference of the queue(s) when it is called in the polling
of flow rule deletion, due to the fact that a flow with Q/RSS action
is always referring to the active Rx queues.

The callback now can only pass one input parameter. Using a global
flag per device to indicate that the user flows flushing is in
progress. Then the reference count of the queue(s) should be
decreased.

Fixes: 3a2f674b6aa8 ("net/mlx5: add queue and RSS HW steering action")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: fix Rx queue control management

With the shared Rx queue feature introduced, the control and private
Rx queue structures are decoupled, each control structure can be
shared for multiple qu

net/mlx5: fix Rx queue control management

With the shared Rx queue feature introduced, the control and private
Rx queue structures are decoupled, each control structure can be
shared for multiple queue for all representors inside a domain.

So it should be only managed by the shared context instead of any
private data of each device. The previous workaround is using a flag
to check the owner (allocator) of the structure and handle it only
on that device closing stage.

A proper formal solution is to add a reference count for each control
structure and only free the structure when there is no reference to
it to get rid of the UAF issue.

Fixes: f957ac996435 ("net/mlx5: workaround list management of Rx queue control")
Fixes: bcc220cb57d7 ("net/mlx5: fix shared Rx queue list management")
CC: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: fix reported Rx/Tx descriptor limits

Currently, `rte_eth_dev_info.rx_desc_lim.nb_max` as well as
`rte_eth_dev_info.tx_desc_lim.nb_max` shows 65535 as the limit,
which results in a few prob

net/mlx5: fix reported Rx/Tx descriptor limits

Currently, `rte_eth_dev_info.rx_desc_lim.nb_max` as well as
`rte_eth_dev_info.tx_desc_lim.nb_max` shows 65535 as the limit,
which results in a few problems:

* It is not the actual Rx/Tx queue limit
* Allocating an Rx queue and passing `rx_desc_lim.nb_max` results in an
integer overflow and 0 ring size:

```
rte_eth_rx_queue_setup(0, 0, rx_desc_lim.nb_max, 0, NULL, mb_pool);
```

Which overflows ring size and generates the following log:
```
mlx5_net: port 0 increased number of descriptors in Rx queue 0 to the
next power of two (0)
```
The same holds for allocating a Tx queue.

Fixes: e60fbd5b24fc ("mlx5: add device configure/start/stop")
Cc: stable@dpdk.org

Signed-off-by: Igor Gutorov <igootorov@gmail.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: workaround list management of Rx queue control

The LIST_REMOVE macro only removes the entry from the list and
updates list itself. The pointers of this entry are not reset to
NULL to preve

net/mlx5: workaround list management of Rx queue control

The LIST_REMOVE macro only removes the entry from the list and
updates list itself. The pointers of this entry are not reset to
NULL to prevent the accessing for the 2nd time.

In the previous fix for the memory accessing, the "rxq_ctrl" was
removed from the list in a device private data when the "refcnt" was
decreased to 0. Under only shared or non-shared queues scenarios,
this was safe since all the "rxq_ctrl" entries were freed or kept.

There is one case that shared and non-shared Rx queues are configured
simultaneously, for example, a hairpin Rx queue cannot be shared.
When closing the port that allocated the shared Rx queues'
"rxq_ctrl", if the next entry is hairpin "rxq_ctrl", the hairpin
"rxq_ctrl" will be freed directly with other resources. When trying
to close the another port sharing the "rxq_ctrl", the LIST_REMOVE
will be called again and cause some UFA issue. If the memory is no
longer mapped, there will be a SIGSEGV.

Adding a flag in the Rx queue private structure to remove the
"rxq_ctrl" from the list only on the port/queue that allocated it.

Fixes: bcc220cb57d7 ("net/mlx5: fix shared Rx queue list management")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow match with external Tx queue

For using external created Tx queues in RTE_FLOW_ITEM_TX_QUEUE,
this commit provides the map and unmap functions to convert the
external created S

net/mlx5: support flow match with external Tx queue

For using external created Tx queues in RTE_FLOW_ITEM_TX_QUEUE,
this commit provides the map and unmap functions to convert the
external created SQ's devx ID to DPDK flow item Tx queue ID.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: rename external queue

Due to external Tx queue will be supported, in order to reuse
the external queue struct, rename the current external Rx
queue to external queue.

Signed-off-by: Suanm

net/mlx5: rename external queue

Due to external Tx queue will be supported, in order to reuse
the external queue struct, rename the current external Rx
queue to external queue.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

drivers: use stdatomic API

Replace the use of gcc builtin __atomic_xxx intrinsics with
corresponding rte_atomic_xxx optional rte stdatomic API.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microso

drivers: use stdatomic API

Replace the use of gcc builtin __atomic_xxx intrinsics with
corresponding rte_atomic_xxx optional rte stdatomic API.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>

show more ...

net/mlx5: remove non-constant size from array cast

Placing a non-constant size in the subscript [size] of a type cast is
causing unnecessary generation of a VLA. Remove size and just provide []
indi

net/mlx5: remove non-constant size from array cast

Placing a non-constant size in the subscript [size] of a type cast is
causing unnecessary generation of a VLA. Remove size and just provide []
indicating the type is an array of unspecified size.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

mempool: rework compile-time check for clang

Clang does not handle casts in static_assert() expressions.
It doesn't like use of floating point to calculate threshold.
Use a different expression with

mempool: rework compile-time check for clang

Clang does not handle casts in static_assert() expressions.
It doesn't like use of floating point to calculate threshold.
Use a different expression with same effect.

Modify comment in mlx5 so that developers don't go searching
for old value.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix MPRQ stride size check

We should only check that MPRQ stride size is bigger than the mbuf size
in case no devarg configuration has been provided. Headroom check was
indtroduced recentl

net/mlx5: fix MPRQ stride size check

We should only check that MPRQ stride size is bigger than the mbuf size
in case no devarg configuration has been provided. Headroom check was
indtroduced recently and removed this condition inadvertently.
Restore this condition and only check if mprq_log_stride_size is not set.

Fixes: e6479f009fbd ("net/mlx5: fix MPRQ stride size for headroom")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix shared Rx queue list management

In shared Rx queue case, the shared control structure could only be
released after the last port's dereference in the group.

There is another managemen

net/mlx5: fix shared Rx queue list management

In shared Rx queue case, the shared control structure could only be
released after the last port's dereference in the group.

There is another management list that holding all of the used Rx
queues' structures for a port. If the reference count of a control
structure is changed to zero during port close, it can be removed
from the list directly without freeing the resource.

Fixes: 09c2555303be ("net/mlx5: support shared Rx queue")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: add global API prefix to public constants

The file rte_pmd_mlx5.h is a public API,
so its components must be prefixed with RTE_PMD_.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

net/mlx5: fix MPRQ stride size for headroom

The space for the headroom is reserved at the end of every MPRQ stride
for the next packet. The Rx burst logic is to copy any overlapping
packet data if t

net/mlx5: fix MPRQ stride size for headroom

The space for the headroom is reserved at the end of every MPRQ stride
for the next packet. The Rx burst logic is to copy any overlapping
packet data if there is an overlap with this reserved headroom space.
But it is not possible if the headroom size is bigger than the whole
stride. Adjust the stride size to make sure the stride size is greater
than the headroom size.

Fixes: 34776af600df ("net/mlx5: fix MPRQ stride devargs adjustment")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support symmetric RSS hash function

This patch supports symmetric hash function that creating same
hash result for bi-direction traffic which having reverse
source and destination IP and L

net/mlx5: support symmetric RSS hash function

This patch supports symmetric hash function that creating same
hash result for bi-direction traffic which having reverse
source and destination IP and L4 port.

Since the hash algorithm is different than spec(XOR), leave a
warning in validation.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: forbid MPRQ restart

The queue restart is only supported by the non-vectorized
single-packet receive queue today. Restarting MPRQ will
result in corrupted packets because of CQE and WQE mis

net/mlx5: forbid MPRQ restart

The queue restart is only supported by the non-vectorized
single-packet receive queue today. Restarting MPRQ will
result in corrupted packets because of CQE and WQE mismatch.
Prevent this by not allowing the MPRQ Rx queue stop.

Fixes: 161d103b231c ("net/mlx5: add queue start and stop")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

rework atomic intrinsics fetch operations

Use __atomic_fetch_{add,and,or,sub,xor} instead of
__atomic_{add,and,or,sub,xor}_fetch adding the necessary code to
allow consumption of the resulting value

rework atomic intrinsics fetch operations

Use __atomic_fetch_{add,and,or,sub,xor} instead of
__atomic_{add,and,or,sub,xor}_fetch adding the necessary code to
allow consumption of the resulting value.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

net/mlx5: support enhanced CQE compression in Rx burst

Enhanced CQE compression changes the structure of the compression block
and the number of miniCQEs per miniCQE array. Adapt to these changes in

net/mlx5: support enhanced CQE compression in Rx burst

Enhanced CQE compression changes the structure of the compression block
and the number of miniCQEs per miniCQE array. Adapt to these changes in
the datapath by defining a new parsing mechanism of a miniCQE array:
1. The title CQE is no longer marked as the compressed one.
Need to copy it for the future miniCQE arrays parsing.
2. Mini CQE arrays now consist of up to 7 miniCQEs and a control block.
The control block contains the number of miniCQEs in the array
as well as an indication that this CQE is compressed.
3. The invalidation of reserved CQEs between miniCQEs arrays is not needed.
4. The owner_bit is replaced the validity_iteration_count for all CQEs.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix maximum LRO message size

The PMD analyzes each Rx queue maximal LRO size and selects one that
fits all queues to configure TIR LRO attribute.
TIR LRO attribute is number of 256 bytes c

net/mlx5: fix maximum LRO message size

The PMD analyzes each Rx queue maximal LRO size and selects one that
fits all queues to configure TIR LRO attribute.
TIR LRO attribute is number of 256 bytes chunks that match the
selected maximal LRO size.

PMD used `priv->max_lro_msg_size` for selected maximal LRO size and
number of TIR chunks.

Fixes: b9f1f4c239 ("net/mlx5: fix port initialization with small LRO")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix port initialization with small LRO

If application provided maximal LRO size was less than expected PMD
minimum, the PMD either crashed with assert, if asserts were enabled,
or proceede

net/mlx5: fix port initialization with small LRO

If application provided maximal LRO size was less than expected PMD
minimum, the PMD either crashed with assert, if asserts were enabled,
or proceeded with port initialization to set port private maximal
LRO size below supported minimum.

The patch terminates port start if LRO size
does not match PMD requirements and TCP LRO offload was requested
at least for one Rx queue.

Fixes: 50c00baff763 ("net/mlx5: limit LRO size to maximum Rx packet")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix shared Rx queue config reuse

There is a check for the configuration match between
all the Rx queues shared among multiple ports in DPDK.
This check ensures that the configuration is th

net/mlx5: fix shared Rx queue config reuse

There is a check for the configuration match between
all the Rx queues shared among multiple ports in DPDK.
This check ensures that the configuration is the same.

The issue is this check takes place before the queue
is released and configured again in case of reconfiguration.
That leads to checking against the old configuration and
preventing the shared Rx queue to start properly.

Release the old configuration and prepare a new Rx queue
before checking that its parameters match the config.

Fixes: 09c2555303be ("net/mlx5: support shared Rx queue")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: create control flow rules with HWS

This patch adds the creation of control flow rules required to receive
default traffic (based on port configuration) with HWS.

Control flow rules are cr

net/mlx5: create control flow rules with HWS

This patch adds the creation of control flow rules required to receive
default traffic (based on port configuration) with HWS.

Control flow rules are created on port start and destroyed on port stop.
Handling of destroying these rules was already implemented before that
patch.

Control flow rules are created if and only if flow isolation mode is
disabled and the creation process goes as follows:

- Port configuration is collected into a set of flags. Each flag
corresponds to a certain Ethernet pattern type, defined by
mlx5_flow_ctrl_rx_eth_pattern_type enumeration. There is a separate
flag for VLAN filtering.

- For each possible Ethernet pattern type and:
- For each possible RSS action configuration:
- If configuration flags do not match this combination, it is
omitted.
- A template table is created using this combination of pattern
and actions template (templates are fetched from hw_ctrl_rx
struct stored in the port's private data).
- Flow rules are created in this table.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix LRO requirements check

One of the conditions to allow LRO offload is the DV configuration.

The function incorrectly checks the DV configuration before initializing
it by the user deva

net/mlx5: fix LRO requirements check

One of the conditions to allow LRO offload is the DV configuration.

The function incorrectly checks the DV configuration before initializing
it by the user devarg; hence, LRO cannot be allowed.

This patch moves this check to mlx5_shared_dev_ctx_args_config, where DV
configuration is initialized.

Fixes: c4b862013598 ("net/mlx5: refactor to detect operation by DevX")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reported-by: Gal Shalom <galshalom@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix stack buffer overflow in drop action

The mlx5_drop_action_create function use mlx5_malloc for allocating
'hrxq', but don't allocate for 'rss_key'. This is wrong and it can
cause buffer

net/mlx5: fix stack buffer overflow in drop action

The mlx5_drop_action_create function use mlx5_malloc for allocating
'hrxq', but don't allocate for 'rss_key'. This is wrong and it can
cause buffer overflow.

Detected with address sanitizer:
0 (/usr/lib64/libasan.so.4+0x7b8e2)
1 in mlx5_devx_tir_attr_set ../drivers/net/mlx5/mlx5_devx.c:765
2 in mlx5_devx_hrxq_new ../drivers/net/mlx5/mlx5_devx.c:800
3 in mlx5_devx_drop_action_create ../drivers/net/mlx5/mlx5_devx.c:1051
4 in mlx5_drop_action_create ../drivers/net/mlx5/mlx5_rxq.c:2846
5 in mlx5_dev_spawn ../drivers/net/mlx5/linux/mlx5_os.c:1743
6 in mlx5_os_pci_probe_pf ../drivers/net/mlx5/linux/mlx5_os.c:2501
7 in mlx5_os_pci_probe ../drivers/net/mlx5/linux/mlx5_os.c:2647
8 in mlx5_os_net_probe ../drivers/net/mlx5/linux/mlx5_os.c:2722
9 in drivers_probe ../drivers/common/mlx5/mlx5_common.c:657
10 in mlx5_common_dev_probe ../drivers/common/mlx5/mlx5_common.c:711
11 in mlx5_common_pci_probe ../drivers/common/mlx5/mlx5_common_pci.c:150
12 in rte_pci_probe_one_driver ../drivers/bus/pci/pci_common.c:269
13 in pci_probe_all_drivers ../drivers/bus/pci/pci_common.c:353
14 in pci_probe ../drivers/bus/pci/pci_common.c:380
15 in rte_bus_probe ../lib/eal/common/eal_common_bus.c:72
16 in rte_eal_init ../lib/eal/linux/eal.c:1286
17 in main ../app/test-pmd/testpmd.c:4112

Fixes: 0c762e81da9b ("net/mlx5: share Rx queue drop action code")
Cc: stable@dpdk.org

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix LRO validation in Rx setup

The mlx5_rx_queue_setup() get LRO offload from user.

When LRO is configured, the LRO flag in rxq_data is set to 1.

This patch adds validation to make sure

net/mlx5: fix LRO validation in Rx setup

The mlx5_rx_queue_setup() get LRO offload from user.

When LRO is configured, the LRO flag in rxq_data is set to 1.

This patch adds validation to make sure the LRO is supported.

Fixes: 17ed314 ("net/mlx5: allow LRO per Rx queue")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...