net/mlx5: fix unneeded stub flow table allocation

The HWS non-template flow API is reusing some implementation of
template API to unify code logic. So for each rule creation, a stub
/ temporary tabl

net/mlx5: fix unneeded stub flow table allocation

The HWS non-template flow API is reusing some implementation of
template API to unify code logic. So for each rule creation, a stub
/ temporary table is used in order to reuse the actions construction.

Since this is temporary and used only internally, there is no need to
save the table permanently. Only parts of them are mandatory, so the
allocation / free from the heap of RTE memory is a waste and causes
a lot of overhead. By using the pre-allocated workspace and set the
needed fields expliticly will save the overhead and help to speed up
the rule insertion rate.

Fixes: 27d171b88031 ("net/mlx5: abstract flow action and enable reconfigure")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

drivers: replace packed attributes

MSVC struct packing is not compatible with GCC. Replace macro
__rte_packed with __rte_packed_begin to push existing pack value
and set packing to 1-byte and macro

drivers: replace packed attributes

MSVC struct packing is not compatible with GCC. Replace macro
__rte_packed with __rte_packed_begin to push existing pack value
and set packing to 1-byte and macro __rte_packed_end to restore
the pack value prior to the push.

Macro __rte_packed_end is deliberately utilized to trigger a
MSVC compiler warning if no existing packing has been pushed allowing
easy identification of locations where the __rte_packed_begin is
missing.

Signed-off-by: Andre Muezerie <andremue@linux.microsoft.com>
Reviewed-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

show more ...

net/mlx5: fix default RSS flows creation order

In both SWS and HWS mode, default ingress RSS flows are always
created via the driver on the root table. In the current driver,
the first created flow

net/mlx5: fix default RSS flows creation order

In both SWS and HWS mode, default ingress RSS flows are always
created via the driver on the root table. In the current driver,
the first created flow rules will be matched firstly when:
1. >= 2 rules can be matched on the root table.
2. the rules have the same priority.

All MC / BC flow rules would have the same priority and discard
the input priority from the user space in the driver. All rules have
a fixed priority 32 when the Ethernet destination MAC is a MC or BC
address.

In SWS non-template API, all the device rules are added into the list
and applied in a reverse order.

This patch syncs default flow rule creation order between SWS and HWS.
The order should be:
1. IPv4(6) + TCP/UDP, if required.
2. IPv4(6) only, if required.
3. None IP traffic.

Fixes: 9fa7c1cddb85 ("net/mlx5: create control flow rules with HWS")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: fix dangling pointer to flow params

Offending commits introduced 2 structures which held temporary
parameters for flow rule created. These parameters were passed
to "rule_acts" buffers sto

net/mlx5: fix dangling pointer to flow params

Offending commits introduced 2 structures which held temporary
parameters for flow rule created. These parameters were passed
to "rule_acts" buffers stored inside template table and
they were used only during a call to mlx5dr_rule_create().
After mlx5dr_rule_create() finished, "rule_acts" buffer is not cleared,
for performance reasons, since on the next flow rule create
it will be overwritten.

GCC 14.2.1, when compiled with -03, issued the following warning:

```
drivers/net/mlx5/mlx5_flow_hw.c:3520:51:
error: storing the address of local variable
'ap' in '*<unknown>.<U2688>.modify_header.data'
[-Werror=dangling-pointer=]
3520 | rule_acts[pos].modify_header.data =
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
3521 | (uint8_t *)ap->mhdr_cmd;
| ~~~~~~~~~~~~~~~~~~~~~~~
drivers/net/mlx5/mlx5_flow_hw.c: In function 'flow_hw_async_flow_create':
drivers/net/mlx5/mlx5_flow_hw.c:3925:43: note: 'ap' declared here
3925 | struct mlx5_flow_hw_action_params ap;
| ^~
drivers/net/mlx5/mlx5_flow_hw.c:3910:59: note: 'table' declared here
3910 | struct rte_flow_template_table *table,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
```

Which is technically correct. This warning could be fixed by
casting "ap->mhdr_cmd" to "void *", but this is just a workaround.

Since what mlx5 PMD cares about is that there is only one instance
of mlx5_flow_hw_action_params per queue, this patch moves this struct
from the stack to mlx5_hw_q struct.
The same is done for mlx5_flow_hw_pattern_params for similar reasons.

Fixes: 1d2744f5028d ("net/mlx5: remove flow action parameters from job")
Fixes: 57fd15fa373d ("net/mlx5: remove flow pattern from job")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5/hws: fix allocation of STCs

STC is a limited resource of the HW, and might get consumed till no more
contexts can be opened.
So, let the user to define the size of how many STCs to allocate

net/mlx5/hws: fix allocation of STCs

STC is a limited resource of the HW, and might get consumed till no more
contexts can be opened.
So, let the user to define the size of how many STCs to allocate per
context.
In case the user has many representors, no need to allocate per each of
them the default value of STCs, otherwise after a certain numbers of
representors no more STC's will remain in the system.

Fixes: b0290e56dd08 ("net/mlx5/hws: add context object")
Cc: stable@dpdk.org

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: support jump to flow table index action

Implement RTE_FLOW_ACTION_TYPE_JUMP_TO_TABLE_INDEX action.
Create the hardware steering jump to matcher action,
associated with the template matcher

net/mlx5: support jump to flow table index action

Implement RTE_FLOW_ACTION_TYPE_JUMP_TO_TABLE_INDEX action.
Create the hardware steering jump to matcher action,
associated with the template matcher. Use this action and
provide the rule index as an offset in the matcher.
Note that it is only supported by the isolated matcher,
i.e. the table insertion type is by index with pattern.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: fix SQ flow item size

Expand the size of struct mlx5_rte_flow_item_sq to 64 bits on 64-bit
systems. This aligns with DPDK's assumption that PMD private data has
pointer size when copying f

net/mlx5: fix SQ flow item size

Expand the size of struct mlx5_rte_flow_item_sq to 64 bits on 64-bit
systems. This aligns with DPDK's assumption that PMD private data has
pointer size when copying flow items with rte_flow_conv.[1]

Previously, the struct was defined as 32 bits, causing DPDK to
incorrectly assign an additional 32 bits when copying
MLX5_RTE_FLOW_ITEM_TYPE_SQ items on 64-bit systems.

This fix ensures proper memory alignment and prevents potential
buffer overflows when DPDK copies MLX5_RTE_FLOW_ITEM_TYPE_SQ items.

[1]:
commit 6cf72047332b ("ethdev: support flow elements with variable length")

Fixes: 75a00812b18f ("net/mlx5: add hardware steering item translation")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: increase number of supported DV sub-flows to 64

Increase the maximum number of supported DV sub-flows from 32 to 64.
This allows more complex flow rules to be created, especially when
usin

net/mlx5: increase number of supported DV sub-flows to 64

Increase the maximum number of supported DV sub-flows from 32 to 64.
This allows more complex flow rules to be created, especially when
using features like sample actions with RSS.

For example, the following testpmd command now works, which previously
exceeded the sub-flow limit:

dpdk-testpmd -a PCI,dv_xmeta_en=1,l3_vxlan_en=1,dv_flow_en=1 -- \
-i --nb-cores=4 --rxq=5 --txq=5

set sample_actions 1 mark id 43704 / \
rss queues 3 0 1 1 end types ipv4 ipv4-other udp tcp ipv4-udp end / \
end

flow create 0 priority 15 group 271 ingress \
pattern mark id spec 16777184 id mask 0xffffff / end \
actions sample ratio 1 index 1 / queue index 0 / end

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: add legacy unicast flow rules management

This patch adds the following internal functions for creation of
unicast DMAC flow rules:

- mlx5_legacy_dmac_flow_create() - simple wrapper over

net/mlx5: add legacy unicast flow rules management

This patch adds the following internal functions for creation of
unicast DMAC flow rules:

- mlx5_legacy_dmac_flow_create() - simple wrapper over
mlx5_ctrl_flow().
- mlx5_legacy_dmac_vlan_flow_create() - simple wrapper over
mlx5_ctrl_flow_vlan().

These will be used as a basis for implementing dynamic
additions of unicast DMAC or unicast DMAC with VLAN
control flow rules when new addresses/VLANs are added.

Also, this path adds the following internal functions
for destructions of unicast DMAC flow rules:

- mlx5_legacy_ctrl_flow_destroy() - assuming a flow rule is on the
control flow rule list, destroy it.
- mlx5_legacy_dmac_flow_destroy() - find and destroy a flow rule
with given unicast DMAC.
- mlx5_legacy_dmac_flow_destroy() - find and destroy a flow rule
with given unicast DMAC and VLAN ID.

These will be used as a basis for implementing dynamic
removals of unicast DMAC or unicast DMAC with VLAN
control flow rules when addresses/VLANs are removed.

At the moment, no relevant flow rules are registered on the list
when working with Verbs or DV flow engine.
This will be added in the follow up commit.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support destroying unicast flow rules

This patch adds support for destroying:

- unicast DMAC control flow rules and
- unicast DMAC with VLAN control flow rules,

without affecting any oth

net/mlx5: support destroying unicast flow rules

This patch adds support for destroying:

- unicast DMAC control flow rules and
- unicast DMAC with VLAN control flow rules,

without affecting any other control flow rules,
when HWS flow engine is used.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: rework creation of unicast flow rules

Rework the code responsible for creation of unicast control flow rules,
to allow creation of:

- unicast DMAC flow rules and
- unicast DMAC with VMAN

net/mlx5: rework creation of unicast flow rules

Rework the code responsible for creation of unicast control flow rules,
to allow creation of:

- unicast DMAC flow rules and
- unicast DMAC with VMAN flow rules,

outside of mlx5_traffic_enable() called when port is started.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: add metadata split for compatibility

The method will not create any new flow rule implicitly during split
stage, but only generate needed items, actions and attributes based
on the detecti

net/mlx5: add metadata split for compatibility

The method will not create any new flow rule implicitly during split
stage, but only generate needed items, actions and attributes based
on the detection.

All the rules will still be created in the flow_hw_list_create().

In the meanwhile, once the mark action is specified in the FDB rule,
a new rule in the NIC Rx will be created to:
1. match the mark value on REG_C_x in FDB and set it into Rx
flow tag field.
2. copy the metadata in REG_C_x' into the REG_B.

If there is no mark, the default rule with only copying metadata
will be hit if there is Queue or RSS action in the NIC Rx rule.

Regarding the NIC Tx, only the metadata is relevant and it will be
copied in NIC Tx from REG_A into some REG_C_x. The current HWS
implementation already has already supported in the default copy
rule or the default SQ miss rule in the NIC Tx root table.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: increase flow pattern template maximum

Until now the number of pattern templates that was
supported per table was limited to 2.
This was the result of the limitation that the table
could o

net/mlx5: increase flow pattern template maximum

Until now the number of pattern templates that was
supported per table was limited to 2.
This was the result of the limitation that the table
could only support 1 matcher.
which meant that we could only support merge of
Ipv4 + TCP and IPv4 + UDP.

With the added ability to use extended hash it is now
possible to use more than 2 pattern templates in a
single table.

Extended match works by creating the hash of the rule
based on the intersection of all pattern templates.
As a result this is good for tables with small number
of rules or that the intersection is very large. for
example ACL table.

Using this feature is not recommended to for use
in tables with large number of rules or with small
intersection. Using this feature on such cases may
result in PPS degradation and rule insertion failures.

This patch increase the max number to 32.

Signed-off-by: Ori Kam <orika@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: store modify field flow action

When template table is created, list of unmasked actions is recorded for
future flow rule insertions.
This patch expands entries for RTE_FLOW_ACTION_TYPE_MOD

net/mlx5: store modify field flow action

When template table is created, list of unmasked actions is recorded for
future flow rule insertions.
This patch expands entries for RTE_FLOW_ACTION_TYPE_MODIFY_FIELD actions
in this list with a copy of the action from the template.
This will be used in follow up commits which add flow rule operation
validation. Specifically, to validate that
RTE_FLOW_ACTION_TYPE_MODIFY_FIELD action passed by the user is correctly
configured.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5: store expected type on flow indirect action

When template table is created, list of unmasked actions is recorded for
future flow rule insertions.
This patch expands entries for RTE_FLOW_AC

net/mlx5: store expected type on flow indirect action

When template table is created, list of unmasked actions is recorded for
future flow rule insertions.
This patch expands entries for RTE_FLOW_ACTION_TYPE_INDIRECT actions in
this list with information about expected indirect action type.
This will be used in follow up commits which add flow rule operation
validation. Specifically, this will be used to verify if indirect action
provided by the user references an action of a correct type.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5: store original actions in flow template

When actions template is created in mlx5 PMD, some actions will be
replaced with another or some actions will be added to implement
required templat

net/mlx5: store original actions in flow template

When actions template is created in mlx5 PMD, some actions will be
replaced with another or some actions will be added to implement
required template semantics on NVIDIA NICs. For example:

- RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID actions is replaced with
modify field action.
- Modify field action is added for preserving metadata
between FDB and NIC domains.
- Modify field action is added when quota action is used to amend
ASO syndrome.

Types of actions and their order in flow rules based on some
actions template must match the original, not modified one.

This patch adds preserving of the original actions for actions template
to allow for easier validation of ordering and types on fast path.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5: store flow pattern template items

Store items which define the pattern template on creation.
This allows validation of items, provided by the user during flow rule
creation, against patter

net/mlx5: store flow pattern template items

Store items which define the pattern template on creation.
This allows validation of items, provided by the user during flow rule
creation, against pattern template.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5: extract flow queue index validation

Extract validation of queue index out of validation of
RTE_FLOW_ACTION_TYPE_QUEUE action to allow reuse in template API
flow rule creation implementatio

net/mlx5: extract flow queue index validation

Extract validation of queue index out of validation of
RTE_FLOW_ACTION_TYPE_QUEUE action to allow reuse in template API
flow rule creation implementation.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

net/mlx5: set modify header as shared flow action

In current implementation, in non template mode,
modify header action is not set as always shared.
Align to HWS implementation, setting modify heade

net/mlx5: set modify header as shared flow action

In current implementation, in non template mode,
modify header action is not set as always shared.
Align to HWS implementation, setting modify header action as always shared.
Optimize mask initialization.

Signed-off-by: Maayan Kashani <mkashani@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: reorganize non-template flow HWS structures

Reorganize rte_flow_hw and rte_flow_nt2hws
structures for better performance, and removed packed.

Signed-off-by: Maayan Kashani <mkashani@nvidi

net/mlx5: reorganize non-template flow HWS structures

Reorganize rte_flow_hw and rte_flow_nt2hws
structures for better performance, and removed packed.

Signed-off-by: Maayan Kashani <mkashani@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: support RSS expansion in non-template HWS setup

The MLX5 PMD expands flow rule with the RSS action in the
non-template environment.

The patch adds RSS flow rule expansion for legacy flow

net/mlx5: support RSS expansion in non-template HWS setup

The MLX5 PMD expands flow rule with the RSS action in the
non-template environment.

The patch adds RSS flow rule expansion for legacy flow rules in
the template setup.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: support bulk actions in non-template flow

Add support for encap/decap/modify header
action for non template API.

Save 1 action per bulk according to action data.
Reuse action if possible.

net/mlx5: support bulk actions in non-template flow

Add support for encap/decap/modify header
action for non template API.

Save 1 action per bulk according to action data.
Reuse action if possible.
Store actions same as for SWS today, use same key structure.

Signed-off-by: Maayan Kashani <mkashani@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: abstract flow action and enable reconfigure

reuse HWS actions translation code in order to create
the actions for lower layer.
All actions are handled as non masked.
Therefore, will be tra

net/mlx5: abstract flow action and enable reconfigure

reuse HWS actions translation code in order to create
the actions for lower layer.
All actions are handled as non masked.
Therefore, will be translated in the rule construct
phase shared with the template code
and not during the action template translate.

Currently there is no option to reconfigure in template API’s.
To support non template API and template API simultaneously,
There is a need to enable the user to reconfigure after default
configuration was applied by the driver on port start.

Fix flow create ptr
In SWS the return flow pointer is the flow index,
in HWS the return flow pointer is the flow data pointer.
Currently the return value of non template code did not
Enable destroy or quary.

Signed-off-by: Maayan Kashani <mkashani@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: introduce HWS for non-template flow API

Implement the frame and needed building
blocks for non template to hws API's.

Added validate, list_create and list_destroy to mlx5_flow_hw_drv_ops.

net/mlx5: introduce HWS for non-template flow API

Implement the frame and needed building
blocks for non template to hws API's.

Added validate, list_create and list_destroy to mlx5_flow_hw_drv_ops.
Rename old list_create/list_destroy functions to legacy_*
and added a call from verbs/dv ops to the legacy functions.

Updated rte_flow_hw as needed.
Added rte_flow_nt2hws structure for non-template rule data.

Signed-off-by: Maayan Kashani <mkashani@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: support flow match with E-Switch manager

Currently, in switch mode, mlx5 PMD only supports match with
dedicated vport. There is a use case which user may want to
offload the rules only to

net/mlx5: support flow match with E-Switch manager

Currently, in switch mode, mlx5 PMD only supports match with
dedicated vport. There is a use case which user may want to
offload the rules only to match with all the packet sent by
application not from vport.

Since the port_id info of packet sent by application is
E-Switch manager, and kernel driver has exposed the E-Switch
manager register value, this commit adds the support of
register matching for E-Switch manager.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...