net/mlx5: fix meter hierarchy with modify header

If any meter in the hierarchy has a policy flow containing set_tag or
modify_field action, the policy flow must match the src port to which
the polic

net/mlx5: fix meter hierarchy with modify header

If any meter in the hierarchy has a policy flow containing set_tag or
modify_field action, the policy flow must match the src port to which
the policy belongs, to determine the order of modify_hdr and
meter action. But the meter hierarchy will not be able to use by
user flow that matches another src port.

To use this type of meter hierarchy for other src ports, we need to add
a new policy flow matching the new src port from the user flow
dynamically. But then it cannot be used by flow matching all ports.

Fixes: ca7e6051e7cb ("net/mlx5: limit meter flow when matching all ports")
Cc: stable@dpdk.org

Signed-off-by: Shun Hao <shunh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix representor item with meter

When creating flow matching port representor item with meter action, it
will fail due to incorrect parsing the item.

This patch fixes this issue by adding

net/mlx5: fix representor item with meter

When creating flow matching port representor item with meter action, it
will fail due to incorrect parsing the item.

This patch fixes this issue by adding the correct item parse for port
representor in validation.

Fixes: 707d5e7d79e0 ("net/mlx5: support flow matching on representor ID")
Cc: stable@dpdk.org

Signed-off-by: Shun Hao <shunh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix hairpin split with set VLAN VID action

Before this patch any flow rule which works on hairpin queues
and which has OF_SET_VLAN_VID action was split into 2 flow rules:

- one subflow fo

net/mlx5: fix hairpin split with set VLAN VID action

Before this patch any flow rule which works on hairpin queues
and which has OF_SET_VLAN_VID action was split into 2 flow rules:

- one subflow for Rx,
- one subflow for Tx.

OF_SET_VLAN_VID action was always placed in the Tx subflow.

Assuming a flow rule which matches VLAN traffic and has both
OF_SET_VLAN_VID action, and MODIFY_FIELD action on VLAN VID,
but no OF_PUSH_VLAN action, the following happened:

- MODIFY_FIELD action was placed in Rx subflow,
- OF_SET_VLAN_VID action was placed in Tx subflow,
- OF_SET_VLAN_VID action is internally compiled to a header modify
command.

This caused the following issues:

1. Since OF_SET_VLAN_VID was placed in Tx subflow, 2 header modify
actions were allocated. One for Rx and one for Tx.
2. If OF_SET_VLAN_VID action was placed before MODIFY_FIELD on VLAN VID,
the flow rule executed header modifications in reverse order.
MODIFY_FIELD actions were executed first in the Rx subflow and
OF_SET_VLAN_VID was executed second in Tx subflow.

This patch fixes this behavior by not splitting hairpin flow rules
if OF_SET_VLAN_VID action is used without OF_PUSH_VLAN.
On top of that, if flow rule is split, the OF_SET_VLAN_VID action
is not moved to Tx subflow (for flow rules mentioned above).

Fixes: 210008309b45 ("net/mlx5: fix VLAN push action on hairpin queue")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: forbid direction attribute in transfer flow rules

Since [1] flow API forbids usage of direction attributes in transfer
flow rules. This patch adapts mlx5 PMD to this requirement.

From thi

net/mlx5: forbid direction attribute in transfer flow rules

Since [1] flow API forbids usage of direction attributes in transfer
flow rules. This patch adapts mlx5 PMD to this requirement.

From this patch, flow rule validation in mxl5 PMD will reject transfer
flow rules with any of the direction attributes set
(i.e. 'ingress' or 'egress').
As a result flow rule can only have one of 'ingress', 'egress' or
'transfer' attributes set.

This patch also changes the following:

- Control flow rules used in FDB are 'transfer' only.
- Checks which assumed that 'transfer' can be used
with 'ingress' and 'egress' are reduced to just checking
for direction attributes, since all attributes are exclusive.
- Flow rules for updating flow_tag are created for both ingress
and transfer flow rules which have MARK action.
- Moves mlx5_flow_validate_attributes() function from generic flow
implementation to legacy Verbs flow engine implementation,
since it was used only there. Function is renamed accordingly.
Also removes checking if E-Switch uses DV in that
function, since if legacy Verbs flow engine is used,
then that is always not the case.

[1] commit bd2a4d4b2e3a ("ethdev: forbid direction attribute in
transfer flow rules")

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix source port checking in sample flow rule

The metadata register C value was lost in FDB egress while doing the
flow sampler on ConnectX-5. The FDB direction checking was decided by
the

net/mlx5: fix source port checking in sample flow rule

The metadata register C value was lost in FDB egress while doing the
flow sampler on ConnectX-5. The FDB direction checking was decided by
the source port in the flow creation. If there's additional port item
was added in the flow match, then the actual source port was changed.

This patch adds the checking for the port id item:
RTE_FLOW_ITEM_TYPE_PORT_ID, RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT,
and RTE_FLOW_ITEM_TYPE_PORT_REPRESENTOR,
then updates FDB egress checking and the source vport metadata
from the port item, also updates the PUSH VLAN, POP VLAN and
flow sampler action validation.

Fixes: 04c0d3f20f54 ("net/mlx5: fix port matching in sample flow rule")
Fixes: 255b8f86eb6e ("net/mlx5: fix E-Switch egress mirror flow validation")
Cc: stable@dpdk.org

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix race condition in counter pool resizing

Counter management structure has array of counter pools. This array is
invalid in management structure initialization and grows on demand.

The

net/mlx5: fix race condition in counter pool resizing

Counter management structure has array of counter pools. This array is
invalid in management structure initialization and grows on demand.

The resizing include:
1. Allocate memory for the new size.
2. Copy the existing data to the new memory.
3. Move the pointer to the new memory.
4. Free the old memory.

The third step can be performed before for this function, and compiler
may do that, but another thread might read the pointer before coping and
read invalid data or even crash.

This patch allocates memory for this array once in management structure
initialization and limit the counters number by 16M.

Fixes: 3aa279157fa0 ("net/mlx5: synchronize flow counter pool creation")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: enable queue flow aging action

As the queue-based aging API has been integrated[1], the flow aging
action support in HWS steering code can be enabled now.

[1]: https://patchwork.dpdk.org/

net/mlx5: enable queue flow aging action

As the queue-based aging API has been integrated[1], the flow aging
action support in HWS steering code can be enabled now.

[1]: https://patchwork.dpdk.org/project/dpdk/cover/
20221026214943.3686635-1-michaelba@nvidia.com/

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix tunnel header with IPIP offload

For the flows with multiple tunnel layers and containing
tunnel decap and modify actions, for example:

... / vxlan / eth / ipv4 proto is 4 / end
action

net/mlx5: fix tunnel header with IPIP offload

For the flows with multiple tunnel layers and containing
tunnel decap and modify actions, for example:

... / vxlan / eth / ipv4 proto is 4 / end
actions raw_decap / modify_field / ...
(note: proto 4 means we have the IP-over-IP tunnel in VXLAN payload)

We have added the multiple tunnel layers validation rejecting
the flows like above mentioned one.

The hardware supports the above match combination till the inner
IP-over-IP header (not including the last one), both for IP-over-IPv4
and IP-over-IPv6, so we should not blindly reject. Also, for the modify
actions following the decap we should set the layer attributes correctly.

This patch reverts the below code changes to support the match, and
adjusts the layers update in case of decap with outer tunnel header.

Fixes: fa06906a48ee ("net/mlx5: fix IPIP multi-tunnel validation")
Cc: stable@dpdk.org

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow matching on representor ID

Add support for port_representor item, it will match on traffic
originated from representor port specified in the pattern. This item
is supported in

net/mlx5: support flow matching on representor ID

Add support for port_representor item, it will match on traffic
originated from representor port specified in the pattern. This item
is supported in FDB steering domain only (in the flow with transfer
attribute).

For example, below flow will redirect the destination of traffic from
ethdev 1 to ethdev 2.

testpmd> ... pattern eth / port_representor port_id is 1 / end actions
represented_port ethdev_port_id 2 / ...

To handle abovementioned item, Tx queue matching is added in the driver,
and the flow will be expanded to number of the Tx queues. If the spec of
port_representor is NULL, the flow will not be expanded and match on
traffic from any representor port.

Signed-off-by: Sean Zhang <xiazhang@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix RSS expansion buffer size

Increase expansion buffer size to accumulate more RSS types.

Fixes: 3f02c7ff6815 ("net/mlx5: fix RSS expansion for inner tunnel VLAN")
Cc: stable@dpdk.org

S

net/mlx5: fix RSS expansion buffer size

Increase expansion buffer size to accumulate more RSS types.

Fixes: 3f02c7ff6815 ("net/mlx5: fix RSS expansion for inner tunnel VLAN")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: support device control of representor matching

In some E-Switch use cases, applications want to receive all traffic
on a single port. Since currently, flow API does not provide a way to
ma

net/mlx5: support device control of representor matching

In some E-Switch use cases, applications want to receive all traffic
on a single port. Since currently, flow API does not provide a way to
match traffic forwarded to any port representor, this patch adds
support for controlling representor matching on ingress flow rules.

Representor matching is controlled through a new device argument
repr_matching_en.

- If representor matching is enabled (default setting),
then each ingress pattern template has an implicit REPRESENTED_PORT
item added. Flow rules based on this pattern template will match
the vport associated with the port on which the rule is created.
- If representor matching is disabled, then there will be no implicit
item added. As a result ingress flow rules will match traffic
coming to any port, not only the port on which the flow rule is
created.

Representor matching is enabled by default, to provide an expected
default behavior.

This patch enables egress flow rules on representors when E-Switch is
enabled in the following configurations:

- repr_matching_en=1 and dv_xmeta_en=4
- repr_matching_en=1 and dv_xmeta_en=0
- repr_matching_en=0 and dv_xmeta_en=0

When representor matching is enabled, the following logic is
implemented:

1. Creating an egress template table in group 0 for each port. These
tables will hold default flow rules defined as follows:

pattern SQ
actions MODIFY_FIELD (set available bits in REG_C_0 to
vport_meta_tag)
MODIFY_FIELD (copy REG_A to REG_C_1, only when
dv_xmeta_en == 4)
JUMP (group 1)

2. Egress pattern templates created by an application have an implicit
MLX5_RTE_FLOW_ITEM_TYPE_TAG item prepended to the pattern, which
matches available bits of REG_C_0.

3. Egress flow rules created by an application have an implicit
MLX5_RTE_FLOW_ITEM_TYPE_TAG item prepended to the pattern, which
matches vport_meta_tag placed in available bits of REG_C_0.

4. Egress template tables created by an application, which are in
group n, are placed in group n + 1.

5. Items and actions related to META are operating on REG_A when
dv_xmeta_en == 0 or REG_C_1 when dv_xmeta_en == 4.

When representor matching is disabled and extended metadata is disabled,
no changes to the current logic are required.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support device control for E-Switch default rule

This patch adds support for fdb_def_rule_en device argument to HW
Steering, which controls:

- the creation of the default FDB jump flow ru

net/mlx5: support device control for E-Switch default rule

This patch adds support for fdb_def_rule_en device argument to HW
Steering, which controls:

- the creation of the default FDB jump flow rule.
- the ability of the user to create transfer flow rules in the root
table.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support async flow action push and pull

The queue based rte_flow_async_action_* functions work the same as
queue based async flow functions. The operations can be pushed
asynchronously, an

net/mlx5: support async flow action push and pull

The queue based rte_flow_async_action_* functions work the same as
queue based async flow functions. The operations can be pushed
asynchronously, and so is the pull.

This commit adds the async action missing push and pull support.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow age action with HWS

Add support for AGE action for HW steering.
This patch includes:

1. Add new structures to manage aging.
2. Initialize all of them in configure function.

net/mlx5: support flow age action with HWS

Add support for AGE action for HW steering.
This patch includes:

1. Add new structures to manage aging.
2. Initialize all of them in configure function.
3. Implement per second aging check using CNT background thread.
4. Enable AGE action in flow create/destroy operations.
5. Implement a queue-based function to report aged flow rules.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow meter mark indirect action with HWS

Add the ability to create an indirect action handle for METER_MARK.
It allows sharing one Meter between several different actions.

Signed-

net/mlx5: support flow meter mark indirect action with HWS

Add the ability to create an indirect action handle for METER_MARK.
It allows sharing one Meter between several different actions.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support connection tracking with HWS

This commit adds the support of connection tracking to HW steering as
SW steering did before.

The difference from SW steering implementation is that i

net/mlx5: support connection tracking with HWS

This commit adds the support of connection tracking to HW steering as
SW steering did before.

The difference from SW steering implementation is that it takes
advantage of HW steering bulk action allocation support, in HW
steering only one single CT pool is needed.

An indexed pool is introduced to record allocated actions from bulk and
CT action state etc. Once one CT action is allocated from bulk, one
indexed object will also be allocated from the indexed pool, similar to
deallocating. That makes mlx5_aso_ct_action can also be managed by that
indexed pool, no need to be reserved from mlx5_aso_ct_pool. The single
CT pool is also saved to mlx5_aso_ct_action struct directly.

The ASO operation functions are shared with SW steering implementation.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow counter action for HWS

This commit adds HW steering counter action support.
The pool mechanism is the basic data structure for the HW steering
counter.

The HW steering's coun

net/mlx5: support flow counter action for HWS

This commit adds HW steering counter action support.
The pool mechanism is the basic data structure for the HW steering
counter.

The HW steering's counter pool is based on the rte_ring of zero-copy
variation.

There are two global rte_rings:
1. free_list:
Store the counters indexes, which are ready for use.
2. wait_reset_list:
Store the counters indexes, which are just freed from the user and
need to query the hardware counter to get the reset value before
this counter can be reused again.

The counter pool also supports cache per HW steering's queues, which are
also based on the rte_ring of zero-copy variation.

The cache can be configured in size, preload, threshold, and fetch size,
they are all exposed via device args.

The main operations of the counter pool are as follows:

- Get one counter from the pool:
1. The user call _get_* API.
2. If the cache is enabled, dequeue one counter index from the local
cache:
2. A: if the dequeued one from the local cache is still in reset
status (counter's query_gen_when_free is equal to pool's query
gen):
I. Flush all counters in the local cache back to global
wait_reset_list.
II. Fetch _fetch_sz_ counters into the cache from the global
free list.
III. Fetch one counter from the cache.
3. If the cache is empty, fetch _fetch_sz_ counters from the global
free list into the cache and fetch one counter from the cache.
- Free one counter into the pool:
1. The user calls _put_* API.
2. Put the counter into the local cache.
3. If the local cache is full:
A: Write back all counters above _threshold_ into the global
wait_reset_list.
B: Also, write back this counter into the global wait_reset_list.

When the local cache is disabled, _get_/_put_ cache directly from/into
global list.

Signed-off-by: Xiaoyu Min <jackmin@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow meter action for HWS

This commit adds meter action for HWS steering.

HW steering meter is based on ASO. The number of meters will
be used by flows should be specified in adva

net/mlx5: support flow meter action for HWS

This commit adds meter action for HWS steering.

HW steering meter is based on ASO. The number of meters will
be used by flows should be specified in advance in the flow
configure API.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: add extended metadata mode for HWS

The new mode 4 of devarg "dv_xmeta_en" is added for HWS only. In this
mode, the Rx / Tx metadata with 32b width copy between FDB and NIC is
supported.

T

net/mlx5: add extended metadata mode for HWS

The new mode 4 of devarg "dv_xmeta_en" is added for HWS only. In this
mode, the Rx / Tx metadata with 32b width copy between FDB and NIC is
supported.

The mark is only supported in NIC and there is no copy supported.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow port action with HWS

This patch implements creating and caching of port action for use with
HW Steering FDB flows.

Actions are created on flow template API configuration and

net/mlx5: support flow port action with HWS

This patch implements creating and caching of port action for use with
HW Steering FDB flows.

Actions are created on flow template API configuration and created
only on the port designated as the master. Attaching and detaching ports
in the same switching domain causes an update to the port actions cache
by, respectively, creating and destroying actions.

A new devarg fdb_def_rule_en is being added and it's used to control
the default dedicated E-Switch rules that are created by the PMD
implicitly or not, and PMD sets this value to 1 by default.

If set to 0, the default E-Switch rule will not be created and the user
can create the specific E-Switch rules on the root table if needed.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix steering engine type check

In the function flow_get_drv_type(), attr will be read in non-HWS mode.
In case the user calls the HWS API in SWS mode, the attr should be
placed in HWS func

net/mlx5: fix steering engine type check

In the function flow_get_drv_type(), attr will be read in non-HWS mode.
In case the user calls the HWS API in SWS mode, the attr should be
placed in HWS functions or it will cause a crash.

Fixes: c40c061a022e ("net/mlx5: add basic flow queue operation")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5/hws: enable hardware steering

Replace stub implementation of HWS with mlx5dr code.

Signed-off-by: Alex Vesker <valex@nvidia.com>

net/mlx5: provide available tag registers

This stores the available tags that can be used by the
application in a global array that will be used to
transfer the TAG item directly from the ID to the

net/mlx5: provide available tag registers

This stores the available tags that can be used by the
application in a global array that will be used to
transfer the TAG item directly from the ID to the REG_C_x
since these can't be changed after startup.

Signed-off-by: Bing Zhao <bingz@nvidia.com>

show more ...

net/mlx5: add port to metadata conversion

This adds conversion functions between both ethdev port_id and IB
context to internal corresponding tag/mask values.

Signed-off-by: Dariusz Sosnowski <dsos

net/mlx5: add port to metadata conversion

This adds conversion functions between both ethdev port_id and IB
context to internal corresponding tag/mask values.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: add hardware steering item translation

This provides shared item tranlsation code for hardware
steering root table flows as they still work under FW steering mode.

Signed-off-by: Suanming

net/mlx5: add hardware steering item translation

This provides shared item tranlsation code for hardware
steering root table flows as they still work under FW steering mode.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>

show more ...