net/mlx5: separate registers usage per port

Current implementation stored REG_C registers available for HWS tags
in PMD global array. As the result, PMD could not work properly with
different port t

net/mlx5: separate registers usage per port

Current implementation stored REG_C registers available for HWS tags
in PMD global array. As the result, PMD could not work properly with
different port types that allocate REG_C registers differently.

The patch stores registers available to a port in the port
shared context. Register values will be assigned according to the port
capabilities.

New function call `flow_hw_get_reg_id_from_ctx()` matches REG_C
register to input DR5 context.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>

show more ...

drivers: use function to compare PCI addresses

Some places were not using the function rte_pci_addr_cmp()
to compare 2 PCI addresses.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by:

drivers: use function to compare PCI addresses

Some places were not using the function rte_pci_addr_cmp()
to compare 2 PCI addresses.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Long Li <longli@microsoft.com>

show more ...

drivers: use macro for PCI address format

Some places were not using the macro PCI_PRI_FMT
as print format of a PCI address.

Note: RTE prefix is missing in the name of some PCI macros.

Signed-off-

drivers: use macro for PCI address format

Some places were not using the macro PCI_PRI_FMT
as print format of a PCI address.

Note: RTE prefix is missing in the name of some PCI macros.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: John Daley <johndale@cisco.com>
Acked-by: Huisong Li <lihuisong@huawei.com>

show more ...

net/mlx5: extend send to kernel action support

The send to kernel action was supported in NIC and FDB tables,
Currently, the send to kernel action is created in NIC RX only.

With some TC rules (exa

net/mlx5: extend send to kernel action support

The send to kernel action was supported in NIC and FDB tables,
Currently, the send to kernel action is created in NIC RX only.

With some TC rules (example: roce packets, redirects into rep ports)
and DPDK rules for the rest of the traffic. Then it needs the
specific rule to re-route the packets into the kernel through the FDB
table.

This patch adds the FDB and NIC-TX tables support for sending to the kernel
action.

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
g

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_clz; new=rte_clz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_ctzll; new=rte_ctz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_ctz; new=rte_ctz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_popcountll; new=rte_popcount64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_popcount; new=rte_popcount32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

Then inclusion of rte_bitops.h was added were necessary.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>

show more ...

net/mlx5: fix drop action attribute validation

Before this patch, DROP flow action was rejected for all egress
flow rules, which was not correct for all cases.

When Verbs flow engine is used (dv_fl

net/mlx5: fix drop action attribute validation

Before this patch, DROP flow action was rejected for all egress
flow rules, which was not correct for all cases.

When Verbs flow engine is used (dv_flow_en=0) DROP flow action
is implemented using IBV_FLOW_SPEC_ACTION_DROP IBV action.
This action is supported on ingress only.
This patch amends the DROP flow action validation to allow it only on
ingress.

When DV flow engine is used (dv_flow_en=1) there are 2 implementation
options for DROP flow action:

- DR drop action (allocated through mlx5dv_dr_action_create_drop() API),
- dedicated drop queue.

When flow rules are created on non-root flow tables DR drop action can
be used on all steering domains. On root flow table however, this action
ca be used if and only if it is supported by rdma-core and kernel
drivers. mlx5 PMD dynamically checks if DR drop action is supported
on root tables during device probing
(it is checked in mlx5_flow_discover_dr_action_support()).
If DR drop action is not supported on root table, then dedicated
drop queue must be used and as a result, DROP flow action on root
is supported only for ingress flow rules.
This patch amends the DROP flow action validation with this logic
for DV flow engine.

This patch also renames the dr_drop_action_en field in device's private
data to dr_root_drop_action_en to align the name with field's meaning.

Fixes: 3c4338a42134 ("net/mlx5: optimize device spawn time with representors")
Fixes: 45633c460c22 ("net/mlx5: workaround drop action with old kernel")
Fixes: da845ae9d7c1 ("net/mlx5: fix drop action for Direct Rules/Verbs")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix isolated mode if no representor matching

In HW steering mode, when running on an E-Switch setup,
mlx5 PMD provides an ability to enable or disable
representor matching (through `repr_m

net/mlx5: fix isolated mode if no representor matching

In HW steering mode, when running on an E-Switch setup,
mlx5 PMD provides an ability to enable or disable
representor matching (through `repr_matching_en` device argument).
If representor matching is enabled, any ingress or egress flow rule,
created on any port representor will match traffic related
to that specific port.
If it is disabled, flow rule created on one of the ports,
will match traffic related to all ports.

As a result, when representor matching is disabled,
PMD cannot correctly create control flow rules for receiving
default traffic according to port configuration.
Since each port representor in the same switch domain,
can have different port configuration and flow rules
do not differentiate between ports, these flow rules cannot be
correctly applied.
In that case, each port works in de facto isolated mode.

This patch makes sure that if representor matching is disabled,
port is forced into isolated mode. Disabling flow isolated is forbidden.

Fixes: 483181f7b6dd ("net/mlx5: support device control of representor matching")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: enable HWS flex item creation

Enable flex item create and destroy with dv_flow_en=2

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

net/mlx5: support aggregated affinity matching

This patch adds the new aggregated affinity item support in PMD:
RTE_FLOW_ITEM_TYPE_AGGR_AFFINITY.

This patch adds the validation function for the new

net/mlx5: support aggregated affinity matching

This patch adds the new aggregated affinity item support in PMD:
RTE_FLOW_ITEM_TYPE_AGGR_AFFINITY.

This patch adds the validation function for the new item,
it works for NIC-RX and FDB rule on ROOT-table only.

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: add LAG Rx port affinity in PRM

This patch adds function to query HCA capability via Devx for
lag_rx_port_affinity.

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ov

net/mlx5: add LAG Rx port affinity in PRM

This patch adds function to query HCA capability via Devx for
lag_rx_port_affinity.

Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix port event cleaning order

The shared IB device (sh) has per port data with filed for interrupt
handler port_id. It used by shared interrupt handler to find the
corresponding rte_eth de

net/mlx5: fix port event cleaning order

The shared IB device (sh) has per port data with filed for interrupt
handler port_id. It used by shared interrupt handler to find the
corresponding rte_eth device by IB port index.
If value is equal or greater RTE_MAX_ETHPORTS it means there is no
subhandler installed for specified IB port index.

When a few ports are created under same sh, the sh is created with the
first port and the interrupt handler port_id is initialized to
RTE_MAX_ETHPORTS for each port.
In port creation, the interrupt handler port_id is updated with the
correct value. Since this updating, the mlx5_dev_interrupt_nl_cb
function uses this port and its priv structure.
However, when the ports are closed, this filed isn't updated and the
interrupt handler continue working until it is uninstalled in SH
destruction.
If mlx5_dev_interrupt_nl_cb is called between port closing and SH
destruction, it uses invalid port causing a crash.

This patch adds interrupt handler port_id updating to the close function
and add memory barrier to make sure it is done before priv reset.

Fixes: 655c3c26c11e ("net/mlx5: fix initial link status detection")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix null check in devargs parsing

The "mlx5_os_parse_eth_devargs()" function parses the ETH devargs into a
specific structure called "eth_da".
It gets structure called "devargs" as a membe

net/mlx5: fix null check in devargs parsing

The "mlx5_os_parse_eth_devargs()" function parses the ETH devargs into a
specific structure called "eth_da".
It gets structure called "devargs" as a member of EAL device containing
the relevant information.

When "devargs" structure is invalid, the function avoids parsing it.
However, when it valid but its field "args" is invalid, the function
tries to parse it and dereference to NULL pointer.

This patch adds check to avoid this NULL dereferencing.

Fixes: 919488fbfa71 ("net/mlx5: support Sub-Function")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: support device control of representor matching

In some E-Switch use cases, applications want to receive all traffic
on a single port. Since currently, flow API does not provide a way to
ma

net/mlx5: support device control of representor matching

In some E-Switch use cases, applications want to receive all traffic
on a single port. Since currently, flow API does not provide a way to
match traffic forwarded to any port representor, this patch adds
support for controlling representor matching on ingress flow rules.

Representor matching is controlled through a new device argument
repr_matching_en.

- If representor matching is enabled (default setting),
then each ingress pattern template has an implicit REPRESENTED_PORT
item added. Flow rules based on this pattern template will match
the vport associated with the port on which the rule is created.
- If representor matching is disabled, then there will be no implicit
item added. As a result ingress flow rules will match traffic
coming to any port, not only the port on which the flow rule is
created.

Representor matching is enabled by default, to provide an expected
default behavior.

This patch enables egress flow rules on representors when E-Switch is
enabled in the following configurations:

- repr_matching_en=1 and dv_xmeta_en=4
- repr_matching_en=1 and dv_xmeta_en=0
- repr_matching_en=0 and dv_xmeta_en=0

When representor matching is enabled, the following logic is
implemented:

1. Creating an egress template table in group 0 for each port. These
tables will hold default flow rules defined as follows:

pattern SQ
actions MODIFY_FIELD (set available bits in REG_C_0 to
vport_meta_tag)
MODIFY_FIELD (copy REG_A to REG_C_1, only when
dv_xmeta_en == 4)
JUMP (group 1)

2. Egress pattern templates created by an application have an implicit
MLX5_RTE_FLOW_ITEM_TYPE_TAG item prepended to the pattern, which
matches available bits of REG_C_0.

3. Egress flow rules created by an application have an implicit
MLX5_RTE_FLOW_ITEM_TYPE_TAG item prepended to the pattern, which
matches vport_meta_tag placed in available bits of REG_C_0.

4. Egress template tables created by an application, which are in
group n, are placed in group n + 1.

5. Items and actions related to META are operating on REG_A when
dv_xmeta_en == 0 or REG_C_1 when dv_xmeta_en == 4.

When representor matching is disabled and extended metadata is disabled,
no changes to the current logic are required.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support device control for E-Switch default rule

This patch adds support for fdb_def_rule_en device argument to HW
Steering, which controls:

- the creation of the default FDB jump flow ru

net/mlx5: support device control for E-Switch default rule

This patch adds support for fdb_def_rule_en device argument to HW
Steering, which controls:

- the creation of the default FDB jump flow rule.
- the ability of the user to create transfer flow rules in the root
table.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support connection tracking with HWS

This commit adds the support of connection tracking to HW steering as
SW steering did before.

The difference from SW steering implementation is that i

net/mlx5: support connection tracking with HWS

This commit adds the support of connection tracking to HW steering as
SW steering did before.

The difference from SW steering implementation is that it takes
advantage of HW steering bulk action allocation support, in HW
steering only one single CT pool is needed.

An indexed pool is introduced to record allocated actions from bulk and
CT action state etc. Once one CT action is allocated from bulk, one
indexed object will also be allocated from the indexed pool, similar to
deallocating. That makes mlx5_aso_ct_action can also be managed by that
indexed pool, no need to be reserved from mlx5_aso_ct_pool. The single
CT pool is also saved to mlx5_aso_ct_action struct directly.

The ASO operation functions are shared with SW steering implementation.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support Direct Rules action template API

This patch adapts mlx5 PMD to changes in mlx5dr API regarding the action
templates. It changes the following:

1. Actions template creation:

-

net/mlx5: support Direct Rules action template API

This patch adapts mlx5 PMD to changes in mlx5dr API regarding the action
templates. It changes the following:

1. Actions template creation:

- Flow actions types are translated to mlx5dr action types in order
to create mlx5dr_action_template object.
- An offset is assigned to each flow action. This offset is used to
predetermine the action's location in the rule_acts array passed
on the rule creation.

2. Template table creation:

- Fixed actions are created and put in the rule_acts cache using
predetermined offsets
- mlx5dr matcher is parametrized by action templates bound to
template table.
- mlx5dr matcher is configured to optimize rule creation based on
passed rule indices.

3. Flow rule creation:

- mlx5dr rule is parametrized by the action template on which these
rule's actions are based.
- Rule index hint is provided to mlx5dr.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: add extended metadata mode for HWS

The new mode 4 of devarg "dv_xmeta_en" is added for HWS only. In this
mode, the Rx / Tx metadata with 32b width copy between FDB and NIC is
supported.

T

net/mlx5: add extended metadata mode for HWS

The new mode 4 of devarg "dv_xmeta_en" is added for HWS only. In this
mode, the Rx / Tx metadata with 32b width copy between FDB and NIC is
supported.

The mark is only supported in NIC and there is no copy supported.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow port action with HWS

This patch implements creating and caching of port action for use with
HW Steering FDB flows.

Actions are created on flow template API configuration and

net/mlx5: support flow port action with HWS

This patch implements creating and caching of port action for use with
HW Steering FDB flows.

Actions are created on flow template API configuration and created
only on the port designated as the master. Attaching and detaching ports
in the same switching domain causes an update to the port actions cache
by, respectively, creating and destroying actions.

A new devarg fdb_def_rule_en is being added and it's used to control
the default dedicated E-Switch rules that are created by the PMD
implicitly or not, and PMD sets this value to 1 by default.

If set to 0, the default E-Switch rule will not be created and the user
can create the specific E-Switch rules on the root table if needed.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support flow modify field with HWS

This patch introduces support for modify_field rte_flow actions in HWS
mode that includes:
- Ingress and egress domains,
- SET and ADD operations,
- u

net/mlx5: support flow modify field with HWS

This patch introduces support for modify_field rte_flow actions in HWS
mode that includes:
- Ingress and egress domains,
- SET and ADD operations,
- usage of arbitrary bit offsets and widths for packet and metadata
fields.

This is implemented in two phases:
1. On flow table creation the hardware commands are generated, based
on rte_flow action templates, and stored alongside action template.

2. On flow rule creation/queueing the hardware commands are updated with
values provided by the user. Any masks over immediate values, provided
in action templates, are applied to these values before enqueueing rules
for creation.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: provide available tag registers

This stores the available tags that can be used by the
application in a global array that will be used to
transfer the TAG item directly from the ID to the

net/mlx5: provide available tag registers

This stores the available tags that can be used by the
application in a global array that will be used to
transfer the TAG item directly from the ID to the REG_C_x
since these can't be changed after startup.

Signed-off-by: Bing Zhao <bingz@nvidia.com>

show more ...

net/mlx5: add port to metadata conversion

This adds conversion functions between both ethdev port_id and IB
context to internal corresponding tag/mask values.

Signed-off-by: Dariusz Sosnowski <dsos

net/mlx5: add port to metadata conversion

This adds conversion functions between both ethdev port_id and IB
context to internal corresponding tag/mask values.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

net/mlx5: add send to kernel action resource holder

Add new structure mlx5_send_to_kernel_action which will hold
together allocated action resource and a reference to used table.
A new structure mem

net/mlx5: add send to kernel action resource holder

Add new structure mlx5_send_to_kernel_action which will hold
together allocated action resource and a reference to used table.
A new structure member of this type added to struct mlx5_dev_ctx_shared.
The member will be initialized upon first created send_to_kernel
action and will be reused for all future actions of this type.
Release of these resources will be done when all shared DR
resources are being released in mlx5_os_free_shared_dr().

Change function flow_dv_tbl_resource_release() from
static to external.

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix LRO requirements check

One of the conditions to allow LRO offload is the DV configuration.

The function incorrectly checks the DV configuration before initializing
it by the user deva

net/mlx5: fix LRO requirements check

One of the conditions to allow LRO offload is the DV configuration.

The function incorrectly checks the DV configuration before initializing
it by the user devarg; hence, LRO cannot be allowed.

This patch moves this check to mlx5_shared_dev_ctx_args_config, where DV
configuration is initialized.

Fixes: c4b862013598 ("net/mlx5: refactor to detect operation by DevX")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reported-by: Gal Shalom <galshalom@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix Verbs FD leak in secondary process

FDs passed from rte_mp_msg are duplicated to the secondary process and
need to be closed.

Fixes: 9a8ab29b84 ("net/mlx5: replace IPC socket with EAL

net/mlx5: fix Verbs FD leak in secondary process

FDs passed from rte_mp_msg are duplicated to the secondary process and
need to be closed.

Fixes: 9a8ab29b84 ("net/mlx5: replace IPC socket with EAL API")
Cc: stable@dpdk.org

Signed-off-by: Long Li <longli@microsoft.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

bus: hide bus object

Make rte_bus opaque for non internal users.
This will make extending this object possible without breaking the ABI.

Introduce a new driver header and move rte_bus definition an

bus: hide bus object

Make rte_bus opaque for non internal users.
This will make extending this object possible without breaking the ABI.

Introduce a new driver header and move rte_bus definition and helpers.
Update drivers and library to use the internal header.

Some applications may have been dereferencing rte_bus objects, mark
this object's accessors as stable.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...