net/mlx5: separate Rx function declarations to another file

The mlx5_rxtx.c file contains a lot of Tx burst functions, each of those
is performance-optimized for the specific set of requested offloa

net/mlx5: separate Rx function declarations to another file

The mlx5_rxtx.c file contains a lot of Tx burst functions, each of those
is performance-optimized for the specific set of requested offloads.
These ones are generated on the basis of the template function and it
takes significant time to compile, just due to a large number of giant
functions generated in the same file and this compilation is not being
done in parallel with using multithreading.

Therefore we can split the mlx5_rxtx.c file into several separate files
to allow different functions to be compiled simultaneously.
In this patch, we separate Rx function declarations to different header
file in preparation for removing them from the source file and as an
optional preparation step for further consolidation of Rx burst
functions.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: fix drop action for Direct Rules/Verbs

There are multiple branches in rdma-core library backing
the rte flows:
- Verbs
- Direct Verbs (DV)
- Direct Rules (DR)

The Verbs API always r

net/mlx5: fix drop action for Direct Rules/Verbs

There are multiple branches in rdma-core library backing
the rte flows:
- Verbs
- Direct Verbs (DV)
- Direct Rules (DR)

The Verbs API always requires the specifying the queue even
if there is the drop action in the flow, though the kernel
optimizes out the actual queue usage for the flows containing
the drop action. The PMD handles the dedicated Rx queue to
provide Verbs API compatibility.

The DV/DR API does not require explicit specifying the queue
at the flow creation, but PMD still specified the dedicated
drop queue as action. It performed the packet forwarding to
the dummy queue (that was not polled at all) causing the
steering pipeline resources usage and degrading the overall
packet processing rate. For example, with inserted flow to
drop all the ingress packets the statistics reported only
15Mpps of 64B packets were received over 100Gbps line.

Since the Direct Rule API for E-Switch was introduced the
rdma-core supports the dedicated drop action, that is recognized
both for DV and DR and can be used for the entire device in
unified fashion, regardless of steering domain. The similar drop
action was introduced for E-Switch, the usage of this one can be
extended for other steering domains, not for E-Switch's one only.

This patch:
- renames esw_drop_action to dr_drop_action to emphasize
the global nature of the variable (not only E-Switch domain)
- specifies this global drop action instead of dedicated
drop queue for the DR/DV flows

Fixes: 34fa7c0268e7 ("net/mlx5: add drop action to Direct Verbs E-Switch")
Fixes: 65b3cd0dc39b ("net/mlx5: create global drop action")
Cc: stable@dpdk.org

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: probe host PF representor with sub-function

To simplify BlueField HPF representor(vf[-1]) probe, this patch allows
probe it with "sf" syntax: "sf[-1]".

Signed-off-by: Xueming Li <xuemingl

net/mlx5: probe host PF representor with sub-function

To simplify BlueField HPF representor(vf[-1]) probe, this patch allows
probe it with "sf" syntax: "sf[-1]".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: save bonding member ports information

Since kernel bonding netdev doesn't provide statistics counter that
reflects all member ports, PMD has to manually summarize counters from
each member

net/mlx5: save bonding member ports information

Since kernel bonding netdev doesn't provide statistics counter that
reflects all member ports, PMD has to manually summarize counters from
each member ports.

As a preparation, this patch collects bonding member port information
and saves to shared context data.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support list of representor PF

To probe representors from different kernel bonding PFs, had to specify
2 separate devargs like this:
-a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,repres

net/mlx5: support list of representor PF

To probe representors from different kernel bonding PFs, had to specify
2 separate devargs like this:
-a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3]

This patch supports range or list of PF section in devargs, so the
alternative short devargs of above is:
-a 03:00.0,representor=pf[0-1]vf[0-3]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: refactor bonding representor probing

To probe representor on 2nd PF of kernel bonding device, had to specify
PF1 BDF in devarg:
<PF1_BDF>,representor=0
When closing bonding device, all r

net/mlx5: refactor bonding representor probing

To probe representor on 2nd PF of kernel bonding device, had to specify
PF1 BDF in devarg:
<PF1_BDF>,representor=0
When closing bonding device, all representors had to be closed together
and this implies all representors have to use primary PF of bonding
device. So after probing representor port on 2nd PF, when locating new
probed device using device argument, the filter used 2nd PF as PCI
address and failed to locate new device.

Conflict happened by using current representor devargs:
- Use PCI BDF to specify representor owner PF
- Use PCI BDF to locate probed representor device.
- PMD uses primary PCI BDF as PCI device.

To resolve such conflicts, new representor syntax is introduced here:
<primary BDF>,representor=pfXvfY
All representors must use primary PF as owner PCI device, PMD internally
locate owner PCI address by checking representor "pfX" part. To EAL, all
representors are registered to primary PCI device, the 2nd PF is hidden
to EAL, thus all search should be consistent.

Same to VF representor, HPF (host PF on BlueField) uses same syntax to
probe, example: representor=pf1vf[0-3,-1]

This patch also adds pf index into kernel bonding representor port name:
<BDF>_<ib_name>_representor_pf<X>vf<Y>

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: revert setting bonding representor to first PF

With kernel bonding, representors on second PF are being probed by
devargs:
<primary_bdf>,representor=pf1vf<N>
No need to save primary PF po

net/mlx5: revert setting bonding representor to first PF

With kernel bonding, representors on second PF are being probed by
devargs:
<primary_bdf>,representor=pf1vf<N>
No need to save primary PF port ID and lookup when probing sibling
ports, revert patch [1]

[1]:
commit e6818853c022 ("net/mlx5: set representor to first PF in bonding mode")

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support sub-function representor

This patch adds support for SF representor. Similar to VF representor,
switch port name of SF representor in phys_port_name sysfs key is
"pf<x>sf<y>".

Dev

net/mlx5: support sub-function representor

This patch adds support for SF representor. Similar to VF representor,
switch port name of SF representor in phys_port_name sysfs key is
"pf<x>sf<y>".

Device representor argument is "representors=sf[list]", list member
could be mix of instance and range. Example:
representors=sf[0,2,4,8-12,-1]

To probe VF representor and SF representor, need to separate into 2
devices:
-a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list]

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: support timestamp format

This patch adds support for the timestamp format settings for
the receive and send queues. If the firmware version x.30.1000
or above is installed and the NIC time

net/mlx5: support timestamp format

This patch adds support for the timestamp format settings for
the receive and send queues. If the firmware version x.30.1000
or above is installed and the NIC timestamps are configured
with the real-time format, the default zero values for newly
added fields cause the queue creation to fail.

The patch queries the timestamp formats supported by the hardware
and sets the configuration values in queue context accordingly.

Fixes: 86fc67fc9315 ("net/mlx5: create advanced RxQ object via DevX")
Fixes: ae18a1ae9692 ("net/mlx5: support Tx hairpin queues")
Fixes: 15c3807e86ab ("common/mlx5: support DevX QP operations")
Cc: stable@dpdk.org

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

common/mlx5: remove extra line feed in log messages

The macro DRV_LOG already includes a terminating line feed character
defined in PMD_DRV_LOG_.
The extra line feeds added in some messages are remo

common/mlx5: remove extra line feed in log messages

The macro DRV_LOG already includes a terminating line feed character
defined in PMD_DRV_LOG_.
The extra line feeds added in some messages are removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

ethdev: introduce representor type

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

ethdev: introduce representor type

To support more representor type, this patch introduces representor type
enum. The enum is subject to be extended to support new representor in
patches upcoming.

For each devarg structure, only one type supported.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>

show more ...

net/mlx5: fix imissed statistics

The imissed port statistic counts packets that were dropped by the
device Rx queues.

In mlx5, the imissed counter summarizes 2 counters:
- packets dropped by the S

net/mlx5: fix imissed statistics

The imissed port statistic counts packets that were dropped by the
device Rx queues.

In mlx5, the imissed counter summarizes 2 counters:
- packets dropped by the SW queue handling counted by SW.
- packets dropped by the HW queues due to "out of buffer" events
detected when no SW buffer is available for the incoming
packets.

There is HW counter object that should be created per device, and all
the Rx queues should be assigned to this counter in configuration time.

This part was missed when the Rx queues were created by DevX what
remained the "out of buffer" counter clean forever in this case.

Add 2 options to assign the DevX Rx queues to queue counter:
- Create queue counter per device by DevX and assign all the
queues to it.
- Query the kernel counter and assign all the queues to it.

Use the first option by default and if it is failed, fallback to the
second option.

Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues")
Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX")
Cc: stable@dpdk.org

Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: check FW miniCQE format capabilities

miniCQE formats for Flow Tag and L3/L4 Header compression are only
supported by Mellanox FW starting version 16.29.392. There is no
point to allow user

net/mlx5: check FW miniCQE format capabilities

miniCQE formats for Flow Tag and L3/L4 Header compression are only
supported by Mellanox FW starting version 16.29.392. There is no
point to allow user to enable these formats if FW cannot provide them.
Check FW capabilities and deny user requests if the selected miniCQE
format is not supported by an underlying NIC.

Fixes: 54c2d46b160f ("net/mlx5: support flow tag and packet header miniCQEs")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

net/mlx5: update flow meter capability flags names

Existing names of the flags denoting flow meter capability are unclear
and may be misleading.

This patch updates the names to align with the updat

net/mlx5: update flow meter capability flags names

Existing names of the flags denoting flow meter capability are unclear
and may be misleading.

This patch updates the names to align with the updated documentation.
Comments were edited, describing the names clearly.

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix multi-process port ID

The device port_id is used for inter-process communication and must
be the same both for primary and secondary process

This IPC port_id was configured with the i

net/mlx5: fix multi-process port ID

The device port_id is used for inter-process communication and must
be the same both for primary and secondary process

This IPC port_id was configured with the invalid temporary value in
port spawn routine. This temporary value was used by the function
rte_eth_dev_get_port_by_name() to check whether the port exists.

This commit corrects the mp port_id with rte_eth_dev port_id.

Fixes: 2eb4d0107acc ("net/mlx5: refactor PCI probing on Linux")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

show more ...

ethdev: make driver-only headers private

The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.

Signed-off-by:

ethdev: make driver-only headers private

The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Steven Webster <steven.webster@windriver.com>

show more ...

net/mlx5: remove CQE padding device argument

The data-path code doesn't take care on 'rxq_cqe_pad_en' and use padded
CQE for any case when the system cache-line size is 128B.

This makes the argumen

net/mlx5: remove CQE padding device argument

The data-path code doesn't take care on 'rxq_cqe_pad_en' and use padded
CQE for any case when the system cache-line size is 128B.

This makes the argument redundant.

Remove it.

Fixes: bc91e8db12cd ("net/mlx5: add 128B padding of Rx completion entry")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix device name size on Windows

Windows Devx interface name is the same as device name with
different size then IF_NAMESIZE. To support it MLX5_NAMESIZE
is defined with IF_NAMESIZE value f

net/mlx5: fix device name size on Windows

Windows Devx interface name is the same as device name with
different size then IF_NAMESIZE. To support it MLX5_NAMESIZE
is defined with IF_NAMESIZE value for Linux and MLX5_FS_NAME_MAX
value for Windows.

Fixes: e9c0b96e3526 ("net/mlx5: move Linux ifname function")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: unify operations for all OS

There are three types of eth_dev_ops: primary, secondary and isolate
represented in three callback tables per OS. In this commit the OS
specific eth dev tables

net/mlx5: unify operations for all OS

There are three types of eth_dev_ops: primary, secondary and isolate
represented in three callback tables per OS. In this commit the OS
specific eth dev tables are unified into shared tables in file mlx5.c.
Starting from this commit all operating systems must implement the same
eth dev APIs. In case an OS does not support an API - it can return in
its implementation an error ENOTSUP.

Fixes: 042f5c94fd3a ("net/mlx5: refactor device operations for Linux")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: prepare MR prototypes for DevX

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well. Rename 'st

net/mlx5: prepare MR prototypes for DevX

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well. Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: extend device attributes getter

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir M

net/mlx5: extend device attributes getter

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: optimize hash list entry memory

Currently, the hash list saves the hash key in the hash entry. And the
key is mostly used to get the bucket index only.

Save the entire 64 bits key to the

net/mlx5: optimize hash list entry memory

Currently, the hash list saves the hash key in the hash entry. And the
key is mostly used to get the bucket index only.

Save the entire 64 bits key to the entry will not be a good option if
the key is only used to get the bucket index. Since 64 bits costs more
memory for the entry, mostly the signature data in the key only uses
32 bits. And in the unregister function, the key in the entry causes
extra bucket index calculation.

This commit saves the bucket index to the entry instead of the hash key.
For the hash list like table, tag and mreg_copy which save the signature
data in the key, the signature data is moved to the resource data struct
itself.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix Verbs memory allocation callback

The rdma-core library uses callbacks to allocate and free memory
from DPDK. The memory allocation callback used the complicated
and incorrect way to ge

net/mlx5: fix Verbs memory allocation callback

The rdma-core library uses callbacks to allocate and free memory
from DPDK. The memory allocation callback used the complicated
and incorrect way to get the NUMA socket ID from the context.
The context was wrong that might result in wrong socket ID
and allocating memory from wrong node.

The callbacks are assigned once as Infinibande device context
is created allowing early access to shared DPDK memory for all
Verbs internal objects need that.

Fixes: 36dabcea78f0 ("net/mlx5: use anonymous Direct Verbs allocator argument")
Fixes: 2eb4d0107acc ("net/mlx5: refactor PCI probing on Linux")
Fixes: 17e19bc4dde7 ("net/mlx5: add IB shared context alloc/free functions")
Cc: stable@dpdk.org

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

common/mlx5: move to formal ASO action API

Existing code uses the previous API offered by rdma-core in order
to create ASO Flow Hit action.

A general API is now formally released, to create ASO act

common/mlx5: move to formal ASO action API

Existing code uses the previous API offered by rdma-core in order
to create ASO Flow Hit action.

A general API is now formally released, to create ASO action of any
type. This patch moves the MLX5 PMD code to use the formal API.

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...

net/mlx5: fix input register for ASO object

Existing code uses the hard-coded value REG_C_5 as input for function
mlx5dv_dr_action_create_flow_hit().

This patch updates function mlx5_flow_get_reg_i

net/mlx5: fix input register for ASO object

Existing code uses the hard-coded value REG_C_5 as input for function
mlx5dv_dr_action_create_flow_hit().

This patch updates function mlx5_flow_get_reg_id() to return the
selected REG_C value for ASO Flow Hit operation.
The returned value is used, after reducing offset REG_C_0, as input
for function mlx5dv_dr_action_create_flow_hit().

Fixes: f935ed4b645a ("net/mlx5: support flow hit action for aging")

Signed-off-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>

show more ...