net/mlx5: fix empty flow error structure

The output flow error parameter is used to indicate the detailed
reason of the failure when calling a rte_flow_* interface. Even
though sometimes the applica

net/mlx5: fix empty flow error structure

The output flow error parameter is used to indicate the detailed
reason of the failure when calling a rte_flow_* interface. Even
though sometimes the application will not check it or use it, the PMD
must fill it in the failure branch before returning. Or else, some
dirty value in the stack, heap will be accessed as a pointer and then
cause a crash.
In this case, when a port is stopped, it is not allowed to insert a
flow from application. The detailed error information should be
filled. If the application needs to check the detailed error reason,
it will get the information but not result in any crash.

Fixes: 40b9e7f65fe1 ("net/mlx5: check device status before creating flow")

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>

show more ...

net/mlx5: fix Rx queue flags on destroying flow

After inserting an offload flow, the software flag information will
be updated based on the flow. When receiving a packet on this queue,
the hardware

net/mlx5: fix Rx queue flags on destroying flow

After inserting an offload flow, the software flag information will
be updated based on the flow. When receiving a packet on this queue,
the hardware packet type bits and the software flag will be used
together to get the inner packet and tunnel header type (if any) from
the global packet type table.
When destroying a flow, the corresponding Rx queue flag needs to be
updated. All flags should be cleared when closing a device because
all control flows and application flows are invalid anymore.
Such behavior is missed when implementing the non-cached mode.

Fixes: 8db7e3b69822 ("net/mlx5: change operations for non-cached flows")

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: set dynamic flow metadata in Rx queues

Using a global mbuf dynamic field for metadata incurs some
performance penalty on a datapath. Store this information in
the Rx queue descriptor for a

net/mlx5: set dynamic flow metadata in Rx queues

Using a global mbuf dynamic field for metadata incurs some
performance penalty on a datapath. Store this information in
the Rx queue descriptor for a better cache locality.

Fixes: a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")
Cc: stable@dpdk.org

Signed-off-by: Alexander Kozyrev <akozyrev@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: fix header modify action validation

The header modify actions number supported now has some limitation,
and it is decided by both driver and hardware. If the configuration
is different or

net/mlx5: fix header modify action validation

The header modify actions number supported now has some limitation,
and it is decided by both driver and hardware. If the configuration
is different or the table to insert the flow is different, the result
might be different if the flow contains header modify actions.
Currently, the actual action number could only be calculated in the
later stage called translate, from user specified value to the driver
format. And the action numbers checking is missed in the flow
validation. So PMD will return incorrect result to indicate the
flow actions are valid by rte_flow_validate but then it will fail
when calling rte_flow_create.

Adding some simple checking in the validation will help to get rid
of this incorrect checking. Most of the actions will only consume 1
SW action field except the MAC address and IPv6 address. And from
SW POV, the maximal action fields for these will be consumed even if
only part of such field will be modified because that there is no
mask in the flow actions and the mask will always be all ONEs.

The metering or extra metadata supports will cost one more action.

Fixes: 9597330c6844 ("net/mlx5: update modify header action translator")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: allocate flow API from indexed pool

This commit allocates rte flow from indexed memory pool.

Allocate rte flow memory from indexed memory pool helps save more than
MALLOC_ELEM_OVERHEAD by

net/mlx5: allocate flow API from indexed pool

This commit allocates rte flow from indexed memory pool.

Allocate rte flow memory from indexed memory pool helps save more than
MALLOC_ELEM_OVERHEAD bytes memory from rte_malloc().

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: optimize flow RSS struct

When destroy the flow with RSS, flow can invoke the queues information
from hrxq index table object, since the queue number and list are both
saved to the index ta

net/mlx5: optimize flow RSS struct

When destroy the flow with RSS, flow can invoke the queues information
from hrxq index table object, since the queue number and list are both
saved to the index table object. No need to save the duplicated data in
rte flow.

Save the RSS description information to the intermediate private data
when create the flow with RSS action helps to save the memory for rte
flow.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: optimize flow director filter memory

This commit is for mlx5 fdir flow memory optimization.

Currently for the fdir member in rte_flow structure. It saves the fdir
memory pointer directly.

net/mlx5: optimize flow director filter memory

This commit is for mlx5 fdir flow memory optimization.

Currently for the fdir member in rte_flow structure. It saves the fdir
memory pointer directly. As fdir is fading away, use one bit help to
indicate the function in the flow and add the content to an extra list
save the memory for the other widely usage cases.

Signed-off-by: Wentao Cui <wentaoc@mellanox.com>
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: convert mark copy resource to indexed

Allocate mark copy resource from indexed pool helps rte flow saves the 4
bytes index instead of 8 bytes pointer. For mark copy resource itself,
it hel

net/mlx5: convert mark copy resource to indexed

Allocate mark copy resource from indexed pool helps rte flow saves the 4
bytes index instead of 8 bytes pointer. For mark copy resource itself,
it helps save MALLOC_ELEM_OVERHEAD bytes from rte_malloc().

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: reorganize flow handle struct

Currently, the mlx5_flow_handle struct is not fully aligned and has some
bits wasted. The members can be optimized and reorganized to save memory.

1. As meta

net/mlx5: reorganize flow handle struct

Currently, the mlx5_flow_handle struct is not fully aligned and has some
bits wasted. The members can be optimized and reorganized to save memory.

1. As metadata and meter is sharing the same flow match id, now the flow
id is limited to 24 bits due to the 8 MSBs are used as for the meter
color. Align the flow id to other bit members to 32 bits to save the
mlx5 flow handle memory.

2. The vlan_vf in struct mlx5_flow_handle_dv was already moved to struct
mlx5_flow_handle. Remove the legacy vlan_vf in struct
mlx5_flow_handle_dv.

3. Reorganize the vlan_vf in mlx5_flow_handle with member SILIST_ENTRY
next to make it align with 8 bytes.

4. Reorganize the header modify in mlx5_flow_handle_dv to ILIST_ENTRY
next to make it align to with bytes.

5. Introduce __rte_pack attribute to make the struct tightly organized.

It will totally save 20 bytes memory for mlx5_flow_handle struct.

For the resource objects which are converted to indexed, align the names
with the prefix of rix_.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: optimize action flags in flow handle

As only limited bits is used in act_flags for flow destroy, it's a bit
expensive to save the whole 64 bits. Move the act_flags out of flow
handle and s

net/mlx5: optimize action flags in flow handle

As only limited bits is used in act_flags for flow destroy, it's a bit
expensive to save the whole 64 bits. Move the act_flags out of flow
handle and save the needed bits for flow destroy to save some bytes for
the flow handle data struct.

The fate action type and mark bits are reserved as they will be used in
flow destroy.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: convert flow dev handle to indexed

This commit converts flow dev handle to indexed.

Change the mlx5 flow handle from pointer to uint32_t saves memory for
flow. With million flow, it saves

net/mlx5: convert flow dev handle to indexed

This commit converts flow dev handle to indexed.

Change the mlx5 flow handle from pointer to uint32_t saves memory for
flow. With million flow, it saves several MBytes memory.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: fix index when creating flow

When creating a flow, usually the creating routine is called in
serial. No parallel execution is supported right now. The same
function will be called only onc

net/mlx5: fix index when creating flow

When creating a flow, usually the creating routine is called in
serial. No parallel execution is supported right now. The same
function will be called only once for a single flow creation.

But there is a special case that the creating routine will be called
nested. If the xmeta feature is enabled and there is FLAG / MARK in
the actions list, some metadata reg copy flow needs to be created
before the original flow is applied to the hardware.
In the flow non-cached mode, resources only for flow creation will
not be saved anymore. The memory space is pre-allocated and reused
for each flow. A global index for each device is used to indicate
the memory address of the resources. If the function is called in a
nested mode, then the index will be reset and make everything get
corrupted.

To solve this, a nested index is introduced to save the position for
the original flow creation. Currently, only one level nested call
of the flow creating routine is supported.

Fixes: e7bfa3596a0a ("net/mlx5: separate the flow handle resource")

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: optimize flow counter handle type

Currently, DV and verbs counters are both changed to indexed. It means
while creating the flow with counter, flow can save the indexed value to
address th

net/mlx5: optimize flow counter handle type

Currently, DV and verbs counters are both changed to indexed. It means
while creating the flow with counter, flow can save the indexed value to
address the counter.

Save the 4 bytes indexed value in the rte_flow instead of 8 bytes
pointer helps to save memory with millions of flows.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: optimize counter release query generation

Query generation was introduced to avoid counter to be reallocated
before the counter statistics be fully updated. Since the counters
be released

net/mlx5: optimize counter release query generation

Query generation was introduced to avoid counter to be reallocated
before the counter statistics be fully updated. Since the counters
be released between query trigger and query handler may miss the
packets arrived in the trigger and handler gap period. In this case,
user can only allocate the counter while pool query_gen is greater
than the counter query_gen + 1 which indicates a new round of query
finished, the statistic is fully updated.

Split the pool query_gen to start_query_gen and end_query_gen helps
to have a better identify for the counter released in the gap period.
And it helps the counter released before query trigger or after query
handler can be reallocated more efficiently.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: check device status before creating flow

By default, flows are categorized into two types of a mlx5 device.
1. The PMD driver will create some default flows to enable the
traffic an

net/mlx5: check device status before creating flow

By default, flows are categorized into two types of a mlx5 device.
1. The PMD driver will create some default flows to enable the
traffic and give some default behaviors on the packets. And
this is transparent to the upper layer application.
2. Other flows will be created in the application based on its
needs.
When in the old cached mode for application flows, it is allowed
to created the flow before the device is started. And when
starting the device, all the flows will be applied to the hardware
and take effect. The cached flows will be also applied in the same
time.
In non-cached mode, all the flows will never be cached when stopping
a device. So it makes no sense to insert any flow into the device
before it is started. Default flows owned by PMD driver are not
affected in this case.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: separate the flow handle resource

Only the members of flow handle structure will be used when trying
to destroy a flow. Other members of mlx5 device flow resource will
only be used for flo

net/mlx5: separate the flow handle resource

Only the members of flow handle structure will be used when trying
to destroy a flow. Other members of mlx5 device flow resource will
only be used for flow creating, and they could be reused for different
flows.
So only the device flow handle structure needs to be saved for further
usage. This could be separated from the whole mlx5 device flow and
stored with a list for each rte flow.
Other members will be pre-allocated with an array, and an index will
be used to help to apply each device flow to the hardware.
The flow handle sizes of Verbs and DV mode will be different, and
some calculation could be done before allocating a verbs handle.
Then the total memory consumption will less for Verbs when there is
no inbox driver being used.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: reorganize flow structures

Common structures used for mlx5 flow creating and destroying are
reorganized in order to separating the parts only for destroying
from all the items.
The "mlx5_f

net/mlx5: reorganize flow structures

Common structures used for mlx5 flow creating and destroying are
reorganized in order to separating the parts only for destroying
from all the items.
The "mlx5_flow" will contain the common items of DV and Verbs flow,
specific items for DV / Verbs only. These items will only be used
when creating a flow.
At the end of "mlx5_flow", a nested structure "mlx5_flow_handle"
located. It contains all the items used both for creating and
destroying a flow. Also, it consists of common items, and DV / Verbs
specific items.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: change operations for non-cached flows

When stopping a mlx5 device, all the flows inserted will be flushed
since they are with non-cached mode. And no more action will be done
for these fl

net/mlx5: change operations for non-cached flows

When stopping a mlx5 device, all the flows inserted will be flushed
since they are with non-cached mode. And no more action will be done
for these flows in the device closing stage.
If the device restarts after stopped, no flow with non-cached mode
will be re-inserted.
The flush operation through rte interface will remain the same, and
all the flows will be flushed actively.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix validation of VXLAN/VXLAN-GPE specs

Trying to create zero spec for vni wasn't allowed, to
avoid matching all packets from previous layer (udp).
This behavior is incorrect, since VXLAN

net/mlx5: fix validation of VXLAN/VXLAN-GPE specs

Trying to create zero spec for vni wasn't allowed, to
avoid matching all packets from previous layer (udp).
This behavior is incorrect, since VXLAN is being identified
through the outer UDP destination port.

Currently, if the user didn't specify outer UDP destination
port the PMD will automatically match only on outer
UDP port of 4798, and if the user want to match on some none
standard port he need to specify it explicitly in the rule.

This removes the limitation of vni spec to be able to match any
vni.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Cc: stable@dpdk.org

Signed-off-by: Raslan Darawsheh <rasland@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix mask used for IPv6 item validation

The nic_mask struct in function mlx5_flow_validate_item_ipv6()
includes hop_limits field.
This is redundant since matching on this item is not suppor

net/mlx5: fix mask used for IPv6 item validation

The nic_mask struct in function mlx5_flow_validate_item_ipv6()
includes hop_limits field.
This is redundant since matching on this item is not supported using
Verbs flow engine (dv_flow_en=0).

(Separate patch will be sent to handle this issue for DV flow engine.)

Setting a rule including match on this field will fail:
testpmd> flow create 0 ingress pattern eth / ipv6 hop is 1 / end actions
drop / end
port_flow_complain(): Caught PMD error type 1 (cause unspecified):
hardware refuses to create flow: Operation not supported

This patch removes the redundant field from nic_mask, to ensure that
such flow rules will be rejected by PMD validation:
testpmd> flow create 0 ingress pattern eth / ipv6 hop is 1 / end actions
drop / end
port_flow_complain(): Caught PMD error type 13 (specific pattern item):
cause: 0x7fffffffa2c8, mask enables non supported bits: Operation not
supported

The related lines in function flow_verbs_translate_item_ipv6() are
removed as well, since they are redundant.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Fixes: 84c406e74524 ("net/mlx5: add flow translate function")
Cc: stable@dpdk.org

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix metadata split with encap action

In order to move the mbuf metadata from the WQE to the FDB steering
domain, the PMD add for each NIC TX flow a new action to copy the
metadata register

net/mlx5: fix metadata split with encap action

In order to move the mbuf metadata from the WQE to the FDB steering
domain, the PMD add for each NIC TX flow a new action to copy the
metadata register REG_A to REG_C_0.

This copy action is considered as modify header action from HW
perspective.

The HW doesn't support to do modify header action after ant
encapsulation action.

The split metadata function wrongly added the copy action in the end of
the original actions list, hence, NIC egress flow with encapsulation
action failed when the PMD worked with dv_xmeta_en mode.

Move the copy action to be before and back to back with the
encapsulation action for the aforementioned case.

Fixes: 71e254bc0294 ("net/mlx5: split Rx flows to provide metadata copy")
Cc: stable@dpdk.org

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...

net/mlx5: fix VLAN actions in meter

Meter suffix subflow only has the port id and tag match item, if VLAN
push and set VLAN id actions exist in the suffix subflow, the user
defined VLAN items is req

net/mlx5: fix VLAN actions in meter

Meter suffix subflow only has the port id and tag match item, if VLAN
push and set VLAN id actions exist in the suffix subflow, the user
defined VLAN items is required for the actions to set a correct VLAN
id.

Currently, the VLAN item stays in the meter prefix subflow. Without
the VLAN item, VLAN id or pcp will not be inherited.

Actions require the VLAN item as below:
RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN
RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID

Add a private VLAN item to copy the user defined VLAN item to the meter
suffix subflow, so the suffix subflow will have the chance to get the
correct original VLAN id and pcp value from the VLAN item.

Fixes: 9ea9b049a960 ("net/mlx5: split meter flow")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix match information in meter

As meter flows are split into three subflows each, the prefix subflow
with meter action color the packet, the meter subflow filters out the
colored packets,

net/mlx5: fix match information in meter

As meter flows are split into three subflows each, the prefix subflow
with meter action color the packet, the meter subflow filters out the
colored packets, the suffix subflow applies all the remaining actions
to the passed packets.

Currently, all the user defined items are matched in the prefix flow.
Flow id tag match item is the only item added to the meter suffix
subflow. Some of the remaining actions to be applied in the suffix
subflow require more information in the match item, or the suffix
subflow will not be created successfully.

Actions require the L3/L4 type in the match items as below:
RTE_FLOW_ACTION_TYPE_SET_TP_SRC
RTE_FLOW_ACTION_TYPE_SET_TP_DST
RTE_FLOW_ACTION_TYPE_DEC_TTL
RTE_FLOW_ACTION_TYPE_SET_TTL
RTE_FLOW_ACTION_TYPE_RSS
RTE_FLOW_ACTION_TYPE_QUEUE

Inherit the match item flags from meter prefix subflow to make actions
in suffix subflow get sufficient information.

Fixes: 9ea9b049a960 ("net/mlx5: split meter flow")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix layer flags missing in metadata

Metadata suffix subflow inherits the RSS needed hash_fields from the
prefix subflow as the suffix subflow only has the tag match item unable
to generate

net/mlx5: fix layer flags missing in metadata

Metadata suffix subflow inherits the RSS needed hash_fields from the
prefix subflow as the suffix subflow only has the tag match item unable
to generate the full original hash_fields for RSS action.

Unfortunately, hash_fields will only be generated if flow has RSS
action. So it means the prefix flow won't generate the hash_fields as
the RSS action has been split to the suffix flow.

Copy the layer flags from prefix subflow to suffix subflow to help the
suffix subflow to generate the correct hash_fields itself.

Fixes: 71e254bc0294 ("net/mlx5: split Rx flows to provide metadata copy")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>

show more ...

net/mlx5: fix L3 VXLAN RSS expansion

The RSS expansion feature was introduced to split RSS flows according to
the adjustment between the RSS types and the flow items.

The expansion function gets an

net/mlx5: fix L3 VXLAN RSS expansion

The RSS expansion feature was introduced to split RSS flows according to
the adjustment between the RSS types and the flow items.

The expansion function gets an item tree for the above adjustment from
the caller which reflects the HW needs.

The standard vxlan header next protocol is always Ethernet while there
are some Mellanox customers who use their own method to allow L3
headers after the vxlan tunnel header.

The expansion tree of mlx5 PMD didn't expect to get L3 headers after the
vxlan header what caused a failure in flow creation when inner RSS is
requested on L3 after vxlan flow.

Add IPV4 and IPV6 as optional headers after vxlan in the RSS expansion
tree to allow L3 tunnel support for vxlan.

Fixes: f4f06e361516 ("net/mlx5: add flow VXLAN item")
Cc: stable@dpdk.org

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>

show more ...