ethdev: add flow rule by index with pattern

Add a new API to enqueue flow rule creation by index with pattern.
The new template table rules insertion type,
index-based insertion with pattern, requir

ethdev: add flow rule by index with pattern

Add a new API to enqueue flow rule creation by index with pattern.
The new template table rules insertion type,
index-based insertion with pattern, requires a new flow rule creation
function with both rule index and pattern provided.
Packets will match on the provided pattern at the provided index.

In testpmd, allow to specify both the rule index and the pattern
in the flow rule creation command line parameters.
Both are needed for rte_flow_async_create_by_index_with_pattern().

flow queue 0 create 0 template_table 2 rule_index 5
pattern_template 0 actions_template 0 postpone no pattern eth / end
actions count / queue index 1 / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

lib: move alignment attribute on types for MSVC

The current location used for __rte_aligned(a) for alignment of types
is not compatible with MSVC. There is only a single location accepted
by both to

lib: move alignment attribute on types for MSVC

The current location used for __rte_aligned(a) for alignment of types
is not compatible with MSVC. There is only a single location accepted
by both toolchains.

The standard offers no alignment facility that compatibly interoperates
with C and C++ but it may be achieved by relocating the placement of
__rte_aligned(a) to the aforementioned location accepted by all currently
supported toolchains.

To allow alignment for both compilers, do the following:

* Expand __rte_aligned(a) to __declspec(align(a)) when building
with MSVC.

* Move __rte_aligned from the end of {struct,union} definitions to
be between {struct,union} and tag.

The placement between {struct,union} and the tag allows the desired
alignment to be imparted on the type regardless of the toolchain being
used for all of GCC, LLVM, MSVC compilers building both C and C++.

Note: this move has an additional benefit as Doxygen is not confused
anymore like for the rte_event_vector struct definition.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

ethdev: add template table resize

Template table creation API sets table flows capacity.
If application needs more flows then the table was designed for,
the following procedures must be completed:

ethdev: add template table resize

Template table creation API sets table flows capacity.
If application needs more flows then the table was designed for,
the following procedures must be completed:
1. Create a new template table with larger flows capacity.
2. Re-create existing flows in the new table and delete flows from
the original table.
3. Destroy original table.

Application cannot always execute that procedure:
* Port may not have sufficient resources to allocate a new table
while maintaining original table.
* Application may not have existing flows "recipes" to re-create
flows in a new table.

The patch defines a new API that allows application to resize
existing template table:

* Resizable template table must be created with the
RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE bit set.

* Application resizes existing table with the
`rte_flow_template_table_resize()` function call.
The table resize procedure updates the table maximal flow number
only. Other table attributes are not affected by the table resize.
** The table resize procedure must not interrupt
existing table flows operations in hardware.
** The table resize procedure must not alter flow handles held by
application.

* After `rte_flow_template_table_resize()` returned, application must
update table flow rules by calling
`rte_flow_async_update_resized()`.
The call reconfigures internal flow resources for the new table
configuration.
The flow update must not interrupt hardware flow operations.

* After table flows were updated, application must call
`rte_flow_template_table_resize_complete()`.
The function releases PMD resources related to the original
table.
Application can start new table resize after
`rte_flow_template_table_resize_complete()` returned.

Testpmd commands:

* Create resizable template table
flow template_table <port-id> create table_id <tbl-id> resizable \
[transfer|ingress|egres] group <group-id> \
rules_number <initial table capacity> \
pattern_template <pt1> [ pattern_template <pt2> [ ... ]] \
actions_template <at1> [ actions_template <at2> [ ... ]]

* Resize table:
flow template_table <tbl-id> resize table_resize_id <tbl-id> \
table_resize_rules_num <new table capacity>

* Queue a flow update:
flow queue <port-id> update_resized <tbl-id> rule <flow-id>

* Complete table resize:
flow template_table <port-id> resize_complete table <tbl-id>

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

ethdev: add encapsulation hash calculation

During encapsulation of a packet, it is possible to change some
outer headers to improve flow distribution.
For example, from VXLAN RFC:
"It is recommended

ethdev: add encapsulation hash calculation

During encapsulation of a packet, it is possible to change some
outer headers to improve flow distribution.
For example, from VXLAN RFC:
"It is recommended that the UDP source port number
be calculated using a hash of fields from the inner packet --
one example being a hash of the inner Ethernet frame's headers.
This is to enable a level of entropy for the ECMP/load-balancing"

The tunnel protocol defines which outer field should hold this hash,
but it doesn't define the hash calculation algorithm.

An application that uses flow offloads gets the first few packets
(exception path) and then decides to offload the flow.
As a result, there are two different paths that a packet from a given
flow may take. SW for the first few packets or HW for the rest.
When the packet goes through the SW, the SW encapsulates the packet
and must use the same hash calculation as the HW will do for
the rest of the packets in this flow.

The new function rte_flow_calc_encap_hash can query the hash value
from the driver for a given packet as if the packet was passed
through the HW.

Testpmd command:
flow hash {port} encap {target field}
pattern {item} [/ {item} [...] ] / end

Testpmd example for VXLAN encapsulation:
flow hash 0 encap hash_field_sport
pattern ipv4 dst is 7.7.7.7 src is 8.8.8.8 /
udp dst is 5678 src is 1234 / end

Signed-off-by: Ori Kam <orika@nvidia.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>

show more ...

ethdev: rework fast path async flow API

This patch reworks the async flow API functions called in data path,
to reduce the overhead during flow operations at the library level.
Main source of the ov

ethdev: rework fast path async flow API

This patch reworks the async flow API functions called in data path,
to reduce the overhead during flow operations at the library level.
Main source of the overhead was indirection and checks done while
ethdev library was fetching rte_flow_ops from a given driver.

This patch introduces rte_flow_fp_ops struct which holds callbacks
to driver's implementation of fast path async flow API functions.
Each driver implementing these functions must populate flow_fp_ops
field inside rte_eth_dev structure with a reference to
its own implementation.
By default, ethdev library provides dummy callbacks with
implementations returning ENOSYS.
Such design provides a few assumptions:

- rte_flow_fp_ops struct for given port is always available.
- Each callback is either:
- Default provided by library.
- Set up by driver.

As a result, no checks for availability of the implementation
are needed at library level in data path.
Any library-level validation checks in async flow API are compiled
if and only if RTE_FLOW_DEBUG macro is defined.

This design was based on changes in ethdev library introduced in [1].

These changes apply only to the following API functions:

- rte_flow_async_create()
- rte_flow_async_create_by_index()
- rte_flow_async_actions_update()
- rte_flow_async_destroy()
- rte_flow_push()
- rte_flow_pull()
- rte_flow_async_action_handle_create()
- rte_flow_async_action_handle_destroy()
- rte_flow_async_action_handle_update()
- rte_flow_async_action_handle_query()
- rte_flow_async_action_handle_query_update()
- rte_flow_async_action_list_handle_create()
- rte_flow_async_action_list_handle_destroy()
- rte_flow_async_action_list_handle_query_update()

This patch also adjusts the mlx5 PMD to the introduced flow API changes.

[1]
commit c87d435a4d79 ("ethdev: copy fast-path API into separate structure")

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>

show more ...

ethdev: add calculate hash function

rte_flow supports insert by index table, see
commit 60261a005dff ("ethdev: add flow template table insertion type").

Using the above table, the application can c

ethdev: add calculate hash function

rte_flow supports insert by index table, see
commit 60261a005dff ("ethdev: add flow template table insertion type").

Using the above table, the application can create rules
that are based on hash.
For example application can create the following logic in order
to create load balancing:
1. Create insert by index table with 2 rules, that hashes based on dmac
2. Insert to index 0 a rule that sends the traffic to port A.
3. Insert to index 1 a rule that sends the traffic to port B.

Let's also assume that before this table, there is a 5 tuple
match table that jumps to the above table.

So each packet that matches one of the 5 tuple rules is RSSed
to port A or B, based on dmac hash.

The issue arises when there is a miss on the 5 tuple table,
which resulted due to the packet being the first packet of this flow, or
fragmented packet or any other reason.
In this case, the application must calculate what would be the
hash calculated by the HW so it can send the packet to the correct
port.

This new API allows applications to calculate the hash value of a given
packet for a given table.

Signed-off-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: set flow group miss actions

Introduce new group set miss actions API:
rte_flow_group_set_miss_actions().

A group's miss actions are a set of actions to be performed
in case of a miss on a g

ethdev: set flow group miss actions

Introduce new group set miss actions API:
rte_flow_group_set_miss_actions().

A group's miss actions are a set of actions to be performed
in case of a miss on a group, meaning a packet didn't hit any rules
in the group. This API function allows a user to set a group's
miss actions.

Add testpmd CLI interface for the group set miss actions API:

flow group 0 group_id 1 ingress set_miss_actions jump group 3 / end
flow group 0 group_id 1 ingress set_miss_actions end

Signed-off-by: Tomer Shmilovich <tshmilovich@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: advertise flow restore in mbuf

As reported by Ilya [1], unconditionally calling
rte_flow_get_restore_info() impacts an application performance for drivers
that do not provide this ops.
It co

ethdev: advertise flow restore in mbuf

As reported by Ilya [1], unconditionally calling
rte_flow_get_restore_info() impacts an application performance for drivers
that do not provide this ops.
It could also impact processing of packets that require no call to
rte_flow_get_restore_info() at all.

Register a dynamic mbuf flag when an application negotiates tunnel
metadata delivery (calling rte_eth_rx_metadata_negotiate() with
RTE_ETH_RX_METADATA_TUNNEL_ID).

Drivers then advertise that metadata can be extracted by setting this
dynamic flag in each mbuf.

The application then calls rte_flow_get_restore_info() only when required.

Link: http://inbox.dpdk.org/dev/5248c2ca-f2a6-3fb0-38b8-7f659bfa40de@ovn.org/

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Tested-by: Ali Alnubani <alialnu@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: add indirect flow list action

Indirect API creates a shared flow action with unique action handle.
Flow rules can access the shared flow action and resources related to
that action through t

ethdev: add indirect flow list action

Indirect API creates a shared flow action with unique action handle.
Flow rules can access the shared flow action and resources related to
that action through the indirect action handle.
In addition, the API allows to update existing shared flow action
configuration. After the update completes, new action configuration
is available to all flows that reference that shared action.

Indirect actions list expands the indirect action API:
• Indirect action list creates a handle for one or several
flow actions, while legacy indirect action handle references
single action only.
Input flow actions arranged in END terminated list.
• Flow rule can provide rule specific configuration parameters to
existing shared handle.
Updates of flow rule specific configuration will not change the base
action configuration.
Base action configuration was set during the action creation.

Indirect action list handle defines 2 types of resources:
• Mutable handle resource can be changed during handle lifespan.
• Immutable handle resource value is set during handle creation
and cannot be changed.

There are 2 types of mutable indirect handle contexts:
• Action mutable context is always shared between all flows
that referenced indirect actions list handle.
Action mutable context can be changed by explicit invocation
of indirect handle update function.
• Flow mutable context is private to a flow.
Flow mutable context can be updated by indirect list handle
flow rule configuration.

flow 1:
/ indirect handle H conf C1 /
| |
| |
| | flow 2:
| | / indirect handle H conf C2 /
| | | |
| | | |
| | | |
=========================================================
^ | | | |
| | V | V
| ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
| flow mutable flow mutable
| context 1 context 2
| ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
indirect | | |
action | | |
context | V V
| -----------------------------------------------------
| action mutable context
| -----------------------------------------------------
v action immutable context
=========================================================

Indirect action types - immutable, action / flow mutable, are mutually
exclusive and depend on the action definition.
For example:
• Indirect METER_MARK policy is immutable action member and profile is
action mutable action member.
• Indirect METER_MARK flow action defines init_color as flow mutable
member.
• Indirect QUOTA flow action does not define flow mutable members.

If indirect list handle was created from a list of actions
A1 / A2 ... An / END
indirect list flow action can update Ai flow mutable context in the
action configuration parameter.
Indirect list action configuration is and array [C1, C2, .., Cn]
where Ci corresponds to Ai in the action handle source.
Ci configuration element points Ai flow mutable update, or it's NULL
if Ai has no flow mutable update.
Indirect list action configuration can be NULL if the action
has no flow mutable updates.

Template API:

Action template format:

template .. indirect_list handle Htmpl conf Ctmpl ..
mask .. indirect_list handle Hmask conf Cmask ..

1 If Htmpl was masked (Hmask != 0), it will be fixed in that template.
Otherwise, indirect action value is set in a flow rule.

2 If Htmpl and Ctmpl[i] were masked (Hmask !=0 and Cmask[i] != 0),
Htmpl's Ai action flow mutable context fill be updated to
Ctmpl[i] values and will be fixed in that template.

Flow rule format:

actions .. indirect_list handle Hflow conf Cflow ..

3 If Htmpl was not masked in actions template, Hflow references an
action of the same type as Htmpl.

4 Cflow[i] updates handle's Ai flow mutable configuration if
the Ci was not masked in action template.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: add flow rule actions update

Introduce the new rte_flow_actions_update() API allowing users
to update the action list in the already existing rule.
Flow rules can be updated now without the

ethdev: add flow rule actions update

Introduce the new rte_flow_actions_update() API allowing users
to update the action list in the already existing rule.
Flow rules can be updated now without the need to destroy
the rule first and create a new one instead.
A single API call ensures that no packets are lost by
guaranteeing atomicity and flow state correctness.
The rte_flow_async_actions_update() is added as well.
The matcher is not updated, only the action list is.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: add atomic flow query and update

Current API allows either query or update indirect flow action.
If indirect action must be conditionally updated according to it's
present state application

ethdev: add atomic flow query and update

Current API allows either query or update indirect flow action.
If indirect action must be conditionally updated according to it's
present state application must first issue action query then
analyze returned data and if needed issue update request.
When the update will be processed, action state can change and
the update can invalidate the action.

Add `rte_flow_action_handle_query_update` function call,
and it's async version `rte_flow_async_action_handle_query_update`
to atomically query and update flow action.

Application can control query and update order, if that is supported
by port hardware, by setting `qu_mode` parameter to
RTE_FLOW_QU_QUERY_FIRST or RTE_FLOW_QU_UPDATE_FIRST.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: add flow template table insertion type

Allow user to specify insertion type used in template tables.
The insertion type is responsible for choosing the appropriate key
value used to map inse

ethdev: add flow template table insertion type

Allow user to specify insertion type used in template tables.
The insertion type is responsible for choosing the appropriate key
value used to map inserted flow rules into a template table.

Flow rules can be inserted by calculating the hash value for
the pattern or inserted by index via the new create_by_index() API.
The idea of the index-based insertion is to avoid additional matches
and simply execute predefined actions after jumping to the index.

This is how the regular pattern-based table works:
1. The hash is calculated on a 5-tuple of a packet.
2. The corresponding entry in the table is checked for collisions.
3. Actions are executed once the final entry is found.
The index-based table skips any lookups for the packet:
1. The index value is taken from a specified field.
2. Actions are executed at the specified index in the table.

The insertion into an already occupied index results in an error.
The old rule must be destroyed first. An index cannot be bigger than
the size of the table, otherwise, the rule is rejected as well.

Add testpmd CLI interface for specifying a template table insertion type.
Available types are: pattern and index.
flow template_table 0 create table_id 0 insertion_type index ...
Allow specifying the rule index instead of the pattern template index:
flow queue 0 create 0 template_table 0 rule_index 5 actions_template 0 ...

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: add queue-based API to report aged flow rules

When application use queue-based flow rule management and operate the
same flow rule on the same queue, e.g create/destroy/query, API of
queryin

ethdev: add queue-based API to report aged flow rules

When application use queue-based flow rule management and operate the
same flow rule on the same queue, e.g create/destroy/query, API of
querying aged flow rules should also have queue id parameter just like
other queue-based flow APIs.

By this way, PMD can work in more optimized way since resources are
isolated by queue and needn't synchronize.

If application do use queue-based flow management but configure port
without RTE_FLOW_PORT_FLAG_STRICT_QUEUE, which means application operate
a given flow rule on different queues, the queue id parameter will
be ignored.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

ethdev: add indirect action async query

As rte_flow_action_handle_create/destroy/update() have their own
asynchronous rte_flow_async_action_handle_create/destroy/update()
version functions to accele

ethdev: add indirect action async query

As rte_flow_action_handle_create/destroy/update() have their own
asynchronous rte_flow_async_action_handle_create/destroy/update()
version functions to accelerate the indirect action operations in
queue based flow engine. Currently, the asynchronous version query
function for indirect action was missing.

Add rte_flow_async_action_handle_query() function corresponding
to rte_flow_action_handle_query(). The new asynchronous version
function enables enqueue the query to the hardware similar as
asynchronous flow management does and returns immediately to free
the CPU for other tasks. Application can get the query results from
rte_flow_pull() when the hardware completes its work.

Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: bring in async indirect actions operations

Queue-based flow rules management mechanism is suitable
not only for flow rules creation/destruction, but also
for speeding up other types of Flow

ethdev: bring in async indirect actions operations

Queue-based flow rules management mechanism is suitable
not only for flow rules creation/destruction, but also
for speeding up other types of Flow API management.
Indirect action object operations may be executed
asynchronously as well. Provide async versions for all
indirect action operations, namely:
rte_flow_async_action_handle_create,
rte_flow_async_action_handle_destroy and
rte_flow_async_action_handle_update.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

ethdev: bring in async queue-based flow rules operations

A new, faster, queue-based flow rules management mechanism is needed for
applications offloading rules inside the datapath. This asynchronous

ethdev: bring in async queue-based flow rules operations

A new, faster, queue-based flow rules management mechanism is needed for
applications offloading rules inside the datapath. This asynchronous
and lockless mechanism frees the CPU for further packet processing and
reduces the performance impact of the flow rules creation/destruction
on the datapath. Note that queues are not thread-safe and the queue
should be accessed from the same thread for all queue operations.
It is the responsibility of the app to sync the queue functions in case
of multi-threaded access to the same queue.

The rte_flow_async_create() function enqueues a flow creation to the
requested queue. It benefits from already configured resources and sets
unique values on top of item and action templates. A flow rule is enqueued
on the specified flow queue and offloaded asynchronously to the hardware.
The function returns immediately to spare CPU for further packet
processing. The application must invoke the rte_flow_pull() function
to complete the flow rule operation offloading, to clear the queue, and to
receive the operation status. The rte_flow_async_destroy() function
enqueues a flow destruction to the requested queue.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

ethdev: add flow item/action templates

Treating every single flow rule as a completely independent and separate
entity negatively impacts the flow rules insertion rate. Oftentimes in an
application,

ethdev: add flow item/action templates

Treating every single flow rule as a completely independent and separate
entity negatively impacts the flow rules insertion rate. Oftentimes in an
application, many flow rules share a common structure (the same item mask
and/or action list) so they can be grouped and classified together.
This knowledge may be used as a source of optimization by a PMD/HW.

The pattern template defines common matching fields (the item mask) without
values. The actions template holds a list of action types that will be used
together in the same rule. The specific values for items and actions will
be given only during the rule creation.

A table combines pattern and actions templates along with shared flow rule
attributes (group ID, priority and traffic direction). This way a PMD/HW
can prepare all the resources needed for efficient flow rules creation in
the datapath. To avoid any hiccups due to memory reallocation, the maximum
number of flow rules is defined at the table creation time.

The flow rule creation is done by selecting a table, a pattern template
and an actions template (which are bound to the table), and setting unique
values for the items and actions.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

ethdev: introduce flow engine configuration

The flow rules creation/destruction at a large scale incurs a performance
penalty and may negatively impact the packet processing when used
as part of the

ethdev: introduce flow engine configuration

The flow rules creation/destruction at a large scale incurs a performance
penalty and may negatively impact the packet processing when used
as part of the datapath logic. This is mainly because software/hardware
resources are allocated and prepared during the flow rule creation.

In order to optimize the insertion rate, PMD may use some hints provided
by the application at the initialization phase. The rte_flow_configure()
function allows to pre-allocate all the needed resources beforehand.
These resources can be used at a later stage without costly allocations.
Every PMD may use only the subset of hints and ignore unused ones or
fail in case the requested configuration is not supported.

The rte_flow_info_get() is available to retrieve the information about
supported pre-configurable resources. Both these functions must be called
before any other usage of the flow API engine.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

ethdev: introduce configurable flexible item

1. Introduction and Retrospective

Nowadays the networks are evolving fast and wide, the network
structures are getting more and more complicated, the ne

ethdev: introduce configurable flexible item

1. Introduction and Retrospective

Nowadays the networks are evolving fast and wide, the network
structures are getting more and more complicated, the new
application areas are emerging. To address these challenges
the new network protocols are continuously being developed,
considered by technical communities, adopted by industry and,
eventually implemented in hardware and software. The DPDK
framework follows the common trends and if we bother
to glance at the RTE Flow API header we see the multiple
new items were introduced during the last years since
the initial release.

The new protocol adoption and implementation process is
not straightforward and takes time, the new protocol passes
development, consideration, adoption, and implementation
phases. The industry tries to mitigate and address the
forthcoming network protocols, for example, many hardware
vendors are implementing flexible and configurable network
protocol parsers. As DPDK developers, could we anticipate
the near future in the same fashion and introduce the similar
flexibility in RTE Flow API?

Let's check what we already have merged in our project, and
we see the nice raw item (rte_flow_item_raw). At the first
glance, it looks superior and we can try to implement a flow
matching on the header of some relatively new tunnel protocol,
say on the GENEVE header with variable length options. And,
under further consideration, we run into the raw item
limitations:

- only fixed size network header can be represented
- the entire network header pattern of fixed format
(header field offsets are fixed) must be provided
- the search for patterns is not robust (the wrong matches
might be triggered), and actually is not supported
by existing PMDs
- no explicitly specified relations with preceding
and following items
- no tunnel hint support

As the result, implementing the support for tunnel protocols
like aforementioned GENEVE with variable extra protocol option
with flow raw item becomes very complicated and would require
multiple flows and multiple raw items chained in the same
flow (by the way, there is no support found for chained raw
items in implemented drivers).

This RFC introduces the dedicated flex item (rte_flow_item_flex)
to handle matches with existing and new network protocol headers
in a unified fashion.

2. Flex Item Life Cycle

Let's assume there are the requirements to support the new
network protocol with RTE Flows. What is given within protocol
specification:

- header format
- header length, (can be variable, depending on options)
- potential presence of extra options following or included
in the header the header
- the relations with preceding protocols. For example,
the GENEVE follows UDP, eCPRI can follow either UDP
or L2 header
- the relations with following protocols. For example,
the next layer after tunnel header can be L2 or L3
- whether the new protocol is a tunnel and the header
is a splitting point between outer and inner layers

The supposed way to operate with flex item:

- application defines the header structures according to
protocol specification

- application calls rte_flow_flex_item_create() with desired
configuration according to the protocol specification, it
creates the flex item object over specified ethernet device
and prepares PMD and underlying hardware to handle flex
item. On item creation call PMD backing the specified
ethernet device returns the opaque handle identifying
the object has been created

- application uses the rte_flow_item_flex with obtained handle
in the flows, the values/masks to match with fields in the
header are specified in the flex item per flow as for regular
items (except that pattern buffer combines all fields)

- flows with flex items match with packets in a regular fashion,
the values and masks for the new protocol header match are
taken from the flex items in the flows

- application destroys flows with flex items

- application calls rte_flow_flex_item_release() as part of
ethernet device API and destroys the flex item object in
PMD and releases the engaged hardware resources

3. Flex Item Structure

The flex item structure is intended to be used as part of the flow
pattern like regular RTE flow items and provides the mask and
value to match with fields of the protocol item was configured
for.

struct rte_flow_item_flex {
void *handle;
uint32_t length;
const uint8_t* pattern;
};

The handle is some opaque object maintained on per device basis
by underlying driver.

The protocol header fields are considered as bit fields, all
offsets and widths are expressed in bits. The pattern is the
buffer containing the bit concatenation of all the fields
presented at item configuration time, in the same order and
same amount. If byte boundary alignment is needed an application
can use a dummy type field, this is just some kind of gap filler.

The length field specifies the pattern buffer length in bytes
and is needed to allow rte_flow_copy() operations. The approach
of multiple pattern pointers and lengths (per field) was
considered and found clumsy - it seems to be much suitable for
the application to maintain the single structure within the
single pattern buffer.

4. Flex Item Configuration

The flex item configuration consists of the following parts:

- header field descriptors:
- next header
- next protocol
- sample to match
- input link descriptors
- output link descriptors

The field descriptors tell the driver and hardware what data should
be extracted from the packet and then control the packet handling
in the flow engine. Besides this, sample fields can be presented
to match with patterns in the flows. Each field is a bit pattern.
It has width, offset from the header beginning, mode of offset
calculation, and offset related parameters.

The next header field is special, no data are actually taken
from the packet, but its offset is used as a pointer to the next
header in the packet, in other words the next header offset
specifies the size of the header being parsed by flex item.

There is one more special field - next protocol, it specifies
where the next protocol identifier is contained and packet data
sampled from this field will be used to determine the next
protocol header type to continue packet parsing. The next
protocol field is like eth_type field in MAC2, or proto field
in IPv4/v6 headers.

The sample fields are used to represent the data be sampled
from the packet and then matched with established flows.

There are several methods supposed to calculate field offset
in runtime depending on configuration and packet content:

- FIELD_MODE_FIXED - fixed offset. The bit offset from
header beginning is permanent and defined by field_base
configuration parameter.

- FIELD_MODE_OFFSET - the field bit offset is extracted
from other header field (indirect offset field). The
resulting field offset to match is calculated from as:

field_base + (*offset_base & offset_mask) << offset_shift

This mode is useful to sample some extra options following
the main header with field containing main header length.
Also, this mode can be used to calculate offset to the
next protocol header, for example - IPv4 header contains
the 4-bit field with IPv4 header length expressed in dwords.
One more example - this mode would allow us to skip GENEVE
header variable length options.

- FIELD_MODE_BITMASK - the field bit offset is extracted
from other header field (indirect offset field), the latter
is considered as bitmask containing some number of one bits,
the resulting field offset to match is calculated as:

field_base + bitcount(*offset_base & offset_mask) << offset_shift

This mode would be useful to skip the GTP header and its
extra options with specified flags.

- FIELD_MODE_DUMMY - dummy field, optionally used for byte
boundary alignment in pattern. Pattern mask and data are
ignored in the match. All configuration parameters besides
field size and offset are ignored.

Note: "*" - means the indirect field offset is calculated
and actual data are extracted from the packet by this
offset (like data are fetched by pointer *p from memory).

The offset mode list can be extended by vendors according to
hardware supported options.

The input link configuration section tells the driver after
what protocols and at what conditions the flex item can follow.
Input link specified the preceding header pattern, for example
for GENEVE it can be UDP item specifying match on destination
port with value 6081. The flex item can follow multiple header
types and multiple input links should be specified. At flow
creation time the item with one of the input link types should
precede the flex item and driver will select the correct flex
item settings, depending on the actual flow pattern.

The output link configuration section tells the driver how
to continue packet parsing after the flex item protocol.
If multiple protocols can follow the flex item header the
flex item should contain the field with the next protocol
identifier and the parsing will be continued depending
on the data contained in this field in the actual packet.

The flex item fields can participate in RSS hash calculation,
the dedicated flag is present in the field description to specify
what fields should be provided for hashing.

5. Flex Item Chaining

If there are multiple protocols supposed to be supported with
flex items in chained fashion - two or more flex items within
the same flow and these ones might be neighbors in the pattern,
it means the flex items are mutual referencing. In this case,
the item that occurred first should be created with empty
output link list or with the list including existing items,
and then the second flex item should be created referencing
the first flex item as input arc, drivers should adjust
the item configuration.

Also, the hardware resources used by flex items to handle
the packet can be limited. If there are multiple flex items
that are supposed to be used within the same flow it would
be nice to provide some hint for the driver that these two
or more flex items are intended for simultaneous usage.
The fields of items should be assigned with hint indices
and these indices from two or more flex items supposed
to be provided within the same flow should be the same
as well. In other words, the field hint index specifies
the group of fields that can be matched simultaneously
within a single flow. If hint indices are specified,
the driver will try to engage not overlapping hardware
resources and provide independent handling of the field
groups with unique indices. If the hint index is zero
the driver assigns resources on its own.

6. Example of New Protocol Handling

Let's suppose we have the requirements to handle the new tunnel
protocol that follows UDP header with destination port 0xFADE
and is followed by MAC header. Let the new protocol header format
be like this:

struct new_protocol_header {
rte_be32 header_length; /* length in dwords, including options */
rte_be32 specific0; /* some protocol data, no intention */
rte_be32 specific1; /* to match in flows on these fields */
rte_be32 crucial; /* data of interest, match is needed */
rte_be32 options[0]; /* optional protocol data, variable length */
};

The supposed flex item configuration:

struct rte_flow_item_flex_field field0 = {
.field_mode = FIELD_MODE_DUMMY, /* Affects match pattern only */
.field_size = 96, /* three dwords from the beginning */
};
struct rte_flow_item_flex_field field1 = {
.field_mode = FIELD_MODE_FIXED,
.field_size = 32, /* Field size is one dword */
.field_base = 96, /* Skip three dwords from the beginning */
};
struct rte_flow_item_udp spec0 = {
.hdr = {
.dst_port = RTE_BE16(0xFADE),
}
};
struct rte_flow_item_udp mask0 = {
.hdr = {
.dst_port = RTE_BE16(0xFFFF),
}
};
struct rte_flow_item_flex_link link0 = {
.item = {
.type = RTE_FLOW_ITEM_TYPE_UDP,
.spec = &spec0,
.mask = &mask0,
};

struct rte_flow_item_flex_conf conf = {
.next_header = {
.tunnel = FLEX_TUNNEL_MODE_SINGLE,
.field_mode = FIELD_MODE_OFFSET,
.field_base = 0,
.offset_base = 0,
.offset_mask = 0xFFFFFFFF,
.offset_shift = 2 /* Expressed in dwords, shift left by 2 */
},
.sample = {
&field0,
&field1,
},
.nb_samples = 2,
.input_link[0] = &link0,
.nb_inputs = 1
};

Let's suppose we have created the flex item successfully, and PMD
returned the handle 0x123456789A. We can use the following item
pattern to match the crucial field in the packet with value 0x00112233:

struct new_protocol_header spec_pattern =
{
.crucial = RTE_BE32(0x00112233),
};
struct new_protocol_header mask_pattern =
{
.crucial = RTE_BE32(0xFFFFFFFF),
};
struct rte_flow_item_flex spec_flex = {
.handle = 0x123456789A
.length = sizeiof(struct new_protocol_header),
.pattern = &spec_pattern,
};
struct rte_flow_item_flex mask_flex = {
.length = sizeof(struct new_protocol_header),
.pattern = &mask_pattern,
};
struct rte_flow_item item_to_match = {
.type = RTE_FLOW_ITEM_TYPE_FLEX,
.spec = &spec_flex,
.mask = &mask_flex,
};

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: query proxy port to manage transfer flows

Not all DPDK ports in a given switching domain may have the
privilege to manage "transfer" flows. Add an API to find a
port with sufficient privileg

ethdev: query proxy port to manage transfer flows

Not all DPDK ports in a given switching domain may have the
privilege to manage "transfer" flows. Add an API to find a
port with sufficient privileges by any port in the domain.

Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ori Kam <orika@nvidia.com>

show more ...

ethdev: introduce indirect flow action

Right now, rte_flow_shared_action_* APIs are used for some shared
actions, like RSS, count. The shared action should be created before
using it inside a flow.

ethdev: introduce indirect flow action

Right now, rte_flow_shared_action_* APIs are used for some shared
actions, like RSS, count. The shared action should be created before
using it inside a flow. These shared actions sometimes are not
really shared but just some indirect actions decoupled from a flow.

The new functions rte_flow_action_handle_* are added to replace
the current shared functions rte_flow_shared_action_*.

There are two types of flow actions:
1. the direct (normal) actions that could be created and stored
within a flow rule. Such action is tied to its flow rule and
cannot be reused.
2. the indirect action, in the past, named shared_action. It is
created from a direct actioni, like count or rss, and then used
in the flow rules with an object handle. The PMD will take care
of the retrieve from indirect action to the direct action
when it is referenced.

The indirect action is accessed (update / query) w/o any flow rule,
just via the action object handle. For example, when querying or
resetting a counter, it could be done out of any flow using this
counter, but only the handle of the counter action object is
required.
The indirect action object could be shared by different flows or
used by a single flow, depending on the direct action type and
the real-life requirements.
The handle of an indirect action object is opaque and defined in
each driver and possibly different per direct action type.

The old name "shared" is improper in a sense and should be replaced.

Since the APIs are changed from "rte_flow_shared_action*" to the new
"rte_flow_action_handle*", the testpmd application code and command
line interfaces also need to be updated to do the adaption.
The testpmd application user guide is also updated. All the "shared
action" related parts are replaced with "indirect action" to have a
correct explanation.

The parameter of "update" interface is also changed. A general
pointer will replace the rte_flow_action struct pointer due to the
facts:
1. Some action may not support fields updating. In the example of a
counter, the only "update" supported should be the reset. So
passing a rte_flow_action struct pointer is meaningless and
there is even no such corresponding action struct. What's more,
if more than one operations should be supported, for some other
action, such pointer parameter may not meet the need.
2. Some action may need conditional or partial update, the current
parameter will not provide the ability to indicate which part(s)
to update.
For different types of indirect action objects, the pointer could
either be the same of rte_flow_action* struct - in order not to
break the current driver implementation, or some wrapper
structures with bits as masks to indicate which part to be
updated, depending on real needs of the corresponding direct
action. For different direct actions, the structures of indirect
action objects updating will be different.

All the underlayer PMD callbacks will be moved to these new APIs.

The RTE_FLOW_ACTION_TYPE_SHARED is kept for now in order not to
break the ABI. All the implementations are changed by using
RTE_FLOW_ACTION_TYPE_INDIRECT.

Since the APIs are changed from "rte_flow_shared_action*" to the new
"rte_flow_action_handle*" and the "update" interface's 3rd input
parameter is changed to generic pointer, the mlx5 PMD that uses these
APIs needs to do the adaption to the new APIs as well.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Andrey Vesnovaty <andreyv@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
m

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
makes it awkward to add features referring to individual libraries in the
build - should the lib names be specified with or without the prefix.
Therefore, we can just remove the library prefix and use the library's
unique name as the directory name, i.e. 'eal' rather than 'librte_eal'

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...