net/ena: support Rx interrupt

In order to support asynchronous Rx in the applications, the driver has
to configure the event file descriptors and configure the HW.

This patch configures appropriate

net/ena: support Rx interrupt

In order to support asynchronous Rx in the applications, the driver has
to configure the event file descriptors and configure the HW.

This patch configures appropriate data structures for the rte_ethdev
layer, adds .rx_queue_intr_enable and .rx_queue_intr_disable API
handlers, and configures IO queues to work in the interrupt mode, if it
was requested by the application.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Artur Rojek <ar@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>

show more ...

net/ena: trigger reset on Tx prepare failure

If the prepare function failed, then it means the descriptors are in the
invalid state.

This condition now triggers the reset, which should be further h

net/ena: trigger reset on Tx prepare failure

If the prepare function failed, then it means the descriptors are in the
invalid state.

This condition now triggers the reset, which should be further handled
by the application.

To notify the application about prepare function failure, the error log
was added. In general, it should never fail in normal conditions, as the
Tx function checks for the available space in the Tx ring before the
preparation even starts.

Fixes: 2081d5e2e92d ("net/ena: add reset routine")
Cc: stable@dpdk.org

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>

show more ...

net/ena: use common debug options

ENA defined its own logger flags for Tx and Rx, but they weren't
technically used anywhere. Those data path loggers weren't used anywhere
after the definition.

Thi

net/ena: use common debug options

ENA defined its own logger flags for Tx and Rx, but they weren't
technically used anywhere. Those data path loggers weren't used anywhere
after the definition.

This commit uses the generic RTE_ETHDEV_DEBUG_RX and RTE_ETHDEV_DEBUG_TX
flags to define PMD_TX_LOG and PMD_RX_LOG which are now being used on
the data path. The PMD_TX_FREE_LOG was removed, as it has no usage in
the current version of the driver.

RTE_ETH_DEBUG_[TR]X now wraps extra checks for the driver state in the
IO path - this saves extra conditionals on the hot path.

ena_com logger is no longer optional (previously it had to be explicitly
enabled by defining this flag: RTE_LIBRTE_ENA_COM_DEBUG). Having this
logger optional makes tracing of ena_com errors much harder.
Due to ena_com design, it's impossible to separate IO path logs
from the management path logs, so for now they will be always enabled.

Default levels for the affected loggers were modified. Hot path loggers
are initialized with the default level of DEBUG instead of NOTICE, as
they have to be explicitly enabled. ena_com logging level was reduced
from NOTICE to WARNING - as it's no longer optional, the driver should
report just a warnings in the ena_com layer.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>

show more ...

net/ena: adjust logs

ENA logs were not consistent regarding the new line character. Few of
them were relying on the new line character added by the PMD_*_LOG
macros, but most were adding the new lin

net/ena: adjust logs

ENA logs were not consistent regarding the new line character. Few of
them were relying on the new line character added by the PMD_*_LOG
macros, but most were adding the new line character by themselves. It
was causing ENA logs to add extra empty line after almost each log.

To unify this behavior, the missing new line characters were added to
the driver logs, and they were removed from the logging macros. After
this patch, every ENA log message should add '\n' at the end.

Moreover, the logging messages were adjusted in terms of wording
(removed unnecessary abbreviations), capitalizing of the words (start
sentences with capital letters, and use 'Tx/Rx' instead of 'tx/TX' etc.

Some of the logs were rephrased to make them more clear for the reader.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shai Brandes <shaibran@amazon.com>

show more ...

net/ena: disable ops not supported by secondary process

For dev_ops not supported by the secondary process, either return -EPERM
or return without doing anything. In both cases log a warning.

It's

net/ena: disable ops not supported by secondary process

For dev_ops not supported by the secondary process, either return -EPERM
or return without doing anything. In both cases log a warning.

It's still application's responsibility to avoid calls like that and
those changes are for debugging/informational purposes.

Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>

show more ...

net/ena: make ethdev references multi-process safe

rte_pci_device and rte_eth_dev are process-local structures. Therefore
ena_adapter::pdev and ena_adapter::rte_dev cannot be used universally.

Both

net/ena: make ethdev references multi-process safe

rte_pci_device and rte_eth_dev are process-local structures. Therefore
ena_adapter::pdev and ena_adapter::rte_dev cannot be used universally.

Both ena_timer_wd_callback and ena_interrupt_handler_rte needs access to
the rte_eth_dev, but as they are being setup and executed in the primary
process, it is safe to pass there the same pointer, which is used for
the device configuration.

In all other cases, except the eth_ena_dev_init(), the rte_eth_dev_data
is used instead.

Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>

show more ...

net/ena: indicate Rx RSS hash presence

To make it possible to the app to determine if the hash was calculated
for the packet or not, the PKT_RX_RSS_HASH should be set in the mbuf's
ol_flags.

As the

net/ena: indicate Rx RSS hash presence

To make it possible to the app to determine if the hash was calculated
for the packet or not, the PKT_RX_RSS_HASH should be set in the mbuf's
ol_flags.

As the PMD wasn't setting that, the application couldn't check if there
is a hash in a proper way.

The hash is valid only if it's UDP or TCP and the IP packet wasn't
fragmented.

Fixes: e5df9f33db00 ("net/ena: fix passing RSS hash to mbuf")
Cc: stable@dpdk.org

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Amit Bernstein <amitbern@amazon.com>

show more ...

net/ena: update version to v2.3.0

This version update contains:
* memcpy mapping to the dpdk-optimized version.
* ena_com (HAL) update to the latest version (from 18.09.2020).
* Bug fixes for

net/ena: update version to v2.3.0

This version update contains:
* memcpy mapping to the dpdk-optimized version.
* ena_com (HAL) update to the latest version (from 18.09.2020).
* Bug fixes for the large LLQ headers and devargs parsing.
* Bug fix for the default ring size.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>

show more ...

net/ena: report default ring size

Remove invalid ring size alignment logic and add default Rx and Tx port
ring sizes to the device info spec.

The logic in lines 1297 and 1371 is invalid. The
RTE_ET

net/ena: report default ring size

Remove invalid ring size alignment logic and add default Rx and Tx port
ring sizes to the device info spec.

The logic in lines 1297 and 1371 is invalid. The
RTE_ETH_DEV_FALLBACK_RX_RINGSIZE (and the TX counterpart) is a value
that rte_eth_rx_queue_setup() will set if
dev_info.default_rxportconf.ring_size is 0 and user provided 0 in
nb_rx_desc argument. However the current code treats it as a hint for
the PMD to change the ring size to internal defaults.

Additionally since the ENA_DEFAULT_RING_SIZE is defined, report it in
the device capabilities so that both rte_ethdev code and the user can
utilize it for device configuration.

Fixes: ea93d37eb49d ("net/ena: add HW queues depth setup")
Cc: stable@dpdk.org

Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>

show more ...

net/ena: fix crash with unsupported device argument

As the documentation of rte_kvargs_parse() states, the valid_keys
argument must be NULL terminated. Lack of this feature may cause
segmentation fa

net/ena: fix crash with unsupported device argument

As the documentation of rte_kvargs_parse() states, the valid_keys
argument must be NULL terminated. Lack of this feature may cause
segmentation fault if the passed devarg will be different then the
supported value.

Fixes: 8a7a73f26cc9 ("net/ena: support large LLQ headers")
Cc: stable@dpdk.org

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

show more ...

net/ena: fix parsing of large LLQ header device argument

The code incorrectly checked the return value of comparison when parsing
the argument key name. The return value of strcmp should be compared

net/ena: fix parsing of large LLQ header device argument

The code incorrectly checked the return value of comparison when parsing
the argument key name. The return value of strcmp should be compared
to 0 to identify a match.

Fixes: 8a7a73f26cc9 ("net/ena: support large LLQ headers")
Cc: stable@dpdk.org

Signed-off-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>

show more ...

net/ena/base: adjust to latest ena-com

1. As memzone uses unique names, changed alloc coherent macro to use
64 bit size atomic variable to increase the memzone name space
2. "handle" param name c

net/ena/base: adjust to latest ena-com

1. As memzone uses unique names, changed alloc coherent macro to use
64 bit size atomic variable to increase the memzone name space
2. "handle" param name change to be consistent with other macros
3. Variable definition displacement
4. Backslash alignment to column 80

Signed-off-by: Amit Bernstein <amitbern@amazon.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Shay Agroskin <shayagr@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>

show more ...

net/ena/base: improve style and comments

List of changes:
* Comment style was adjusted for the functions
* The keys_num at "struct ena_admin_feature_rss_flow_hash_control" was
renamed to the

net/ena/base: improve style and comments

List of changes:
* Comment style was adjusted for the functions
* The keys_num at "struct ena_admin_feature_rss_flow_hash_control" was
renamed to the key_parts to better describe it's meaning
* The RSS indirection table was called "REDIRECTION" -> changed to
INDIRECTION
* Change AENQ field "syndrom" -> "syndrome"
* Calculate number of the RSS key parts or whole key by using the
common way: sizeof of the first element of the RSS key
* Add description of the "enum ena_admin_aq_feature_id"
* Rename "map_rx_buf_bidirectional" field as "rx_buf_mirroring"
* Other minor style fixes (remove extra spaces, add missing line break,
improve indentation)
* Remove unused macros ENA_ADMIN_EXTRA_PROPERTIES_*
* Restructure the "if {} else if {} else" conditional statement for
setting up the meta descriptor

Fixes: 99ecfbf845b3 ("ena: import communication layer")
Fixes: b68309be44c0 ("net/ena/base: update communication layer for the ENAv2")
Fixes: b2b02edeb0d6 ("net/ena/base: upgrade HAL for new HW features")
Cc: stable@dpdk.org

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>

show more ...

log: register with standardized names

Let's try to enforce the convention where most drivers use a pmd. logtype
with their class reflected in it, and libraries use a lib. logtype.

Introduce two new

log: register with standardized names

Let's try to enforce the convention where most drivers use a pmd. logtype
with their class reflected in it, and libraries use a lib. logtype.

Introduce two new macros:
- RTE_LOG_REGISTER_DEFAULT can be used when a single logtype is
used in a component. It is associated to the default name provided
by the build system,
- RTE_LOG_REGISTER_SUFFIX can be used when multiple logtypes are used,
and then the passed name is appended to the default name,

RTE_LOG_REGISTER is left untouched for existing external users
and for components that do not comply with the convention.

There is a new Meson variable log_prefix to adapt the default name
for baseband (pmd.bb.), bus (no pmd.) and mempool (no pmd.) classes.

Note: achieved with below commands + reverted change on net/bonding +
edits on crypto/virtio, compress/mlx5, regex/mlx5

$ git grep -l RTE_LOG_REGISTER drivers/ |
while read file; do
pattern=${file##drivers/};
class=${pattern%%/*};
pattern=${pattern#$class/};
drv=${pattern%%/*};
case "$class" in
baseband) pattern=pmd.bb.$drv;;
bus) pattern=bus.$drv;;
mempool) pattern=mempool.$drv;;
*) pattern=pmd.$class.$drv;;
esac
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
done

$ git grep -l RTE_LOG_REGISTER lib/ |
while read file; do
pattern=${file##lib/};
pattern=lib.${pattern%%/*};
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
done

Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

net/ena: fix releasing Tx ring mbufs

When ena_tx_queue_release_bufs() frees the mbufs it does not clear
the mbuf pointers. So, when the device starts and stops multiple
times it can cause the appli

net/ena: fix releasing Tx ring mbufs

When ena_tx_queue_release_bufs() frees the mbufs it does not clear
the mbuf pointers. So, when the device starts and stops multiple
times it can cause the application to receive duplicate mbufs for
two different packets. Fix the issue by clearing the mbuf pointer.

Also, while tracking down the "double free" issue the ena calls to
allocate and free mbufs in bulk were migrated to the mbuf based APIs
so the common mbuf alloc/free routines are exercised.

Fixes: 79405ee17585 ("net/ena: fix out of order completion")
Fixes: 1173fca25af9 ("ena: add polling-mode driver")
Cc: stable@dpdk.org

Signed-off-by: David Harton <dharton@cisco.com>
Acked-by: Michal Krawczyk <mk@semihalf.com>

show more ...

drivers/net: redefine array size macros

Replace copies of size(arr)/size(arr[0]) with RTE_DIM().
Eventually all of these macro definitions should be removed.

Signed-off-by: Andrew Boyer <aboyer@pen

drivers/net: redefine array size macros

Replace copies of size(arr)/size(arr[0]) with RTE_DIM().
Eventually all of these macro definitions should be removed.

Signed-off-by: Andrew Boyer <aboyer@pensando.io>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

show more ...

net/ena: prevent double doorbell

Add per-tx-ring flag for packets that were pushed to HW but await
doorbell. That is to prevent a situation when a doorbell is sent due to
reaching Tx burst threshold

net/ena: prevent double doorbell

Add per-tx-ring flag for packets that were pushed to HW but await
doorbell. That is to prevent a situation when a doorbell is sent due to
reaching Tx burst threshold and next send fails (e.g., due to queue
full). In such case we shouldn't send another doorbell because there are
no actual packets waiting for transmission.

Fixes: c7519ea5eb8d ("net/ena: call additional doorbells if needed")
Cc: stable@dpdk.org

Signed-off-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>

show more ...

net/ena: fix Tx SQ free space assessment

Before starting transmission of Tx burst, the driver checked the
available space in the sq and limited the number of packets for
transmission accordingly.
Th

net/ena: fix Tx SQ free space assessment

Before starting transmission of Tx burst, the driver checked the
available space in the sq and limited the number of packets for
transmission accordingly.
The calculation was incorrect for fragmented packets and potentially had
significantly limited the length of Tx bursts.

This patch removes the assessment and pushes packets to the sq as long
as the burst is not exhausted and space is available in the sq.

Correct evaluation of the required space isn't possible before the burst
because it depends on the number of segments of each packet.
This patch adds per-packet space evaluation for each packet before
attempting to process it. In case there is not enough queue space, the
burst will just stop without error.

Fixes: 2061fe41f212 ("net/ena: linearize Tx mbuf")
Cc: stable@dpdk.org

Signed-off-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>

show more ...

net/ena: validate Rx req ID upon acquiring descriptor

Instead of verifying the Rx descriptor each time it's being used in the
driver code, now the verification happens on the HAL side.

This simplif

net/ena: validate Rx req ID upon acquiring descriptor

Instead of verifying the Rx descriptor each time it's being used in the
driver code, now the verification happens on the HAL side.

This simplifies code a lot as instead of doing 2 validations, only
single one is needed. The driver have to check the rc value returned
by the ena_com upon reading the Rx descriptor and trigger the reset
if needed. It was previously the responsibility of the
validate_rx_req_id() function.

As part of the change, the version of the driver was bumped to v2.2.1.

Fixes: 2061fe41f212 ("net/ena: linearize Tx mbuf")
Cc: stable@dpdk.org

Signed-off-by: Ido Segev <idose@amazon.com>
Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>

show more ...

net/ena: fix Tx doorbell statistics

Increment Tx doorbell statistics on tx_pkt_burst
after writing to doorbell and in case max burst size achieved

Fixes: c7519ea5eb8d ("net/ena: call additional doo

net/ena: fix Tx doorbell statistics

Increment Tx doorbell statistics on tx_pkt_burst
after writing to doorbell and in case max burst size achieved

Fixes: c7519ea5eb8d ("net/ena: call additional doorbells if needed")
Cc: stable@dpdk.org

Signed-off-by: Amit Bernstein <amitbern@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>

show more ...

net/ena: flush Rx buffers memory pool cache

As the refill called as part of ena_start(), we end up the refill
progress with stuck buffers at the caller core cache.

Calling to flush the cache result

net/ena: flush Rx buffers memory pool cache

As the refill called as part of ena_start(), we end up the refill
progress with stuck buffers at the caller core cache.

Calling to flush the cache results with invalidate this cache and free
those stuck buffers.

Fixes: 1173fca25af9 ("ena: add polling-mode driver")
Cc: stable@dpdk.org

Signed-off-by: Ido Segev <idose@amazon.com>
Reviewed-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>

show more ...

ethdev: make driver-only headers private

The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.

Signed-off-by:

ethdev: make driver-only headers private

The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Steven Webster <steven.webster@windriver.com>

show more ...

net/ena: upgrade driver version to v2.2.0

The v2.2.0 adds support for network interface metrics, includes some bug
fixes and updates HAL to the latest version.

Signed-off-by: Michal Krawczyk <mk@se

net/ena: upgrade driver version to v2.2.0

The v2.2.0 adds support for network interface metrics, includes some bug
fixes and updates HAL to the latest version.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>

show more ...

net/ena: change name of supported PCI device IDs

The ID 0xEC21 is not associated with LLQ feature of the device, so it
would be misleading for the user. Because of that, the current
identifier is mo

net/ena: change name of supported PCI device IDs

The ID 0xEC21 is not associated with LLQ feature of the device, so it
would be misleading for the user. Because of that, the current
identifier is more precise.

Together with code update, the documentation was changed to reflect
current changes

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>

show more ...

net/ena: fix setting Rx checksum flags in mbuf

The driver was never setting PKT_RX_*_CKSUM_GOOD flags, so the only way
of checking if the checksum was checked was by testing for the
PKT_RX_*_CKSUM_B

net/ena: fix setting Rx checksum flags in mbuf

The driver was never setting PKT_RX_*_CKSUM_GOOD flags, so the only way
of checking if the checksum was checked was by testing for the
PKT_RX_*_CKSUM_BAD. In that situation, the application couldn't detect
if the checksum was valid or unknown, as unknown flag is equal to 0.

Moreover, the l3_csum_err value is only valid if the l3_proto is
indicating IPv4, so it shouldn't be checked for other protocols.

Fixes: 1173fca25af9 ("ena: add polling-mode driver")
Cc: stable@dpdk.org

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>

show more ...