ena: Trigger reset when mbuf is NULL error happens

The mbuf is NULL issue happens when the device sends the driver
a completion with a wrong request id.
Trigger a reset whenever this happens.

Appro

ena: Trigger reset when mbuf is NULL error happens

The mbuf is NULL issue happens when the device sends the driver
a completion with a wrong request id.
Trigger a reset whenever this happens.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add differentiation for missing TX completions reset

This commit adds differentiation for a reset caused by missing tx
completions, by verifying if the driver didn't receive tx
completions caus

ena: Add differentiation for missing TX completions reset

This commit adds differentiation for a reset caused by missing tx
completions, by verifying if the driver didn't receive tx
completions caused by missing interrupts.
The cleanup_running field was added to ena_ring because
cleanup_task.ta_pending is zeroed before ena_cleanup() runs.

Also ena_increment_reset_counter() API was added in order to support
only incrementing the reset counter.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add reset reason for corrupted RX cdescs

RX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit

ena: Add reset reason for corrupted RX cdescs

RX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit reset reason
ENA_REGS_RESET_RX_DESCRIPTOR_MALFORMED.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit

ena: Add reset reason for corrupted TX cdescs

TX completion descriptors may sometimes contain errors due
to corruption. Upon identifying such a case, the driver will
trigger a reset with an explicit reset reason
ENA_REGS_RESET_TX_DESCRIPTOR_MALFORMED.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Update license signatures to 2024

This commit updates all the license signatures to 2024.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Update the license dating to 2023

Some of the files are using outdated linceses.
Update the license to be 2023.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Remove CQ tail pointer update API

This commit removes the usage of this API from the freebsd driver since
the relevant functionality is not supported by the device.

Approved by: cperciva (ment

ena: Remove CQ tail pointer update API

This commit removes the usage of this API from the freebsd driver since
the relevant functionality is not supported by the device.

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Update ena_com_update_intr_reg API usage

This commit fixes the usage of this function to be compatible with the
new API introduced by ena-com update to v2.7.0

Approved by: cperciva (mentor)
MF

ena: Update ena_com_update_intr_reg API usage

This commit fixes the usage of this function to be compatible with the
new API introduced by ena-com update to v2.7.0

Approved by: cperciva (mentor)
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Mechanically convert ena(4) to DrvAPI

Reviewed by: mw
Differential Revision: https://reviews.freebsd.org/D37837

ena: Align req_id and qid print order

In most places, the req_id is printed first, and the qid is printed as a
second. To align the driver, one printout was reworked and the print
order of those var

ena: Align req_id and qid print order

In most places, the req_id is printed first, and the qid is printed as a
second. To align the driver, one printout was reworked and the print
order of those variables was changed.

Suggested by: rpokala
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Make first_interrupt a uint8_t

We do not have atomic(9) routines for bools, and it is not guaranteed
that sizeof(bool) is 1.

This fixes the KASAN and KMSAN kernel builds, which fail because th

ena: Make first_interrupt a uint8_t

We do not have atomic(9) routines for bools, and it is not guaranteed
that sizeof(bool) is 1.

This fixes the KASAN and KMSAN kernel builds, which fail because the
compiler refuses to silently cast a _Bool * to a uint8_t * when calling
the atomic(9) sanitizer interceptors.

Reviewed by: Dawid Górecki <dgr@semihalf.com>
MFC after: 2 weeks
Fixes: 0ac122c388d9 ("ena: Use atomic_load/store functions for first_interrupt variable")
Differential Revision: https://reviews.freebsd.org/D35683

show more ...

ena: Align names of constants

Most of the constants in ena.h file were prefixed with ENA_*, while
others did not have this prefix. Align the constants by prefixing the
remaining constants with ENA.

ena: Align names of constants

Most of the constants in ena.h file were prefixed with ENA_*, while
others did not have this prefix. Align the constants by prefixing the
remaining constants with ENA.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Remove write-only datapath variable

The ena_qid variable value is never used. It can be safely removed.
That also silences the compilation warning.

Obtained from: Semihalf
MFC after: 2 weeks
S

ena: Remove write-only datapath variable

The ena_qid variable value is never used. It can be safely removed.
That also silences the compilation warning.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Fix styling issues

Align code style with FreeBSD style(9) guidelines.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: Use atomic_load/store functions for first_interrupt variable

Surround cases of possible simultaneous access to the first_interrupt
variable with atomic_load/store functions.

Obtained from: Sem

ena: Use atomic_load/store functions for first_interrupt variable

Surround cases of possible simultaneous access to the first_interrupt
variable with atomic_load/store functions.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Store ticks of last Tx cleanup

Store timestamp of last cleanup in Tx ring structure. This does not
change anything during normal operation of the driver but could be
useful when the device fail

ena: Store ticks of last Tx cleanup

Store timestamp of last cleanup in Tx ring structure. This does not
change anything during normal operation of the driver but could be
useful when the device fails for some reason.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Extend debug prints for invalid req_id resets

Print information about qid if req_id is invalid. Add information about
qid and req_id if mbuf is invalid.

Obtained from: Semihalf
MFC after: 2 we

ena: Extend debug prints for invalid req_id resets

Print information about qid if req_id is invalid. Add information about
qid and req_id if mbuf is invalid.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add ena_ring_tx_doorbell() function

Add ena_ring_tx_doorbell function to remove code duplication.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

ena: rework tx req_id validation logic

Since `ena_com_tx_comp_req_id_get` already checks for `req_id` validity,
the logic was exiting early, never giving `validate_tx_req_id` a chance
to trigger dev

ena: rework tx req_id validation logic

Since `ena_com_tx_comp_req_id_get` already checks for `req_id` validity,
the logic was exiting early, never giving `validate_tx_req_id` a chance
to trigger device reset.
Rewrite the logic so that device reset is called based on return value
of `ena_com_tx_comp_req_id_get` instead.

Submitted by: Artur Rojek <ar@semihalf.com>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: properly handle IPv6 L4 checksum offload

ena_tx_csum function did not check if IPv6 checksum offload was
requested it only checked checksum offloading flags for IPv4 packets.
Because of that, w

ena: properly handle IPv6 L4 checksum offload

ena_tx_csum function did not check if IPv6 checksum offload was
requested it only checked checksum offloading flags for IPv4 packets.
Because of that, when encountering CSUM_IP6_* flags, the function simply
returned without actually setting checksum offloading in ena_ctx.
Check CUSM_IP6_* flags to enable IPv6 checksum offload.

Additionally, only IPv4 header was being parsed regardless of EtherType
field, because of that, value of L4 protocol read when actually trying
to send IPv6 packets was wrong. Use ip6_lasthdr function to get length
of all IPv6 headers and payload protocol.

Set the DF flag to 1 in order to allow the device to offload the IPv6
checksum calculation and achieve optimal performance.

Add CSUM6_OFFLOAD and CSUM_OFFLOAD definitions into ena_datapath.h.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Implement full RSS reconfiguration

Bind RX/TX queues and MSI-X vectors to matching CPUs based on the RSS
bucket entries.

Introduce sysctls for the following RSS functionality:
- rss.indir_tabl

ena: Implement full RSS reconfiguration

Bind RX/TX queues and MSI-X vectors to matching CPUs based on the RSS
bucket entries.

Introduce sysctls for the following RSS functionality:
- rss.indir_table: indirection table mapping
- rss.indir_table_size: indirection table size
- rss.key: RSS hash key (if Toeplitz used)

Said sysctls are only available when compiled without `option RSS`, as
kernel-side RSS support currently doesn't offer RSS reconfiguration.

Migrate the hash algorithm from CRC32 to Toeplitz and change the initial
hash value to 0x0 in order to match the standard Toeplitz implementation.
Provide helpers for hash key inversion required for HW operations.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add missing statistics

Provide the following sysctl statistics in order to stay aligned with
the Linux driver:
* rx_ring.csum_good
* tx_ring.unmask_interrupt_num

Also rename the 'bad_csum' sta

ena: Add missing statistics

Provide the following sysctl statistics in order to stay aligned with
the Linux driver:
* rx_ring.csum_good
* tx_ring.unmask_interrupt_num

Also rename the 'bad_csum' statistic name to 'csum_bad' for alignment.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add locking assertions

ENA silently assumed that ena_up, ena_down and ena_start_xmit routines
should be called within locked context. Driver's logic heavily assumes
on concurrent access to thos

ena: Add locking assertions

ENA silently assumed that ena_up, ena_down and ena_start_xmit routines
should be called within locked context. Driver's logic heavily assumes
on concurrent access to those routines, so for safety and better
documentation about this assumption, the locking assertions were added
to the above functions.

The assertion was added only for the main steps (skipping the helper
functions) which can be called from multiple places including the kernel
and the driver itself.

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...

ena: Add extra log messages

Stay aligned with the Linux driver by adding the following logs:
* inform the user about retrying queue creation
* warn on non-empty ena_tx_buffer.mbuf prior to ena_tx_ma

ena: Add extra log messages

Stay aligned with the Linux driver by adding the following logs:
* inform the user about retrying queue creation
* warn on non-empty ena_tx_buffer.mbuf prior to ena_tx_map_mbuf

Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.

show more ...