net/ionic: fix mbuf double-free when emptying array

The bulk-allocation array is used back to front, so we need to free
everything before the marker, not after it. Flip ionic_empty_array()
so that i

net/ionic: fix mbuf double-free when emptying array

The bulk-allocation array is used back to front, so we need to free
everything before the marker, not after it. Flip ionic_empty_array()
so that it frees from 0 to the provided index. Adjust the callers
as needed.

Fixes: 218afd825bca ("net/ionic: do bulk allocations of Rx mbufs")
CC: stable@dpdk.org

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

drivers: move alignment attribute on types for MSVC

Move location of __rte_aligned(a) to new conventional location. The new
placement between {struct,union} and the tag allows the desired
alignment

drivers: move alignment attribute on types for MSVC

Move location of __rte_aligned(a) to new conventional location. The new
placement between {struct,union} and the tag allows the desired
alignment to be imparted on the type regardless of the toolchain being
used for both C and C++. Additionally, it avoids confusion by Doxygen
when generating documentation.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>

show more ...

net/ionic: optimize device start

Split the queue_start operation into first-half and second-half helpers.

This allows us to batch up the queue commands during dev_start(), reducing
the outage windo

net/ionic: optimize device start

Split the queue_start operation into first-half and second-half helpers.

This allows us to batch up the queue commands during dev_start(), reducing
the outage window when restarting the process by about 1ms per queue.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: optimize device stop

Split the queue_stop operation into first-half and second-half helpers.
Move the command context from the stack into each Rx/Tx queue struct.
Expose some needed admin

net/ionic: optimize device stop

Split the queue_stop operation into first-half and second-half helpers.
Move the command context from the stack into each Rx/Tx queue struct.
Expose some needed adminq interfaces.

This allows us to batch up the queue commands during dev_stop(), reducing
the outage window when restarting the process by about 1ms per queue.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: fix missing volatile type for cqe pointers

This memory may be changed by the hardware, so the volatile
keyword is required for correctness.

Fixes: e86a6fcc7cf3 ("net/ionic: add optimized

net/ionic: fix missing volatile type for cqe pointers

This memory may be changed by the hardware, so the volatile
keyword is required for correctness.

Fixes: e86a6fcc7cf3 ("net/ionic: add optimized non-scattered Rx/Tx")
Cc: stable@dpdk.org

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>

show more ...

net/ionic: add completion queue entries processed stats

When completion coalescing is turned on in the FW, there will be
fewer CQE than Tx packets. Expose the stat through debug logging.

Signed-off

net/ionic: add completion queue entries processed stats

When completion coalescing is turned on in the FW, there will be
fewer CQE than Tx packets. Expose the stat through debug logging.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

drivers/net: return number of supported packet types

Missing "RTE_PTYPE_UNKNOWN" ptype causes buffer overflow.
Enhance code such that the dev_supported_ptypes_get()
function pointer now returns the

drivers/net: return number of supported packet types

Missing "RTE_PTYPE_UNKNOWN" ptype causes buffer overflow.
Enhance code such that the dev_supported_ptypes_get()
function pointer now returns the number of elements to
eliminate the need for "RTE_PTYPE_UNKNOWN" as the last item.

Same applied to 'buffer_split_supported_hdr_ptypes_get()' dev_ops too.

Signed-off-by: Sivaramakrishnan Venkat <venkatx.sivaramakrishnan@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@amd.com>

show more ...

net/ionic: add watchdogs to protect each queue type

Ring the doorbell again for the following scenarios:
* No receives posted but Rx queue not empty after deadline
* No transmits posted but Tx wor

net/ionic: add watchdogs to protect each queue type

Ring the doorbell again for the following scenarios:
* No receives posted but Rx queue not empty after deadline
* No transmits posted but Tx work still pending after deadline
* Admin queue work still pending after deadline

This will help the queues recover in the extremely rare case that
a doorbell is missed by the FW.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: add optimized non-scattered Rx/Tx

The code is very similar, but the simple case can skip a few branches
in the hot path. This improves PPS when 10KB mbufs are used.

S/G is enabled on the

net/ionic: add optimized non-scattered Rx/Tx

The code is very similar, but the simple case can skip a few branches
in the hot path. This improves PPS when 10KB mbufs are used.

S/G is enabled on the Rx side by offload DEV_RX_OFFLOAD_SCATTER.
S/G is enabled on the Tx side by offload DEV_TX_OFFLOAD_MULTI_SEGS.

S/G is automatically enabled on the Rx side if the provided mbufs are
too small to hold the maximum possible frame.

To enable S/G in testpmd, add these args:
--rx-offloads=0x2000 --tx-offloads=0x8000

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: R Mohamed Shah <mohamedshah.r@amd.com>

show more ...

net/ionic: allow to specify Tx free threshold

Some clients have opinions about how often to flush the
transmit ring.

The default value is the number of Tx descriptors minus the
default Tx burst siz

net/ionic: allow to specify Tx free threshold

Some clients have opinions about how often to flush the
transmit ring.

The default value is the number of Tx descriptors minus the
default Tx burst size.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: add Q-in-CMB option

When 'ionic_cmb' is set to '1', queue memory will be allocated from
the device's onboard memory (Controller Memory Buffer). In some
configurations, this will dramatica

net/ionic: add Q-in-CMB option

When 'ionic_cmb' is set to '1', queue memory will be allocated from
the device's onboard memory (Controller Memory Buffer). In some
configurations, this will dramatically reduce packet latency and
increase PPS.

Add the WC_ACTIVATE flag to the PCI driver flags.
Write combining must be enabled to achieve the maximum PPS.

When the queue is in the CMB, descriptors cannot be prefetched.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Neel Patel <neel.patel@amd.com>

show more ...

net/ionic: support Tx descriptor status

This may be useful for clients.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Allen Hubbe <allen.hubbe@amd.com>

net/ionic: support Rx descriptor status

These may be useful for clients.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Allen Hubbe <allen.hubbe@amd.com>

net/ionic: advertise supported packet types

This improves performance, since clients may be able to skip SW
packet classification.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: add lookup table for checksum flags

This improves performance by reducing branching.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: add lookup table for packet type

This improves performance by reducing branching.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: do bulk allocations of Rx mbufs

Do bulk allocations to improve performance.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: support mbuf fast free

Use a put() rather than a free() in the optimized case.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: overhaul Tx for performance

Linearize Tx mbuf chains in the info array.
This avoids walking the mbuf chain during flush.
Move a few branches out of the hot path.

Signed-off-by: Andrew Bo

net/ionic: overhaul Tx for performance

Linearize Tx mbuf chains in the info array.
This avoids walking the mbuf chain during flush.
Move a few branches out of the hot path.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: overhaul Rx for performance

Linearize RX mbuf chains in the expanded info array.
Clean one and fill one per CQE (completions are not coalesced).
Touch the mbufs as little as possible in t

net/ionic: overhaul Rx for performance

Linearize RX mbuf chains in the expanded info array.
Clean one and fill one per CQE (completions are not coalesced).
Touch the mbufs as little as possible in the fill stage.
When touching the mbuf in the clean stage, use the rearm_data unions.
Ring the doorbell once at the end of the bulk clean/fill.

Signed-off-by: Neel Patel <neel.patel@amd.com>
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: do one-time init of Rx descriptors

These fields don't need to be set in the hot path.
This improves performance.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: use helper variable in Tx

This improves readability.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

net/ionic: precalculate segment lengths on Rx side

The first (header) segment includes the standard headroom.
Subsequent segments do not.

Store the fragment counts in the queue structure.

Precalcu

net/ionic: precalculate segment lengths on Rx side

The first (header) segment includes the standard headroom.
Subsequent segments do not.

Store the fragment counts in the queue structure.

Precalculating improves performance by reducing
how much work must be done in the hot path.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: free all buffers during Rx queue stop

Free all of the mbufs in the receive queue when the queue is
stopped. This will allow them to be resized when the MTU is
changed.

Signed-off-by: And

net/ionic: free all buffers during Rx queue stop

Free all of the mbufs in the receive queue when the queue is
stopped. This will allow them to be resized when the MTU is
changed.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...

net/ionic: replace void pointer with actual type

This makes the code safer by helping the compiler catch errors.
Rename the variables, too; they're not callbacks anymore.

Signed-off-by: Andrew Boye

net/ionic: replace void pointer with actual type

This makes the code safer by helping the compiler catch errors.
Rename the variables, too; they're not callbacks anymore.

Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>

show more ...