virtio: optimize stats counters performance

Optimized the performance of updating the virtio statistics counters by
reducing the number of branches.

Ordered the packet size comparisons according to

virtio: optimize stats counters performance

Optimized the performance of updating the virtio statistics counters by
reducing the number of branches.

Ordered the packet size comparisons according to the probability with
typical internet traffic mix.

Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Huisong Li <lihuisong@huawei.com>
Reviewed-by: Chenbo Xia <chenbox@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

use C11 alignof

Replace use of __alignof__(e) (resp. __alignof__(T) with C11
alignof(typeof(e)) (resp. alignof(T)) to improve portability
between toolchains.

Signed-off-by: Tyler Retzlaff <roretzla

use C11 alignof

Replace use of __alignof__(e) (resp. __alignof__(T) with C11
alignof(typeof(e)) (resp. alignof(T)) to improve portability
between toolchains.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Volodymyr Fialko <vfialko@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

show more ...

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
g

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_clz; new=rte_clz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_ctzll; new=rte_ctz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_ctz; new=rte_ctz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_popcountll; new=rte_popcount64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_popcount; new=rte_popcount32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

Then inclusion of rte_bitops.h was added were necessary.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>

show more ...

net/virtio: deduce IP length for TSO checksum

The length of TSO payload could not fit into 16 bits provided by the
IPv4 total length and IPv6 payload length fields. Thus, deduce it
from the length o

net/virtio: deduce IP length for TSO checksum

The length of TSO payload could not fit into 16 bits provided by the
IPv4 total length and IPv6 payload length fields. Thus, deduce it
from the length of the packet.

Fixes: 696573046e9e ("net/virtio: support TSO")
Cc: stable@dpdk.org

Signed-off-by: Boleslav Stankevich <boleslav.stankevich@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: alloc Rx SW ring only if vectorized path

This patch only allocates the SW ring when vectorized
datapath is used. It also moves the SW ring and fake mbuf
in the virtnet_rx struct since th

net/virtio: alloc Rx SW ring only if vectorized path

This patch only allocates the SW ring when vectorized
datapath is used. It also moves the SW ring and fake mbuf
in the virtnet_rx struct since this is Rx-only.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: remove unused queue ID from Rx queue

This patch removes the queue ID field in virtnet_rx struct.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <che

net/virtio: remove unused queue ID from Rx queue

This patch removes the queue ID field in virtnet_rx struct.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: remove unused fields from Tx queue

The port and queue IDs are not used in virtnet_tx struct,
this patch removes them.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewe

net/virtio: remove unused fields from Tx queue

The port and queue IDs are not used in virtnet_tx struct,
this patch removes them.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: remove port ID info from Rx queue

The port ID information is duplicated in several places.
This patch removes it from the virtnet_rx struct as it can
be found in virtio_hw struct.

Signe

net/virtio: remove port ID info from Rx queue

The port ID information is duplicated in several places.
This patch removes it from the virtnet_rx struct as it can
be found in virtio_hw struct.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: refactor virtqueue headers allocation

This patch refactors virtqueue initialization by moving
its headers allocation and deallocation in dedicated
function.

While at it, it renames the

net/virtio: refactor virtqueue headers allocation

This patch refactors virtqueue initialization by moving
its headers allocation and deallocation in dedicated
function.

While at it, it renames the memzone metadata and address
pointers in the virtnet_tx and virtnet_ctl structures to
remove redundant virtio_net_ prefix.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: add queue and port ID in some logs

Add the queue id and/or the port id in some logs, so it is easier to
understand what happens.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Rev

net/virtio: add queue and port ID in some logs

Add the queue id and/or the port id in some logs, so it is easier to
understand what happens.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: fix Tx queue 0 overriden by queue 128

Both Rx queue and Tx queue are VirtQ in virtio, VQ index is 256 for Tx
queue 128. Uint8 type of TxQ VQ index overflows and overrides Tx queue 0
data

net/virtio: fix Tx queue 0 overriden by queue 128

Both Rx queue and Tx queue are VirtQ in virtio, VQ index is 256 for Tx
queue 128. Uint8 type of TxQ VQ index overflows and overrides Tx queue 0
data.

This patch fixes VQ index type with uint16 type.

Fixes: c1f86306a026 ("virtio: add new driver")
Cc: stable@dpdk.org

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

fix spelling in comments and strings

The tool comes from https://github.com/jsoref

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

mbuf: add namespace to offload flags

Fix the mbuf offload flags namespace by adding an RTE_ prefix to the
name. The old flags remain usable, but a deprecation warning is issued
at compilation.

Sign

mbuf: add namespace to offload flags

Fix the mbuf offload flags namespace by adding an RTE_ prefix to the
name. The old flags remain usable, but a deprecation warning is issued
at compilation.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Somnath Kotur <somnath.kotur@broadcom.com>

show more ...

mbuf: mark old VLAN offload flags as deprecated

The flags PKT_TX_VLAN_PKT and PKT_TX_QINQ_PKT are
marked as deprecated since commit 380a7aab1ae2 ("mbuf: rename deprecated
VLAN flags") (2017). But th

mbuf: mark old VLAN offload flags as deprecated

The flags PKT_TX_VLAN_PKT and PKT_TX_QINQ_PKT are
marked as deprecated since commit 380a7aab1ae2 ("mbuf: rename deprecated
VLAN flags") (2017). But they were not using the RTE_DEPRECATED
macro, because it did not exist at this time. Add it, and replace
usage of these flags.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

show more ...

ethdev: remove legacy Rx descriptor done API

rte_eth_rx_descriptor_status() should be used as a replacement.

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Ferruh Yigi

ethdev: remove legacy Rx descriptor done API

rte_eth_rx_descriptor_status() should be used as a replacement.

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

net/virtio: revert forcing IOVA as VA mode for virtio-user

This patch removes the simplification in Virtio descriptors
handling, where their buffer addresses are IOVAs for Virtio
PCI devices, and VA

net/virtio: revert forcing IOVA as VA mode for virtio-user

This patch removes the simplification in Virtio descriptors
handling, where their buffer addresses are IOVAs for Virtio
PCI devices, and VA-only for Virtio-user devices, which
added a requirement on Virtio-user that it only supported
IOVA as VA.

This change introduced a regression for applications using
Virtio-user and other physical PMDs that require IOVA as PA
because they don't use an IOMMU.

This patch reverts to the old behaviour, but needed to be
reworked because of the refactoring that happened in v21.02.

Fixes: 17043a2909bb ("net/virtio: force IOVA as VA mode for virtio-user")
Cc: stable@dpdk.org

Reported-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Tested-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

net/virtio: do not use PMD log type

Fixes: 1982462eadea ("net/virtio: add Rx free threshold setting")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxi

net/virtio: do not use PMD log type

Fixes: 1982462eadea ("net/virtio: add Rx free threshold setting")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: remove blank lines in log

The macros PMD_*_LOG already include the line feed character.
Redundant \n are removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David

net/virtio: remove blank lines in log

The macros PMD_*_LOG already include the line feed character.
Redundant \n are removed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

net/virtio: fix split queue vectorized Rx

Descriptors number may be set less than queue size for split queue
vectorized Rx path. Pointers to mbufs for received packets are
obtained from SW ring, tha

net/virtio: fix split queue vectorized Rx

Descriptors number may be set less than queue size for split queue
vectorized Rx path. Pointers to mbufs for received packets are
obtained from SW ring, that is initially filled with them in the end
of queue setup in virtio_dev_rx_queue_setup_finish(). The begin of the
SW ring filled up to the size of descriptors number. At queue size
offset from the begin of the SW ring pointers to some fake mbuf are also
set for wrapping purpose. So the ring may contains the hole of invalid
pointers from descriptors number offset to queue size offset, and split
vectorized Rx routines could write to the invalid addresses since they
use the ring up to the queue size. Fix this by setting descriptors
number to queue size on Rx queue setup.

Fixes: fc3d66212fed ("virtio: add vector Rx")
Cc: stable@dpdk.org

Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: remove handling of zero desc on Rx setup

Rx queue setup callback allows to use the whole ring when
descriptor number argument equals zero. There's no point to
handle zero in any way sinc

net/virtio: remove handling of zero desc on Rx setup

Rx queue setup callback allows to use the whole ring when
descriptor number argument equals zero. There's no point to
handle zero in any way since RTE Rx queue setup function
rte_eth_rx_queue_setup() doesn't pass zero using fallback
values.

Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: fix mbuf count on Rx queue setup

Rx queue setup finish function may report wrong number of
allocated mbufs in case of in-order feature. Fix the
function to not ignore allocation error an

net/virtio: fix mbuf count on Rx queue setup

Rx queue setup finish function may report wrong number of
allocated mbufs in case of in-order feature. Fix the
function to not ignore allocation error and count only
successfully allocated number of buffers.

Fixes: e5f456a98d3c ("net/virtio: support in-order Rx and Tx")
Cc: stable@dpdk.org

Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: fix Rx scatter offload

Report Rx scatter offload capability depending on VIRTIO_NET_F_MRG_RXBUF.

If Rx scatter is not requested, ensure that provided Rx buffers on
each Rx queue are big

net/virtio: fix Rx scatter offload

Report Rx scatter offload capability depending on VIRTIO_NET_F_MRG_RXBUF.

If Rx scatter is not requested, ensure that provided Rx buffers on
each Rx queue are big enough to fit Rx packets up to configured MTU.

Fixes: ce17eddefc20 ("ethdev: introduce Rx queue offloads API")
Cc: stable@dpdk.org

Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: fix refill order in packed ring datapath

The front-end should refill the descriptor with the mbuf indicated by
the buff_id rather then the index of used descriptor. Back-end may
return b

net/virtio: fix refill order in packed ring datapath

The front-end should refill the descriptor with the mbuf indicated by
the buff_id rather then the index of used descriptor. Back-end may
return buffers out of order if async copy mode is enabled.

When initializing rxq, refill the descriptors in order as buff_id is
not available at that time.

Fixes: a76290c8f1cf ("net/virtio: implement Rx path for packed queues")
Cc: stable@dpdk.org

Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Signed-off-by: Marvin Liu <yong.liu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: refactor Tx offload helper

Purely cosmetic but it is rather odd to have an "offload" helper that
checks if it actually must do something.
We already have the same checks in most callers,

net/virtio: refactor Tx offload helper

Purely cosmetic but it is rather odd to have an "offload" helper that
checks if it actually must do something.
We already have the same checks in most callers, so move this branch
in them.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Flavio Leitner <fbl@sysclose.org>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

net/virtio: allocate fake mbuf in Rx queue

While it is worth clarifying whether the fake mbuf
in virtnet_rx struct is really necessary, it is sure
that it heavily impacts cache usage by being part o

net/virtio: allocate fake mbuf in Rx queue

While it is worth clarifying whether the fake mbuf
in virtnet_rx struct is really necessary, it is sure
that it heavily impacts cache usage by being part of
the struct. Indeed, it uses two cachelines, and
requires alignment on a cacheline.

Before this series, it means it took 120 bytes in
virtnet_rx struct:

struct virtnet_rx {
struct virtqueue *vq; /*0 8*/

/* XXX 56 bytes hole, try to pack */

/* --- cacheline 1 boundary (64 bytes) --- */
struct rte_mbuf fake_mbuf __attribute__((__aligned__(64))); /*64 128*/
/* --- cacheline 3 boundary (192 bytes) --- */

This patch allocates it using malloc in order to optimize
virtnet_rx cache usage and so virtqueue cache usage.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Tested-by: Balazs Nemeth <bnemeth@redhat.com>

show more ...