vhost: add API to set max queue pairs

This patch introduces a new rte_vhost_driver_set_max_queues
API as preliminary work for multiqueue support with VDUSE.

Indeed, with VDUSE we need to pre-alloca

vhost: add API to set max queue pairs

This patch introduces a new rte_vhost_driver_set_max_queues
API as preliminary work for multiqueue support with VDUSE.

Indeed, with VDUSE we need to pre-allocate the vrings at
device creation time, so we need such API not to allocate
the 128 queue pairs supported by the Vhost library.

Calling the API is optional, 128 queue pairs remaining the
default.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: add operation to offload the interrupt kick

This patch adds an operation callback which gets called every time the
library wants to call eventfd_write(). This eventfd_write() call could
resul

vhost: add operation to offload the interrupt kick

This patch adds an operation callback which gets called every time the
library wants to call eventfd_write(). This eventfd_write() call could
result in a system call, which could potentially block the PMD thread.

The callback function can decide whether it's ok to handle the
eventfd_write() now or have the newly introduced function,
rte_vhost_notify_guest(), called at a later time.

This can be used by 3rd party applications, like OVS, to avoid system
calls being called as part of the PMD threads.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: add non-blocking API for posting interrupt

Vhost-user library locks all VQ's access lock when processing
vring based messages, such as SET_VRING_KICK and SET_VRING_CALL,
and the data processi

vhost: add non-blocking API for posting interrupt

Vhost-user library locks all VQ's access lock when processing
vring based messages, such as SET_VRING_KICK and SET_VRING_CALL,
and the data processing thread may already be started, e.g: SPDK
vhost-blk and vhost-scsi will start the data processing thread
when one vring is ready, then deadlock may happen when SPDK is
posting interrupts to VM. Here, we add a new API which allows
caller to try again later for this case.

Bugzilla ID: 1015
Fixes: c5736998305d ("vhost: fix missing virtqueue lock protection")
Cc: stable@dpdk.org

Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: introduce DMA vChannel unconfiguration

Add a new API rte_vhost_async_dma_unconfigure() to unconfigure DMA
vChannels in vhost async data path. Lock protection are also added
to protect DMA vCh

vhost: introduce DMA vChannel unconfiguration

Add a new API rte_vhost_async_dma_unconfigure() to unconfigure DMA
vChannels in vhost async data path. Lock protection are also added
to protect DMA vChannel configuration and unconfiguration
from concurrent calls.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

doc: fix readability in vhost guide

fix grammar issues and readbility in vhost library programmer guide

Fixes: 768274ebbd5e ("vhost: avoid populate guest memory")
Cc: stable@dpdk.org

Signed-off-by

doc: fix readability in vhost guide

fix grammar issues and readbility in vhost library programmer guide

Fixes: 768274ebbd5e ("vhost: avoid populate guest memory")
Cc: stable@dpdk.org

Signed-off-by: Herakliusz Lipiec <herakliusz.lipiec@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

doc: clean vhost async note

This patch moves the 'Recommended IOVA mode in async datapath'
section under 'Vhost asynchronous data path' as a sub-section,
which makes the doc cleaner.

Signed-off-by:

doc: clean vhost async note

This patch moves the 'Recommended IOVA mode in async datapath'
section under 'Vhost asynchronous data path' as a sub-section,
which makes the doc cleaner.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

doc: add vhost async enqueue API usage

This patch updates the correct usage for async enqueue APIs.
The rte_vhost_poll_enqueue_completed() needs to be
called in time to notify the guest of completed

doc: add vhost async enqueue API usage

This patch updates the correct usage for async enqueue APIs.
The rte_vhost_poll_enqueue_completed() needs to be
called in time to notify the guest of completed packets and
avoid packet loss.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: support clear in-flight packets for async dequeue

rte_vhost_clear_queue_thread_unsafe() supports to clear
in-flight packets for async enqueue only. But after
supporting async dequeue, this AP

vhost: support clear in-flight packets for async dequeue

rte_vhost_clear_queue_thread_unsafe() supports to clear
in-flight packets for async enqueue only. But after
supporting async dequeue, this API should support async dequeue too.

This patch also adds the thread-safe version of this API,
the difference between the two API is that thread safety uses lock.

These APIs maybe used to clean up packets in the async channel
to prevent packet loss when the device state changes or
when the device is destroyed.

Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Jiayu Hu <jiayu.hu@intel.com>

show more ...

vhost: get vDPA device type

Vhost backend of different devices have different features.
Add an API to get vDPA device type, net device or blk device
currently, so users can set different features fo

vhost: get vDPA device type

Vhost backend of different devices have different features.
Add an API to get vDPA device type, net device or blk device
currently, so users can set different features for different
kinds of devices.

Signed-off-by: Andy Pei <andy.pei@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: support async dequeue for split ring

This patch implements asynchronous dequeue data path for vhost split
ring, a new API rte_vhost_async_try_dequeue_burst() is introduced.

Signed-off-by: Xu

vhost: support async dequeue for split ring

This patch implements asynchronous dequeue data path for vhost split
ring, a new API rte_vhost_async_try_dequeue_burst() is introduced.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: support per-virtqueue statistics

This patch introduces new APIs for the application
to query and reset per-virtqueue statistics. The
patch also introduces generic counters.

Signed-off-by: Ma

vhost: support per-virtqueue statistics

This patch introduces new APIs for the application
to query and reset per-virtqueue statistics. The
patch also introduces generic counters.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: add unsafe API to check in-flight packets

In async data path, when vring state changes or device is destroyed,
it is necessary to know the number of in-flight packets in DMA engine.
This patc

vhost: add unsafe API to check in-flight packets

In async data path, when vring state changes or device is destroyed,
it is necessary to know the number of in-flight packets in DMA engine.
This patch provides a thread unsafe API to return the number of
in-flight packets for a vhost queue without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: integrate dmadev in asynchronous data-path

Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
abstraction layer and simplify application logics, this patch integrates
dma

vhost: integrate dmadev in asynchronous data-path

Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
abstraction layer and simplify application logics, this patch integrates
dmadev in asynchronous data path.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

doc: update recommended IOVA mode for async vhost

DPDK 21.11 adds vfio support for DMA device in vhost. This patch
updates recommended IOVA mode in async datapath.

Signed-off-by: Xuan Ding <xuan.di

doc: update recommended IOVA mode for async vhost

DPDK 21.11 adds vfio support for DMA device in vhost. This patch
updates recommended IOVA mode in async datapath.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: rename driver callbacks struct

As previously announced, this patch renames struct
vhost_device_ops to struct rte_vhost_device_ops.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>

vhost: rename driver callbacks struct

As previously announced, this patch renames struct
vhost_device_ops to struct rte_vhost_device_ops.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: remove copy threshold for async path

Copy threshold has been introduced in async vhost data
path to select the appropriate copy engine to do copies
for higher efficiency.

However, it may cau

vhost: remove copy threshold for async path

Copy threshold has been introduced in async vhost data
path to select the appropriate copy engine to do copies
for higher efficiency.

However, it may cause packets ordering issues and also
introduces performance unpredictability.

Therefore, this patch removes copy threshold support in
async vhost data path.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: add unsafe async API to clear packets

Applications need to stop DMA transfers and finish all the inflight
packets when in VM memory hot-plug case and async vhost is used. This
patch is to pro

vhost: add unsafe async API to clear packets

Applications need to stop DMA transfers and finish all the inflight
packets when in VM memory hot-plug case and async vhost is used. This
patch is to provide an unsafe API to clear inflight packets which
are submitted to DMA engine in vhost async data path. Update the
program guide and release notes for virtqueue inflight packets clear
API in vhost lib.

Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: add thread-unsafe async registration

This patch adds thread unsafe version for async register and
unregister functions.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Chenbo Xia <

vhost: add thread-unsafe async registration

This patch adds thread unsafe version for async register and
unregister functions.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: rework async configuration structure

This patch reworks the async configuration structure to improve code
readability. In addition, add preserved padding fields on the structure
for future us

vhost: rework async configuration structure

This patch reworks the async configuration structure to improve code
readability. In addition, add preserved padding fields on the structure
for future usage.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: allow to check in-flight packets for async vhost

This patch allows to check the amount of in-flight packets
for the vhost queue using async acceleration.

Signed-off-by: Jiayu Hu <jiayu.hu@in

vhost: allow to check in-flight packets for async vhost

This patch allows to check the amount of in-flight packets
for the vhost queue using async acceleration.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: fix offload flags in Rx path

The vhost library currently configures Tx offloading (PKT_TX_*) on any
packet received from a guest virtio device which asks for some offloading.

This is problem

vhost: fix offload flags in Rx path

The vhost library currently configures Tx offloading (PKT_TX_*) on any
packet received from a guest virtio device which asks for some offloading.

This is problematic, as Tx offloading is something that the application
must ask for: the application needs to configure devices
to support every used offloads (ip, tcp checksumming, tso..), and the
various l2/l3/l4 lengths must be set following any processing that
happened in the application itself.

On the other hand, the received packets are not marked wrt current
packet l3/l4 checksumming info.

Copy virtio rx processing to fix those offload flags with some
differences:
- accept VIRTIO_NET_HDR_GSO_ECN and VIRTIO_NET_HDR_GSO_UDP,
- ignore anything but the VIRTIO_NET_HDR_F_NEEDS_CSUM flag (to comply with
the virtio spec),

Some applications might rely on the current behavior, so it is left
untouched by default.
A new RTE_VHOST_USER_NET_COMPLIANT_OL_FLAGS flag is added to enable the
new behavior.

The vhost example has been updated for the new behavior: TSO is applied to
any packet marked LRO.

Fixes: 859b480d5afd ("vhost: add guest offload setting")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

doc: update async vhost register/unregister

This patch is to update programmer guide for register/unregister
copy devices in vhost.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Maxime

doc: update async vhost register/unregister

This patch is to update programmer guide for register/unregister
copy devices in vhost.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: enhance async enqueue for small packets

Async enqueue offloads large copies to DMA devices, and small copies
are still performed by the CPU. However, it requires users to get
enqueue complete

vhost: enhance async enqueue for small packets

Async enqueue offloads large copies to DMA devices, and small copies
are still performed by the CPU. However, it requires users to get
enqueue completed packets by rte_vhost_poll_enqueue_completed(), even
if they are completed by the CPU when rte_vhost_submit_enqueue_burst()
returns. This design incurs extra overheads of tracking completed
pktmbufs and function calls, thus degrading performance on small packets.

This patch enhances async enqueue for small packets by enabling
rte_vhost_submit_enqueue_burst() to return completed packets.

Signed-off-by: Jiayu Hu <jiayu.hu@intel.com>
Tested-by: Yinan Wang <yinan.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: remove dequeue zero-copy support

Dequeue zero-copy removal was announced in DPDK v20.08.
This feature brings constraints which makes the maintenance
of the Vhost library difficult. Its limita

vhost: remove dequeue zero-copy support

Dequeue zero-copy removal was announced in DPDK v20.08.
This feature brings constraints which makes the maintenance
of the Vhost library difficult. Its limitations makes it also
difficult to use by the applications (Tx vring starvation).

Removing it makes it easier to add new features, and also remove
some code in the hot path, which should bring a performance
improvement for the standard path.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

doc: describe async API in vhost guide

Update vhost guides to document vhost async APIs

Signed-off-by: Patrick Fu <patrick.fu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by:

doc: describe async API in vhost guide

Update vhost guides to document vhost async APIs

Signed-off-by: Patrick Fu <patrick.fu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...