lib: replace logging helpers

This is a preparation step before the next change.

Many libraries have their own logging helpers that do not add a newline
in their format string.
Some previous changes

lib: replace logging helpers

This is a preparation step before the next change.

Many libraries have their own logging helpers that do not add a newline
in their format string.
Some previous changes fixed places where some of those helpers are
called without a trailing newline.
Using RTE_LOG_LINE in the existing helpers will ensure we don't
introduce new issues in the future.

The problem is that if we simply convert to the RTE_LOG_LINE helper,
a future fix may introduce a regression since the logging helper
change won't be backported.

To address this concern, rename existing helpers: backporting a call to
them will trigger some conflict or build issue in LTS branches.

Note:
- bpf and vhost that still has some debug multilines messages, a direct
call to RTE_LOG/RTE_LOG_DP is used: this will make it easier to notice
such special cases,
- about previously publicly exposed logging helpers, when such helper is
not publicly used (iow in public inline API), it is removed from the
public API (this is the case for the member library),

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

vhost: improve log for memory dumping configuration

Add the device name as a prefix of logs associated to madvise() calls.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephe

vhost: improve log for memory dumping configuration

Add the device name as a prefix of logs associated to madvise() calls.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
g

use abstracted bit count functions

Now that DPDK provides such bit count functions, make use of them.

This patch was prepared with a "brutal" commandline:

$ old=__builtin_clzll; new=rte_clz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_clz; new=rte_clz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_ctzll; new=rte_ctz64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_ctz; new=rte_ctz32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

$ old=__builtin_popcountll; new=rte_popcount64;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"
$ old=__builtin_popcount; new=rte_popcount32;
git grep -lw $old :^lib/eal/include/rte_bitops.h |
xargs sed -i -e "s#\<$old\>#$new#g"

Then inclusion of rte_bitops.h was added were necessary.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>

show more ...

vhost: add IOTLB cache entry removal callback

VDUSE will need to munmap() the IOTLB entry on removal
from the cache, as it performs mmap() before insertion.

This patch introduces a callback that VD

vhost: add IOTLB cache entry removal callback

VDUSE will need to munmap() the IOTLB entry on removal
from the cache, as it performs mmap() before insertion.

This patch introduces a callback that VDUSE layer will
implement to achieve this.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: add page size info to IOTLB entry

VDUSE will close the file descriptor after having mapped
the shared memory, so it will not be possible to get the
page size afterwards.

This patch adds an n

vhost: add page size info to IOTLB entry

VDUSE will close the file descriptor after having mapped
the shared memory, so it will not be possible to get the
page size afterwards.

This patch adds an new page_shift field to the IOTLB entry,
so that the information will be passed at IOTLB cache
insertion time. The information is stored as a bit shift
value so that IOTLB entry keeps fitting in a single
cacheline.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: add offset field to IOTLB entries

This patch is a preliminary work to prepare for VDUSE
support, for which we need to keep track of the mmaped base
address and offset in order to be able to u

vhost: add offset field to IOTLB entries

This patch is a preliminary work to prepare for VDUSE
support, for which we need to keep track of the mmaped base
address and offset in order to be able to unmap it later
when IOTLB entry is invalidated.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: change to single IOTLB cache per device

This patch simplifies IOTLB implementation and improves
IOTLB memory consumption by having a single IOTLB cache
per device, instead of having one per q

vhost: change to single IOTLB cache per device

This patch simplifies IOTLB implementation and improves
IOTLB memory consumption by having a single IOTLB cache
per device, instead of having one per queue.

In order to not impact performance, it keeps an IOTLB lock
per virtqueue, so that there is no contention between
multiple queue trying to acquire it.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: skip unneeded pages in dump with IOTLB

On IOTLB entry removal, previous fixes took care of not
marking pages shared with other IOTLB entries as DONTDUMP.

However, if an IOTLB entry is spanne

vhost: skip unneeded pages in dump with IOTLB

On IOTLB entry removal, previous fixes took care of not
marking pages shared with other IOTLB entries as DONTDUMP.

However, if an IOTLB entry is spanned on multiple pages,
the other pages were kept as DODUMP while they might not
have been shared with other entries, increasing needlessly
the coredump size.

This patch addresses this issue by excluding only the
shared pages from madvise's DONTDUMP.

Fixes: dea092d0addb ("vhost: fix madvise arguments alignment")
Cc: stable@dpdk.org

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: add helper for IOTLB entries shared page check

This patch introduces a helper to check whether two IOTLB
entries share a page.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Ack

vhost: add helper for IOTLB entries shared page check

This patch introduces a helper to check whether two IOTLB
entries share a page.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: add helper of IOTLB entries coredump

This patch reworks IOTLB code to extract madvise-related
bits into dedicated helper. This refactoring improves code
sharing.

Signed-off-by: Maxime Coquel

vhost: add helper of IOTLB entries coredump

This patch reworks IOTLB code to extract madvise-related
bits into dedicated helper. This refactoring improves code
sharing.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: fix IOTLB entries overlap check with previous entry

Commit 22b6d0ac691a ("vhost: fix madvise IOTLB entries pages overlap check")
fixed the check to ensure the entry to be removed does not
ove

vhost: fix IOTLB entries overlap check with previous entry

Commit 22b6d0ac691a ("vhost: fix madvise IOTLB entries pages overlap check")
fixed the check to ensure the entry to be removed does not
overlap with the next one in the IOTLB cache before marking
it as DONTDUMP with madvise(). This is not enough, because
the same issue is present when comparing with the previous
entry in the cache, where the end address of the previous
entry should be used, not the start one.

Fixes: dea092d0addb ("vhost: fix madvise arguments alignment")
Cc: stable@dpdk.org

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: fix madvise IOTLB entries pages overlap check

At removal time, when testing whether the IOTLB entry has
shared pages with the previous and next entries in the
cache, it checks whether the sta

vhost: fix madvise IOTLB entries pages overlap check

At removal time, when testing whether the IOTLB entry has
shared pages with the previous and next entries in the
cache, it checks whether the start address of the entry to
be removed is on the same page as the start address of the
next entry in the cache.

This is not correct, as an entry could cover several page
so the end address of the entry to be remove should be
used. This patch address this issue.

Note there is another issue not fixed by this patch, but
delayed to next release given its minor impact and the
complexity of the fix it requires. If a removed IOTLB entry
is spanned on several pages and one of the pages is shared
with another entry, all the pages will remain as DODUMP
while only the shared page should be. It would result in
non-shared pages to be part of the coredump while it would
not be needed.

Fixes: dea092d0addb ("vhost: fix madvise arguments alignment")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: fix madvise arguments alignment

The arguments passed to madvise should be aligned to the alignment of
the backing memory. Now we keep track of each regions alignment and use
then when setting

vhost: fix madvise arguments alignment

The arguments passed to madvise should be aligned to the alignment of
the backing memory. Now we keep track of each regions alignment and use
then when setting coredump preferences. To facilitate this, a new member
was added to rte_vhost_mem_region. A new function was added to easily
translate memory address back to region alignment. Unneeded calls to
madvise were reduced, as the cache removal case should already be
covered by the cache insertion case. The previously inline function
mem_set_dump was removed from a header file and made not inline.

Fixes: 338ad77c9ed3 ("vhost: exclude VM hugepages from coredumps")

Signed-off-by: Mike Pattrick <mkp@redhat.com>
Tested-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: exclude VM hugepages from coredumps

Currently if an application wants to include shared hugepages in
coredumps in conjunction with the vhost library, the coredump will be
larger than expected

vhost: exclude VM hugepages from coredumps

Currently if an application wants to include shared hugepages in
coredumps in conjunction with the vhost library, the coredump will be
larger than expected and include unneeded virtual machine memory.

This patch will mark all vhost huge pages as DONTDUMP, except for some
select pages used by DPDK.

Signed-off-by: Mike Pattrick <mkp@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: reduce memory footprint when IOMMU is disabled

If an application does not request IOMMU support, we can avoid
allocating a IOMMU pool.

This saves 112kB (IOTLB_CACHE_SIZE * sizeof(struct vhos

vhost: reduce memory footprint when IOMMU is disabled

If an application does not request IOMMU support, we can avoid
allocating a IOMMU pool.

This saves 112kB (IOTLB_CACHE_SIZE * sizeof(struct vhost_iotlb_entry))
per vq.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

vhost: stop using mempool for IOTLB cache

A mempool consumes 3 memzones (with the default ring mempool driver).
The default DPDK configuration allows RTE_MAX_MEMZONE (2560) memzones.

Assuming there

vhost: stop using mempool for IOTLB cache

A mempool consumes 3 memzones (with the default ring mempool driver).
The default DPDK configuration allows RTE_MAX_MEMZONE (2560) memzones.

Assuming there is no other memzones that means that we can have a
maximum of 853 mempools.

In the vhost library, the IOTLB cache code so far was requesting a
mempool per vq, which means that at the maximum, the vhost library
could request mempools for 426 qps.

This limit was recently reached on big systems with a lot of virtio
ports (and multiqueue in use).

While the limit on mempool count could be something we fix at the DPDK
project level, there is no reason to use mempools for the IOTLB cache:
- the IOTLB cache entries do not need to be DMA-able and are only used
by the current process (in multiprocess context),
- getting/putting objects from/in the mempool is always associated with
some other locks, so some level of lock contention is already present,

We can convert to a malloc'd pool with objects put in a free list
protected by a spinlock.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: keep a reference to virtqueue index

Having a back reference to the index of the vq in the dev->virtqueue[]
array makes it possible to unify the internal API, with only passing dev
and vq.
It

vhost: keep a reference to virtqueue index

Having a back reference to the index of the vq in the dev->virtqueue[]
array makes it possible to unify the internal API, with only passing dev
and vq.
It also allows displaying the vq index in log messages.

Remove virtqueue index checks where unneeded (like in static helpers
called from a loop on all available virtqueue).
Move virtqueue index validity checks the sooner possible.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

vhost: prefix logs with context

We recently improved the log messages in the vhost library, adding some
context that helps filtering for a given vhost-user device.
However, some parts of the code we

vhost: prefix logs with context

We recently improved the log messages in the vhost library, adding some
context that helps filtering for a given vhost-user device.
However, some parts of the code were missed, and some later code changes
broke this new convention (fixes were sent previous to this patch).

Change the VHOST_LOG_CONFIG/DATA helpers and always ask for a string
used as context. This should help limit regressions on this topic.

Most of the time, the context is the vhost-user device socket path.
For the rest when a vhost-user device can not be related, generic
names were chosen:
- "dma", for vhost-user async DMA operations,
- "device", for vhost-user device creation and lookup,
- "thread", for threads management,

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

show more ...

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free functions
found by nullfree.cocci

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

vhost: differentiate IOTLB logs

Same logging messages were used for both IOTLB cache
insertion failure and IOTLB pending insertion failure.

This patch differentiate them to ease logs analysis.

Sug

vhost: differentiate IOTLB logs

Same logging messages were used for both IOTLB cache
insertion failure and IOTLB pending insertion failure.

This patch differentiate them to ease logs analysis.

Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

vhost: improve IOTLB logs

This patch adds IOTLB mempool name when logging debug
or error messages, and also prepends the socket path.
to all the logs.

Signed-off-by: Maxime Coquelin <maxime.coqueli

vhost: improve IOTLB logs

This patch adds IOTLB mempool name when logging debug
or error messages, and also prepends the socket path.
to all the logs.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

mempool: add namespace to flags

Fix the mempool flags namespace by adding an RTE_ prefix to the name.
The old flags remain usable, to be deprecated in the future.

Flag MEMPOOL_F_NON_IO added in the

mempool: add namespace to flags

Fix the mempool flags namespace by adding an RTE_ prefix to the name.
The old flags remain usable, to be deprecated in the future.

Flag MEMPOOL_F_NON_IO added in the release is just renamed to have RTE_
prefix.

Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>

show more ...

eal: remove sys/queue.h from public headers

Currently there are some public headers that include 'sys/queue.h', which
is not POSIX, but usually provided by the Linux/BSD system library.
(Not in POSI

eal: remove sys/queue.h from public headers

Currently there are some public headers that include 'sys/queue.h', which
is not POSIX, but usually provided by the Linux/BSD system library.
(Not in POSIX.1, POSIX.1-2001, or POSIX.1-2008. Present on the BSDs.)
The file is missing on Windows. During the Windows build, DPDK uses a
bundled copy, so building a DPDK library works fine. But when OVS or other
applications use DPDK as a library, because some DPDK public headers
include 'sys/queue.h', on Windows, it triggers an error due to no such
file.

One solution is to install the 'lib/eal/windows/include/sys/queue.h' into
Windows environment, such as [1]. However, this means DPDK exports the
functionalities of 'sys/queue.h' into the environment, which might cause
symbols, macros, headers clashing with other applications.

The patch fixes it by removing the "#include <sys/queue.h>" from
DPDK public headers, so programs including DPDK headers don't depend
on the system to provide 'sys/queue.h'. When these public headers use
macros such as TAILQ_xxx, we replace it by the ones with RTE_ prefix.
For Windows, we copy the definitions from <sys/queue.h> to rte_os.h
in Windows EAL. Note that these RTE_ macros are compatible with
<sys/queue.h>, both at the level of API (to use with <sys/queue.h>
macros in C files) and ABI (to avoid breaking it).

Additionally, the TAILQ_FOREACH_SAFE is not part of <sys/queue.h>,
the patch replaces it with RTE_TAILQ_FOREACH_SAFE.

[1] http://mails.dpdk.org/archives/dev/2021-August/216304.html

Suggested-by: Nick Connolly <nick.connolly@mayadata.io>
Suggested-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Signed-off-by: William Tu <u9012063@gmail.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

show more ...

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
m

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
makes it awkward to add features referring to individual libraries in the
build - should the lib names be specified with or without the prefix.
Therefore, we can just remove the library prefix and use the library's
unique name as the directory name, i.e. 'eal' rather than 'librte_eal'

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...