log: rework syslog handling

Refactor how syslog is handled, make it common to Linux and FreeBSD
The syslog facility property is better handled in lib/log
rather than in eal.
This also adds syslog su

log: rework syslog handling

Refactor how syslog is handled, make it common to Linux and FreeBSD
The syslog facility property is better handled in lib/log
rather than in eal.
This also adds syslog support to FreeBSD.

Log to syslog only if option is specified.
If no --syslog is given then use only standard error.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

doc: add lcore variables guide

Add lcore variables programmer's guide. This guide gives both an
overview of the API, its implementation, and alternatives to the use
of lcore variables for maintainin

doc: add lcore variables guide

Add lcore variables programmer's guide. This guide gives both an
overview of the API, its implementation, and alternatives to the use
of lcore variables for maintaining per-lcore id data.

It has pictures, too.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Reviewed-by: Luka Jankovic <luka.jankovic@ericsson.com>

show more ...

eal: add static per-lcore memory allocation facility

Introduce DPDK per-lcore id variables, or lcore variables for short.

An lcore variable has one value for every current and future lcore
id-equip

eal: add static per-lcore memory allocation facility

Introduce DPDK per-lcore id variables, or lcore variables for short.

An lcore variable has one value for every current and future lcore
id-equipped thread.

The primary <rte_lcore_var.h> use case is for statically allocating
small, frequently-accessed data structures, for which one instance
should exist for each lcore.

Lcore variables are similar to thread-local storage (TLS, e.g., C11
_Thread_local), but decoupling the values' life time with that of the
threads.

Lcore variables are also similar in terms of functionality provided by
FreeBSD kernel's DPCPU_*() family of macros and the associated
build-time machinery. DPCPU uses linker scripts, which effectively
prevents the reuse of its, otherwise seemingly viable, approach.

The currently-prevailing way to solve the same problem as lcore
variables is to keep a module's per-lcore data as RTE_MAX_LCORE-sized
array of cache-aligned, RTE_CACHE_GUARDed structs. The benefit of
lcore variables over this approach is that data related to the same
lcore now is close (spatially, in memory), rather than data used by
the same module, which in turn avoid excessive use of padding,
polluting caches with unused data.

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

doc: reorganize prog guide

Create categories in the index of the programmer's guide,
sort chapters and rewrite some titles for consistency.

Subdirectories are created for ethdev and eventdev
for gr

doc: reorganize prog guide

Create categories in the index of the programmer's guide,
sort chapters and rewrite some titles for consistency.

Subdirectories are created for ethdev and eventdev
for grouping the files together.

Useless link anchors at the beginning of files are removed,
the corresponding :ref: are replaced with :doc: links.

Signed-off-by: David Young <dave@youngcopy.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

doc: use ordered lists

Prefer automatically ordered lists by using #.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Dar

doc: use ordered lists

Prefer automatically ordered lists by using #.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>

show more ...

eal: remove deprecated thread functions

The deprecated functions rte_thread_setname() and rte_ctrl_thread_create()
are replaced with the new rte_thread API:

rte_thread_setname()
can be replaced wi

eal: remove deprecated thread functions

The deprecated functions rte_thread_setname() and rte_ctrl_thread_create()
are replaced with the new rte_thread API:

rte_thread_setname()
can be replaced with
rte_thread_set_name()
or rte_thread_set_prefixed_name()

rte_ctrl_thread_create()
can be replaced with
rte_thread_create_control()
or rte_thread_create_internal_control()

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>

show more ...

log: separate logging functions out of EAL

Move the logging capability to a separate library, free from EAL. Rename
files as appropriate, and use meson.build to select the correct file to
be built f

log: separate logging functions out of EAL

Move the logging capability to a separate library, free from EAL. Rename
files as appropriate, and use meson.build to select the correct file to
be built for each operating system, rather than having a subdir per-os.
Add new documentation section in programmers guide to cover logging in
more detail.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

show more ...

remove KNI

The KNI driver had design flaws such as calling userspace with kernel
mutex held that made it prone to deadlock. The design also introduced
security risks because the kernel driver truste

remove KNI

The KNI driver had design flaws such as calling userspace with kernel
mutex held that made it prone to deadlock. The design also introduced
security risks because the kernel driver trusted the userspace
(DPDK) kni interface. The kernel driver was never reviewed by
the upstream kernel community and would never have been accepted.

And since the Linux kernel API is not stable, it was a continual
source of maintenance issues especially with distribution kernels.

There are better ways to inject packets into the kernel such as
virtio_user, tap and XDP drivers. All of these do not need out of
tree kernel drivers.

The deprecation was announced in 22.11 release (see links for the
the techboard discussions); and users were directed to alternatives
there.

Link: https://mails.dpdk.org/archives/dev/2021-January/197077.html
Link: https://mails.dpdk.org/archives/dev/2022-June/243596.html

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

enable lock check

Now that a lot of components can be compiled with the lock checks,
invert the logic and opt out for components not ready yet:
- drivers/bus/dpaa,
- drivers/common/cnxk,
- drivers/c

enable lock check

Now that a lot of components can be compiled with the lock checks,
invert the logic and opt out for components not ready yet:
- drivers/bus/dpaa,
- drivers/common/cnxk,
- drivers/common/mlx5,
- drivers/event/cnxk,
- drivers/net/bnx2x,
- drivers/net/bnxt,
- drivers/net/cnxk,
- drivers/net/enic,
- drivers/net/hns3,
- drivers/net/mlx5,
- lib/ipsec,
- lib/timer,

The FreeBSD pthread API has been annotated but Linux glibc does not have
those annotations. Disable lock checks for FreeBSD where pthread_mutex_*
are used:
- drivers/net/failsafe,
- drivers/net/hinic,
- lib/eal,
- lib/ethdev,

Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: Sachin Saxena <sachin.saxena@oss.nxp.com>

show more ...

eal: annotate spinlock, rwlock and seqlock

clang offers some thread safety checks, statically verifying that locks
are taken and released in the code.
To use those checks, the full code leading to t

eal: annotate spinlock, rwlock and seqlock

clang offers some thread safety checks, statically verifying that locks
are taken and released in the code.
To use those checks, the full code leading to taking or releasing locks
must be annotated with some attributes.

Wrap those attributes into our own set of macros.

rwlock, seqlock and the "normal" spinlock are instrumented.

Those checks might be of interest out of DPDK, but it requires that the
including application locks are annotated.
On the other hand, applications out there might have been using
those same checks.
To be on the safe side, keep this instrumentation under a
RTE_ANNOTATE_LOCKS internal build flag.

A component may en/disable this check by setting
annotate_locks = true/false in its meson.build.

Note:
Doxygen preprocessor does not understand trailing function attributes
(this can be observed with the rte_seqlock.h header).
One would think that expanding the annotation macros to a noop in
rte_lock_annotations.h would be enough (since RTE_ANNOTATE_LOCKS is not
set during doxygen processing)). Unfortunately, the use of
EXPAND_ONLY_PREDEF defeats this.

Removing EXPAND_ONLY_PREDEF entirely is not an option as it would expand
all other DPDK macros.

The chosen solution is to expand the annotation macros explicitly to a
noop in PREDEFINED.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Chenbo Xia <chenbo.xia@intel.com>

show more ...

doc: add signal safety warning

The DPDK is not designed to be used from a signal handler.
Add a notice in the documentation describing this limitation,
similar to Linux signal-safety manual page.

B

doc: add signal safety warning

The DPDK is not designed to be used from a signal handler.
Add a notice in the documentation describing this limitation,
similar to Linux signal-safety manual page.

Bugzilla ID: 1030

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

doc: add more instructions for running as non-root

The guide to run DPDK applications as non-root in Linux
did not provide specific instructions to configure the required access
and did not explain

doc: add more instructions for running as non-root

The guide to run DPDK applications as non-root in Linux
did not provide specific instructions to configure the required access
and did not explain why each bit is needed.
The latter is important because running as non-root
is one of the ways to tighten security and grant minimal permissions.

Cc: stable@dpdk.org

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

eal/linux: allocate worker lcore stacks in hugepages

Add support for using hugepages for worker lcore stack memory. The
intent is to improve performance by reducing stack memory related TLB
misses a

eal/linux: allocate worker lcore stacks in hugepages

Add support for using hugepages for worker lcore stack memory. The
intent is to improve performance by reducing stack memory related TLB
misses and also by using memory local to the NUMA node of each lcore.

EAL option '--huge-worker-stack[=stack-size-in-kbytes]' is added to allow
the feature to be enabled at runtime. If the size is not specified,
the system pthread stack size will be used.

Signed-off-by: Don Wallwork <donw@xsightlabs.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

eal: extend --huge-unlink for hugepage file reuse

Expose Linux EAL ability to reuse existing hugepage files
via --huge-unlink=never switch.
Default behavior is unchanged, it can also be specified
us

eal: extend --huge-unlink for hugepage file reuse

Expose Linux EAL ability to reuse existing hugepage files
via --huge-unlink=never switch.
Default behavior is unchanged, it can also be specified
using --huge-unlink=existing for consistency.
Old --huge-unlink switch is kept,
it is an alias for --huge-unlink=always.
Add a test case for the --huge-unlink=never mode.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

mem: add dirty malloc element support

EAL malloc layer assumed all free elements content
is filled with zeros ("clean"), as opposed to uninitialized ("dirty").
This assumption was ensured in two way

mem: add dirty malloc element support

EAL malloc layer assumed all free elements content
is filled with zeros ("clean"), as opposed to uninitialized ("dirty").
This assumption was ensured in two ways:
1. EAL memalloc layer always returned clean memory.
2. Freed memory was cleared before returning into the heap.

Clearing the memory can be as slow as around 14 GiB/s.
To save doing so, memalloc layer is allowed to return dirty memory.
Such segments being marked with RTE_MEMSEG_FLAG_DIRTY.
The allocator tracks elements that contain dirty memory
using the new flag in the element header.
When clean memory is requested via rte_zmalloc*()
and the suitable element is dirty, it is cleared on allocation.
When memory is deallocated, the freed element is joined
with adjacent free elements, and the dirty flag is updated:

a) If the joint element contains dirty parts, it is dirty:

dirty + freed + dirty = dirty => no need to clean
freed + dirty = dirty the freed memory

Dirty parts may be large (e.g. initial allocation),
so clearing them could create unpredictable slowdown.

b) If the only dirty part of the joint element
is the freed memory, the joint element can be made clean:

clean + freed + clean = clean => freed memory
clean + freed = clean must be cleared
freed + clean = clean
freed = clean

This logic naturally reproduces the old behavior
and always applies in modes when EAL memalloc layer
returns only clean segments.

As a result, memory is either cleared on free, as before,
or it will be cleared on allocation if need be, but never twice.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

doc: add hugepage mapping details

Hugepage mapping is a layer of EAL malloc builds upon.
There were implicit references to its details,
like mentions of segment file descriptors,
but no explicit des

doc: add hugepage mapping details

Hugepage mapping is a layer of EAL malloc builds upon.
There were implicit references to its details,
like mentions of segment file descriptors,
but no explicit description of its modes and operation.
Add an overview of mechanics used on ech supported OS.
Convert memory management subsections from list items
to level 4 headers: they are big and important enough.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

fix spelling in comments and strings

The tool comes from https://github.com/jsoref

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

doc: fix a typo in EAL guide

Change from "how many segments each segment can have" to
"how many segments each segment list can have".

Fixes: b31739328354 ("doc: update guides for memory subsystem")

doc: fix a typo in EAL guide

Change from "how many segments each segment can have" to
"how many segments each segment list can have".

Fixes: b31739328354 ("doc: update guides for memory subsystem")
Cc: stable@dpdk.org

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

show more ...

doc: fix spelling

Spell checked and corrected documentation.
If there are any errors, or I have changed something that wasn't an error
please reach out to me so I can update the dictionary.

Cc: sta

doc: fix spelling

Spell checked and corrected documentation.
If there are any errors, or I have changed something that wasn't an error
please reach out to me so I can update the dictionary.

Cc: stable@dpdk.org

Signed-off-by: Henry Nadeau <hnadeau@iol.unh.edu>

show more ...

eal: replace blacklist/whitelist options

Replace -w / --pci-whitelist with -a / --allow options
and --pci-blacklist with --block.
The -b short option remains unchanged.

Allow the old options for no

eal: replace blacklist/whitelist options

Replace -w / --pci-whitelist with -a / --allow options
and --pci-blacklist with --block.
The -b short option remains unchanged.

Allow the old options for now, but print a nag
warning since old options are deprecated.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Luca Boccassi <bluca@debian.org>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

eal: rename lcore master and slave

Replace master lcore with main lcore and
replace slave lcore with worker lcore.

Keep the old functions and macros but mark them as deprecated
for this release.

T

eal: rename lcore master and slave

Replace master lcore with main lcore and
replace slave lcore with worker lcore.

Keep the old functions and macros but mark them as deprecated
for this release.

The "--master-lcore" command line option is also deprecated
and any usage will print a warning and use "--main-lcore"
as replacement.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

eal: control max SIMD bitwidth

This patch adds a max SIMD bitwidth EAL configuration. The API allows
for an app to set this value. It can also be set using EAL argument
--force-max-simd-bitwidth, wh

eal: control max SIMD bitwidth

This patch adds a max SIMD bitwidth EAL configuration. The API allows
for an app to set this value. It can also be set using EAL argument
--force-max-simd-bitwidth, which will lock the value and override any
modifications made by the app.

Each arch has a define for the default SIMD bitwidth value, this is used
on EAL init to set the config max SIMD bitwidth.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>

show more ...

doc: remove references to make from prog guide

Make is no longer supported for compiling DPDK, references are now
removed in the documentation.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Re

doc: remove references to make from prog guide

Make is no longer supported for compiling DPDK, references are now
removed in the documentation.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>

show more ...

eal: register non-EAL threads as lcores

DPDK allows calling some part of its API from a non-EAL thread but this
has some limitations.
OVS (and other applications) has its own thread management but s

eal: register non-EAL threads as lcores

DPDK allows calling some part of its API from a non-EAL thread but this
has some limitations.
OVS (and other applications) has its own thread management but still
want to avoid such limitations by hacking RTE_PER_LCORE(_lcore_id) and
faking EAL threads potentially unknown of some DPDK component.

Introduce a new API to register non-EAL thread and associate them to a
free lcore with a new NON_EAL role.
This role denotes lcores that do not run DPDK mainloop and as such
prevents use of rte_eal_wait_lcore() and consorts.

Multiprocess is not supported as the need for cohabitation with this new
feature is unclear at the moment.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

show more ...

doc: fix a typo in EAL guide

The correct name for virt2memseg API is `rte_mem_virt2memseg`, not
`rte_virt2memseg`.

Fixes: 950e8fb4e194 ("mem: allow registering external memory areas")
Cc: stable@dp

doc: fix a typo in EAL guide

The correct name for virt2memseg API is `rte_mem_virt2memseg`, not
`rte_virt2memseg`.

Fixes: 950e8fb4e194 ("mem: allow registering external memory areas")
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...