log: colorize log output

Like dmesg, colorize the log output (unless redirected to file).
Timestamp is green, the subsystem is in yellow and the message is red
if urgent, boldface if an error, and n

log: colorize log output

Like dmesg, colorize the log output (unless redirected to file).
Timestamp is green, the subsystem is in yellow and the message is red
if urgent, boldface if an error, and normal for info and debug messages.

The default is to not use color since it may disturb automatic tests
and other embedded usage.

Note: add fileno() and isatty() to the Windows wrapper.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

log: add timestamp option

When debugging driver or startup issues, it is useful to have a timestamp
on each message printed.
The messages in syslog already have a timestamp, but often syslog is not

log: add timestamp option

When debugging driver or startup issues, it is useful to have a timestamp
on each message printed.
The messages in syslog already have a timestamp, but often syslog is not
available during testing.

There are multiple timestamp formats similar to Linux dmesg.
The default is time relative since startup (when first step of logging
initialization is done by constructor).
Other alternative formats are delta, ctime, reltime and iso formats.

Example:
$ dpdk-testpmd --log-timestamp -- -i
[ 0.008610] EAL: Detected CPU lcores: 8
[ 0.008634] EAL: Detected NUMA nodes: 1
[ 0.008792] EAL: Detected static linkage of DPDK
[ 0.010620] EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
[ 0.012618] EAL: Selected IOVA mode 'VA'
[ 0.016675] testpmd: No probed ethernet devices
Interactive-mode selected

Note: Windows does not have localtime_r but it does have a similar
function that can be used instead. Add it to rte_os_shim.h.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

log: rework syslog handling

Refactor how syslog is handled, make it common to Linux and FreeBSD
The syslog facility property is better handled in lib/log
rather than in eal.
This also adds syslog su

log: rework syslog handling

Refactor how syslog is handled, make it common to Linux and FreeBSD
The syslog facility property is better handled in lib/log
rather than in eal.
This also adds syslog support to FreeBSD.

Log to syslog only if option is specified.
If no --syslog is given then use only standard error.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

eal: introduce log options parser

Rename existing log level option parser and add a little helper tracking
all options related to logs.
This will make it easier to add more log options in the next c

eal: introduce log options parser

Rename existing log level option parser and add a little helper tracking
all options related to logs.
This will make it easier to add more log options in the next commits.

Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

eal: factorize parser for log level

The code to parse for log-level option should be same on all OS variants.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Tyler Retzlaff

eal: factorize parser for log level

The code to parse for log-level option should be same on all OS variants.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

eal: fix logs for '--lcores'

The argument is '--lcores' but it is being logged as '--lcore'
in several places.

e.g.
$ dpdk-testpmd -l 8,10,12 --lcores 8,10,12
EAL: Option --lcore is ignored, becaus

eal: fix logs for '--lcores'

The argument is '--lcores' but it is being logged as '--lcore'
in several places.

e.g.
$ dpdk-testpmd -l 8,10,12 --lcores 8,10,12
EAL: Option --lcore is ignored, because (-l) is set!

Fix the spelling in logs and tests.

Fixes: 117eaa70584b ("eal: add error check for core options")
Fixes: 128ee4c26d87 ("app/test: add unit tests for --lcores option")

Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>

show more ...

eal: verify strdup return

Add verify strdup() return value logic.

Fixes: 293c53d8b23c ("eal: add telemetry callbacks")
Fixes: 0d0f478d0483 ("eal/linux: add uevent parse and process")
Cc: stable@dpd

eal: verify strdup return

Add verify strdup() return value logic.

Fixes: 293c53d8b23c ("eal: add telemetry callbacks")
Fixes: 0d0f478d0483 ("eal/linux: add uevent parse and process")
Cc: stable@dpdk.org

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

lib: add more logging helpers

Add helpers for logging messages in libraries instead of calling
RTE_LOG() directly.
Those helpers take care of adding a \n: this will make the transition to
RTE_LOG_LI

lib: add more logging helpers

Add helpers for logging messages in libraries instead of calling
RTE_LOG() directly.
Those helpers take care of adding a \n: this will make the transition to
RTE_LOG_LINE trivial.

Note:
- for acl and sched libraries that still has some debug multilines
messages, a direct call to RTE_LOG is used: this will make it easier to
notice such special cases,

Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

lib: add newline in logs

Fix places leading to a log message not terminated with a newline.

Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminge

lib: add newline in logs

Fix places leading to a log message not terminated with a newline.

Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

show more ...

lib: remove pthread.h from includes

The header files should have the minimum embedded includes.
The file pthread.h can logically be removed from
rte_per_lcore.h and rte_ethdev_core.h files.

Signed-

lib: remove pthread.h from includes

The header files should have the minimum embedded includes.
The file pthread.h can logically be removed from
rte_per_lcore.h and rte_ethdev_core.h files.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>

show more ...

log: separate logging functions out of EAL

Move the logging capability to a separate library, free from EAL. Rename
files as appropriate, and use meson.build to select the correct file to
be built f

log: separate logging functions out of EAL

Move the logging capability to a separate library, free from EAL. Rename
files as appropriate, and use meson.build to select the correct file to
be built for each operating system, rather than having a subdir per-os.
Add new documentation section in programmers guide to cover logging in
more detail.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>

show more ...

eal: use macros for help option

The macros OPT_HELP and OPT_HELP_NUM were not used where appropriate.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richards

eal: use macros for help option

The macros OPT_HELP and OPT_HELP_NUM were not used where appropriate.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

eal: stop using pthread for lcores and control threads

Replace the use of pthread_t in struct lcore_config with the EAL
rte_thread_t type.

Replace the direct use of pthread_create(), pthread_self()

eal: stop using pthread for lcores and control threads

Replace the use of pthread_t in struct lcore_config with the EAL
rte_thread_t type.

Replace the direct use of pthread_create(), pthread_self()
pthread_getaffinity_np() and pthread_setaffinity_np().

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

eal: export coremask parsing helper

DLB2 has a need to parse a user supplied coremask as part
of an optimization that associates optimal core/resource
pairs. Therefore eal_parse_coremask has been re

eal: export coremask parsing helper

DLB2 has a need to parse a user supplied coremask as part
of an optimization that associates optimal core/resource
pairs. Therefore eal_parse_coremask has been renamed
to rte_eal_parse_coremask and exported but kept internal.

Signed-off-by: Abdullah Sevincer <abdullah.sevincer@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

eal/linux: allocate worker lcore stacks in hugepages

Add support for using hugepages for worker lcore stack memory. The
intent is to improve performance by reducing stack memory related TLB
misses a

eal/linux: allocate worker lcore stacks in hugepages

Add support for using hugepages for worker lcore stack memory. The
intent is to improve performance by reducing stack memory related TLB
misses and also by using memory local to the NUMA node of each lcore.

EAL option '--huge-worker-stack[=stack-size-in-kbytes]' is added to allow
the feature to be enabled at runtime. If the size is not specified,
the system pthread stack size will be used.

Signed-off-by: Don Wallwork <donw@xsightlabs.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

lib: remove unneeded header includes

These header includes have been flagged by the iwyu_tool
and removed.

Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free

remove unnecessary null checks

Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.

Remove redundant NULL pointer checks before free functions
found by nullfree.cocci

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

eal: extend --huge-unlink for hugepage file reuse

Expose Linux EAL ability to reuse existing hugepage files
via --huge-unlink=never switch.
Default behavior is unchanged, it can also be specified
us

eal: extend --huge-unlink for hugepage file reuse

Expose Linux EAL ability to reuse existing hugepage files
via --huge-unlink=never switch.
Default behavior is unchanged, it can also be specified
using --huge-unlink=existing for consistency.
Old --huge-unlink switch is kept,
it is an alias for --huge-unlink=always.
Add a test case for the --huge-unlink=never mode.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

eal/linux: allow hugepage file reuse

Linux EAL ensured that mapped hugepages are clean
by always mapping from newly created files:
existing hugepage backing files were always removed.
In this case,

eal/linux: allow hugepage file reuse

Linux EAL ensured that mapped hugepages are clean
by always mapping from newly created files:
existing hugepage backing files were always removed.
In this case, the kernel clears the page to prevent data leaks,
because the mapped memory may contain leftover data
from the previous process that was using this memory.
Clearing takes the bulk of the time spent in mmap(2),
increasing EAL initialization time.

Introduce a mode to keep existing files and reuse them
in order to speed up initial memory allocation in EAL.
Hugepages mapped from such files may contain data
left by the previous process that used this memory,
so RTE_MEMSEG_FLAG_DIRTY is set for their segments.
If multiple hugepages are mapped from the same file:
1. When fallocate(2) is used, all memory mapped from this file
is considered dirty, because it is unknown
which parts of the file are holes.
2. When ftruncate(3) is used, memory mapped from this file
is considered dirty unless the file is extended
to create a new mapping, which implies clean memory.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

eal: refactor --huge-unlink storage

In preparation to extend --huge-unlink option semantics
refactor how it is stored in the internal configuration.
It makes future changes more isolated.

Signed-of

eal: refactor --huge-unlink storage

In preparation to extend --huge-unlink option semantics
refactor how it is stored in the internal configuration.
It makes future changes more isolated.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

eal: suggest using --lcores option

If the user requests to use an lcore above 128 using -l,
the eal will exit with "EAL: invalid core list syntax" and
very little else useful information.

This patc

eal: suggest using --lcores option

If the user requests to use an lcore above 128 using -l,
the eal will exit with "EAL: invalid core list syntax" and
very little else useful information.

This patch adds some extra information suggesting to use --lcores
so that physical cores above RTE_MAX_LCORE (default 128) can be
used. This is achieved by using the --lcores option by mapping
the logical cores in the application to physical cores.

For example, if "-l 12-16,130,132" is used, we see the following
additional output on the command line:

EAL: lcore 132 >= RTE_MAX_LCORE (128)
EAL: lcore 133 >= RTE_MAX_LCORE (128)
EAL: To use high physical core ids, please use --lcores to map them
to lcore ids below RTE_MAX_LCORE,
EAL: e.g. --lcores 0@12,1@13,2@14,3@15,4@16,5@132,6@133

The same is added to -c option parsing.

For example, if "-c 0x300000000000000000000000000000000" is
used, we see the following additional output on the command line:

EAL: lcore 128 >= RTE_MAX_LCORE (128)
EAL: lcore 129 >= RTE_MAX_LCORE (128)
EAL: To use high physical core ids, please use --lcores to map them
to lcore ids below RTE_MAX_LCORE,
EAL: e.g. --lcores 0@128,1@129

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

eal: remove sys/queue.h from public headers

Currently there are some public headers that include 'sys/queue.h', which
is not POSIX, but usually provided by the Linux/BSD system library.
(Not in POSI

eal: remove sys/queue.h from public headers

Currently there are some public headers that include 'sys/queue.h', which
is not POSIX, but usually provided by the Linux/BSD system library.
(Not in POSIX.1, POSIX.1-2001, or POSIX.1-2008. Present on the BSDs.)
The file is missing on Windows. During the Windows build, DPDK uses a
bundled copy, so building a DPDK library works fine. But when OVS or other
applications use DPDK as a library, because some DPDK public headers
include 'sys/queue.h', on Windows, it triggers an error due to no such
file.

One solution is to install the 'lib/eal/windows/include/sys/queue.h' into
Windows environment, such as [1]. However, this means DPDK exports the
functionalities of 'sys/queue.h' into the environment, which might cause
symbols, macros, headers clashing with other applications.

The patch fixes it by removing the "#include <sys/queue.h>" from
DPDK public headers, so programs including DPDK headers don't depend
on the system to provide 'sys/queue.h'. When these public headers use
macros such as TAILQ_xxx, we replace it by the ones with RTE_ prefix.
For Windows, we copy the definitions from <sys/queue.h> to rte_os.h
in Windows EAL. Note that these RTE_ macros are compatible with
<sys/queue.h>, both at the level of API (to use with <sys/queue.h>
macros in C files) and ABI (to avoid breaking it).

Additionally, the TAILQ_FOREACH_SAFE is not part of <sys/queue.h>,
the patch replaces it with RTE_TAILQ_FOREACH_SAFE.

[1] http://mails.dpdk.org/archives/dev/2021-August/216304.html

Suggested-by: Nick Connolly <nick.connolly@mayadata.io>
Suggested-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Signed-off-by: William Tu <u9012063@gmail.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>

show more ...

eal: fix memory leak when saving arguments

This patch fixes a memleak which was reported in Bugzilla within the
eal_save_args function. This was caused by the function mistakenly
adding -- to the ea

eal: fix memory leak when saving arguments

This patch fixes a memleak which was reported in Bugzilla within the
eal_save_args function. This was caused by the function mistakenly
adding -- to the eal args instead of breaking beforehand.

Bugzilla ID: 722
Fixes: 293c53d8b23c ("eal: add telemetry callbacks")

Reported-by: Zhihong Peng <zhihongx.peng@intel.com>
Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Signed-off-by: Conor Fogarty <conor.fogarty@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...

eal: remove deprecated noninclusive API

New API for these were added in 20.11 and the old API was retained
but marked deprecated. Since 21.11 is the next LTS, it is time
to remove the deprecated one

eal: remove deprecated noninclusive API

New API for these were added in 20.11 and the old API was retained
but marked deprecated. Since 21.11 is the next LTS, it is time
to remove the deprecated ones.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

eal: fix leak in shared lib mode detection

This is reported by our internal covscan:

1. dpdk-20.11/lib/librte_eal/common/eal_common_options.c:508: alloc_fn:
Storage is returned from allocation func

eal: fix leak in shared lib mode detection

This is reported by our internal covscan:

1. dpdk-20.11/lib/librte_eal/common/eal_common_options.c:508: alloc_fn:
Storage is returned from allocation function "dlopen".
6. dpdk-20.11/lib/librte_eal/common/eal_common_options.c:508:
leaked_storage: Failing to save or free storage allocated by
"dlopen("librte_eal.so.21.0", 5)" leaks it.

# 506| * shared library is not already loaded i.e. it's
# statically linked.)
# 507| */
# 508|-> if (dlopen("librte_eal.so."ABI_VERSION, RTLD_LAZY |
# RTLD_NOLOAD) != NULL &&
# 509| *default_solib_dir != '\0' &&
# 510| stat(default_solib_dir, &sb) == 0 &&

This leak is not an issue per se, but on the other hand, this is easy
to fix and I prefer not having to waive this warning later.

Fixes: 06c7871dde01 ("eal: restrict default plugin path to shared lib mode")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...