mem: remove type argument from internal routines

The type argument is carried through malloc heap routines but
never used there. It is only used in rte_malloc for tracing.

Signed-off-by: Stephen He

mem: remove type argument from internal routines

The type argument is carried through malloc heap routines but
never used there. It is only used in rte_malloc for tracing.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

lib: add more logging helpers

Add helpers for logging messages in libraries instead of calling
RTE_LOG() directly.
Those helpers take care of adding a \n: this will make the transition to
RTE_LOG_LI

lib: add more logging helpers

Add helpers for logging messages in libraries instead of calling
RTE_LOG() directly.
Those helpers take care of adding a \n: this will make the transition to
RTE_LOG_LINE trivial.

Note:
- for acl and sched libraries that still has some debug multilines
messages, a direct call to RTE_LOG is used: this will make it easier to
notice such special cases,

Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

eal: enhance NUMA affinity heuristic

When a DPDK application is started on only one numa node, memory is
allocated for only one socket. When interrupt threads use memory,
memory may not be found on

eal: enhance NUMA affinity heuristic

When a DPDK application is started on only one numa node, memory is
allocated for only one socket. When interrupt threads use memory,
memory may not be found on the socket where the interrupt thread
is currently located, and memory has to be reallocated on the hugepage,
this operation will lead to performance degradation.

Fixes: 705356f0811f ("eal: simplify control thread creation")
Fixes: 770d41bf3309 ("malloc: fix allocation with unknown socket ID")

Signed-off-by: Kaisen You <kaisenx.you@intel.com>
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

malloc: simplify condition

The alt_elem is initialized to null.
Checking only size hint flag is sufficient to return a proper value.
Removed the redundant check.

Signed-off-by: Ruifeng Wang <ruifen

malloc: simplify condition

The alt_elem is initialized to null.
Checking only size hint flag is sufficient to return a proper value.
Removed the redundant check.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Feifei Wang <feifei.wang2@arm.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

mem: rework malloc heap init

rte_eal_memory_init() and rte_eal_malloc_heap_init() must be called in
a common section taking rte_mcfg_mem_read_lock().
Split rte_eal_malloc_heap_init() in two so that

mem: rework malloc heap init

rte_eal_memory_init() and rte_eal_malloc_heap_init() must be called in
a common section taking rte_mcfg_mem_read_lock().
Split rte_eal_malloc_heap_init() in two so that the mem lock is taken
in rte_eal_init() making lock checks trivial (once annotated in the next
patch).

Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

malloc: rework heap destroy

The destroy helper has been reworked to zero all the heap object but
leave the lock untouched. The heap lock is then released through the
standard API.

Signed-off-by: Da

malloc: rework heap destroy

The destroy helper has been reworked to zero all the heap object but
leave the lock untouched. The heap lock is then released through the
standard API.

Signed-off-by: David Marchand <david.marchand@redhat.com>

show more ...

malloc: fix storage size for some allocations

The amount of memory to allocate from the system for heap expansion
was calculated in a way that may yield one page more than needed.
This could hit the

malloc: fix storage size for some allocations

The amount of memory to allocate from the system for heap expansion
was calculated in a way that may yield one page more than needed.
This could hit the allocation limit from the system or EAL.
The allocation would fail despite enough memory being available.

In response to mail:
https://inbox.dpdk.org/dev/CAEYuUWCnRZNwxiOHEeTHw0Gy9aFJRLZtvAG9g=smuUvUEMcFXg@mail.gmail.com/

A reproducer has been provided by Dmitry, see:
https://inbox.dpdk.org/dev/20220922015212.03bfde66@sovereign/

Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug")
Cc: stable@dpdk.org

Signed-off-by: Fidaullah Noonari <fidaullah.noonari@emumba.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

show more ...

malloc: fix allocation of almost hugepage size

If called to allocate memory of size is between multiple of hugepage
size minus malloc_header_len and hugepage size, rte_malloc fails.

This fix replac

malloc: fix allocation of almost hugepage size

If called to allocate memory of size is between multiple of hugepage
size minus malloc_header_len and hugepage size, rte_malloc fails.

This fix replaces malloc_elem_trailer_len with malloc_elem_overhead in
try_expand_heap() to include malloc_elem_header_len when calculating
n_seg.

Bugzilla ID: 800
Fixes: 07dcbfe0101f ("malloc: support multiprocess memory hotplug")
Cc: stable@dpdk.org

Signed-off-by: Fidaullah Noonari <fidaullah.noonari@emumba.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>

show more ...

malloc: fix ASan handling for unmapped memory

Currently, when we free previously allocated memory, we mark the area as
"freed" for ASan purposes (flag 0xfd). However, sometimes, freeing a
malloc ele

malloc: fix ASan handling for unmapped memory

Currently, when we free previously allocated memory, we mark the area as
"freed" for ASan purposes (flag 0xfd). However, sometimes, freeing a
malloc element will cause pages to be unmapped from memory and re-backed
with anonymous memory again. This may cause ASan's "use-after-free"
error down the line, because the allocator will try to write into
memory areas recently marked as "freed".

To fix this, we need to mark the unmapped memory area as "available",
and fixup surrounding malloc element header/trailers to enable later
malloc routines to safely write into new malloc elements' headers or
trailers.

Bugzilla ID: 994
Fixes: 6cc51b1293ce ("mem: instrument allocator for ASan")
Cc: stable@dpdk.org

Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

lib: remove unneeded header includes

These header includes have been flagged by the iwyu_tool
and removed.

Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>

mem: cleanup multiprocess resources

The mp action resources in malloc should be cleaned up via
rte_eal_cleanup.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burak

mem: cleanup multiprocess resources

The mp action resources in malloc should be cleaned up via
rte_eal_cleanup.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

mem: add dirty malloc element support

EAL malloc layer assumed all free elements content
is filled with zeros ("clean"), as opposed to uninitialized ("dirty").
This assumption was ensured in two way

mem: add dirty malloc element support

EAL malloc layer assumed all free elements content
is filled with zeros ("clean"), as opposed to uninitialized ("dirty").
This assumption was ensured in two ways:
1. EAL memalloc layer always returned clean memory.
2. Freed memory was cleared before returning into the heap.

Clearing the memory can be as slow as around 14 GiB/s.
To save doing so, memalloc layer is allowed to return dirty memory.
Such segments being marked with RTE_MEMSEG_FLAG_DIRTY.
The allocator tracks elements that contain dirty memory
using the new flag in the element header.
When clean memory is requested via rte_zmalloc*()
and the suitable element is dirty, it is cleared on allocation.
When memory is deallocated, the freed element is joined
with adjacent free elements, and the dirty flag is updated:

a) If the joint element contains dirty parts, it is dirty:

dirty + freed + dirty = dirty => no need to clean
freed + dirty = dirty the freed memory

Dirty parts may be large (e.g. initial allocation),
so clearing them could create unpredictable slowdown.

b) If the only dirty part of the joint element
is the freed memory, the joint element can be made clean:

clean + freed + clean = clean => freed memory
clean + freed = clean must be cleared
freed + clean = clean
freed = clean

This logic naturally reproduces the old behavior
and always applies in modes when EAL memalloc layer
returns only clean segments.

As a result, memory is either cleared on free, as before,
or it will be cleared on allocation if need be, but never twice.

Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

malloc: fix allocation with unknown socket ID

When using rte_malloc() from a thread which is not bound to a numa
socket (the typical case is a control thread, but it can also happen
on a dataplane t

malloc: fix allocation with unknown socket ID

When using rte_malloc() from a thread which is not bound to a numa
socket (the typical case is a control thread, but it can also happen
on a dataplane thread if its cpu affinity is on cores attached to
several sockets), the used heap is the one from numa socket 0, which
may not have available memory.

Fix this by selecting the first socket which has available memory.

Note: malloc_get_numa_socket() is only used from one .c file, so move
it there, and remove the inline keyword.

Fixes: b94580d6887e ("malloc: avoid unknown socket id")
Cc: stable@dpdk.org

Signed-off-by: Ilyes Ben Hamouda <ilyes.ben_hamouda@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: David Marchand <david.marchand@redhat.com>

show more ...

mem: instrument allocator for ASan

This patch adds necessary hooks in the memory allocator for ASan.

This feature is currently available in DPDK only on Linux x86_64.
If other OS/architectures want

mem: instrument allocator for ASan

This patch adds necessary hooks in the memory allocator for ASan.

This feature is currently available in DPDK only on Linux x86_64.
If other OS/architectures want to support it, ASAN_SHADOW_OFFSET must be
defined and RTE_MALLOC_ASAN must be set accordingly in meson.

Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

show more ...

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
m

lib: remove librte_ prefix from directory names

There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
makes it awkward to add features referring to individual libraries in the
build - should the lib names be specified with or without the prefix.
Therefore, we can just remove the library prefix and use the library's
unique name as the directory name, i.e. 'eal' rather than 'librte_eal'

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

show more ...