1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4.. _Mempool_Library: 5 6Mempool Library 7=============== 8 9A memory pool is an allocator of a fixed-sized object. 10In the DPDK, it is identified by name and uses a mempool handler to store free objects. 11The default mempool handler is ring based. 12It provides some other optional services such as a per-core object cache and 13an alignment helper to ensure that objects are padded to spread them equally on all DRAM or DDR3 channels. 14 15This library is used by the :ref:`Mbuf Library <Mbuf_Library>`. 16 17Cookies 18------- 19 20In debug mode, cookies are added at the beginning and end of allocated blocks. 21The allocated objects then contain overwrite protection fields to help debugging buffer overflows. 22 23Debug mode is disabled by default, 24but can be enabled by setting ``RTE_LIBRTE_MEMPOOL_DEBUG`` in ``config/rte_config.h``. 25 26Stats 27----- 28 29In stats mode, statistics about get from/put in the pool are stored in the mempool structure. 30Statistics are per-lcore to avoid concurrent access to statistics counters. 31 32Stats mode is disabled by default, 33but can be enabled by setting ``RTE_LIBRTE_MEMPOOL_STATS`` in ``config/rte_config.h``. 34 35Memory Alignment Constraints on x86 architecture 36------------------------------------------------ 37 38Depending on hardware memory configuration on X86 architecture, performance can be greatly improved by adding a specific padding between objects. 39The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded. 40 41This is particularly true for packet buffers when doing L3 forwarding or flow classification. 42Only the first 64 bytes are accessed, so performance can be increased by spreading the start addresses of objects among the different channels. 43 44The number of ranks on any DIMM is the number of independent sets of DRAMs that can be accessed for the full data bit-width of the DIMM. 45The ranks cannot be accessed simultaneously since they share the same data path. 46The physical layout of the DRAM chips on the DIMM itself does not necessarily relate to the number of ranks. 47 48When running an application, the EAL command line options provide the ability to add the number of memory channels and ranks. 49 50.. note:: 51 52 The command line must always have the number of memory channels specified for the processor. 53 54Examples of alignment for different DIMM architectures are shown in 55:numref:`figure_memory-management` and :numref:`figure_memory-management2`. 56 57.. _figure_memory-management: 58 59.. figure:: img/memory-management.* 60 61 Two Channels and Quad-ranked DIMM Example 62 63 64In this case, the assumption is that a packet is 16 blocks of 64 bytes, which is not true. 65 66The Intel® 5520 chipset has three channels, so in most cases, 67no padding is required between objects (except for objects whose size are n x 3 x 64 bytes blocks). 68 69.. _figure_memory-management2: 70 71.. figure:: img/memory-management2.* 72 73 Three Channels and Two Dual-ranked DIMM Example 74 75 76When creating a new pool, the user can specify to use this feature or not. 77 78.. _mempool_local_cache: 79 80Local Cache 81----------- 82 83In terms of CPU usage, the cost of multiple cores accessing a memory pool's ring of free buffers may be high 84since each access requires a compare-and-set (CAS) operation. 85To avoid having too many access requests to the memory pool's ring, 86the memory pool allocator can maintain a per-core cache and do bulk requests to the memory pool's ring, 87via the cache with many fewer locks on the actual memory pool structure. 88In this way, each core has full access to its own cache (with locks) of free objects and 89only when the cache fills does the core need to shuffle some of the free objects back to the pools ring or 90obtain more objects when the cache is empty. 91 92While this may mean a number of buffers may sit idle on some core's cache, 93the speed at which a core can access its own cache for a specific memory pool without locks provides performance gains. 94 95The cache is composed of a small, per-core table of pointers and its length (used as a stack). 96This internal cache can be enabled or disabled at creation of the pool. 97 98The maximum size of the cache is static and is defined at compilation time (RTE_MEMPOOL_CACHE_MAX_SIZE). 99 100:numref:`figure_mempool` shows a cache in operation. 101 102.. _figure_mempool: 103 104.. figure:: img/mempool.* 105 106 A mempool in Memory with its Associated Ring 107 108Alternatively to the internal default per-lcore local cache, an application can create and manage external caches through the ``rte_mempool_cache_create()``, ``rte_mempool_cache_free()`` and ``rte_mempool_cache_flush()`` calls. 109These user-owned caches can be explicitly passed to ``rte_mempool_generic_put()`` and ``rte_mempool_generic_get()``. 110The ``rte_mempool_default_cache()`` call returns the default internal cache if any. 111In contrast to the default caches, user-owned caches can be used by unregistered non-EAL threads too. 112 113.. _Mempool_Handlers: 114 115Mempool Handlers 116------------------------ 117 118This allows external memory subsystems, such as external hardware memory 119management systems and software based memory allocators, to be used with DPDK. 120 121There are two aspects to a mempool handler. 122 123* Adding the code for your new mempool operations (ops). This is achieved by 124 adding a new mempool ops code, and using the ``RTE_MEMPOOL_REGISTER_OPS`` macro. 125 126* Using the new API to call ``rte_mempool_create_empty()`` and 127 ``rte_mempool_set_ops_byname()`` to create a new mempool and specifying which 128 ops to use. 129 130Several different mempool handlers may be used in the same application. A new 131mempool can be created by using the ``rte_mempool_create_empty()`` function, 132then using ``rte_mempool_set_ops_byname()`` to point the mempool to the 133relevant mempool handler callback (ops) structure. 134 135Legacy applications may continue to use the old ``rte_mempool_create()`` API 136call, which uses a ring based mempool handler by default. These applications 137will need to be modified to use a new mempool handler. 138 139For applications that use ``rte_pktmbuf_create()``, there is a config setting 140(``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of 141an alternative mempool handler. 142 143 .. note:: 144 145 When running a DPDK application with shared libraries, mempool handler 146 shared objects specified with the '-d' EAL command-line parameter are 147 dynamically loaded. When running a multi-process application with shared 148 libraries, the -d arguments for mempool handlers *must be specified in the 149 same order for all processes* to ensure correct operation. 150 151 152Use Cases 153--------- 154 155All allocations that require a high level of performance should use a pool-based memory allocator. 156Below are some examples: 157 158* :ref:`Mbuf Library <Mbuf_Library>` 159 160* :ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` , for logging service 161 162* Any application that needs to allocate fixed-sized objects in the data plane and that will be continuously utilized by the system. 163