1.. BSD LICENSE 2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved. 3 All rights reserved. 4 5 Redistribution and use in source and binary forms, with or without 6 modification, are permitted provided that the following conditions 7 are met: 8 9 * Redistributions of source code must retain the above copyright 10 notice, this list of conditions and the following disclaimer. 11 * Redistributions in binary form must reproduce the above copyright 12 notice, this list of conditions and the following disclaimer in 13 the documentation and/or other materials provided with the 14 distribution. 15 * Neither the name of Intel Corporation nor the names of its 16 contributors may be used to endorse or promote products derived 17 from this software without specific prior written permission. 18 19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 31.. _Mempool_Library: 32 33Mempool Library 34=============== 35 36A memory pool is an allocator of a fixed-sized object. 37In the Intel® DPDK, it is identified by name and uses a ring to store free objects. 38It provides some other optional services such as a per-core object cache and 39an alignment helper to ensure that objects are padded to spread them equally on all DRAM or DDR3 channels. 40 41This library is used by the 42:ref:`Mbuf Library <Mbuf_Library>` and the 43:ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` (for logging history). 44 45Cookies 46------- 47 48In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), cookies are added at the beginning and end of allocated blocks. 49The allocated objects then contain overwrite protection fields to help debugging buffer overflows. 50 51Stats 52----- 53 54In debug mode (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), 55statistics about get from/put in the pool are stored in the mempool structure. 56Statistics are per-lcore to avoid concurrent access to statistics counters. 57 58Memory Alignment Constraints 59---------------------------- 60 61Depending on hardware memory configuration, performance can be greatly improved by adding a specific padding between objects. 62The objective is to ensure that the beginning of each object starts on a different channel and rank in memory so that all channels are equally loaded. 63 64This is particularly true for packet buffers when doing L3 forwarding or flow classification. 65Only the first 64 bytes are accessed, so performance can be increased by spreading the start addresses of objects among the different channels. 66 67The number of ranks on any DIMM is the number of independent sets of DRAMs that can be accessed for the full data bit-width of the DIMM. 68The ranks cannot be accessed simultaneously since they share the same data path. 69The physical layout of the DRAM chips on the DIMM itself does not necessarily relate to the number of ranks. 70 71When running an application, the EAL command line options provide the ability to add the number of memory channels and ranks. 72 73.. note:: 74 75 The command line must always have the number of memory channels specified for the processor. 76 77Examples of alignment for different DIMM architectures are shown in Figure 5 and Figure 6. 78 79.. _pg_figure_5: 80 81**Figure 5. Two Channels and Quad-ranked DIMM Example** 82 83.. image19_png has been replaced 84 85|memory-management| 86 87In this case, the assumption is that a packet is 16 blocks of 64 bytes, which is not true. 88 89The Intel® 5520 chipset has three channels, so in most cases, 90no padding is required between objects (except for objects whose size are n x 3 x 64 bytes blocks). 91 92.. _pg_figure_6: 93 94**Figure 6. Three Channels and Two Dual-ranked DIMM Example** 95 96.. image20_png has been replaced 97 98|memory-management2| 99 100When creating a new pool, the user can specify to use this feature or not. 101 102Local Cache 103----------- 104 105In terms of CPU usage, the cost of multiple cores accessing a memory pool's ring of free buffers may be high 106since each access requires a compare-and-set (CAS) operation. 107To avoid having too many access requests to the memory pool's ring, 108the memory pool allocator can maintain a per-core cache and do bulk requests to the memory pool's ring, 109via the cache with many fewer locks on the actual memory pool structure. 110In this way, each core has full access to its own cache (with locks) of free objects and 111only when the cache fills does the core need to shuffle some of the free objects back to the pools ring or 112obtain more objects when the cache is empty. 113 114While this may mean a number of buffers may sit idle on some core's cache, 115the speed at which a core can access its own cache for a specific memory pool without locks provides performance gains. 116 117The cache is composed of a small, per-core table of pointers and its length (used as a stack). 118This cache can be enabled or disabled at creation of the pool. 119 120The maximum size of the cache is static and is defined at compilation time (CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE). 121 122Figure 7 shows a cache in operation. 123 124.. _pg_figure_7: 125 126**Figure 7. A mempool in Memory with its Associated Ring** 127 128.. image21_png has been replaced 129 130|mempool| 131 132Use Cases 133--------- 134 135All allocations that require a high level of performance should use a pool-based memory allocator. 136Below are some examples: 137 138* :ref:`Mbuf Library <Mbuf_Library>` 139 140* :ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` , for logging service 141 142* Any application that needs to allocate fixed-sized objects in the data plane and that will be continuously utilized by the system. 143 144.. |memory-management| image:: img/memory-management.svg 145 146.. |memory-management2| image:: img/memory-management2.svg 147 148.. |mempool| image:: img/mempool.svg 149