15630257fSFerruh Yigit.. SPDX-License-Identifier: BSD-3-Clause 25630257fSFerruh Yigit Copyright(c) 2010-2014 Intel Corporation. 31ab07743SBernard Iremonger 4c265d586SBruce Richardson.. include:: <isonum.txt> 5c265d586SBruce Richardson 61ab07743SBernard IremongerSystem Requirements 71ab07743SBernard Iremonger=================== 81ab07743SBernard Iremonger 92e486e26SSiobhan ButlerThis chapter describes the packages required to compile the DPDK. 101ab07743SBernard Iremonger 11df6aaab3SChao ZhuBIOS Setting Prerequisite on x86 12df6aaab3SChao Zhu-------------------------------- 131ab07743SBernard Iremonger 142e486e26SSiobhan ButlerFor the majority of platforms, no special BIOS settings are needed to use basic DPDK functionality. 151ab07743SBernard IremongerHowever, for additional HPET timer and power management functionality, 16d239f17dSShahaf Shulerand high performance of small packets, BIOS setting changes may be needed. 1729c67340SJohn McNamaraConsult the section on :ref:`Enabling Additional Functionality <Enabling_Additional_Functionality>` 181ab07743SBernard Iremongerfor more information on the required changes. 191ab07743SBernard Iremonger 202e486e26SSiobhan ButlerCompilation of the DPDK 212e486e26SSiobhan Butler----------------------- 221ab07743SBernard Iremonger 2306673193SJohn McNamara**Required Tools and Libraries:** 241ab07743SBernard Iremonger 251ab07743SBernard Iremonger.. note:: 261ab07743SBernard Iremonger 27e4f11376SHarry van Haaren The setup commands and installed packages needed on various systems may be different. 28e4f11376SHarry van Haaren For details on Linux distributions and the versions tested, please consult the DPDK Release Notes. 291ab07743SBernard Iremonger 3033d66940SBruce Richardson* General development tools including a C compiler supporting the C11 standard, 3133d66940SBruce Richardson including standard atomics, for example: GCC (version 5.0+) or Clang (version 3.6+), 32167c0860SBruce Richardson and ``pkg-config`` or ``pkgconf`` to be used when building end-user binaries against DPDK. 331ab07743SBernard Iremonger 34f6412d75SBruce Richardson * For RHEL/Fedora systems these can be installed using ``dnf groupinstall "Development Tools"`` 35f6412d75SBruce Richardson * For Ubuntu/Debian systems these can be installed using ``apt install build-essential`` 36bc1db4f4SThomas Monjalon * For Alpine Linux, ``apk add alpine-sdk bsd-compat-headers`` 37167c0860SBruce Richardson 38167c0860SBruce Richardson.. note:: 39167c0860SBruce Richardson 40167c0860SBruce Richardson pkg-config 0.27, supplied with RHEL-7, 41167c0860SBruce Richardson does not process the Libs.private section correctly, 42167c0860SBruce Richardson resulting in statically linked applications not being linked properly. 43167c0860SBruce Richardson Use an updated version of ``pkg-config`` or ``pkgconf`` instead when building applications 441ab07743SBernard Iremonger 450ce3cf4aSRobin Jarry* Python 3.6 or later. 461ab07743SBernard Iremonger 47*6f3dbd30SBruce Richardson* Meson (version 0.57+) and ninja 48df6aaab3SChao Zhu 49f6412d75SBruce Richardson * ``meson`` & ``ninja-build`` packages in most Linux distributions 50f6412d75SBruce Richardson * If the packaged version is below the minimum version, the latest versions 51f6412d75SBruce Richardson can be installed from Python's "pip" repository: ``pip3 install meson ninja`` 52d3aa5274SDaniel Mrzyglod 53f0f93a7aSDmitry Kozlyuk* ``pyelftools`` (version 0.22+) 54f0f93a7aSDmitry Kozlyuk 5563a9293fSKevin Traynor * For Fedora systems it can be installed using ``dnf install python-pyelftools`` 5663a9293fSKevin Traynor * For RHEL/CentOS systems it can be installed using ``pip3 install pyelftools`` 57f0f93a7aSDmitry Kozlyuk * For Ubuntu/Debian it can be installed using ``apt install python3-pyelftools`` 58c6bd9f4bSThomas Monjalon * For Alpine Linux, ``apk add py3-elftools`` 59f0f93a7aSDmitry Kozlyuk 60ee57170cSYong Wang* Library for handling NUMA (Non Uniform Memory Access). 61ee57170cSYong Wang 62f6412d75SBruce Richardson * ``numactl-devel`` in RHEL/Fedora; 63f6412d75SBruce Richardson * ``libnuma-dev`` in Debian/Ubuntu; 64c6bd9f4bSThomas Monjalon * ``numactl-dev`` in Alpine Linux 6506673193SJohn McNamara 66e090fdb4SJohn McNamara.. note:: 67e090fdb4SJohn McNamara 68e090fdb4SJohn McNamara Please ensure that the latest patches are applied to third party libraries 69e090fdb4SJohn McNamara and software to avoid any known vulnerabilities. 70e090fdb4SJohn McNamara 711ab07743SBernard Iremonger 721ab07743SBernard Iremonger**Optional Tools:** 731ab07743SBernard Iremonger 74c265d586SBruce Richardson* Intel\ |reg| C++ Compiler (icc). For installation, additional libraries may be required. 751ab07743SBernard Iremonger See the icc Installation Guide found in the Documentation directory under the compiler installation. 761ab07743SBernard Iremonger 77c265d586SBruce Richardson* IBM\ |reg| Advance ToolChain for Powerlinux. This is a set of open source development tools and runtime libraries 78df6aaab3SChao Zhu which allows users to take leading edge advantage of IBM's latest POWER hardware features on Linux. To install 79df6aaab3SChao Zhu it, see the IBM official installation document. 80df6aaab3SChao Zhu 81f6412d75SBruce Richardson**Additional Libraries** 821ab07743SBernard Iremonger 83f6412d75SBruce RichardsonA number of DPDK components, such as libraries and poll-mode drivers (PMDs) have additional dependencies. 843cc6ecfdSCiara PowerFor DPDK builds, the presence or absence of these dependencies will be automatically detected 853cc6ecfdSCiara Powerenabling or disabling the relevant components appropriately. 86f6412d75SBruce Richardson 87f6412d75SBruce RichardsonIn each case, the relevant library development package (``-devel`` or ``-dev``) is needed to build the DPDK components. 88f6412d75SBruce Richardson 89f6412d75SBruce RichardsonFor libraries the additional dependencies include: 90f6412d75SBruce Richardson 91f6412d75SBruce Richardson* libarchive: for some unit tests using tar to get their resources. 92f6412d75SBruce Richardson 93f6412d75SBruce Richardson* libelf: to compile and use the bpf library. 94f6412d75SBruce Richardson 95f6412d75SBruce RichardsonFor poll-mode drivers, the additional dependencies for each driver can be 96f6412d75SBruce Richardsonfound in that driver's documentation in the relevant DPDK guide document, 97f6412d75SBruce Richardsone.g. :doc:`../nics/index` 9818aa3272SThomas Monjalon 992e486e26SSiobhan ButlerRunning DPDK Applications 1002e486e26SSiobhan Butler------------------------- 1011ab07743SBernard Iremonger 102c053d9e9SSarosh ArifTo run a DPDK application, some customization may be required on the target machine. 1031ab07743SBernard Iremonger 1041ab07743SBernard IremongerSystem Software 1051ab07743SBernard Iremonger~~~~~~~~~~~~~~~ 1061ab07743SBernard Iremonger 1071ab07743SBernard Iremonger**Required:** 1081ab07743SBernard Iremonger 109eb334762SStephen Hemminger* Kernel version >= 4.19 11090f3229fSStephen Hemminger 11190f3229fSStephen Hemminger The kernel version required is based on the oldest long term stable kernel available 11290f3229fSStephen Hemminger at kernel.org when the DPDK version is in development. 113d993b012SThomas Monjalon Compatibility for recent distribution kernels will be kept, notably RHEL/CentOS 7. 1141ab07743SBernard Iremonger 11529c67340SJohn McNamara The kernel version in use can be checked using the command:: 1161ab07743SBernard Iremonger 1171ab07743SBernard Iremonger uname -r 1181ab07743SBernard Iremonger 1191ab07743SBernard Iremonger* glibc >= 2.7 (for features related to cpuset) 1201ab07743SBernard Iremonger 12129c67340SJohn McNamara The version can be checked using the ``ldd --version`` command. 1221ab07743SBernard Iremonger 1231ab07743SBernard Iremonger* Kernel configuration 1241ab07743SBernard Iremonger 12529c67340SJohn McNamara In the Fedora OS and other common distributions, such as Ubuntu, or Red Hat Enterprise Linux, 1262e486e26SSiobhan Butler the vendor supplied kernel configurations can be used to run most DPDK applications. 1271ab07743SBernard Iremonger 1282e486e26SSiobhan Butler For other kernel builds, options which should be enabled for DPDK include: 1291ab07743SBernard Iremonger 1301ab07743SBernard Iremonger * HUGETLBFS 1311ab07743SBernard Iremonger 1321ab07743SBernard Iremonger * PROC_PAGE_MONITOR support 1331ab07743SBernard Iremonger 1341ab07743SBernard Iremonger * HPET and HPET_MMAP configuration options should also be enabled if HPET support is required. 13529c67340SJohn McNamara See the section on :ref:`High Precision Event Timer (HPET) Functionality <High_Precision_Event_Timer>` for more details. 1361ab07743SBernard Iremonger 137b932ebcbSQian Xu.. _linux_gsg_hugepages: 138b932ebcbSQian Xu 13929c67340SJohn McNamaraUse of Hugepages in the Linux Environment 14029c67340SJohn McNamara~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1411ab07743SBernard Iremonger 1421ab07743SBernard IremongerHugepage support is required for the large memory pool allocation used for packet buffers 14329c67340SJohn McNamara(the HUGETLBFS option must be enabled in the running kernel as indicated the previous section). 1441ab07743SBernard IremongerBy using hugepage allocations, performance is increased since fewer pages are needed, 1451ab07743SBernard Iremongerand therefore less Translation Lookaside Buffers (TLBs, high speed translation caches), 1461ab07743SBernard Iremongerwhich reduce the time it takes to translate a virtual page address to a physical page address. 1471ab07743SBernard IremongerWithout hugepages, high TLB miss rates would occur with the standard 4k page size, slowing performance. 1481ab07743SBernard Iremonger 1492e486e26SSiobhan ButlerReserving Hugepages for DPDK Use 1502e486e26SSiobhan Butler^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1511ab07743SBernard Iremonger 1528397cac7SAnatoly BurakovThe reservation of hugepages can be performed at run time. 1538397cac7SAnatoly BurakovThis is done by echoing the number of hugepages required 1548397cac7SAnatoly Burakovto a ``nr_hugepages`` file in the ``/sys/kernel/`` directory 1558397cac7SAnatoly Burakovcorresponding to a specific page size (in Kilobytes). 1568397cac7SAnatoly BurakovFor a single-node system, the command to use is as follows 1578397cac7SAnatoly Burakov(assuming that 1024 of 2MB pages are required):: 1588397cac7SAnatoly Burakov 1598397cac7SAnatoly Burakov echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages 1608397cac7SAnatoly Burakov 1618397cac7SAnatoly BurakovOn a NUMA machine, the above command will usually divide the number of hugepages 1628397cac7SAnatoly Burakovequally across all NUMA nodes (assuming there is enough memory on all NUMA nodes). 1638397cac7SAnatoly BurakovHowever, pages can also be reserved explicitly on individual NUMA nodes 1648397cac7SAnatoly Burakovusing a ``nr_hugepages`` file in the ``/sys/devices/`` directory:: 1658397cac7SAnatoly Burakov 1668397cac7SAnatoly Burakov echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages 1678397cac7SAnatoly Burakov echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages 1688397cac7SAnatoly Burakov 169de34aaa9SThomas MonjalonThe tool ``dpdk-hugepages.py`` can be used to manage hugepages. 170de34aaa9SThomas Monjalon 1718397cac7SAnatoly Burakov.. note:: 1728397cac7SAnatoly Burakov 1738397cac7SAnatoly Burakov Some kernel versions may not allow reserving 1 GB hugepages at run time, 1748397cac7SAnatoly Burakov so reserving them at boot time may be the only option. 1758397cac7SAnatoly Burakov Please see below for instructions. 1768397cac7SAnatoly Burakov 1778397cac7SAnatoly Burakov**Alternative:** 1788397cac7SAnatoly Burakov 1798397cac7SAnatoly BurakovIn the general case, reserving hugepages at run time is perfectly fine, 1808397cac7SAnatoly Burakovbut in use cases where having lots of physically contiguous memory is required, 1818397cac7SAnatoly Burakovit is preferable to reserve hugepages at boot time, 1828397cac7SAnatoly Burakovas that will help in preventing physical memory from becoming heavily fragmented. 1838397cac7SAnatoly Burakov 18429c67340SJohn McNamaraTo reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line. 1851ab07743SBernard Iremonger 18629c67340SJohn McNamaraFor 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use:: 1871ab07743SBernard Iremonger 1881ab07743SBernard Iremonger hugepages=1024 1891ab07743SBernard Iremonger 1901ab07743SBernard IremongerFor other hugepage sizes, for example 1G pages, the size must be specified explicitly and 1911ab07743SBernard Iremongercan also be optionally set as the default hugepage size for the system. 19229c67340SJohn McNamaraFor example, to reserve 4G of hugepage memory in the form of four 1G pages, the following options should be passed to the kernel:: 1931ab07743SBernard Iremonger 1941ab07743SBernard Iremonger default_hugepagesz=1G hugepagesz=1G hugepages=4 1951ab07743SBernard Iremonger 1961ab07743SBernard Iremonger.. note:: 1971ab07743SBernard Iremonger 198df6aaab3SChao Zhu The hugepage sizes that a CPU supports can be determined from the CPU flags on Intel architecture. 1991ab07743SBernard Iremonger If pse exists, 2M hugepages are supported; if pdpe1gb exists, 1G hugepages are supported. 200df6aaab3SChao Zhu On IBM Power architecture, the supported hugepage sizes are 16MB and 16GB. 2011ab07743SBernard Iremonger 2021ab07743SBernard Iremonger.. note:: 2031ab07743SBernard Iremonger 2041ab07743SBernard Iremonger For 64-bit applications, it is recommended to use 1 GB hugepages if the platform supports them. 2051ab07743SBernard Iremonger 2061ab07743SBernard IremongerIn the case of a dual-socket NUMA system, 2071ab07743SBernard Iremongerthe number of hugepages reserved at boot time is generally divided equally between the two sockets 2081ab07743SBernard Iremonger(on the assumption that sufficient memory is present on both sockets). 2091ab07743SBernard Iremonger 210e91ae7e0SRami RosenSee the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options. 2111ab07743SBernard Iremonger 2122e486e26SSiobhan ButlerUsing Hugepages with the DPDK 2132e486e26SSiobhan Butler^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2141ab07743SBernard Iremonger 2158397cac7SAnatoly BurakovIf secondary process support is not required, DPDK is able to use hugepages 2168397cac7SAnatoly Burakovwithout any configuration by using "in-memory" mode. 2178397cac7SAnatoly BurakovPlease see :doc:`linux_eal_parameters` for more details. 2188397cac7SAnatoly Burakov 2198397cac7SAnatoly BurakovIf secondary process support is required, 2208397cac7SAnatoly Burakovmount points for hugepages need to be created. 2218397cac7SAnatoly BurakovOn modern Linux distributions, a default mount point for hugepages 2228397cac7SAnatoly Burakovis provided by the system and is located at ``/dev/hugepages``. 2238397cac7SAnatoly BurakovThis mount point will use the default hugepage size 2248397cac7SAnatoly Burakovset by the kernel parameters as described above. 2258397cac7SAnatoly Burakov 2268397cac7SAnatoly BurakovHowever, in order to use hugepage sizes other than the default, it is necessary 2278397cac7SAnatoly Burakovto manually create mount points for those hugepage sizes (e.g. 1GB pages). 2288397cac7SAnatoly Burakov 2298397cac7SAnatoly BurakovTo make the hugepages of size 1GB available for DPDK use, 2308397cac7SAnatoly Burakovfollowing steps must be performed:: 2311ab07743SBernard Iremonger 2321ab07743SBernard Iremonger mkdir /mnt/huge 2338397cac7SAnatoly Burakov mount -t hugetlbfs pagesize=1GB /mnt/huge 2341ab07743SBernard Iremonger 23529c67340SJohn McNamaraThe mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file:: 2361ab07743SBernard Iremonger 2378397cac7SAnatoly Burakov nodev /mnt/huge hugetlbfs pagesize=1GB 0 0 238