15630257fSFerruh Yigit.. SPDX-License-Identifier: BSD-3-Clause 25630257fSFerruh Yigit Copyright(c) 2010-2014 Intel Corporation. 31ab07743SBernard Iremonger 45c7cb088SCiara PowerRunning Sample Applications 55c7cb088SCiara Power=========================== 61ab07743SBernard Iremonger 7c053d9e9SSarosh ArifThe chapter describes how to compile and run applications in a DPDK environment. 81ab07743SBernard IremongerIt also provides a pointer to where sample applications are stored. 91ab07743SBernard Iremonger 101ab07743SBernard IremongerCompiling a Sample Application 111ab07743SBernard Iremonger------------------------------ 121ab07743SBernard Iremonger 133cc6ecfdSCiara PowerPlease refer to :ref:`building_app_using_installed_dpdk` for detail on compiling sample apps. 141ab07743SBernard Iremonger 151ab07743SBernard IremongerRunning a Sample Application 161ab07743SBernard Iremonger---------------------------- 171ab07743SBernard Iremonger 181ab07743SBernard Iremonger.. warning:: 191ab07743SBernard Iremonger 201ee5af82SShahaf Shuler Before running the application make sure: 211ab07743SBernard Iremonger 221ee5af82SShahaf Shuler - Hugepages setup is done. 231ee5af82SShahaf Shuler - Any kernel driver being used is loaded. 241ee5af82SShahaf Shuler - In case needed, ports being used by the application should be 251ee5af82SShahaf Shuler bound to the corresponding kernel driver. 261ab07743SBernard Iremonger 271ee5af82SShahaf Shuler refer to :ref:`linux_gsg_linux_drivers` for more details. 281ab07743SBernard Iremonger 292e486e26SSiobhan ButlerThe application is linked with the DPDK target environment's Environmental Abstraction Layer (EAL) library, 302e486e26SSiobhan Butlerwhich provides some options that are generic to every DPDK application. 311ab07743SBernard Iremonger 321ab07743SBernard IremongerThe following is the list of options that can be given to the EAL: 331ab07743SBernard Iremonger 341ab07743SBernard Iremonger.. code-block:: console 351ab07743SBernard Iremonger 3635b09d76SKeith Wiles ./rte-app [-c COREMASK | -l CORELIST] [-n NUM] [-b <domain:bus:devid.func>] \ 37fe2497f9SReshma Pattan [--socket-mem=MB,...] [-d LIB.so|DIR] [-m MB] [-r NUM] [-v] [--file-prefix] \ 38f26ab687SJianfeng Tan [--proc-type <primary|secondary|auto>] 391ab07743SBernard Iremonger 401ab07743SBernard IremongerThe EAL options are as follows: 411ab07743SBernard Iremonger 4235b09d76SKeith Wiles* ``-c COREMASK`` or ``-l CORELIST``: 4329c67340SJohn McNamara An hexadecimal bit mask of the cores to run on. Note that core numbering can 4435b09d76SKeith Wiles change between platforms and should be determined beforehand. The corelist is 4535b09d76SKeith Wiles a set of core numbers instead of a bitmap core mask. 461ab07743SBernard Iremonger 4729c67340SJohn McNamara* ``-n NUM``: 4829c67340SJohn McNamara Number of memory channels per processor socket. 491ab07743SBernard Iremonger 5029c67340SJohn McNamara* ``-b <domain:bus:devid.func>``: 51db27370bSStephen Hemminger Blocklisting of ports; prevent EAL from using specified PCI device 5229c67340SJohn McNamara (multiple ``-b`` options are allowed). 531ab07743SBernard Iremonger 5429c67340SJohn McNamara* ``--use-device``: 5529c67340SJohn McNamara use the specified Ethernet device(s) only. Use comma-separate 5629c67340SJohn McNamara ``[domain:]bus:devid.func`` values. Cannot be used with ``-b`` option. 571ab07743SBernard Iremonger 5829c67340SJohn McNamara* ``--socket-mem``: 59b3173932SAnatoly Burakov Memory to allocate from hugepages on specific sockets. In dynamic memory mode, 60b3173932SAnatoly Burakov this memory will also be pinned (i.e. not released back to the system until 61b3173932SAnatoly Burakov application closes). 621ab07743SBernard Iremonger 63e4348122SAnatoly Burakov* ``--socket-limit``: 64e4348122SAnatoly Burakov Limit maximum memory available for allocation on each socket. Does not support 65e4348122SAnatoly Burakov legacy memory mode. 66e4348122SAnatoly Burakov 67fe2497f9SReshma Pattan* ``-d``: 68fe2497f9SReshma Pattan Add a driver or driver directory to be loaded. 69f8dbaebbSSean Morrissey The application should use this option to load the PMDs 70fe2497f9SReshma Pattan that are built as shared libraries. 71fe2497f9SReshma Pattan 7229c67340SJohn McNamara* ``-m MB``: 7329c67340SJohn McNamara Memory to allocate from hugepages, regardless of processor socket. It is 7429c67340SJohn McNamara recommended that ``--socket-mem`` be used instead of this option. 751ab07743SBernard Iremonger 7629c67340SJohn McNamara* ``-r NUM``: 7729c67340SJohn McNamara Number of memory ranks. 781ab07743SBernard Iremonger 7929c67340SJohn McNamara* ``-v``: 8029c67340SJohn McNamara Display version information on startup. 811ab07743SBernard Iremonger 8229c67340SJohn McNamara* ``--huge-dir``: 8329c67340SJohn McNamara The directory where hugetlbfs is mounted. 841ab07743SBernard Iremonger 8565c2bbf4SDavid Marchand* ``--mbuf-pool-ops-name``: 86a103a97eSSantosh Shukla Pool ops name for mbuf to use. 87a103a97eSSantosh Shukla 8829c67340SJohn McNamara* ``--file-prefix``: 8929c67340SJohn McNamara The prefix text used for hugepage filenames. 901ab07743SBernard Iremonger 9129c67340SJohn McNamara* ``--proc-type``: 9229c67340SJohn McNamara The type of process instance. 931ab07743SBernard Iremonger 9429c67340SJohn McNamara* ``--vmware-tsc-map``: 9529c67340SJohn McNamara Use VMware TSC map instead of native RDTSC. 961ab07743SBernard Iremonger 9729c67340SJohn McNamara* ``--base-virtaddr``: 9829c67340SJohn McNamara Specify base virtual address. 991ab07743SBernard Iremonger 10029c67340SJohn McNamara* ``--vfio-intr``: 10129c67340SJohn McNamara Specify interrupt type to be used by VFIO (has no effect if VFIO is not used). 1021ab07743SBernard Iremonger 103b3173932SAnatoly Burakov* ``--legacy-mem``: 104b3173932SAnatoly Burakov Run DPDK in legacy memory mode (disable memory reserve/unreserve at runtime, 105b3173932SAnatoly Burakov but provide more IOVA-contiguous memory). 106b3173932SAnatoly Burakov 107b3173932SAnatoly Burakov* ``--single-file-segments``: 108b3173932SAnatoly Burakov Store memory segments in fewer files (dynamic memory mode only - does not 109b3173932SAnatoly Burakov affect legacy memory mode). 110b3173932SAnatoly Burakov 11135b09d76SKeith WilesThe ``-c`` or ``-l`` and option is mandatory; the others are optional. 1121ab07743SBernard Iremonger 1132e486e26SSiobhan ButlerCopy the DPDK application binary to your target, then run the application as follows 1141ab07743SBernard Iremonger(assuming the platform has four memory channels per processor socket, 11529c67340SJohn McNamaraand that cores 0-3 are present and are to be used for running the application):: 1161ab07743SBernard Iremonger 1175c7cb088SCiara Power ./dpdk-helloworld -l 0-3 -n 4 1181ab07743SBernard Iremonger 1191ab07743SBernard Iremonger.. note:: 1201ab07743SBernard Iremonger 12129c67340SJohn McNamara The ``--proc-type`` and ``--file-prefix`` EAL options are used for running 12229c67340SJohn McNamara multiple DPDK processes. See the "Multi-process Sample Application" 12329c67340SJohn McNamara chapter in the *DPDK Sample Applications User Guide* and the *DPDK 12429c67340SJohn McNamara Programmers Guide* for more details. 1251ab07743SBernard Iremonger 1261ab07743SBernard IremongerLogical Core Use by Applications 1271ab07743SBernard Iremonger~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1281ab07743SBernard Iremonger 12935b09d76SKeith WilesThe coremask (-c 0x0f) or corelist (-l 0-3) parameter is always mandatory for DPDK applications. 13035b09d76SKeith WilesEach bit of the mask corresponds to the equivalent logical core number as reported by Linux. The preferred corelist option is a cleaner method to define cores to be used. 1311ab07743SBernard IremongerSince these logical core numbers, and their mapping to specific cores on specific NUMA sockets, can vary from platform to platform, 13235b09d76SKeith Wilesit is recommended that the core layout for each platform be considered when choosing the coremask/corelist to use in each case. 1331ab07743SBernard Iremonger 134c053d9e9SSarosh ArifOn initialization of the EAL layer by a DPDK application, the logical cores to be used and their socket location are displayed. 13529c67340SJohn McNamaraThis information can also be determined for all cores on the system by examining the ``/proc/cpuinfo`` file, for example, by running cat ``/proc/cpuinfo``. 1361ab07743SBernard IremongerThe physical id attribute listed for each processor indicates the CPU socket to which it belongs. 1371ab07743SBernard IremongerThis can be useful when using other processors to understand the mapping of the logical cores to the sockets. 1381ab07743SBernard Iremonger 1391ab07743SBernard Iremonger.. note:: 1401ab07743SBernard Iremonger 1414821fa10SThomas Monjalon A more graphical view of the logical core layout 1424821fa10SThomas Monjalon may be obtained using the ``lstopo`` Linux utility. 1434821fa10SThomas Monjalon On Fedora, this may be installed and run using the following commands:: 1441ab07743SBernard Iremonger 1451ab07743SBernard Iremonger sudo yum install hwloc 1464821fa10SThomas Monjalon lstopo 1474821fa10SThomas Monjalon 1484821fa10SThomas Monjalon This command produces a quite short textual output:: 1494821fa10SThomas Monjalon 1504821fa10SThomas Monjalon lstopo-no-graphics --merge 1511ab07743SBernard Iremonger 1521ab07743SBernard Iremonger.. warning:: 1531ab07743SBernard Iremonger 15435b09d76SKeith Wiles The logical core layout can change between different board layouts and should be checked before selecting an application coremask/corelist. 1551ab07743SBernard Iremonger 1561ab07743SBernard IremongerHugepage Memory Use by Applications 1571ab07743SBernard Iremonger~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1581ab07743SBernard Iremonger 1591ab07743SBernard IremongerWhen running an application, it is recommended to use the same amount of memory as that allocated for hugepages. 1602e486e26SSiobhan ButlerThis is done automatically by the DPDK application at startup, 16129c67340SJohn McNamaraif no ``-m`` or ``--socket-mem`` parameter is passed to it when run. 1621ab07743SBernard Iremonger 16329c67340SJohn McNamaraIf more memory is requested by explicitly passing a ``-m`` or ``--socket-mem`` value, the application fails. 16429c67340SJohn McNamaraHowever, the application itself can also fail if the user requests less memory than the reserved amount of hugepage-memory, particularly if using the ``-m`` option. 1651ab07743SBernard IremongerThe reason is as follows. 1661ab07743SBernard IremongerSuppose the system has 1024 reserved 2 MB pages in socket 0 and 1024 in socket 1. 1671ab07743SBernard IremongerIf the user requests 128 MB of memory, the 64 pages may not match the constraints: 1681ab07743SBernard Iremonger 1691ab07743SBernard Iremonger* The hugepage memory by be given to the application by the kernel in socket 1 only. 1701ab07743SBernard Iremonger In this case, if the application attempts to create an object, such as a ring or memory pool in socket 0, it fails. 17129c67340SJohn McNamara To avoid this issue, it is recommended that the ``--socket-mem`` option be used instead of the ``-m`` option. 1721ab07743SBernard Iremonger 1732e486e26SSiobhan Butler* These pages can be located anywhere in physical memory, and, although the DPDK EAL will attempt to allocate memory in contiguous blocks, 1741ab07743SBernard Iremonger it is possible that the pages will not be contiguous. In this case, the application is not able to allocate big memory pools. 1751ab07743SBernard Iremonger 1761ab07743SBernard IremongerThe socket-mem option can be used to request specific amounts of memory for specific sockets. 17729c67340SJohn McNamaraThis is accomplished by supplying the ``--socket-mem`` flag followed by amounts of memory requested on each socket, 17829c67340SJohn McNamarafor example, supply ``--socket-mem=0,512`` to try and reserve 512 MB for socket 1 only. 17929c67340SJohn McNamaraSimilarly, on a four socket system, to allocate 1 GB memory on each of sockets 0 and 2 only, the parameter ``--socket-mem=1024,0,1024`` can be used. 1801ab07743SBernard IremongerNo memory will be reserved on any CPU socket that is not explicitly referenced, for example, socket 3 in this case. 1812e486e26SSiobhan ButlerIf the DPDK cannot allocate enough memory on each socket, the EAL initialization fails. 1821ab07743SBernard Iremonger 183*036f72d0SDmitry KozlyukWhether hugepages are included in core dump is controlled by ``/proc/<pid>/coredump_filter``. 184*036f72d0SDmitry KozlyukIt is ``33`` (hexadecimal) by default, which means that hugepages are excluded from core dump. 185*036f72d0SDmitry KozlyukThis setting is per-process and is inherited. 186*036f72d0SDmitry KozlyukRefer to ``core(5)`` for details. 187*036f72d0SDmitry KozlyukTo include mapped hugepages in core dump, set bit 6 (``0x40``) in the parent process 188*036f72d0SDmitry Kozlyukor shell before running a DPDK application: 189*036f72d0SDmitry Kozlyuk 190*036f72d0SDmitry Kozlyuk.. code-block:: shell 191*036f72d0SDmitry Kozlyuk 192*036f72d0SDmitry Kozlyuk echo 0x73 > /proc/self/coredump_filter 193*036f72d0SDmitry Kozlyuk ./dpdk-application ... 194*036f72d0SDmitry Kozlyuk 195*036f72d0SDmitry Kozlyuk.. note:: 196*036f72d0SDmitry Kozlyuk 197*036f72d0SDmitry Kozlyuk Including hugepages in core dump file increases its size, 198*036f72d0SDmitry Kozlyuk which may fill the storage or overload the transport. 199*036f72d0SDmitry Kozlyuk Hugepages typically hold data processed by the application, 200*036f72d0SDmitry Kozlyuk like network packets, which may contain sensitive information. 201*036f72d0SDmitry Kozlyuk 2021ab07743SBernard IremongerAdditional Sample Applications 2031ab07743SBernard Iremonger------------------------------ 2041ab07743SBernard Iremonger 2055c7cb088SCiara PowerAdditional sample applications are included in the DPDK examples directory. 2061ab07743SBernard IremongerThese sample applications may be built and run in a manner similar to that described in earlier sections in this manual. 2072e486e26SSiobhan ButlerIn addition, see the *DPDK Sample Applications User Guide* for a description of the application, 2081ab07743SBernard Iremongerspecific instructions on compilation and execution and some explanation of the code. 209