1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4.. _building_from_source: 5 6Compiling the DPDK Target from Source 7===================================== 8 9Prerequisites 10------------- 11 12The following FreeBSD packages are required to build DPDK: 13 14* meson 15* ninja 16* pkgconf 17* py37-pyelftools 18 19These can be installed using (as root):: 20 21 pkg install meson pkgconf py37-pyelftools 22 23To compile the required kernel modules for memory management and working 24with physical NIC devices, the kernel sources for FreeBSD also 25need to be installed. If not already present on the system, these can be 26installed via commands like the following, for FreeBSD 12.1 on x86_64:: 27 28 fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE/src.txz 29 tar -C / -xJvf src.txz 30 31To enable the telemetry library in DPDK, the jansson library also needs to 32be installed, and can be installed via:: 33 34 pkg install jansson 35 36Individual drivers may have additional requirements. Consult the relevant 37driver guide for any driver-specific requirements of interest. 38 39Building DPDK 40------------- 41 42The following commands can be used to build and install DPDK on a system. 43The final, install, step generally needs to be run as root:: 44 45 meson build 46 cd build 47 ninja 48 ninja install 49 50This will install the DPDK libraries and drivers to `/usr/local/lib` with a 51pkg-config file `libdpdk.pc` installed to `/usr/local/lib/pkgconfig`. The 52DPDK test applications, such as `dpdk-testpmd` are installed to 53`/usr/local/bin`. To use these applications, it is recommended that the 54`contigmem` and `nic_uio` kernel modules be loaded first, as described in 55the next section. 56 57.. note:: 58 59 It is recommended that pkg-config be used to query information 60 about the compiler and linker flags needed to build applications 61 against DPDK. In some cases, the path `/usr/local/lib/pkgconfig` 62 may not be in the default search paths for `.pc` files, which means 63 that queries for DPDK information may fail. This can be fixed by 64 setting the appropriate path in `PKG_CONFIG_PATH` environment 65 variable. 66 67 68.. _loading_contigmem: 69 70Loading the DPDK contigmem Module 71--------------------------------- 72 73To run a DPDK application, physically contiguous memory is required. 74In the absence of non-transparent superpages, the included sources for the 75contigmem kernel module provides the ability to present contiguous blocks of 76memory for the DPDK to use. The contigmem module must be loaded into the 77running kernel before any DPDK is run. Once DPDK is installed on the 78system, the module can be found in the `/boot/modules` directory. 79 80The amount of physically contiguous memory along with the number of physically 81contiguous blocks to be reserved by the module can be set at runtime prior to 82module loading using:: 83 84 kenv hw.contigmem.num_buffers=n 85 kenv hw.contigmem.buffer_size=m 86 87The kernel environment variables can also be specified during boot by placing the 88following in ``/boot/loader.conf``: 89 90.. code-block:: shell 91 92 hw.contigmem.num_buffers=n 93 hw.contigmem.buffer_size=m 94 95The variables can be inspected using the following command:: 96 97 sysctl -a hw.contigmem 98 99Where n is the number of blocks and m is the size in bytes of each area of 100contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) 101each is set during module load if they are not specified in the environment. 102 103The module can then be loaded using kldload:: 104 105 kldload contigmem 106 107It is advisable to include the loading of the contigmem module during the boot 108process to avoid issues with potential memory fragmentation during later system 109up time. This can be achieved by placing lines similar to the following into 110``/boot/loader.conf``: 111 112.. code-block:: shell 113 114 hw.contigmem.num_buffers=1 115 hw.contigmem.buffer_size=1073741824 116 contigmem_load="YES" 117 118.. note:: 119 120 The contigmem_load directive should be placed after any definitions of 121 ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values 122 are not to be used. 123 124An error such as:: 125 126 kldload: can't load <build_dir>/kernel/freebsd/contigmem.ko: 127 Exec format error 128 129is generally attributed to not having enough contiguous memory 130available and can be verified via dmesg or ``/var/log/messages``:: 131 132 kernel: contigmalloc failed for buffer <n> 133 134To avoid this error, reduce the number of buffers or the buffer size. 135 136.. _loading_nic_uio: 137 138Loading the DPDK nic_uio Module 139------------------------------- 140 141After loading the contigmem module, the ``nic_uio`` module must also be loaded into the 142running kernel prior to running any DPDK application, e.g. using:: 143 144 kldload nic_uio 145 146.. note:: 147 148 If the ports to be used are currently bound to a existing kernel driver 149 then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the 150 module. Setting this value is described in the next section below. 151 152Currently loaded modules can be seen by using the ``kldstat`` command and a module 153can be removed from the running kernel by using ``kldunload <module_name>``. 154 155To load the module during boot place the following into ``/boot/loader.conf``: 156 157.. code-block:: shell 158 159 nic_uio_load="YES" 160 161.. note:: 162 163 ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. 164 165By default, the ``nic_uio`` module will take ownership of network ports if they are 166recognized DPDK devices and are not owned by another module. However, since 167the FreeBSD kernel includes support, either built-in, or via a separate driver 168module, for most network card devices, it is likely that the ports to be used are 169already bound to a driver other than ``nic_uio``. The following sub-section describe 170how to query and modify the device ownership of the ports to be used by 171DPDK applications. 172 173.. _binding_network_ports: 174 175Binding Network Ports to the nic_uio Module 176~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 177 178Device ownership can be viewed using the pciconf -l command. The example below shows 179four Intel® 82599 network ports under ``if_ixgbe`` module ownership. 180 181.. code-block:: none 182 183 pciconf -l 184 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 185 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 186 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 187 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 188 189The first column constitutes three components: 190 191#. Device name: ``ixN`` 192 193#. Unit name: ``pci0`` 194 195#. Selector (Bus:Device:Function): ``1:0:0`` 196 197Where no driver is associated with a device, the device name will be ``none``. 198 199By default, the FreeBSD kernel will include built-in drivers for the most common 200devices; a kernel rebuild would normally be required to either remove the drivers 201or configure them as loadable modules. 202 203To avoid building a custom kernel, the ``nic_uio`` module can detach a network port 204from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` 205kernel environment variable prior to loading ``nic_uio``, as follows:: 206 207 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 208 209Where a comma separated list of selectors is set, the list must not contain any 210whitespace. 211 212For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module 213upon loading, use the following command:: 214 215 kenv hw.nic_uio.bdfs="2:0:0,2:0:1" 216 217The variable can also be specified during boot by placing the following into 218``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as 219shown: 220 221.. code-block:: shell 222 223 hw.nic_uio.bdfs="2:0:0,2:0:1" 224 nic_uio_load="YES" 225 226Binding Network Ports Back to their Original Kernel Driver 227~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 228 229If the original driver for a network port has been compiled into the kernel, 230it is necessary to reboot FreeBSD to restore the original device binding. Before 231doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. 232 233If rebinding to a driver that is a loadable module, the network port binding can 234be reset without rebooting. To do so, unload both the target kernel module and the 235``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) 236value, and reload the two drivers - first the original kernel driver, and then 237the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are 238ports that are still to be bound to it. 239 240Example commands to perform these steps are shown below:: 241 242 kldunload nic_uio 243 kldunload <original_driver> 244 245 # To clear the value completely: 246 kenv -u hw.nic_uio.bdfs 247 248 # To update the list of ports to bind: 249 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 250 251 kldload <original_driver> 252 253 kldload nic_uio # optional 254