1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4.. include:: <isonum.txt> 5 6.. _building_from_source: 7 8Compiling the DPDK Target from Source 9===================================== 10 11Prerequisites 12------------- 13 14The following FreeBSD packages are required to build DPDK: 15 16* meson 17* ninja 18* pkgconf 19* py38-pyelftools 20 21.. note: 22 23 The specific package for pyelftools is dependent on the version of python in use, 24 Python 3.8 being the version at type of writing, hence the ``py38`` prefix. 25 26These can be installed using (as root):: 27 28 pkg install meson pkgconf py38-pyelftools 29 30To compile the required kernel modules for memory management and working 31with physical NIC devices, the kernel sources for FreeBSD also 32need to be installed. If not already present on the system, these can be 33installed via commands like the following, for FreeBSD 12.1 on x86_64:: 34 35 fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE/src.txz 36 tar -C / -xJvf src.txz 37 38Individual drivers may have additional requirements. Consult the relevant 39driver guide for any driver-specific requirements of interest. 40 41Building DPDK 42------------- 43 44The following commands can be used to build and install DPDK on a system. 45The final, install, step generally needs to be run as root:: 46 47 meson setup build 48 cd build 49 ninja 50 ninja install 51 52This will install the DPDK libraries and drivers to `/usr/local/lib` with a 53pkg-config file `libdpdk.pc` installed to `/usr/local/lib/pkgconfig`. The 54DPDK test applications, such as `dpdk-testpmd` are installed to 55`/usr/local/bin`. To use these applications, it is recommended that the 56`contigmem` and `nic_uio` kernel modules be loaded first, as described in 57the next section. 58 59.. note:: 60 61 It is recommended that pkg-config be used to query information 62 about the compiler and linker flags needed to build applications 63 against DPDK. In some cases, the path `/usr/local/lib/pkgconfig` 64 may not be in the default search paths for `.pc` files, which means 65 that queries for DPDK information may fail. This can be fixed by 66 setting the appropriate path in `PKG_CONFIG_PATH` environment 67 variable. 68 69 70.. _loading_contigmem: 71 72Loading the DPDK contigmem Module 73--------------------------------- 74 75To run a DPDK application, physically contiguous memory is required. 76In the absence of non-transparent superpages, the included sources for the 77contigmem kernel module provides the ability to present contiguous blocks of 78memory for the DPDK to use. The contigmem module must be loaded into the 79running kernel before any DPDK is run. Once DPDK is installed on the 80system, the module can be found in the `/boot/modules` directory. 81 82The amount of physically contiguous memory along with the number of physically 83contiguous blocks to be reserved by the module can be set at runtime prior to 84module loading using:: 85 86 kenv hw.contigmem.num_buffers=n 87 kenv hw.contigmem.buffer_size=m 88 89The kernel environment variables can also be specified during boot by placing the 90following in ``/boot/loader.conf``: 91 92.. code-block:: shell 93 94 hw.contigmem.num_buffers=n 95 hw.contigmem.buffer_size=m 96 97The variables can be inspected using the following command:: 98 99 sysctl -a hw.contigmem 100 101Where n is the number of blocks and m is the size in bytes of each area of 102contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) 103each is set during module load if they are not specified in the environment. 104 105The module can then be loaded using kldload:: 106 107 kldload contigmem 108 109It is advisable to include the loading of the contigmem module during the boot 110process to avoid issues with potential memory fragmentation during later system 111up time. This can be achieved by placing lines similar to the following into 112``/boot/loader.conf``: 113 114.. code-block:: shell 115 116 hw.contigmem.num_buffers=1 117 hw.contigmem.buffer_size=1073741824 118 contigmem_load="YES" 119 120.. note:: 121 122 The contigmem_load directive should be placed after any definitions of 123 ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values 124 are not to be used. 125 126An error such as:: 127 128 kldload: can't load <build_dir>/kernel/freebsd/contigmem.ko: 129 Exec format error 130 131is generally attributed to not having enough contiguous memory 132available and can be verified via dmesg or ``/var/log/messages``:: 133 134 kernel: contigmalloc failed for buffer <n> 135 136To avoid this error, reduce the number of buffers or the buffer size. 137 138.. _loading_nic_uio: 139 140Loading the DPDK nic_uio Module 141------------------------------- 142 143After loading the contigmem module, the ``nic_uio`` module must also be loaded into the 144running kernel prior to running any DPDK application, e.g. using:: 145 146 kldload nic_uio 147 148.. note:: 149 150 If the ports to be used are currently bound to a existing kernel driver 151 then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the 152 module. Setting this value is described in the next section below. 153 154Currently loaded modules can be seen by using the ``kldstat`` command and a module 155can be removed from the running kernel by using ``kldunload <module_name>``. 156 157To load the module during boot place the following into ``/boot/loader.conf``: 158 159.. code-block:: shell 160 161 nic_uio_load="YES" 162 163.. note:: 164 165 ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. 166 167By default, the ``nic_uio`` module will take ownership of network ports if they are 168recognized DPDK devices and are not owned by another module. However, since 169the FreeBSD kernel includes support, either built-in, or via a separate driver 170module, for most network card devices, it is likely that the ports to be used are 171already bound to a driver other than ``nic_uio``. The following sub-section describe 172how to query and modify the device ownership of the ports to be used by 173DPDK applications. 174 175.. _binding_network_ports: 176 177Binding Network Ports to the nic_uio Module 178~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 179 180Device ownership can be viewed using the pciconf -l command. The example below shows 181four Intel\ |reg| 82599 network ports under ``if_ixgbe`` module ownership. 182 183.. code-block:: none 184 185 pciconf -l 186 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 187 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 188 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 189 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 190 191The first column constitutes three components: 192 193#. Device name: ``ixN`` 194 195#. Unit name: ``pci0`` 196 197#. Selector (Bus:Device:Function): ``1:0:0`` 198 199Where no driver is associated with a device, the device name will be ``none``. 200 201By default, the FreeBSD kernel will include built-in drivers for the most common 202devices; a kernel rebuild would normally be required to either remove the drivers 203or configure them as loadable modules. 204 205To avoid building a custom kernel, the ``nic_uio`` module can detach a network port 206from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` 207kernel environment variable prior to loading ``nic_uio``, as follows:: 208 209 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 210 211Where a comma separated list of selectors is set, the list must not contain any 212whitespace. 213 214For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module 215upon loading, use the following command:: 216 217 kenv hw.nic_uio.bdfs="2:0:0,2:0:1" 218 219The variable can also be specified during boot by placing the following into 220``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as 221shown: 222 223.. code-block:: shell 224 225 hw.nic_uio.bdfs="2:0:0,2:0:1" 226 nic_uio_load="YES" 227 228Binding Network Ports Back to their Original Kernel Driver 229~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 230 231If the original driver for a network port has been compiled into the kernel, 232it is necessary to reboot FreeBSD to restore the original device binding. Before 233doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. 234 235If rebinding to a driver that is a loadable module, the network port binding can 236be reset without rebooting. To do so, unload both the target kernel module and the 237``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) 238value, and reload the two drivers - first the original kernel driver, and then 239the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are 240ports that are still to be bound to it. 241 242Example commands to perform these steps are shown below:: 243 244 kldunload nic_uio 245 kldunload <original_driver> 246 247 # To clear the value completely: 248 kenv -u hw.nic_uio.bdfs 249 250 # To update the list of ports to bind: 251 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 252 253 kldload <original_driver> 254 255 kldload nic_uio # optional 256