1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4.. _building_from_source: 5 6Compiling the DPDK Target from Source 7===================================== 8 9Prerequisites 10------------- 11 12The following FreeBSD packages are required to build DPDK: 13 14* meson 15* ninja 16* pkgconf 17 18These can be installed using (as root):: 19 20 pkg install meson pkgconf 21 22To compile the required kernel modules for memory management and working 23with physical NIC devices, the kernel sources for FreeBSD also 24need to be installed. If not already present on the system, these can be 25installed via commands like the following, for FreeBSD 12.1 on x86_64:: 26 27 fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE/src.txz 28 tar -C / -xJvf src.txz 29 30To enable the telemetry library in DPDK, the jansson library also needs to 31be installed, and can be installed via:: 32 33 pkg install jansson 34 35Individual drivers may have additional requirements. Consult the relevant 36driver guide for any driver-specific requirements of interest. 37 38Building DPDK 39------------- 40 41The following commands can be used to build and install DPDK on a system. 42The final, install, step generally needs to be run as root:: 43 44 meson build 45 cd build 46 ninja 47 ninja install 48 49This will install the DPDK libraries and drivers to `/usr/local/lib` with a 50pkg-config file `libdpdk.pc` installed to `/usr/local/lib/pkgconfig`. The 51DPDK test applications, such as `dpdk-testpmd` are installed to 52`/usr/local/bin`. To use these applications, it is recommended that the 53`contigmem` and `nic_uio` kernel modules be loaded first, as described in 54the next section. 55 56.. note:: 57 58 It is recommended that pkg-config be used to query information 59 about the compiler and linker flags needed to build applications 60 against DPDK. In some cases, the path `/usr/local/lib/pkgconfig` 61 may not be in the default search paths for `.pc` files, which means 62 that queries for DPDK information may fail. This can be fixed by 63 setting the appropriate path in `PKG_CONFIG_PATH` environment 64 variable. 65 66 67.. _loading_contigmem: 68 69Loading the DPDK contigmem Module 70--------------------------------- 71 72To run a DPDK application, physically contiguous memory is required. 73In the absence of non-transparent superpages, the included sources for the 74contigmem kernel module provides the ability to present contiguous blocks of 75memory for the DPDK to use. The contigmem module must be loaded into the 76running kernel before any DPDK is run. Once DPDK is installed on the 77system, the module can be found in the `/boot/modules` directory. 78 79The amount of physically contiguous memory along with the number of physically 80contiguous blocks to be reserved by the module can be set at runtime prior to 81module loading using:: 82 83 kenv hw.contigmem.num_buffers=n 84 kenv hw.contigmem.buffer_size=m 85 86The kernel environment variables can also be specified during boot by placing the 87following in ``/boot/loader.conf``: 88 89.. code-block:: shell 90 91 hw.contigmem.num_buffers=n 92 hw.contigmem.buffer_size=m 93 94The variables can be inspected using the following command:: 95 96 sysctl -a hw.contigmem 97 98Where n is the number of blocks and m is the size in bytes of each area of 99contiguous memory. A default of two buffers of size 1073741824 bytes (1 Gigabyte) 100each is set during module load if they are not specified in the environment. 101 102The module can then be loaded using kldload:: 103 104 kldload contigmem 105 106It is advisable to include the loading of the contigmem module during the boot 107process to avoid issues with potential memory fragmentation during later system 108up time. This can be achieved by placing lines similar to the following into 109``/boot/loader.conf``: 110 111.. code-block:: shell 112 113 hw.contigmem.num_buffers=1 114 hw.contigmem.buffer_size=1073741824 115 contigmem_load="YES" 116 117.. note:: 118 119 The contigmem_load directive should be placed after any definitions of 120 ``hw.contigmem.num_buffers`` and ``hw.contigmem.buffer_size`` if the default values 121 are not to be used. 122 123An error such as:: 124 125 kldload: can't load ./x86_64-native-freebsd-gcc/kmod/contigmem.ko: 126 Exec format error 127 128is generally attributed to not having enough contiguous memory 129available and can be verified via dmesg or ``/var/log/messages``:: 130 131 kernel: contigmalloc failed for buffer <n> 132 133To avoid this error, reduce the number of buffers or the buffer size. 134 135.. _loading_nic_uio: 136 137Loading the DPDK nic_uio Module 138------------------------------- 139 140After loading the contigmem module, the ``nic_uio`` module must also be loaded into the 141running kernel prior to running any DPDK application, e.g. using:: 142 143 kldload nic_uio 144 145.. note:: 146 147 If the ports to be used are currently bound to a existing kernel driver 148 then the ``hw.nic_uio.bdfs sysctl`` value will need to be set before loading the 149 module. Setting this value is described in the next section below. 150 151Currently loaded modules can be seen by using the ``kldstat`` command and a module 152can be removed from the running kernel by using ``kldunload <module_name>``. 153 154To load the module during boot place the following into ``/boot/loader.conf``: 155 156.. code-block:: shell 157 158 nic_uio_load="YES" 159 160.. note:: 161 162 ``nic_uio_load="YES"`` must appear after the contigmem_load directive, if it exists. 163 164By default, the ``nic_uio`` module will take ownership of network ports if they are 165recognized DPDK devices and are not owned by another module. However, since 166the FreeBSD kernel includes support, either built-in, or via a separate driver 167module, for most network card devices, it is likely that the ports to be used are 168already bound to a driver other than ``nic_uio``. The following sub-section describe 169how to query and modify the device ownership of the ports to be used by 170DPDK applications. 171 172.. _binding_network_ports: 173 174Binding Network Ports to the nic_uio Module 175~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 176 177Device ownership can be viewed using the pciconf -l command. The example below shows 178four Intel® 82599 network ports under ``if_ixgbe`` module ownership. 179 180.. code-block:: none 181 182 pciconf -l 183 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 184 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 185 ix2@pci0:2:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 186 ix3@pci0:2:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 187 188The first column constitutes three components: 189 190#. Device name: ``ixN`` 191 192#. Unit name: ``pci0`` 193 194#. Selector (Bus:Device:Function): ``1:0:0`` 195 196Where no driver is associated with a device, the device name will be ``none``. 197 198By default, the FreeBSD kernel will include built-in drivers for the most common 199devices; a kernel rebuild would normally be required to either remove the drivers 200or configure them as loadable modules. 201 202To avoid building a custom kernel, the ``nic_uio`` module can detach a network port 203from its current device driver. This is achieved by setting the ``hw.nic_uio.bdfs`` 204kernel environment variable prior to loading ``nic_uio``, as follows:: 205 206 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 207 208Where a comma separated list of selectors is set, the list must not contain any 209whitespace. 210 211For example to re-bind ``ix2@pci0:2:0:0`` and ``ix3@pci0:2:0:1`` to the ``nic_uio`` module 212upon loading, use the following command:: 213 214 kenv hw.nic_uio.bdfs="2:0:0,2:0:1" 215 216The variable can also be specified during boot by placing the following into 217``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line - as 218shown: 219 220.. code-block:: shell 221 222 hw.nic_uio.bdfs="2:0:0,2:0:1" 223 nic_uio_load="YES" 224 225Binding Network Ports Back to their Original Kernel Driver 226~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 227 228If the original driver for a network port has been compiled into the kernel, 229it is necessary to reboot FreeBSD to restore the original device binding. Before 230doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. 231 232If rebinding to a driver that is a loadable module, the network port binding can 233be reset without rebooting. To do so, unload both the target kernel module and the 234``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment (kenv) 235value, and reload the two drivers - first the original kernel driver, and then 236the ``nic_uio driver``. Note: the latter does not need to be reloaded unless there are 237ports that are still to be bound to it. 238 239Example commands to perform these steps are shown below:: 240 241 kldunload nic_uio 242 kldunload <original_driver> 243 244 # To clear the value completely: 245 kenv -u hw.nic_uio.bdfs 246 247 # To update the list of ports to bind: 248 kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." 249 250 kldload <original_driver> 251 252 kldload nic_uio # optional 253