1.. BSD LICENSE 2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved. 3 All rights reserved. 4 5 Redistribution and use in source and binary forms, with or without 6 modification, are permitted provided that the following conditions 7 are met: 8 9 * Redistributions of source code must retain the above copyright 10 notice, this list of conditions and the following disclaimer. 11 * Redistributions in binary form must reproduce the above copyright 12 notice, this list of conditions and the following disclaimer in 13 the documentation and/or other materials provided with the 14 distribution. 15 * Neither the name of Intel Corporation nor the names of its 16 contributors may be used to endorse or promote products derived 17 from this software without specific prior written permission. 18 19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 31Compiling the Intel® DPDK Target from Source 32============================================ 33 34.. note:: 35 36 Parts of this process can also be done using the setup script described in Chapter 6 of this document. 37 38Install the Intel® DPDK and Browse Sources 39------------------------------------------ 40 41First, uncompress the archive and move to the uncompressed Intel® DPDK source directory: 42 43.. code-block:: console 44 45 user@host:~$ unzip DPDK-<version>.zip 46 user@host:~$ cd DPDK-<version> 47 user@host:~/DPDK-<version>$ ls 48 app/ config/ examples/ lib/ LICENSE.GPL LICENSE.LGPL Makefile mk/ scripts/ tools/ 49 50The Intel® DPDK is composed of several directories: 51 52* lib: Source code of Intel® DPDK libraries 53 54* app: Source code of Intel® DPDK applications (automatic tests) 55 56* examples: Source code of Intel® DPDK application examples 57 58* config, tools, scripts, mk: Framework-related makefiles, scripts and configuration 59 60Installation of Intel® DPDK Target Environments 61----------------------------------------------- 62 63The format of a Intel® DPDK target is: 64 65 ARCH-MACHINE-EXECENV-TOOLCHAIN 66 67where: 68 69* ARCH can be: i686, x86_64 70 71* MACHINE can be: native, ivshmem 72 73* EXECENV can be: linuxapp, bsdapp 74 75* TOOLCHAIN can be: gcc, icc 76 77The targets to be installed depend on the 32-bit and/or 64-bit packages and compilers installed on the host. 78Available targets can be found in the DPDK/config directory. 79The defconfig\_ prefix should not be used. 80 81.. note:: 82 83 Configuration files are provided with the RTE_MACHINE optimization level set. 84 Within the configuration files, the RTE_MACHINE configuration value is set to native, 85 which means that the compiled software is tuned for the platform on which it is built. 86 For more information on this setting, and its possible values, see the *Intel® DPDK Programmers Guide*. 87 88When using the Intel® C++ Compiler (icc), one of the following commands should be invoked for 64-bit or 32-bit use respectively. 89Notice that the shell scripts update the $PATH variable and therefore should not be performed in the same session. 90Also, verify the compiler's installation directory since the path may be different: 91 92.. code-block:: console 93 94 source /opt/intel/bin/iccvars.sh intel64 95 source /opt/intel/bin/iccvars.sh ia32 96 97To install and make targets, use the make install T=<target> command in the top-level Intel® DPDK directory. 98 99For example, to compile a 64-bit target using icc, run: 100 101.. code-block:: console 102 103 make install T=x86_64-native-linuxapp-icc 104 105To compile a 32-bit build using gcc, the make command should be: 106 107.. code-block:: console 108 109 make install T=i686-native-linuxapp-gcc 110 111To compile all 64-bit targets using gcc, use: 112 113.. code-block:: console 114 115 make install T=x86_64*gcc 116 117To compile all 64-bit targets using both gcc and icc, use: 118 119.. code-block:: console 120 121 make install T=x86_64-* 122 123.. note:: 124 125 The wildcard operator (*) can be used to create multiple targets at the same time. 126 127To prepare a target without building it, for example, if the configuration changes need to be made before compilation, 128use the make config T=<target> command: 129 130.. code-block:: console 131 132 make config T=x86_64-native-linuxapp-gcc 133 134.. warning:: 135 136 The igb_uio module must be compiled with the same kernel as the one running on the target. 137 If the Intel® DPDK is not being built on the target machine, 138 the RTE_KERNELDIR environment variable should be used to point the compilation at a copy of the kernel version to be used on the target machine. 139 140Once the target environment is created, the user may move to the target environment directory and continue to make code changes and re-compile. 141The user may also make modifications to the compile-time Intel® DPDK configuration by editing the .config file in the build directory. 142(This is a build-local copy of the defconfig file from the top- level config directory). 143 144.. code-block:: console 145 146 cd x86_64-native-linuxapp-gcc 147 vi .config 148 make 149 150In addition, the make clean command can be used to remove any existing compiled files for a subsequent full, clean rebuild of the code. 151 152Browsing the Installed Intel® DPDK Environment Target 153----------------------------------------------------- 154 155Once a target is created it contains all libraries and header files for the Intel® DPDK environment that are required to build customer applications. 156In addition, the test and testpmd applications are built under the build/app directory, which may be used for testing. 157In the case of Linux, a kmod directory is also present that contains a module to install: 158 159.. code-block:: console 160 161 $ ls x86_64-native-linuxapp-gcc 162 app build hostapp include kmod lib Makefile 163 164Loading the Intel® DPDK igb_uio Module 165-------------------------------------- 166 167To run any Intel® DPDK application, the igb_uio module can be loaded into the running kernel. 168The module is found in the kmod sub-directory of the Intel® DPDK target directory. 169This module should be loaded using the insmod command as shown below (assuming that the current directory is the Intel® DPDK target directory). 170In many cases, the uio support in the Linux* kernel is compiled as a module rather than as part of the kernel, 171so it is often necessary to load the uio module first: 172 173.. code-block:: console 174 175 sudo modprobe uio 176 sudo insmod kmod/igb_uio.ko 177 178Since Intel® DPDK release 1.7 provides VFIO support, compilation and use of igb_uio module has become optional for platforms that support using VFIO. 179 180Loading VFIO Module 181------------------- 182 183To run an Intel® DPDK application and make use of VFIO, the vfio-pci module must be loaded: 184 185.. code-block:: console 186 187 sudo modprobe vfio-pci 188 189Note that in order to use VFIO, your kernel must support it. 190VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default, 191however please consult your distributions documentation to make sure that is the case. 192 193Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d). 194 195For proper operation of VFIO when running Intel® DPDK applications as a non-privileged user, correct permissions should also be set up. 196This can be done by using the Intel® DPDK setup script (called setup.sh and located in the tools directory). 197 198Binding and Unbinding Network Ports to/from the igb_uioor VFIO Modules 199---------------------------------------------------------------------- 200 201As of release 1.4, Intel® DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. 202Instead, all ports that are to be used by an Intel® DPDK application must be bound to the igb_uio or vfio-pci module before the application is run. 203Any network ports under Linux* control will be ignored by the Intel® DPDK poll-mode drivers and cannot be used by the application. 204 205.. warning:: 206 207 The Intel® DPDK will, by default, no longer automatically unbind network ports from the kernel driver at startup. 208 Any ports to be used by an Intel® DPDK application must be unbound from Linux* control and bound to the igb_uio or vfio-pci module before the application is run. 209 210To bind ports to the igb_uio or vfio-pci module for Intel® DPDK use, and then subsequently return ports to Linux* control, 211a utility script called dpdk_nic _bind.py is provided in the tools subdirectory. 212This utility can be used to provide a view of the current state of the network ports on the system, 213and to bind and unbind those ports from the different kernel modules, including igb_uio and vfio-pci. 214The following are some examples of how the script can be used. 215A full description of the script and its parameters can be obtained by calling the script with the --help or --usage options. 216 217.. warning:: 218 219 Due to the way VFIO works, there are certain limitations to which devices can be used with VFIO. 220 Mainly it comes down to how IOMMU groups work. 221 Any Virtual Function device can be used with VFIO on its own, but physical devices will require either all ports bound to VFIO, 222 or some of them bound to VFIO while others not being bound to anything at all. 223 224 If your device is behind a PCI-to-PCI bridge, the bridge will then be part of the IOMMU group in which your device is in. 225 Therefore, the bridge driver should also be unbound from the bridge PCI device for VFIO to work with devices behind the bridge. 226 227.. warning:: 228 229 While any user can run the dpdk_nic_bind.py script to view the status of the network ports, 230 binding or unbinding network ports requires root privileges. 231 232To see the status of all network ports on the system: 233 234.. code-block:: console 235 236 root@host:DPDK# ./tools/dpdk_nic_bind.py --status 237 238 Network devices using IGB_UIO driver 239 ==================================== 240 0000:82:00.0 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe 241 0000:82:00.1 '82599EB 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe 242 243 Network devices using kernel driver 244 =================================== 245 0000:04:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio *Active* 246 0000:04:00.1 'I350 Gigabit Network Connection' if=eth1 drv=igb unused=igb_uio 247 0000:04:00.2 'I350 Gigabit Network Connection' if=eth2 drv=igb unused=igb_uio 248 0000:04:00.3 'I350 Gigabit Network Connection' if=eth3 drv=igb unused=igb_uio 249 250 Other network devices 251 ===================== 252 <none> 253 254To bind device eth1, 04:00.1, to the igb_uio driver: 255 256.. code-block:: console 257 258 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=igb_uio 04:00.1 259 260or, alternatively, 261 262.. code-block:: console 263 264 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=igb_uio eth1 265 266To restore device 82:00.0 to its original kernel binding: 267 268.. code-block:: console 269 270 root@host:DPDK# ./tools/dpdk_nic_bind.py --bind=ixgbe 82:00.0 271