1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2021 ARM Corporation. 3 4Cross compiling DPDK for aarch64 and aarch32 5============================================ 6 7This chapter describes how to cross compile DPDK for aarch64 on x86 build 8machine and compile 32-bit aarch32 DPDK on aarch64 build machine. 9 10.. note:: 11 12 Whilst it is recommended to natively build DPDK on aarch64 (just 13 like with x86), it is also possible to cross compile DPDK for aarch64. 14 An aarch64 cross compiler GNU toolchain or an LLVM/clang toolchain 15 may be used for cross-compilation. 16 17 18Prerequisites 19------------- 20 21NUMA library 22~~~~~~~~~~~~ 23 24NUMA is required by most modern machines, not needed for non-NUMA architectures. 25 26.. note:: 27 28 For compiling the NUMA lib, run libtool --version to ensure the libtool version >= 2.2, 29 otherwise the compilation will fail with errors. 30 31.. code-block:: console 32 33 git clone https://github.com/numactl/numactl.git 34 cd numactl 35 git checkout v2.0.13 -b v2.0.13 36 ./autogen.sh 37 autoconf -i 38 ./configure --host=aarch64-linux-gnu CC=aarch64-none-linux-gnu-gcc --prefix=<numa install dir> 39 make install 40 41.. note:: 42 43 The compiler is ``aarch64-none-linux-gnu-gcc`` if you download GCC 44 using the below guide. If you're using a different compiler, 45 make sure you're using the proper executable name. 46 47The numa header files and lib file is generated in the include and lib folder 48respectively under ``<numa install dir>``. 49 50Meson prerequisites 51~~~~~~~~~~~~~~~~~~~ 52 53Meson depends on pkgconfig to find the dependencies. 54The package ``pkg-config-aarch64-linux-gnu`` is required for aarch64. 55To install it in Ubuntu:: 56 57 sudo apt install pkg-config-aarch64-linux-gnu 58 59For aarch32, install ``pkg-config-arm-linux-gnueabihf``:: 60 61 sudo apt install pkg-config-arm-linux-gnueabihf 62 63 64GNU toolchain 65------------- 66 67.. _obtain_GNU_toolchain: 68 69Get the cross toolchain 70~~~~~~~~~~~~~~~~~~~~~~~ 71 72The latest GNU cross compiler toolchain can be downloaded from: 73https://developer.arm.com/open-source/gnu-toolchain/gnu-a/downloads. 74 75It is always recommended to check and get the latest compiler tool 76from the page and use it to generate better code. 77As of this writing 9.2-2019.12 is the newest, 78the following description is an example of this version. 79 80For aarch64:: 81 82 wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz 83 tar -xvf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz 84 export PATH=$PATH:<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin 85 86For aarch32:: 87 88 wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf.tar.xz 89 tar -xvf gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf.tar.xz 90 export PATH=$PATH:<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf/bin 91 92.. note:: 93 94 For the host requirements and other info, refer to the release note section: 95 https://releases.linaro.org/components/toolchain/binaries/ 96 97.. _augment_the_gnu_toolchain_with_numa_support: 98 99Augment the GNU toolchain with NUMA support 100~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 101 102Copy the NUMA header files and lib to the cross compiler's directories: 103 104.. code-block:: console 105 106 cp <numa_install_dir>/include/numa*.h <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc/usr/include/ 107 cp <numa_install_dir>/lib/libnuma.a <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/lib/gcc/aarch64-none-linux-gnu/9.2.1/ 108 cp <numa_install_dir>/lib/libnuma.so <cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/lib/gcc/aarch64-none-linux-gnu/9.2.1/ 109 110.. note:: 111 112 Using LDFLAGS and CFLAGS is not a viable alternative to copying the files. 113 The Meson docs say it is not recommended, as there are many caveats 114 to their use with Meson, especially when rebuilding the project. 115 A viable alternative would be to use the ``c_args`` and ``c_link_args`` 116 options with Meson 0.51.0 and higher: 117 118 .. code-block:: console 119 120 -Dc_args=-I<numa_install_dir>/include -Dc_link_args=-L<numa_install_dir>/lib 121 122For Meson versions lower than 0.51.0, the ``c_args`` and ``c_link_args`` 123options do not apply to cross compilation. 124However, the compiler/linker flags may be added to cross files under [properties]: 125 126.. code-block:: console 127 128 c_args = ['-I<numa_install_dir>/include'] 129 c_link_args = ['-L<numa_install_dir>/lib'] 130 131Cross Compiling DPDK with GNU toolchain using Meson 132~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 133 134.. note:: 135 136 The names of GCC binaries in cross files differ from the downloaded ones, 137 which have an extra ``-none-`` in their name. 138 Please modify the cross file binaries accordingly 139 when using the downloaded cross compilers. 140 141 An example cross file with modified names and added NUMA paths 142 would look like this: 143 144 .. code-block:: console 145 146 [binaries] 147 c = 'aarch64-none-linux-gnu-gcc' 148 cpp = 'aarch64-none-linux-gnu-cpp' 149 ar = 'aarch64-none-linux-gnu-gcc-ar' 150 strip = 'aarch64-none-linux-gnu-strip' 151 pkgconfig = 'aarch64-linux-gnu-pkg-config' # the downloaded binaries 152 # do not contain a pkgconfig binary, so it is not modified 153 pcap-config = '' 154 155 [host_machine] 156 system = 'linux' 157 cpu_family = 'aarch64' 158 cpu = 'armv8-a' 159 endian = 'little' 160 161 [properties] 162 # Generate binaries that are portable across all Armv8 machines 163 platform = 'generic' 164 c_args = ['-I<numa_install_dir>/include'] # replace <numa_install_dir> 165 c_link_args = ['-L<numa_install_dir>/lib'] # with your path 166 167To cross-compile DPDK on a desired target machine we can use the following 168command:: 169 170 meson setup cross-build --cross-file <target_machine_configuration> 171 ninja -C cross-build 172 173For example if the target machine is aarch64 we can use the following 174command, provided the cross file has been modified accordingly:: 175 176 meson setup aarch64-build-gcc --cross-file config/arm/arm64_armv8_linux_gcc 177 ninja -C aarch64-build-gcc 178 179If the target machine is aarch32 we can use the following command, 180provided the cross file has been modified accordingly:: 181 182 meson setup aarch32-build --cross-file config/arm/arm32_armv8_linux_gcc 183 ninja -C aarch32-build 184 185LLVM/Clang toolchain 186-------------------- 187 188Obtain the cross tool chain 189~~~~~~~~~~~~~~~~~~~~~~~~~~~ 190 191The latest LLVM/Clang cross compiler toolchain can be downloaded from: 192https://developer.arm.com/tools-and-software/open-source-software/developer-tools/llvm-toolchain. 193 194.. code-block:: console 195 196 # Ubuntu binaries 197 wget https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz 198 199The LLVM/Clang toolchain does not implement the standard c library. 200The GNU toolchain ships an implementation we can use. 201Refer to obtain_GNU_toolchain_ to get the GNU toolchain. 202 203Unzip and add into the PATH 204~~~~~~~~~~~~~~~~~~~~~~~~~~~ 205 206.. code-block:: console 207 208 tar -xvf clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz 209 export PATH=$PATH:<cross_install_dir>/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04/bin 210 211Cross Compiling DPDK with LLVM/Clang toolchain using Meson 212~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 213 214.. note:: 215 216 To use the NUMA library follow the same steps as for 217 augment_the_gnu_toolchain_with_numa_support_. 218 219The paths to GNU stdlib must be specified in a cross file. 220Augmenting the default cross-file's ``c_args`` and ``c_link_args`` 221``config/arm/arm64_armv8_linux_clang_ubuntu1804`` would look like this: 222 223.. code-block:: console 224 225 ... 226 c_args = ['-target', 'aarch64-linux-gnu', '--sysroot', '<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc'] 227 c_link_args = ['-target', 'aarch64-linux-gnu', '-fuse-ld=lld', '--sysroot', '<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/aarch64-none-linux-gnu/libc', '--gcc-toolchain=<cross_install_dir>/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu'] 228 229Assuming the file with augmented ``c_args`` and ``c_link_args`` 230is named ``arm64_armv8_linux_clang``, 231use the following command to cross-compile DPDK for the target machine:: 232 233 meson setup aarch64-build-clang --cross-file config/arm/arm64_armv8_linux_clang 234 ninja -C aarch64-build-clang 235 236Cross Compiling DPDK with LLVM/Clang toolchain using Meson on Ubuntu 18.04 237~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 238 239On most popular Linux distribution it is not necessary to download 240the toolchains, but rather use the packages provided by said distributions. 241On Ubuntu 18.04, these packages are needed: 242 243.. code-block:: console 244 245 sudo apt-get install pkg-config-aarch64-linux-gnu clang llvm llvm-dev lld 246 libc6-dev-arm64-cross libatomic1-arm64-cross libgcc-8-dev-arm64-cross 247 248Use the following command to cross-compile DPDK for the target machine:: 249 250 meson setup aarch64-build-clang --cross-file config/arm/arm64_armv8_linux_clang_ubuntu1804 251 ninja -C aarch64-build-clang 252 253Building for an aarch64 SoC on an aarch64 build machine 254------------------------------------------------------- 255 256If you wish to build on an aarch64 build machine for a different aarch64 SoC, 257you don't need a separate cross toolchain, just a different set of 258configuration options. To build for an aarch64 SoC, use the -Dplatform meson 259option:: 260 261 meson setup soc_build -Dplatform=<target_soc> 262 263Substitute <target_soc> with one of the supported SoCs 264 265.. literalinclude:: ../../../config/arm/meson.build 266 :start-after: Start of SoCs list 267 :end-before: End of SoCs list 268 269These SoCs are also used in cross files, e.g.:: 270 271 [properties] 272 # Generate binaries that are portable across all Armv8 machines 273 platform = 'generic' 274 275Supported SoC configuration 276--------------------------- 277 278The SoC configuration is a combination of implementer and CPU part number 279configuration and SoC-specific configuration:: 280 281 soc_<name> = { 282 'description': 'SoC Description', # mandatory 283 'implementer': <implementer_id>, # mandatory 284 'part_number': <part_number>, # mandatory 285 'numa': false, # optional, specify for non-NUMA SoCs 286 'enable_drivers': 'common/*,bus/*', # optional, comma-separated list of 287 # drivers to build, wildcards are accepted 288 'disable_drivers': 'crypto/*', # optional, comma-separated list of 289 # drivers to disable, wildcards are accepted 290 'flags': [ 291 ['RTE_MAX_LCORE', '16'], 292 ['RTE_MAX_NUMA_NODES', '1'] 293 ] # optional, list of DPDK options that will be added 294 # or overwritten 295 } 296 297Where <implementer_id> is a key defined in the implementers dictionary 298in config/arm/meson.build (e.g. 0x41) and part_number is a key defined 299in implementers[<implementer_id>]['part_number_config'] dictionary 300(i.e. the part number must be defined for the implementer, 301e.g. for 0x41, a valid value is 0xd49, which is the neoverse-n2 SoC). 302