1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019 Intel Corporation. 3 4Enable DPDK on OpenWrt 5====================== 6 7This document describes how to enable Data Plane Development Kit (DPDK) on 8OpenWrt in both a virtual and physical x86 environment. 9 10Introduction 11------------ 12 13The OpenWrt project is a well-known source-based router OS which provides a 14fully writable filesystem with package management. 15 16Build OpenWrt 17------------- 18 19You can obtain OpenWrt image through https://downloads.openwrt.org/releases. 20To fully customize your own OpenWrt, it is highly recommended to build it from 21the source code. You can clone the OpenWrt source code as follows: 22 23.. code-block:: console 24 25 git clone https://git.openwrt.org/openwrt/openwrt.git 26 27OpenWrt configuration 28~~~~~~~~~~~~~~~~~~~~~ 29 30* Select ``x86`` in ``Target System`` 31* Select ``x86_64`` in ``Subtarget`` 32* Select ``Build the OpenWrt SDK`` for cross-compilation environment 33* Select ``Use glibc`` in ``Advanced configuration options (for developers)`` 34 then ``ToolChain Options`` and ``C Library implementation`` 35 36Kernel configuration 37~~~~~~~~~~~~~~~~~~~~ 38 39The following configurations should be enabled: 40 41* ``CONFIG_VFIO_IOMMU_TYPE1=y`` 42* ``CONFIG_VFIO_VIRQFD=y`` 43* ``CONFIG_VFIO=y`` 44* ``CONFIG_VFIO_NOIOMMU=y`` 45* ``CONFIG_VFIO_PCI=y`` 46* ``CONFIG_VFIO_PCI_MMAP=y`` 47* ``CONFIG_HUGETLBFS=y`` 48* ``CONFIG_HUGETLB_PAGE=y`` 49* ``CONFIG_PROC_PAGE_MONITOR=y`` 50 51Build steps 52~~~~~~~~~~~ 53 54For detailed OpenWrt build steps and prerequisites, please refer to the 55`OpenWrt build guide 56<https://openwrt.org/docs/guide-developer/build-system/use-buildsystem>`_. 57 58After the build is completed, you can find the images and SDK in 59``<OpenWrt Root>/bin/targets/x86/64-glibc/``. 60 61 62DPDK Cross Compilation for OpenWrt 63---------------------------------- 64 65Pre-requisites 66~~~~~~~~~~~~~~ 67 68NUMA is required to run DPDK in x86. 69 70.. note:: 71 72 For compiling the NUMA lib, run ``libtool --version`` to ensure the libtool 73 version >= 2.2, otherwise the compilation will fail with errors. 74 75.. code-block:: console 76 77 git clone https://github.com/numactl/numactl.git 78 cd numactl 79 git checkout v2.0.13 -b v2.0.13 80 ./autogen.sh 81 autoconf -i 82 export PATH=<OpenWrt SDK>/glibc/openwrt-sdk-x86-64_gcc-8.3.0_glibc.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-8.3.0_glibc/bin/:$PATH 83 ./configure CC=x86_64-openwrt-linux-gnu-gcc --prefix=<OpenWrt SDK toolchain dir> 84 make install 85 86The numa header files and lib file is generated in the include and lib folder 87respectively under <OpenWrt SDK toolchain dir>. 88 89Build DPDK 90~~~~~~~~~~ 91 92To cross compile with meson build, you need to write a customized cross file 93first. 94 95.. code-block:: console 96 97 [binaries] 98 c = 'x86_64-openwrt-linux-gcc' 99 cpp = 'x86_64-openwrt-linux-cpp' 100 ar = 'x86_64-openwrt-linux-ar' 101 strip = 'x86_64-openwrt-linux-strip' 102 103 meson builddir --cross-file openwrt-cross 104 ninja -C builddir 105 106.. note:: 107 108 For compiling the igb_uio with the kernel version used in target machine, 109 you need to explicitly specify kernel_dir in meson_options.txt. 110 111Running DPDK application on OpenWrt 112----------------------------------- 113 114Virtual machine 115~~~~~~~~~~~~~~~ 116 117* Extract the boot image 118 119.. code-block:: console 120 121 gzip -d openwrt-x86-64-combined-ext4.img.gz 122 123* Launch Qemu 124 125.. code-block:: console 126 127 qemu-system-x86_64 \ 128 -cpu host \ 129 -smp 8 \ 130 -enable-kvm \ 131 -M q35 \ 132 -m 2048M \ 133 -object memory-backend-file,id=mem,size=2048M,mem-path=/tmp/hugepages,share=on \ 134 -drive file=<Your OpenWrt images folder>/openwrt-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 \ 135 -device ide-hd,drive=d0,bus=ide.0 \ 136 -net nic,vlan=0 \ 137 -net nic,vlan=1 \ 138 -net user,vlan=1 \ 139 -display none \ 140 141 142Physical machine 143~~~~~~~~~~~~~~~~ 144 145You can use the ``dd`` tool to write the OpenWrt image to the drive you 146want to write the image on. 147 148.. code-block:: console 149 150 dd if=openwrt-18.06.1-x86-64-combined-squashfs.img of=/dev/sdX 151 152Where sdX is name of the drive. (You can find it though ``fdisk -l``) 153 154Running DPDK 155~~~~~~~~~~~~ 156 157More detailed info about how to run a DPDK application please refer to 158``Running DPDK Applications`` section of :ref:`the DPDK documentation <linux_gsg>`. 159 160.. note:: 161 162 You need to install pre-built NUMA libraries (including soft link) 163 to /usr/lib64 in OpenWrt. 164