1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019 Intel Corporation 3 4Intel(R) FPGA LTE FEC Poll Mode Driver 5====================================== 6 7The BBDEV FPGA LTE FEC poll mode driver (PMD) supports an FPGA implementation of a VRAN 8Turbo Encode / Decode LTE wireless acceleration function, using Intel's PCI-e and FPGA 9based Vista Creek device. 10 11Features 12-------- 13 14FPGA LTE FEC PMD supports the following features: 15 16- Turbo Encode in the DL with total throughput of 4.5 Gbits/s 17- Turbo Decode in the UL with total throughput of 1.5 Gbits/s assuming 8 decoder iterations 18- 8 VFs per PF (physical device) 19- Maximum of 32 UL queues per VF 20- Maximum of 32 DL queues per VF 21- PCIe Gen-3 x8 Interface 22- MSI-X 23- SR-IOV 24 25 26FPGA LTE FEC PMD supports the following BBDEV capabilities: 27 28* For the turbo encode operation: 29 - ``RTE_BBDEV_TURBO_CRC_24B_ATTACH`` : set to attach CRC24B to CB(s) 30 - ``RTE_BBDEV_TURBO_RATE_MATCH`` : if set then do not do Rate Match bypass 31 - ``RTE_BBDEV_TURBO_ENC_INTERRUPTS`` : set for encoder dequeue interrupts 32 33 34* For the turbo decode operation: 35 - ``RTE_BBDEV_TURBO_CRC_TYPE_24B`` : check CRC24B from CB(s) 36 - ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE`` : perform subblock de-interleave 37 - ``RTE_BBDEV_TURBO_DEC_INTERRUPTS`` : set for decoder dequeue interrupts 38 - ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_IN`` : set if negative LLR encoder i/p is supported 39 - ``RTE_BBDEV_TURBO_DEC_TB_CRC_24B_KEEP`` : keep CRC24B bits appended while decoding 40 41 42Limitations 43----------- 44 45FPGA LTE FEC does not support the following: 46 47- Scatter-Gather function 48 49 50Installation 51-------------- 52 53Section 3 of the DPDK manual provides instuctions on installing and compiling DPDK. The 54default set of bbdev compile flags may be found in config/common_base, where for example 55the flag to build the FPGA LTE FEC device, ``CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_LTE_FEC``, is already 56set. It is assumed DPDK has been compiled using for instance: 57 58.. code-block:: console 59 60 make install T=x86_64-native-linuxapp-gcc 61 62 63DPDK requires hugepages to be configured as detailed in section 2 of the DPDK manual. 64The bbdev test application has been tested with a configuration 40 x 1GB hugepages. The 65hugepage configuration of a server may be examined using: 66 67.. code-block:: console 68 69 grep Huge* /proc/meminfo 70 71 72Initialization 73-------------- 74 75When the device first powers up, its PCI Physical Functions (PF) can be listed through this command: 76 77.. code-block:: console 78 79 sudo lspci -vd1172:5052 80 81The physical and virtual functions are compatible with Linux UIO drivers: 82``vfio`` and ``igb_uio``. However, in order to work the FPGA LTE FEC device firstly needs 83to be bound to one of these linux drivers through DPDK. 84 85 86Bind PF UIO driver(s) 87~~~~~~~~~~~~~~~~~~~~~ 88 89Install the DPDK igb_uio driver, bind it with the PF PCI device ID and use 90``lspci`` to confirm the PF device is under use by ``igb_uio`` DPDK UIO driver. 91 92The igb_uio driver may be bound to the PF PCI device using one of three methods: 93 94 951. PCI functions (physical or virtual, depending on the use case) can be bound to 96the UIO driver by repeating this command for every function. 97 98.. code-block:: console 99 100 cd <dpdk-top-level-directory> 101 insmod ./build/kmod/igb_uio.ko 102 echo "1172 5052" > /sys/bus/pci/drivers/igb_uio/new_id 103 lspci -vd1172: 104 105 1062. Another way to bind PF with DPDK UIO driver is by using the ``dpdk-devbind.py`` tool 107 108.. code-block:: console 109 110 cd <dpdk-top-level-directory> 111 ./usertools/dpdk-devbind.py -b igb_uio 0000:06:00.0 112 113where the PCI device ID (example: 0000:06:00.0) is obtained using lspci -vd1172: 114 115 1163. A third way to bind is to use ``dpdk-setup.sh`` tool 117 118.. code-block:: console 119 120 cd <dpdk-top-level-directory> 121 ./usertools/dpdk-setup.sh 122 123 select 'Bind Ethernet/Crypto/Baseband device to IGB UIO module' 124 or 125 select 'Bind Ethernet/Crypto/Baseband device to VFIO module' depending on driver required 126 enter PCI device ID 127 select 'Display current Ethernet/Crypto/Baseband device settings' to confirm binding 128 129 130In the same way the FPGA LTE FEC PF can be bound with vfio, but vfio driver does not 131support SR-IOV configuration right out of the box, so it will need to be patched. 132 133 134Enable Virtual Functions 135~~~~~~~~~~~~~~~~~~~~~~~~ 136 137Now, it should be visible in the printouts that PCI PF is under igb_uio control 138"``Kernel driver in use: igb_uio``" 139 140To show the number of available VFs on the device, read ``sriov_totalvfs`` file.. 141 142.. code-block:: console 143 144 cat /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_totalvfs 145 146 where 0000\:<b>\:<d>.<f> is the PCI device ID 147 148 149To enable VFs via igb_uio, echo the number of virtual functions intended to 150enable to ``max_vfs`` file.. 151 152.. code-block:: console 153 154 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/max_vfs 155 156 157Afterwards, all VFs must be bound to appropriate UIO drivers as required, same 158way it was done with the physical function previously. 159 160Enabling SR-IOV via vfio driver is pretty much the same, except that the file 161name is different: 162 163.. code-block:: console 164 165 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_numvfs 166 167 168Configure the VFs through PF 169~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 170 171The PCI virtual functions must be configured before working or getting assigned 172to VMs/Containers. The configuration involves allocating the number of hardware 173queues, priorities, load balance, bandwidth and other settings necessary for the 174device to perform FEC functions. 175 176This configuration needs to be executed at least once after reboot or PCI FLR and can 177be achieved by using the function ``fpga_lte_fec_configure()``, which sets up the 178parameters defined in ``fpga_lte_fec_conf`` structure: 179 180.. code-block:: c 181 182 struct fpga_lte_fec_conf { 183 bool pf_mode_en; 184 uint8_t vf_ul_queues_number[FPGA_LTE_FEC_NUM_VFS]; 185 uint8_t vf_dl_queues_number[FPGA_LTE_FEC_NUM_VFS]; 186 uint8_t ul_bandwidth; 187 uint8_t dl_bandwidth; 188 uint8_t ul_load_balance; 189 uint8_t dl_load_balance; 190 uint16_t flr_time_out; 191 }; 192 193- ``pf_mode_en``: identifies whether only PF is to be used, or the VFs. PF and 194 VFs are mutually exclusive and cannot run simultaneously. 195 Set to 1 for PF mode enabled. 196 If PF mode is enabled all queues available in the device are assigned 197 exclusively to PF and 0 queues given to VFs. 198 199- ``vf_*l_queues_number``: defines the hardware queue mapping for every VF. 200 201- ``*l_bandwidth``: in case of congestion on PCIe interface. The device 202 allocates different bandwidth to UL and DL. The weight is configured by this 203 setting. The unit of weight is 3 code blocks. For example, if the code block 204 cbps (code block per second) ratio between UL and DL is 12:1, then the 205 configuration value should be set to 36:3. The schedule algorithm is based 206 on code block regardless the length of each block. 207 208- ``*l_load_balance``: hardware queues are load-balanced in a round-robin 209 fashion. Queues get filled first-in first-out until they reach a pre-defined 210 watermark level, if exceeded, they won't get assigned new code blocks.. 211 This watermark is defined by this setting. 212 213 If all hardware queues exceeds the watermark, no code blocks will be 214 streamed in from UL/DL code block FIFO. 215 216- ``flr_time_out``: specifies how many 16.384us to be FLR time out. The 217 time_out = flr_time_out x 16.384us. For instance, if you want to set 10ms for 218 the FLR time out then set this setting to 0x262=610. 219 220 221An example configuration code calling the function ``fpga_lte_fec_configure()`` is shown 222below: 223 224.. code-block:: c 225 226 struct fpga_lte_fec_conf conf; 227 unsigned int i; 228 229 memset(&conf, 0, sizeof(struct fpga_lte_fec_conf)); 230 conf.pf_mode_en = 1; 231 232 for (i = 0; i < FPGA_LTE_FEC_NUM_VFS; ++i) { 233 conf.vf_ul_queues_number[i] = 4; 234 conf.vf_dl_queues_number[i] = 4; 235 } 236 conf.ul_bandwidth = 12; 237 conf.dl_bandwidth = 5; 238 conf.dl_load_balance = 64; 239 conf.ul_load_balance = 64; 240 241 /* setup FPGA PF */ 242 ret = fpga_lte_fec_configure(info->dev_name, &conf); 243 TEST_ASSERT_SUCCESS(ret, 244 "Failed to configure 4G FPGA PF for bbdev %s", 245 info->dev_name); 246 247 248Test Application 249---------------- 250 251BBDEV provides a test application, ``test-bbdev.py`` and range of test data for testing 252the functionality of FPGA LTE FEC turbo encode and turbo decode, depending on the device's 253capabilities. The test application is located under app->test-bbdev folder and has the 254following options: 255 256.. code-block:: console 257 258 "-p", "--testapp-path": specifies path to the bbdev test app. 259 "-e", "--eal-params" : EAL arguments which are passed to the test app. 260 "-t", "--timeout" : Timeout in seconds (default=300). 261 "-c", "--test-cases" : Defines test cases to run. Run all if not specified. 262 "-v", "--test-vector" : Test vector path (default=dpdk_path+/app/test-bbdev/test_vectors/bbdev_null.data). 263 "-n", "--num-ops" : Number of operations to process on device (default=32). 264 "-b", "--burst-size" : Operations enqueue/dequeue burst size (default=32). 265 "-l", "--num-lcores" : Number of lcores to run (default=16). 266 "-i", "--init-device" : Initialise PF device with default values. 267 268 269To execute the test application tool using simple turbo decode or turbo encode data, 270type one of the following: 271 272.. code-block:: console 273 274 ./test-bbdev.py -c validation -n 64 -b 8 -v ./turbo_dec_default.data 275 ./test-bbdev.py -c validation -n 64 -b 8 -v ./turbo_enc_default.data 276 277 278The test application ``test-bbdev.py``, supports the ability to configure the PF device with 279a default set of values, if the "-i" or "- -init-device" option is included. The default values 280are defined in test_bbdev_perf.c as: 281 282- VF_UL_QUEUE_VALUE 4 283- VF_DL_QUEUE_VALUE 4 284- UL_BANDWIDTH 3 285- DL_BANDWIDTH 3 286- UL_LOAD_BALANCE 128 287- DL_LOAD_BALANCE 128 288- FLR_TIMEOUT 610 289 290 291Test Vectors 292~~~~~~~~~~~~ 293 294In addition to the simple turbo decoder and turbo encoder tests, bbdev also provides 295a range of additional tests under the test_vectors folder, which may be useful. The results 296of these tests will depend on the FPGA LTE FEC capabilities: 297 298* turbo decoder tests: 299 - ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 300 - ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data`` 301 - ``turbo_dec_c1_k6144_r0_e34560_negllr.data`` 302 - ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data`` 303 - ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data`` 304 - ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data`` 305 306 307* turbo encoder tests: 308 - ``turbo_enc_c1_k40_r0_e1190_rm.data`` 309 - ``turbo_enc_c1_k40_r0_e1194_rm.data`` 310 - ``turbo_enc_c1_k40_r0_e1196_rm.data`` 311 - ``turbo_enc_c1_k40_r0_e272_rm.data`` 312 - ``turbo_enc_c1_k6144_r0_e18444.data`` 313 - ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 314 - ``turbo_enc_c2_k5952_r0_e17868_crc24b.data`` 315 - ``turbo_enc_c3_k4800_r2_e14412_crc24b.data`` 316 - ``turbo_enc_c4_k4800_r2_e14412_crc24b.data`` 317