1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019 Intel Corporation 3 4Intel(R) FPGA LTE FEC Poll Mode Driver 5====================================== 6 7The BBDEV FPGA LTE FEC poll mode driver (PMD) supports an FPGA implementation of a VRAN 8Turbo Encode / Decode LTE wireless acceleration function, using Intel's PCI-e and FPGA 9based Vista Creek device. 10 11Features 12-------- 13 14FPGA LTE FEC PMD supports the following features: 15 16- Turbo Encode in the DL with total throughput of 4.5 Gbits/s 17- Turbo Decode in the UL with total throughput of 1.5 Gbits/s assuming 8 decoder iterations 18- 8 VFs per PF (physical device) 19- Maximum of 32 UL queues per VF 20- Maximum of 32 DL queues per VF 21- PCIe Gen-3 x8 Interface 22- MSI-X 23- SR-IOV 24 25 26FPGA LTE FEC PMD supports the following BBDEV capabilities: 27 28* For the turbo encode operation: 29 - ``RTE_BBDEV_TURBO_CRC_24B_ATTACH`` : set to attach CRC24B to CB(s) 30 - ``RTE_BBDEV_TURBO_RATE_MATCH`` : if set then do not do Rate Match bypass 31 - ``RTE_BBDEV_TURBO_ENC_INTERRUPTS`` : set for encoder dequeue interrupts 32 33 34* For the turbo decode operation: 35 - ``RTE_BBDEV_TURBO_CRC_TYPE_24B`` : check CRC24B from CB(s) 36 - ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE`` : perform subblock de-interleave 37 - ``RTE_BBDEV_TURBO_DEC_INTERRUPTS`` : set for decoder dequeue interrupts 38 - ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_IN`` : set if negative LLR encoder i/p is supported 39 - ``RTE_BBDEV_TURBO_DEC_TB_CRC_24B_KEEP`` : keep CRC24B bits appended while decoding 40 41 42Limitations 43----------- 44 45FPGA LTE FEC does not support the following: 46 47- Scatter-Gather function 48 49 50Installation 51-------------- 52 53Section 3 of the DPDK manual provides instructions on installing and compiling DPDK. 54 55DPDK requires hugepages to be configured as detailed in section 2 of the DPDK manual. 56The bbdev test application has been tested with a configuration 40 x 1GB hugepages. The 57hugepage configuration of a server may be examined using: 58 59.. code-block:: console 60 61 grep Huge* /proc/meminfo 62 63 64Initialization 65-------------- 66 67When the device first powers up, its PCI Physical Functions (PF) can be listed through this command: 68 69.. code-block:: console 70 71 sudo lspci -vd1172:5052 72 73The physical and virtual functions are compatible with Linux UIO drivers: 74``vfio`` and ``igb_uio``. However, in order to work the FPGA LTE FEC device firstly needs 75to be bound to one of these linux drivers through DPDK. 76 77 78Bind PF UIO driver(s) 79~~~~~~~~~~~~~~~~~~~~~ 80 81Install the DPDK igb_uio driver, bind it with the PF PCI device ID and use 82``lspci`` to confirm the PF device is under use by ``igb_uio`` DPDK UIO driver. 83 84The igb_uio driver may be bound to the PF PCI device using one of two methods: 85 86 871. PCI functions (physical or virtual, depending on the use case) can be bound to 88the UIO driver by repeating this command for every function. 89 90.. code-block:: console 91 92 insmod igb_uio.ko 93 echo "1172 5052" > /sys/bus/pci/drivers/igb_uio/new_id 94 lspci -vd1172: 95 96 972. Another way to bind PF with DPDK UIO driver is by using the ``dpdk-devbind.py`` tool 98 99.. code-block:: console 100 101 cd <dpdk-top-level-directory> 102 ./usertools/dpdk-devbind.py -b igb_uio 0000:06:00.0 103 104where the PCI device ID (example: 0000:06:00.0) is obtained using lspci -vd1172: 105 106 107In the same way the FPGA LTE FEC PF can be bound with vfio, but vfio driver does not 108support SR-IOV configuration right out of the box, so it will need to be patched. 109 110 111Enable Virtual Functions 112~~~~~~~~~~~~~~~~~~~~~~~~ 113 114Now, it should be visible in the printouts that PCI PF is under igb_uio control 115"``Kernel driver in use: igb_uio``" 116 117To show the number of available VFs on the device, read ``sriov_totalvfs`` file.. 118 119.. code-block:: console 120 121 cat /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_totalvfs 122 123 where 0000\:<b>\:<d>.<f> is the PCI device ID 124 125 126To enable VFs via igb_uio, echo the number of virtual functions intended to 127enable to ``max_vfs`` file.. 128 129.. code-block:: console 130 131 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/max_vfs 132 133 134Afterwards, all VFs must be bound to appropriate UIO drivers as required, same 135way it was done with the physical function previously. 136 137Enabling SR-IOV via vfio driver is pretty much the same, except that the file 138name is different: 139 140.. code-block:: console 141 142 echo <num-of-vfs> > /sys/bus/pci/devices/0000\:<b>\:<d>.<f>/sriov_numvfs 143 144 145Configure the VFs through PF 146~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 147 148The PCI virtual functions must be configured before working or getting assigned 149to VMs/Containers. The configuration involves allocating the number of hardware 150queues, priorities, load balance, bandwidth and other settings necessary for the 151device to perform FEC functions. 152 153This configuration needs to be executed at least once after reboot or PCI FLR and can 154be achieved by using the function ``rte_fpga_lte_fec_configure()``, which sets up the 155parameters defined in ``rte_fpga_lte_fec_conf`` structure: 156 157.. code-block:: c 158 159 struct rte_fpga_lte_fec_conf { 160 bool pf_mode_en; 161 uint8_t vf_ul_queues_number[FPGA_LTE_FEC_NUM_VFS]; 162 uint8_t vf_dl_queues_number[FPGA_LTE_FEC_NUM_VFS]; 163 uint8_t ul_bandwidth; 164 uint8_t dl_bandwidth; 165 uint8_t ul_load_balance; 166 uint8_t dl_load_balance; 167 uint16_t flr_time_out; 168 }; 169 170- ``pf_mode_en``: identifies whether only PF is to be used, or the VFs. PF and 171 VFs are mutually exclusive and cannot run simultaneously. 172 Set to 1 for PF mode enabled. 173 If PF mode is enabled all queues available in the device are assigned 174 exclusively to PF and 0 queues given to VFs. 175 176- ``vf_*l_queues_number``: defines the hardware queue mapping for every VF. 177 178- ``*l_bandwidth``: in case of congestion on PCIe interface. The device 179 allocates different bandwidth to UL and DL. The weight is configured by this 180 setting. The unit of weight is 3 code blocks. For example, if the code block 181 cbps (code block per second) ratio between UL and DL is 12:1, then the 182 configuration value should be set to 36:3. The schedule algorithm is based 183 on code block regardless the length of each block. 184 185- ``*l_load_balance``: hardware queues are load-balanced in a round-robin 186 fashion. Queues get filled first-in first-out until they reach a pre-defined 187 watermark level, if exceeded, they won't get assigned new code blocks.. 188 This watermark is defined by this setting. 189 190 If all hardware queues exceeds the watermark, no code blocks will be 191 streamed in from UL/DL code block FIFO. 192 193- ``flr_time_out``: specifies how many 16.384us to be FLR time out. The 194 time_out = flr_time_out x 16.384us. For instance, if you want to set 10ms for 195 the FLR time out then set this setting to 0x262=610. 196 197 198An example configuration code calling the function ``rte_fpga_lte_fec_configure()`` is shown 199below: 200 201.. code-block:: c 202 203 struct rte_fpga_lte_fec_conf conf; 204 unsigned int i; 205 206 memset(&conf, 0, sizeof(struct rte_fpga_lte_fec_conf)); 207 conf.pf_mode_en = 1; 208 209 for (i = 0; i < FPGA_LTE_FEC_NUM_VFS; ++i) { 210 conf.vf_ul_queues_number[i] = 4; 211 conf.vf_dl_queues_number[i] = 4; 212 } 213 conf.ul_bandwidth = 12; 214 conf.dl_bandwidth = 5; 215 conf.dl_load_balance = 64; 216 conf.ul_load_balance = 64; 217 218 /* setup FPGA PF */ 219 ret = rte_fpga_lte_fec_configure(info->dev_name, &conf); 220 TEST_ASSERT_SUCCESS(ret, 221 "Failed to configure 4G FPGA PF for bbdev %s", 222 info->dev_name); 223 224 225Test Application 226---------------- 227 228BBDEV provides a test application, ``test-bbdev.py`` and range of test data for testing 229the functionality of FPGA LTE FEC turbo encode and turbo decode, depending on the device's 230capabilities. The test application is located under app->test-bbdev folder and has the 231following options: 232 233.. code-block:: console 234 235 "-p", "--testapp-path": specifies path to the bbdev test app. 236 "-e", "--eal-params" : EAL arguments which are passed to the test app. 237 "-t", "--timeout" : Timeout in seconds (default=300). 238 "-c", "--test-cases" : Defines test cases to run. Run all if not specified. 239 "-v", "--test-vector" : Test vector path (default=dpdk_path+/app/test-bbdev/test_vectors/bbdev_null.data). 240 "-n", "--num-ops" : Number of operations to process on device (default=32). 241 "-b", "--burst-size" : Operations enqueue/dequeue burst size (default=32). 242 "-l", "--num-lcores" : Number of lcores to run (default=16). 243 "-i", "--init-device" : Initialise PF device with default values. 244 245 246To execute the test application tool using simple turbo decode or turbo encode data, 247type one of the following: 248 249.. code-block:: console 250 251 ./test-bbdev.py -c validation -n 64 -b 8 -v ./turbo_dec_default.data 252 ./test-bbdev.py -c validation -n 64 -b 8 -v ./turbo_enc_default.data 253 254 255The test application ``test-bbdev.py``, supports the ability to configure the PF device with 256a default set of values, if the "-i" or "- -init-device" option is included. The default values 257are defined in test_bbdev_perf.c as: 258 259- VF_UL_QUEUE_VALUE 4 260- VF_DL_QUEUE_VALUE 4 261- UL_BANDWIDTH 3 262- DL_BANDWIDTH 3 263- UL_LOAD_BALANCE 128 264- DL_LOAD_BALANCE 128 265- FLR_TIMEOUT 610 266 267 268Test Vectors 269~~~~~~~~~~~~ 270 271In addition to the simple turbo decoder and turbo encoder tests, bbdev also provides 272a range of additional tests under the test_vectors folder, which may be useful. The results 273of these tests will depend on the FPGA LTE FEC capabilities: 274 275* turbo decoder tests: 276 - ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 277 - ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data`` 278 - ``turbo_dec_c1_k6144_r0_e34560_negllr.data`` 279 - ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data`` 280 - ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data`` 281 - ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data`` 282 283 284* turbo encoder tests: 285 - ``turbo_enc_c1_k40_r0_e1190_rm.data`` 286 - ``turbo_enc_c1_k40_r0_e1194_rm.data`` 287 - ``turbo_enc_c1_k40_r0_e1196_rm.data`` 288 - ``turbo_enc_c1_k40_r0_e272_rm.data`` 289 - ``turbo_enc_c1_k6144_r0_e18444.data`` 290 - ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 291 - ``turbo_enc_c2_k5952_r0_e17868_crc24b.data`` 292 - ``turbo_enc_c3_k4800_r2_e14412_crc24b.data`` 293 - ``turbo_enc_c4_k4800_r2_e14412_crc24b.data`` 294 295 296Alternate Baseband Device configuration tool 297~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 298 299On top of the embedded configuration feature supported in test-bbdev using "- -init-device" 300option, there is also a tool available to perform that device configuration using a companion 301application. 302The ``pf_bb_config`` application notably enables then to run bbdev-test from the VF 303and not only limited to the PF as captured above. 304 305See for more details: https://github.com/intel/pf-bb-config 306 307Specifically for the BBDEV FPGA LTE FEC PMD, the command below can be used: 308 309.. code-block:: console 310 311 ./pf_bb_config FPGA_LTE -c fpga_lte/fpga_lte_config_vf.cfg 312 ./test-bbdev.py -e="-c 0xff0 -a${VF_PCI_ADDR}" -c validation -n 64 -b 32 -l 1 -v ./turbo_dec_default.data 313