1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation 3 4dpdk-test-bbdev Application 5=========================== 6 7The ``dpdk-test-bbdev`` tool is a Data Plane Development Kit (DPDK) utility that 8allows measuring performance parameters of PMDs available in the bbdev framework. 9Available tests available for execution are: latency, throughput, validation and 10sanity tests. Execution of tests can be customized using various parameters 11passed to a python running script. 12 13Compiling the Application 14------------------------- 15 16**Step 1: PMD setting** 17 18The ``dpdk-test-bbdev`` tool depends on crypto device drivers PMD which 19are disabled by default in the build configuration file ``common_base``. 20The bbdevice drivers PMD which should be tested can be enabled by setting 21 22 ``CONFIG_RTE_LIBRTE_PMD_<name>=y`` 23 24Setting example for (*baseband_turbo_sw*) PMD 25 26 ``CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW=y`` 27 28**Step 2: Build the application** 29 30Execute the ``dpdk-setup.sh`` script to build the DPDK library together with the 31``dpdk-test-bbdev`` application. 32 33Initially, the user must select a DPDK target to choose the correct target type 34and compiler options to use when building the libraries. 35The user must have all libraries, modules, updates and compilers installed 36in the system prior to this, as described in the earlier chapters in this 37Getting Started Guide. 38 39Running the Application 40----------------------- 41 42The tool application has a number of command line options: 43 44.. code-block:: console 45 46 python test-bbdev.py [-h] [-p TESTAPP_PATH] [-e EAL_PARAMS] [-t TIMEOUT] 47 [-c TEST_CASE [TEST_CASE ...]] 48 [-v TEST_VECTOR [TEST_VECTOR...]] [-n NUM_OPS] 49 [-b BURST_SIZE [BURST_SIZE ...]] [-l NUM_LCORES] 50 51command-line Options 52~~~~~~~~~~~~~~~~~~~~ 53 54The following are the command-line options: 55 56``-h, --help`` 57 Shows help message and exit. 58 59``-p TESTAPP_PATH, --testapp_path TESTAPP_PATH`` 60 Indicates the path to the bbdev test app. If not specified path is set based 61 on *$RTE_SDK* environment variable concatenated with "*/build/app/testbbdev*". 62 63``-e EAL_PARAMS, --eal_params EAL_PARAMS`` 64 Specifies EAL arguments which are passed to the test app. For more details, 65 refer to DPDK documentation at 66 http://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html. 67 68``-t TIMEOUT, --timeout TIMEOUT`` 69 Specifies timeout in seconds. If not specified timeout is set to 300 seconds. 70 71``-c TEST_CASE [TEST_CASE ...], --test_cases TEST_CASE [TEST_CASE ...]`` 72 Defines test cases to run. If not specified all available tests are run. 73 74 **Example usage:** 75 76 ``./test-bbdev.py -c validation`` 77 Runs validation test suite 78 79 ``./test-bbdev.py -c latency throughput`` 80 Runs latency and throughput test suites 81 82``-v TEST_VECTOR [TEST_VECTOR ...], --test_vector TEST_VECTOR [TEST_VECTOR ...]`` 83 Specifies paths to the test vector files. If not specified path is set based 84 on *$RTE_SDK* environment variable concatenated with 85 "*/app/test-bbdev/test_vectors/bbdev_null.data*" and indicates default 86 data file. 87 88 **Example usage:** 89 90 ``./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_dec_test1.data`` 91 Fills vector based on turbo_dec_test1.data file and runs all tests 92 93 ``./test-bbdev.py -v turbo_dec_test1.data turbo_enc_test2.data`` 94 The bbdev test app is executed twice. First time vector is filled based on 95 *turbo_dec_test1.data* file and second time based on 96 *turb_enc_test2.data* file. For both executions all tests are run. 97 98``-n NUM_OPS, --num_ops NUM_OPS`` 99 Specifies number of operations to process on device. If not specified num_ops 100 is set to 32 operations. 101 102``-l NUM_LCORES, --num_lcores NUM_LCORES`` 103 Specifies number of lcores to run. If not specified num_lcores is set 104 according to value from RTE configuration (EAL coremask) 105 106``-b BURST_SIZE [BURST_SIZE ...], --burst-size BURST_SIZE [BURST_SIZE ...]`` 107 Specifies operations enqueue/dequeue burst size. If not specified burst_size is 108 set to 32. Maximum is 512. 109 110Test Cases 111~~~~~~~~~~ 112 113There are 6 main test cases that can be executed using testbbdev tool: 114 115* Sanity checks [-c unittest] 116 - Performs sanity checks on BBDEV interface, validating basic functionality 117 118* Validation tests [-c validation] 119 - Performs full operation of enqueue and dequeue 120 - Compares the dequeued data buffer with a expected values in the test 121 vector (TV) being used 122 - Fails if any dequeued value does not match the data in the TV 123 124* Offload Cost measurement [-c offload] 125 - Measures the CPU cycles consumed from the receipt of a user enqueue 126 until it is put on the device queue 127 - The test measures 4 metrics 128 (a) *SW Enq Offload Cost*: Software only enqueue offload cost, the cycle 129 counts and time (us) from the point the enqueue API is called until 130 the point the operation is put on the accelerator queue. 131 (b) *Acc Enq Offload Cost*: The cycle count and time (us) from the 132 point the operation is put on the accelerator queue until the return 133 from enqueue. 134 (c) *SW Deq Offload Cost*: Software dequeue cost, the cycle counts and 135 time (us) consumed to dequeue one operation. 136 (d) *Empty Queue Enq Offload Cost*: The cycle count and time (us) 137 consumed to dequeue from an empty queue. 138 139* Latency measurement [-c latency] 140 - Measures the time consumed from the first enqueue until the first 141 appearance of a dequeued result 142 - This measurement represents the full latency of a bbdev operation 143 (encode or decode) to execute 144 145* Poll-mode Throughput measurement [-c throughput] 146 - Performs full operation of enqueue and dequeue 147 - Executes in poll mode 148 - Measures the achieved throughput on a subset or all available CPU cores 149 - Dequeued data is not validated against expected values stored in TV 150 - Results are printed in million operations per second and million bits 151 per second 152 153* Interrupt-mode Throughput [-c interrupt] 154 - Similar to Throughput test case, but using interrupts. No polling. 155 156 157Parameter Globbing 158~~~~~~~~~~~~~~~~~~ 159 160Thanks to the globbing functionality in python test-bbdev.py script allows to 161run tests with different set of vector files without giving all of them explicitly. 162 163**Example usage:** 164 165.. code-block:: console 166 167 ./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_<enc/dec>_c<c>_k<k>_r<r>_e<e>_<extra-info>.data 168 169It runs all tests with following vectors: 170 171- ``bbdev_null.data`` 172 173- ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data`` 174 175- ``turbo_enc_c1_k40_r0_e1196_rm.data`` 176 177- ``turbo_enc_c2_k5952_r0_e17868_crc24b.data`` 178 179- ``turbo_dec_c1_k40_r0_e17280_sbd_negllr.data`` 180 181- ``turbo_dec_c1_k6144_r0_e34560_sbd_posllr.data`` 182 183- ``turbo_enc_c1_k40_r0_e272_rm.data`` 184 185- ``turbo_enc_c3_k4800_r2_e14412_crc24b.data`` 186 187- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 188 189- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data`` 190 191- ``turbo_enc_c1_k6144_r0_e120_rm_rvidx.data`` 192 193- ``turbo_enc_c4_k4800_r2_e14412_crc24b.data`` 194 195- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data`` 196 197- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data`` 198 199- ``turbo_enc_c1_k6144_r0_e18444.data`` 200 201- ``turbo_dec_c1_k6144_r0_e34560_negllr.data`` 202 203- ``turbo_enc_c1_k40_r0_e1190_rm.data`` 204 205- ``turbo_enc_c1_k6144_r0_e18448_crc24a.data`` 206 207- ``turbo_dec_c1_k6144_r0_e34560_posllr.data`` 208 209- ``turbo_enc_c1_k40_r0_e1194_rm.data`` 210 211- ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 212 213.. code-block:: console 214 215 ./test-bbdev.py -v app/test-bbdev/turbo_*_default.data 216 217It runs all tests with "default" vectors. 218 219* ``turbo_dec_default.data`` is a soft link to 220 ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 221 222* ``turbo_enc_default.data`` is a soft link to 223 ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 224 225 226Running Tests 227------------- 228 229Shortened tree of isg_cid-wireless_dpdk_ae with dpdk compiled for 230x86_64-native-linux-icc target: 231 232:: 233 234 |-- app 235 |-- test-bbdev 236 |-- test_vectors 237 |-- bbdev_null.data 238 |-- turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data 239 |-- turbo_enc_c1_k40_r0_e1196_rm.data 240 |-- turbo_enc_c2_k5952_r0_e17868_crc24b.data 241 |-- turbo_dec_c1_k40_r0_e17280_sbd_negllr.data 242 |-- turbo_dec_c1_k6144_r0_e34560_sbd_posllr.data 243 |-- turbo_enc_c1_k40_r0_e272_rm.data 244 |-- turbo_enc_c3_k4800_r2_e14412_crc24b.data 245 |-- turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data 246 |-- turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data 247 |-- turbo_enc_c1_k6144_r0_e120_rm_rvidx.data 248 |-- turbo_enc_c4_k4800_r2_e14412_crc24b.data 249 |-- turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data 250 |-- turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data 251 |-- turbo_enc_c1_k6144_r0_e18444.data 252 |-- turbo_dec_c1_k6144_r0_e34560_negllr.data 253 |-- turbo_enc_c1_k40_r0_e1190_rm.data 254 |-- turbo_enc_c1_k6144_r0_e18448_crc24a.data 255 |-- turbo_dec_c1_k6144_r0_e34560_posllr.data 256 |-- turbo_enc_c1_k40_r0_e1194_rm.data 257 |-- turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data 258 259 |-- x86_64-native-linux-icc 260 |-- app 261 |-- testbbdev 262 263All bbdev devices 264~~~~~~~~~~~~~~~~~ 265 266.. code-block:: console 267 268 ./test-bbdev.py -p ../../x86_64-native-linux-icc/app/testbbdev 269 -v turbo_dec_default.data 270 271It runs all available tests using the test vector filled based on 272*turbo_dec_default.data* file. 273By default number of operations to process on device is set to 32, timeout is 274set to 300s and operations enqueue/dequeue burst size is set to 32. 275Moreover a bbdev (*baseband_null*) device will be created. 276 277baseband turbo_sw device 278~~~~~~~~~~~~~~~~~~~~~~~~ 279 280.. code-block:: console 281 282 ./test-bbdev.py -p ../../x86_64-native-linux-icc/app/testbbdev 283 -e="--vdev=baseband_turbo_sw" -t 120 -c validation 284 -v ./test_vectors/turbo_* -n 64 -b 8 32 285 286It runs **validation** test for each vector file that matches the given pattern. 287Number of operations to process on device is set to 64 and operations timeout is 288set to 120s and enqueue/dequeue burst size is set to 8 and to 32. 289Moreover a bbdev (*baseband_turbo_sw*) device will be created. 290 291 292bbdev null device 293~~~~~~~~~~~~~~~~~ 294 295Executing bbdev null device with *bbdev_null.data* helps in measuring the 296overhead introduced by the bbdev framework. 297 298.. code-block:: console 299 300 ./test-bbdev.py -e="--vdev=baseband_null0" 301 -v ./test_vectors/bbdev_null.data 302 303**Note:** 304 305baseband_null device does not have to be defined explicitly as it is created by default. 306 307 308 309Test Vector files 310----------------- 311 312Test Vector files contain the data which is used to set turbo decoder/encoder 313parameters and buffers for validation purpose. New test vector files should be 314stored in ``app/test-bbdev/test_vectors/`` directory. Detailed description of 315the syntax of the test vector files is in the following section. 316 317 318Basic principles for test vector files 319~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 320Line started with ``#`` is treated as a comment and is ignored. 321 322If variable is a chain of values, values should be separated by a comma. If 323assignment is split into several lines, each line (except the last one) has to 324be ended with a comma. 325There is no comma after last value in last line. Correct assignment should 326look like the following: 327 328.. parsed-literal:: 329 330 variable = 331 value, value, value, value, 332 value, value 333 334In case where variable is a single value correct assignment looks like the 335following: 336 337.. parsed-literal:: 338 339 variable = 340 value 341 342Length of chain variable is calculated by parser. Can not be defined 343explicitly. 344 345Variable op_type has to be defined as a first variable in file. It specifies 346what type of operations will be executed. For decoder op_type has to be set to 347``RTE_BBDEV_OP_TURBO_DEC`` and for encoder to ``RTE_BBDEV_OP_TURBO_ENC``. 348 349Full details of the meaning and valid values for the below fields are 350documented in *rte_bbdev_op.h* 351 352 353Turbo decoder test vectors template 354~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 355 356For turbo decoder it has to be always set to ``RTE_BBDEV_OP_TURBO_DEC`` 357 358.. parsed-literal:: 359 360 op_type = 361 RTE_BBDEV_OP_TURBO_DEC 362 363Chain of uint32_t values. Note that it is possible to define more than one 364input/output entries which will result in chaining two or more data structures 365for *segmented Transport Blocks* 366 367.. parsed-literal:: 368 369 input0 = 370 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100, 371 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00 372 373Chain of uint32_t values 374 375.. parsed-literal:: 376 377 input1 = 378 0x7f7f0000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 379 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 380 381Chain of uint32_t values 382 383.. parsed-literal:: 384 385 input2 = 386 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 387 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 388 389Chain of uint32_t values 390 391.. parsed-literal:: 392 393 hard_output0 = 394 0xa7d6732e 395 396Chain of uint32_t values 397 398.. parsed-literal:: 399 400 hard_output1 = 401 0xa61 402 403Chain of uint32_t values 404 405.. parsed-literal:: 406 407 soft_output0 = 408 0x817f817f, 0x7f817f7f, 0x81818181, 0x817f7f81, 0x7f818181, 0x8181817f, 0x817f817f, 0x8181817f 409 410Chain of uint32_t values 411 412.. parsed-literal:: 413 414 soft_output1 = 415 0x817f7f81, 0x7f7f7f81, 0x7f7f8181 416 417uint32_t value 418 419.. parsed-literal:: 420 421 e = 422 44 423 424uint16_t value 425 426.. parsed-literal:: 427 428 k = 429 40 430 431uint8_t value 432 433.. parsed-literal:: 434 435 rv_index = 436 0 437 438uint8_t value 439 440.. parsed-literal:: 441 442 iter_max = 443 8 444 445uint8_t value 446 447.. parsed-literal:: 448 449 iter_min = 450 4 451 452uint8_t value 453 454.. parsed-literal:: 455 456 expected_iter_count = 457 8 458 459uint8_t value 460 461.. parsed-literal:: 462 463 ext_scale = 464 15 465 466uint8_t value 467 468.. parsed-literal:: 469 470 num_maps = 471 0 472 473Chain of flags for turbo decoder operation. Following flags can be used: 474 475- ``RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE`` 476 477- ``RTE_BBDEV_TURBO_CRC_TYPE_24B`` 478 479- ``RTE_BBDEV_TURBO_EQUALIZER`` 480 481- ``RTE_BBDEV_TURBO_SOFT_OUT_SATURATE`` 482 483- ``RTE_BBDEV_TURBO_HALF_ITERATION_EVEN`` 484 485- ``RTE_BBDEV_TURBO_CONTINUE_CRC_MATCH`` 486 487- ``RTE_BBDEV_TURBO_SOFT_OUTPUT`` 488 489- ``RTE_BBDEV_TURBO_EARLY_TERMINATION`` 490 491- ``RTE_BBDEV_TURBO_DEC_INTERRUPTS`` 492 493- ``RTE_BBDEV_TURBO_POS_LLR_1_BIT_IN`` 494 495- ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_IN`` 496 497- ``RTE_BBDEV_TURBO_POS_LLR_1_BIT_SOFT_OUT`` 498 499- ``RTE_BBDEV_TURBO_NEG_LLR_1_BIT_SOFT_OUT`` 500 501- ``RTE_BBDEV_TURBO_MAP_DEC`` 502 503Example: 504 505 .. parsed-literal:: 506 507 op_flags = 508 RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE, RTE_BBDEV_TURBO_EQUALIZER, 509 RTE_BBDEV_TURBO_SOFT_OUTPUT 510 511Chain of operation statuses that are expected after operation is performed. 512Following statuses can be used: 513 514- ``DMA`` 515 516- ``FCW`` 517 518- ``CRC`` 519 520- ``OK`` 521 522``OK`` means no errors are expected. Cannot be used with other values. 523 524.. parsed-literal:: 525 526 expected_status = 527 FCW, CRC 528 529 530Turbo encoder test vectors template 531~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 532 533For turbo encoder it has to be always set to ``RTE_BBDEV_OP_TURBO_ENC`` 534 535.. parsed-literal:: 536 537 op_type = 538 RTE_BBDEV_OP_TURBO_ENC 539 540Chain of uint32_t values 541 542.. parsed-literal:: 543 544 input0 = 545 0x11d2bcac, 0x4d 546 547Chain of uint32_t values 548 549.. parsed-literal:: 550 551 output0 = 552 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 553 0xa25b7f47, 0x2aa224ce, 0x79f2 554 555uint32_t value 556 557.. parsed-literal:: 558 559 e = 560 272 561 562uint16_t value 563 564.. parsed-literal:: 565 566 k = 567 40 568 569uint16_t value 570 571.. parsed-literal:: 572 573 ncb = 574 192 575 576uint8_t value 577 578.. parsed-literal:: 579 580 rv_index = 581 0 582 583Chain of flags for turbo encoder operation. Following flags can be used: 584 585- ``RTE_BBDEV_TURBO_RV_INDEX_BYPASS`` 586 587- ``RTE_BBDEV_TURBO_RATE_MATCH`` 588 589- ``RTE_BBDEV_TURBO_CRC_24B_ATTACH`` 590 591- ``RTE_BBDEV_TURBO_CRC_24A_ATTACH`` 592 593- ``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` 594 595``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` is used to indicate the parser to 596force the input data to be memory split and formed as a segmented mbuf. 597 598 599.. parsed-literal:: 600 601 op_flags = 602 RTE_BBDEV_TURBO_RATE_MATCH 603 604Chain of operation statuses that are expected after operation is performed. 605Following statuses can be used: 606 607- ``DMA`` 608 609- ``FCW`` 610 611- ``OK`` 612 613``OK`` means no errors are expected. Cannot be used with other values. 614 615.. parsed-literal:: 616 617 expected_status = 618 OK 619