1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation 3 4dpdk-test-bbdev Application 5=========================== 6 7The ``dpdk-test-bbdev`` tool is a Data Plane Development Kit (DPDK) utility that 8allows measuring performance parameters of PMDs available in the bbdev framework. 9Tests available for execution are: latency, throughput, validation, 10bler and sanity tests. Execution of tests can be customized using various 11parameters passed to a python running script. 12 13 14Running the Application 15----------------------- 16 17The tool application has a number of command line options: 18 19.. code-block:: console 20 21 test-bbdev.py [-h] [-p TESTAPP_PATH] [-e EAL_PARAMS] [-t TIMEOUT] 22 [-c TEST_CASE [TEST_CASE ...]] 23 [-v TEST_VECTOR [TEST_VECTOR...]] [-n NUM_OPS] 24 [-b BURST_SIZE [BURST_SIZE ...]] [-l NUM_LCORES] 25 [-t MAX_ITERS [MAX_ITERS ...]] 26 [-s SNR [SNR ...]] 27 28command-line Options 29~~~~~~~~~~~~~~~~~~~~ 30 31The following are the command-line options: 32 33``-h, --help`` 34 Shows help message and exit. 35 36``-p TESTAPP_PATH, --testapp_path TESTAPP_PATH`` 37 Indicates the path to the bbdev test app. If not specified path is set based 38 on "../.." concatenated with "*/build/app/dpdk-test-bbdev*". 39 40``-e EAL_PARAMS, --eal_params EAL_PARAMS`` 41 Specifies EAL arguments which are passed to the test app. For more details, 42 refer to DPDK documentation at :doc:`../linux_gsg/linux_eal_parameters`. 43 44``-t TIMEOUT, --timeout TIMEOUT`` 45 Specifies timeout in seconds. If not specified timeout is set to 300 seconds. 46 47``-c TEST_CASE [TEST_CASE ...], --test_cases TEST_CASE [TEST_CASE ...]`` 48 Defines test cases to run. If not specified all available tests are run. 49 50 **Example usage:** 51 52 ``./test-bbdev.py -c validation`` 53 Runs validation test suite 54 55 ``./test-bbdev.py -c latency throughput`` 56 Runs latency and throughput test suites 57 58``-v TEST_VECTOR [TEST_VECTOR ...], --test_vector TEST_VECTOR [TEST_VECTOR ...]`` 59 Specifies paths to the test vector files. If not specified path is set based 60 on "../.." concatenated with "*/app/test-bbdev/test_vectors/bbdev_null.data*" 61 and indicates default data file. 62 63 **Example usage:** 64 65 ``./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_dec_test1.data`` 66 Fills vector based on turbo_dec_test1.data file and runs all tests 67 68 ``./test-bbdev.py -v turbo_dec_test1.data turbo_enc_test2.data`` 69 The bbdev test app is executed twice. First time vector is filled based on 70 *turbo_dec_test1.data* file and second time based on 71 *turb_enc_test2.data* file. For both executions all tests are run. 72 73``-n NUM_OPS, --num_ops NUM_OPS`` 74 Specifies number of operations to process on device. If not specified num_ops 75 is set to 32 operations. 76 77``-l NUM_LCORES, --num_lcores NUM_LCORES`` 78 Specifies number of lcores to run. If not specified num_lcores is set 79 according to value from RTE configuration (EAL coremask) 80 81``-b BURST_SIZE [BURST_SIZE ...], --burst-size BURST_SIZE [BURST_SIZE ...]`` 82 Specifies operations enqueue/dequeue burst size. If not specified burst_size is 83 set to 32. Maximum is 512. 84 85``-t MAX_ITERS [MAX_ITERS ...], --iter_max MAX_ITERS [MAX_ITERS ...]`` 86 Specifies LDPC decoder operations maximum number of iterations for throughput 87 and bler tests. If not specified iter_max is set to 6. 88 89``-s SNR [SNR ...], --snr SNR [SNR ...]`` 90 Specifies for LDPC decoder operations the SNR in dB used when generating LLRs 91 for bler tests. If not specified snr is set to 0 dB. 92 93Test Cases 94~~~~~~~~~~ 95 96There are 7 main test cases that can be executed using testbbdev tool: 97 98* Sanity checks [-c unittest] 99 - Performs sanity checks on BBDEV interface, validating basic functionality 100 101* Validation tests [-c validation] 102 - Performs full operation of enqueue and dequeue 103 - Compares the dequeued data buffer with a expected values in the test 104 vector (TV) being used 105 - Fails if any dequeued value does not match the data in the TV 106 107* Offload Cost measurement [-c offload] 108 - Measures the CPU cycles consumed from the receipt of a user enqueue 109 until it is put on the device queue 110 - The test measures 4 metrics 111 (a) *SW Enq Offload Cost*: Software only enqueue offload cost, the cycle 112 counts and time (us) from the point the enqueue API is called until 113 the point the operation is put on the accelerator queue. 114 (b) *Acc Enq Offload Cost*: The cycle count and time (us) from the 115 point the operation is put on the accelerator queue until the return 116 from enqueue. 117 (c) *SW Deq Offload Cost*: Software dequeue cost, the cycle counts and 118 time (us) consumed to dequeue one operation. 119 (d) *Empty Queue Enq Offload Cost*: The cycle count and time (us) 120 consumed to dequeue from an empty queue. 121 122* Latency measurement [-c latency] 123 - Measures the time consumed from the first enqueue until the first 124 appearance of a dequeued result 125 - This measurement represents the full latency of a bbdev operation 126 (encode or decode) to execute 127 128* Poll-mode Throughput measurement [-c throughput] 129 - Performs full operation of enqueue and dequeue 130 - Executes in poll mode 131 - Measures the achieved throughput on a subset or all available CPU cores 132 - Dequeued data is not validated against expected values stored in TV 133 - Results are printed in million operations per second and million bits 134 per second 135 136* BLER measurement [-c bler] 137 - Performs full operation of enqueue and dequeue 138 - Measures the achieved throughput on a subset or all available CPU cores 139 - Computed BLER (Block Error Rate, ratio of blocks not decoded at a given 140 SNR) in % based on the total number of operations. 141 142* Interrupt-mode Throughput [-c interrupt] 143 - Similar to Throughput test case, but using interrupts. No polling. 144 145 146Parameter Globbing 147~~~~~~~~~~~~~~~~~~ 148 149Thanks to the globbing functionality in python test-bbdev.py script allows to 150run tests with different set of vector files without giving all of them explicitly. 151 152**Example usage for 4G:** 153 154.. code-block:: console 155 156 ./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_<enc/dec>_c<c>_k<k>_r<r>_e<e>_<extra-info>.data 157 158It runs all tests with following vectors: 159 160- ``bbdev_null.data`` 161 162- ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data`` 163 164- ``turbo_enc_c1_k40_r0_e1196_rm.data`` 165 166- ``turbo_enc_c2_k5952_r0_e17868_crc24b.data`` 167 168- ``turbo_dec_c1_k40_r0_e17280_sbd_negllr.data`` 169 170- ``turbo_dec_c1_k6144_r0_e34560_sbd_posllr.data`` 171 172- ``turbo_enc_c1_k40_r0_e272_rm.data`` 173 174- ``turbo_enc_c3_k4800_r2_e14412_crc24b.data`` 175 176- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 177 178- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data`` 179 180- ``turbo_enc_c1_k6144_r0_e120_rm_rvidx.data`` 181 182- ``turbo_enc_c4_k4800_r2_e14412_crc24b.data`` 183 184- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data`` 185 186- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data`` 187 188- ``turbo_enc_c1_k6144_r0_e18444.data`` 189 190- ``turbo_dec_c1_k6144_r0_e34560_negllr.data`` 191 192- ``turbo_enc_c1_k40_r0_e1190_rm.data`` 193 194- ``turbo_enc_c1_k6144_r0_e18448_crc24a.data`` 195 196- ``turbo_dec_c1_k6144_r0_e34560_posllr.data`` 197 198- ``turbo_enc_c1_k40_r0_e1194_rm.data`` 199 200- ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 201 202.. code-block:: console 203 204 ./test-bbdev.py -v app/test-bbdev/turbo_*_default.data 205 206It runs all tests with "default" vectors. 207 208* ``turbo_dec_default.data`` is a soft link to 209 ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 210 211* ``turbo_enc_default.data`` is a soft link to 212 ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 213 214* ``ldpc_dec_default.data`` is a soft link to 215 ``ldpc_dec_v6563.data`` 216 217* ``ldpc_enc_default.data`` is a soft link to 218 ``ldpc_enc_c1_k8148_r0_e9372_rm.data`` 219 220Running Tests 221------------- 222 223All default reference test-vectors are stored in the test_vector 224directory below. 225The prefix trivially defines which type of operation is included : 226turbo_enc, turbo_dec, ldpc_enc, ldpc_dec. 227The details of the configuration are captured in the file but some 228vector name refer more explicitly processing specificity such as 229'HARQ' when HARQ retransmission is used, 'loopback' when the data 230is purely read/written for external DDR, lbrm when limited buffer 231rate matching is expected, or crc_fail when a CRC failure is expected. 232They are chosen to have a good coverage across sizes and processing 233parameters while still keeping their number limited as part of sanity 234regression. 235 236Shortened tree of isg_cid-wireless_dpdk_ae with dpdk compiled and output 237to the build directory: 238 239:: 240 241 |-- app 242 |-- test-bbdev 243 |-- test_vectors 244 245 |-- build 246 |-- app 247 |-- dpdk-test-bbdev 248 249All bbdev devices 250~~~~~~~~~~~~~~~~~ 251 252.. code-block:: console 253 254 ./test-bbdev.py -p ../../build/app/dpdk-test-bbdev 255 -v turbo_dec_default.data 256 257It runs all available tests using the test vector filled based on 258*turbo_dec_default.data* file. 259By default number of operations to process on device is set to 32, timeout is 260set to 300s and operations enqueue/dequeue burst size is set to 32. 261Moreover a bbdev (*baseband_null*) device will be created. 262 263baseband turbo_sw device 264~~~~~~~~~~~~~~~~~~~~~~~~ 265 266.. code-block:: console 267 268 ./test-bbdev.py -p ../../build/app/dpdk-test-bbdev 269 -e="--vdev=baseband_turbo_sw" -t 120 -c validation 270 -v ./test_vectors/* -n 64 -b 8 32 271 272It runs **validation** test for each vector file that matches the given pattern. 273Number of operations to process on device is set to 64 and operations timeout is 274set to 120s and enqueue/dequeue burst size is set to 8 and to 32. 275Moreover a bbdev (*baseband_turbo_sw*) device will be created. 276 277 278bbdev null device 279~~~~~~~~~~~~~~~~~ 280 281Executing bbdev null device with *bbdev_null.data* helps in measuring the 282overhead introduced by the bbdev framework. 283 284.. code-block:: console 285 286 ./test-bbdev.py -e="--vdev=baseband_null0" 287 -v ./test_vectors/bbdev_null.data 288 289**Note:** 290 291baseband_null device does not have to be defined explicitly as it is created by default. 292 293 294 295Test Vector files 296----------------- 297 298Test Vector files contain the data which is used to set turbo decoder/encoder 299parameters and buffers for validation purpose. New test vector files should be 300stored in ``app/test-bbdev/test_vectors/`` directory. Detailed description of 301the syntax of the test vector files is in the following section. 302 303 304Basic principles for test vector files 305~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 306Line started with ``#`` is treated as a comment and is ignored. 307 308If variable is a chain of values, values should be separated by a comma. If 309assignment is split into several lines, each line (except the last one) has to 310be ended with a comma. 311There is no comma after last value in last line. Correct assignment should 312look like the following: 313 314.. parsed-literal:: 315 316 variable = 317 value, value, value, value, 318 value, value 319 320In case where variable is a single value correct assignment looks like the 321following: 322 323.. parsed-literal:: 324 325 variable = 326 value 327 328Length of chain variable is calculated by parser. Can not be defined 329explicitly. 330 331Variable op_type has to be defined as a first variable in file. It specifies 332what type of operations will be executed. For 4G decoder op_type has to be set to 333``RTE_BBDEV_OP_TURBO_DEC`` and for 4G encoder to ``RTE_BBDEV_OP_TURBO_ENC``. 334 335Full details of the meaning and valid values for the below fields are 336documented in *rte_bbdev_op.h* 337 338 339Turbo decoder test vectors template 340~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 341 342For turbo decoder it has to be always set to ``RTE_BBDEV_OP_TURBO_DEC`` 343 344.. parsed-literal:: 345 346 op_type = 347 RTE_BBDEV_OP_TURBO_DEC 348 349Chain of uint32_t values. Note that it is possible to define more than one 350input/output entries which will result in chaining two or more data structures 351for *segmented Transport Blocks* 352 353.. parsed-literal:: 354 355 input0 = 356 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100, 357 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00 358 359Chain of uint32_t values 360 361.. parsed-literal:: 362 363 input1 = 364 0x7f7f0000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 365 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 366 367Chain of uint32_t values 368 369.. parsed-literal:: 370 371 input2 = 372 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 373 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 374 375Chain of uint32_t values 376 377.. parsed-literal:: 378 379 hard_output0 = 380 0xa7d6732e 381 382Chain of uint32_t values 383 384.. parsed-literal:: 385 386 hard_output1 = 387 0xa61 388 389Chain of uint32_t values 390 391.. parsed-literal:: 392 393 soft_output0 = 394 0x817f817f, 0x7f817f7f, 0x81818181, 0x817f7f81, 0x7f818181, 0x8181817f, 0x817f817f, 0x8181817f 395 396Chain of uint32_t values 397 398.. parsed-literal:: 399 400 soft_output1 = 401 0x817f7f81, 0x7f7f7f81, 0x7f7f8181 402 403uint32_t value 404 405.. parsed-literal:: 406 407 e = 408 44 409 410uint16_t value 411 412.. parsed-literal:: 413 414 k = 415 40 416 417uint8_t value 418 419.. parsed-literal:: 420 421 rv_index = 422 0 423 424uint8_t value 425 426.. parsed-literal:: 427 428 iter_max = 429 8 430 431uint8_t value 432 433.. parsed-literal:: 434 435 iter_min = 436 4 437 438uint8_t value 439 440.. parsed-literal:: 441 442 expected_iter_count = 443 8 444 445uint8_t value 446 447.. parsed-literal:: 448 449 ext_scale = 450 15 451 452uint8_t value 453 454.. parsed-literal:: 455 456 num_maps = 457 0 458 459Chain of flags for LDPC decoder operation based on the rte_bbdev_op_td_flag_bitmasks: 460 461Example: 462 463 .. parsed-literal:: 464 465 op_flags = 466 RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE, RTE_BBDEV_TURBO_EQUALIZER, 467 RTE_BBDEV_TURBO_SOFT_OUTPUT 468 469Chain of operation statuses that are expected after operation is performed. 470Following statuses can be used: 471 472- ``DMA`` 473 474- ``FCW`` 475 476- ``CRC`` 477 478- ``OK`` 479 480``OK`` means no errors are expected. Cannot be used with other values. 481 482.. parsed-literal:: 483 484 expected_status = 485 FCW, CRC 486 487 488Turbo encoder test vectors template 489~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 490 491For turbo encoder it has to be always set to ``RTE_BBDEV_OP_TURBO_ENC`` 492 493.. parsed-literal:: 494 495 op_type = 496 RTE_BBDEV_OP_TURBO_ENC 497 498Chain of uint32_t values 499 500.. parsed-literal:: 501 502 input0 = 503 0x11d2bcac, 0x4d 504 505Chain of uint32_t values 506 507.. parsed-literal:: 508 509 output0 = 510 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 511 0xa25b7f47, 0x2aa224ce, 0x79f2 512 513uint32_t value 514 515.. parsed-literal:: 516 517 e = 518 272 519 520uint16_t value 521 522.. parsed-literal:: 523 524 k = 525 40 526 527uint16_t value 528 529.. parsed-literal:: 530 531 ncb = 532 192 533 534uint8_t value 535 536.. parsed-literal:: 537 538 rv_index = 539 0 540 541Chain of flags for LDPC decoder operation based on the rte_bbdev_op_te_flag_bitmasks: 542 543``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` is used to indicate the parser to 544force the input data to be memory split and formed as a segmented mbuf. 545 546 547.. parsed-literal:: 548 549 op_flags = 550 RTE_BBDEV_TURBO_RATE_MATCH 551 552Chain of operation statuses that are expected after operation is performed. 553Following statuses can be used: 554 555- ``DMA`` 556 557- ``FCW`` 558 559- ``OK`` 560 561``OK`` means no errors are expected. Cannot be used with other values. 562 563.. parsed-literal:: 564 565 expected_status = 566 OK 567 568LDPC decoder test vectors template 569~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 570 571For LDPC decoder it has to be always set to ``RTE_BBDEV_OP_LDPC_DEC`` 572 573.. parsed-literal:: 574 575 op_type = 576 RTE_BBDEV_OP_LDPC_DEC 577 578Chain of uint32_t values. Note that it is possible to define more than one 579input/output entries which will result in chaining two or more data structures 580for *segmented Transport Blocks* 581 582.. parsed-literal:: 583 584 input0 = 585 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100, 586 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00 587 588.. parsed-literal:: 589 590 output0 = 591 0xa7d6732e 592 593uint8_t value 594 595.. parsed-literal:: 596 597 basegraph= 598 1 599 600uint16_t value 601 602.. parsed-literal:: 603 604 z_c= 605 224 606 607uint16_t value 608 609.. parsed-literal:: 610 611 n_cb= 612 14784 613 614uint8_t value 615 616.. parsed-literal:: 617 618 q_m= 619 1 620 621uint16_t value 622 623.. parsed-literal:: 624 625 n_filler= 626 40 627 628uint32_t value 629 630.. parsed-literal:: 631 632 e= 633 13072 634 635uint8_t value 636 637.. parsed-literal:: 638 639 rv_index= 640 2 641 642uint8_t value 643 644.. parsed-literal:: 645 code_block_mode= 646 1 647 648uint8_t value 649 650.. parsed-literal:: 651 652 iter_max= 653 20 654 655uint8_t value 656 657.. parsed-literal:: 658 659 expected_iter_count= 660 8 661 662 663Chain of flags for LDPC decoder operation based on the rte_bbdev_op_ldpcdec_flag_bitmasks: 664 665Example: 666 667 .. parsed-literal:: 668 669 op_flags = 670 RTE_BBDEV_LDPC_ITERATION_STOP_ENABLE, RTE_BBDEV_LDPC_HQ_COMBINE_OUT_ENABLE, 671 RTE_BBDEV_LDPC_HQ_COMBINE_IN_ENABLE, RTE_BBDEV_LDPC_HARQ_6BIT_COMPRESSION 672 673Chain of operation statuses that are expected after operation is performed. 674Following statuses can be used: 675 676- ``OK`` : No error reported. 677 678- ``SYN`` : LDPC syndrome parity check is failing. 679 680- ``CRC`` : CRC parity check is failing when CRC check operation is included. 681 682- ``SYNCRC`` : Both CRC and LDPC syndromes parity checks are failing. 683 684``OK`` means no errors are expected. Cannot be used with other values. 685 686.. parsed-literal:: 687 688 expected_status = 689 CRC 690 691 692LDPC encoder test vectors template 693~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 694 695For turbo encoder it has to be always set to ``RTE_BBDEV_OP_LDPC_ENC`` 696 697.. parsed-literal:: 698 699 op_type = 700 RTE_BBDEV_OP_LDPC_ENC 701 702Chain of uint32_t values 703 704.. parsed-literal:: 705 706 input0 = 707 0x11d2bcac, 0x4d 708 709Chain of uint32_t values 710 711.. parsed-literal:: 712 713 output0 = 714 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 715 0xa25b7f47, 0x2aa224ce, 0x79f2 716 717 718uint8_t value 719 720.. parsed-literal:: 721 722 basegraph= 723 1 724 725uint16_t value 726 727.. parsed-literal:: 728 729 z_c= 730 52 731 732uint16_t value 733 734.. parsed-literal:: 735 736 n_cb= 737 3432 738 739uint8_t value 740 741.. parsed-literal:: 742 743 q_m= 744 6 745 746uint16_t value 747 748.. parsed-literal:: 749 750 n_filler= 751 0 752 753uint32_t value 754 755.. parsed-literal:: 756 757 e = 758 1380 759 760uint8_t value 761 762.. parsed-literal:: 763 764 rv_index = 765 1 766 767uint8_t value 768 769.. parsed-literal:: 770 771 code_block_mode = 772 1 773 774 775Chain of flags for LDPC encoder operation based on the 776rte_bbdev_op_ldpcenc_flag_bitmasks: 777 778.. parsed-literal:: 779 780 op_flags = 781 RTE_BBDEV_LDPC_RATE_MATCH 782 783Chain of operation statuses that are expected after operation is performed. 784Following statuses can be used: 785 786- ``DMA`` 787 788- ``FCW`` 789 790- ``OK`` 791 792``OK`` means no errors are expected. Cannot be used with other values. 793 794.. parsed-literal:: 795 796 expected_status = 797 OK 798