1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation 3 4dpdk-test-bbdev Application 5=========================== 6 7The ``dpdk-test-bbdev`` tool is a Data Plane Development Kit (DPDK) utility that 8allows measuring performance parameters of PMDs available in the bbdev framework. 9Tests available for execution are: latency, throughput, validation, 10bler and sanity tests. Execution of tests can be customized using various 11parameters passed to a python running script. 12 13 14Running the Application 15----------------------- 16 17The tool application has a number of command line options: 18 19.. code-block:: console 20 21 test-bbdev.py [-h] [-p TESTAPP_PATH] [-e EAL_PARAMS] [-t TIMEOUT] 22 [-c TEST_CASE [TEST_CASE ...]] 23 [-v TEST_VECTOR [TEST_VECTOR...]] [-n NUM_OPS] 24 [-b BURST_SIZE [BURST_SIZE ...]] [-l NUM_LCORES] 25 [-t MAX_ITERS [MAX_ITERS ...]] 26 [-s SNR [SNR ...]] 27 28command-line Options 29~~~~~~~~~~~~~~~~~~~~ 30 31The following are the command-line options: 32 33``-h, --help`` 34 Shows help message and exit. 35 36``-p TESTAPP_PATH, --testapp_path TESTAPP_PATH`` 37 Indicates the path to the bbdev test app. If not specified path is set based 38 on "../.." concatenated with "*/build/app/dpdk-test-bbdev*". 39 40``-e EAL_PARAMS, --eal_params EAL_PARAMS`` 41 Specifies EAL arguments which are passed to the test app. For more details, 42 refer to DPDK documentation at :doc:`../linux_gsg/linux_eal_parameters`. 43 44``-t TIMEOUT, --timeout TIMEOUT`` 45 Specifies timeout in seconds. If not specified timeout is set to 300 seconds. 46 47``-c TEST_CASE [TEST_CASE ...], --test_cases TEST_CASE [TEST_CASE ...]`` 48 Defines test cases to run. If not specified all available tests are run. 49 50 **Example usage:** 51 52 ``./test-bbdev.py -c validation`` 53 Runs validation test suite 54 55 ``./test-bbdev.py -c latency throughput`` 56 Runs latency and throughput test suites 57 58``-v TEST_VECTOR [TEST_VECTOR ...], --test_vector TEST_VECTOR [TEST_VECTOR ...]`` 59 Specifies paths to the test vector files. If not specified path is set based 60 on "../.." concatenated with "*/app/test-bbdev/test_vectors/bbdev_null.data*" 61 and indicates default data file. 62 63 **Example usage:** 64 65 ``./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_dec_test1.data`` 66 Fills vector based on turbo_dec_test1.data file and runs all tests 67 68 ``./test-bbdev.py -v turbo_dec_test1.data turbo_enc_test2.data`` 69 The bbdev test app is executed twice. First time vector is filled based on 70 *turbo_dec_test1.data* file and second time based on 71 *turb_enc_test2.data* file. For both executions all tests are run. 72 73``-n NUM_OPS, --num_ops NUM_OPS`` 74 Specifies number of operations to process on device. If not specified num_ops 75 is set to 32 operations. 76 77``-l NUM_LCORES, --num_lcores NUM_LCORES`` 78 Specifies number of lcores to run. If not specified num_lcores is set 79 according to value from RTE configuration (EAL coremask) 80 81``-b BURST_SIZE [BURST_SIZE ...], --burst-size BURST_SIZE [BURST_SIZE ...]`` 82 Specifies operations enqueue/dequeue burst size. If not specified burst_size is 83 set to 32. Maximum is 512. 84 85``-t MAX_ITERS [MAX_ITERS ...], --iter_max MAX_ITERS [MAX_ITERS ...]`` 86 Specifies LDPC decoder operations maximum number of iterations for throughput 87 and bler tests. If not specified iter_max is set to 6. 88 89``-s SNR [SNR ...], --snr SNR [SNR ...]`` 90 Specifies for LDPC decoder operations the SNR in dB used when generating LLRs 91 for bler tests. If not specified snr is set to 0 dB. 92 93Test Cases 94~~~~~~~~~~ 95 96There are 7 main test cases that can be executed using testbbdev tool: 97 98* Sanity checks [-c unittest] 99 - Performs sanity checks on BBDEV interface, validating basic functionality 100 101* Validation tests [-c validation] 102 - Performs full operation of enqueue and dequeue 103 - Compares the dequeued data buffer with a expected values in the test 104 vector (TV) being used 105 - Fails if any dequeued value does not match the data in the TV 106 107* Offload Cost measurement [-c offload] 108 - Measures the CPU cycles consumed from the receipt of a user enqueue 109 until it is put on the device queue 110 - The test measures 4 metrics 111 (a) *SW Enq Offload Cost*: Software only enqueue offload cost, the cycle 112 counts and time (us) from the point the enqueue API is called until 113 the point the operation is put on the accelerator queue. 114 (b) *Acc Enq Offload Cost*: The cycle count and time (us) from the 115 point the operation is put on the accelerator queue until the return 116 from enqueue. 117 (c) *SW Deq Offload Cost*: Software dequeue cost, the cycle counts and 118 time (us) consumed to dequeue one operation. 119 (d) *Empty Queue Enq Offload Cost*: The cycle count and time (us) 120 consumed to dequeue from an empty queue. 121 122* Latency measurement [-c latency] 123 - Measures the time consumed from the first enqueue until the first 124 appearance of a dequeued result 125 - This measurement represents the full latency of a bbdev operation 126 (encode or decode) to execute 127 128* Poll-mode Throughput measurement [-c throughput] 129 - Performs full operation of enqueue and dequeue 130 - Executes in poll mode 131 - Measures the achieved throughput on a subset or all available CPU cores 132 - Dequeued data is not validated against expected values stored in TV 133 - Results are printed in million operations per second and million bits 134 per second 135 136* BLER measurement [-c bler] 137 - Performs full operation of enqueue and dequeue 138 - Measures the achieved throughput on a subset or all available CPU cores 139 - Computed BLER (Block Error Rate, ratio of blocks not decoded at a given 140 SNR) in % based on the total number of operations. 141 142* Interrupt-mode Throughput [-c interrupt] 143 - Similar to Throughput test case, but using interrupts. No polling. 144 145 146Parameter Globbing 147~~~~~~~~~~~~~~~~~~ 148 149Thanks to the globbing functionality in python test-bbdev.py script allows to 150run tests with different set of vector files without giving all of them explicitly. 151 152**Example usage for 4G:** 153 154.. code-block:: console 155 156 ./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_<enc/dec>_c<c>_k<k>_r<r>_e<e>_<extra-info>.data 157 158It runs all tests with following vectors: 159 160- ``bbdev_null.data`` 161 162- ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data`` 163 164- ``turbo_enc_c1_k40_r0_e1196_rm.data`` 165 166- ``turbo_enc_c2_k5952_r0_e17868_crc24b.data`` 167 168- ``turbo_dec_c1_k40_r0_e17280_sbd_negllr.data`` 169 170- ``turbo_dec_c1_k6144_r0_e34560_sbd_posllr.data`` 171 172- ``turbo_enc_c1_k40_r0_e272_rm.data`` 173 174- ``turbo_enc_c3_k4800_r2_e14412_crc24b.data`` 175 176- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 177 178- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data`` 179 180- ``turbo_enc_c1_k6144_r0_e120_rm_rvidx.data`` 181 182- ``turbo_enc_c4_k4800_r2_e14412_crc24b.data`` 183 184- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data`` 185 186- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data`` 187 188- ``turbo_enc_c1_k6144_r0_e18444.data`` 189 190- ``turbo_dec_c1_k6144_r0_e34560_negllr.data`` 191 192- ``turbo_enc_c1_k40_r0_e1190_rm.data`` 193 194- ``turbo_enc_c1_k6144_r0_e18448_crc24a.data`` 195 196- ``turbo_dec_c1_k6144_r0_e34560_posllr.data`` 197 198- ``turbo_enc_c1_k40_r0_e1194_rm.data`` 199 200- ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 201 202.. code-block:: console 203 204 ./test-bbdev.py -v app/test-bbdev/turbo_*_default.data 205 206It runs all tests with "default" vectors. 207 208* ``turbo_dec_default.data`` is a soft link to 209 ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data`` 210 211* ``turbo_enc_default.data`` is a soft link to 212 ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data`` 213 214* ``ldpc_dec_default.data`` is a soft link to 215 ``ldpc_dec_v6563.data`` 216 217* ``ldpc_enc_default.data`` is a soft link to 218 ``ldpc_enc_c1_k8148_r0_e9372_rm.data`` 219 220Running Tests 221------------- 222 223All default reference test-vectors are stored in the test_vector 224directory below. 225The prefix trivially defines which type of operation is included : 226turbo_enc, turbo_dec, ldpc_enc, ldpc_dec. 227The details of the configuration are captured in the file but some 228vector name refer more explicitly processing specificity such as 229'HARQ' when HARQ retransmission is used, 'loopback' when the data 230is purely read/written for external DDR, lbrm when limited buffer 231rate matching is expected, or crc_fail when a CRC failure is expected. 232They are chosen to have a good coverage across sizes and processing 233parameters while still keeping their number limited as part of sanity 234regression. 235 236Shortened tree of isg_cid-wireless_dpdk_ae with dpdk compiled and output 237to the build directory: 238 239:: 240 241 |-- app 242 |-- test-bbdev 243 |-- test_vectors 244 245 |-- build 246 |-- app 247 |-- dpdk-test-bbdev 248 249All bbdev devices 250~~~~~~~~~~~~~~~~~ 251 252.. code-block:: console 253 254 ./test-bbdev.py -p ../../build/app/dpdk-test-bbdev 255 -v turbo_dec_default.data 256 257It runs all available tests using the test vector filled based on 258*turbo_dec_default.data* file. 259By default number of operations to process on device is set to 32, timeout is 260set to 300s and operations enqueue/dequeue burst size is set to 32. 261Moreover a bbdev (*baseband_null*) device will be created. 262 263baseband turbo_sw device 264~~~~~~~~~~~~~~~~~~~~~~~~ 265 266.. code-block:: console 267 268 ./test-bbdev.py -p ../../build/app/dpdk-test-bbdev 269 -e="--vdev=baseband_turbo_sw" -t 120 -c validation 270 -v ./test_vectors/* -n 64 -b 8 32 271 272It runs **validation** test for each vector file that matches the given pattern. 273Number of operations to process on device is set to 64 and operations timeout is 274set to 120s and enqueue/dequeue burst size is set to 8 and to 32. 275Moreover a bbdev (*baseband_turbo_sw*) device will be created. 276 277 278bbdev null device 279~~~~~~~~~~~~~~~~~ 280 281Executing bbdev null device with *bbdev_null.data* helps in measuring the 282overhead introduced by the bbdev framework. 283 284.. code-block:: console 285 286 ./test-bbdev.py -e="--vdev=baseband_null0" 287 -v ./test_vectors/bbdev_null.data 288 289**Note:** 290 291baseband_null device does not have to be defined explicitly as it is created by default. 292 293 294 295Test Vector files 296----------------- 297 298Test Vector files contain the data which is used to set turbo decoder/encoder 299parameters and buffers for validation purpose. New test vector files should be 300stored in ``app/test-bbdev/test_vectors/`` directory. Detailed description of 301the syntax of the test vector files is in the following section. 302 303 304Basic principles for test vector files 305~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 306Line started with ``#`` is treated as a comment and is ignored. 307 308If variable is a chain of values, values should be separated by a comma. If 309assignment is split into several lines, each line (except the last one) has to 310be ended with a comma. 311There is no comma after last value in last line. Correct assignment should 312look like the following: 313 314.. parsed-literal:: 315 316 variable = 317 value, value, value, value, 318 value, value 319 320In case where variable is a single value correct assignment looks like the 321following: 322 323.. parsed-literal:: 324 325 variable = 326 value 327 328Length of chain variable is calculated by parser. Can not be defined 329explicitly. 330 331Variable op_type has to be defined as a first variable in file. It specifies 332what type of operations will be executed. For 4G decoder op_type has to be set to 333``RTE_BBDEV_OP_TURBO_DEC`` and for 4G encoder to ``RTE_BBDEV_OP_TURBO_ENC``. 334 335Bbdev-test adjusts the byte endianness based on the PMD capability (data_endianness) 336and all the test vectors input/output data are assumed to be LE by default 337 338Full details of the meaning and valid values for the below fields are 339documented in *rte_bbdev_op.h* 340 341 342Turbo decoder test vectors template 343~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 344 345For turbo decoder it has to be always set to ``RTE_BBDEV_OP_TURBO_DEC`` 346 347.. parsed-literal:: 348 349 op_type = 350 RTE_BBDEV_OP_TURBO_DEC 351 352Chain of uint32_t values. Note that it is possible to define more than one 353input/output entries which will result in chaining two or more data structures 354for *segmented Transport Blocks* 355 356.. parsed-literal:: 357 358 input0 = 359 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100, 360 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00 361 362Chain of uint32_t values 363 364.. parsed-literal:: 365 366 input1 = 367 0x7f7f0000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 368 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 369 370Chain of uint32_t values 371 372.. parsed-literal:: 373 374 input2 = 375 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 376 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000 377 378Chain of uint32_t values 379 380.. parsed-literal:: 381 382 hard_output0 = 383 0xa7d6732e 384 385Chain of uint32_t values 386 387.. parsed-literal:: 388 389 hard_output1 = 390 0xa61 391 392Chain of uint32_t values 393 394.. parsed-literal:: 395 396 soft_output0 = 397 0x817f817f, 0x7f817f7f, 0x81818181, 0x817f7f81, 0x7f818181, 0x8181817f, 0x817f817f, 0x8181817f 398 399Chain of uint32_t values 400 401.. parsed-literal:: 402 403 soft_output1 = 404 0x817f7f81, 0x7f7f7f81, 0x7f7f8181 405 406uint32_t value 407 408.. parsed-literal:: 409 410 e = 411 44 412 413uint16_t value 414 415.. parsed-literal:: 416 417 k = 418 40 419 420uint8_t value 421 422.. parsed-literal:: 423 424 rv_index = 425 0 426 427uint8_t value 428 429.. parsed-literal:: 430 431 iter_max = 432 8 433 434uint8_t value 435 436.. parsed-literal:: 437 438 iter_min = 439 4 440 441uint8_t value 442 443.. parsed-literal:: 444 445 expected_iter_count = 446 8 447 448uint8_t value 449 450.. parsed-literal:: 451 452 ext_scale = 453 15 454 455uint8_t value 456 457.. parsed-literal:: 458 459 num_maps = 460 0 461 462Chain of flags for LDPC decoder operation based on the rte_bbdev_op_td_flag_bitmasks: 463 464Example: 465 466 .. parsed-literal:: 467 468 op_flags = 469 RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE, RTE_BBDEV_TURBO_EQUALIZER, 470 RTE_BBDEV_TURBO_SOFT_OUTPUT 471 472Chain of operation statuses that are expected after operation is performed. 473Following statuses can be used: 474 475- ``DMA`` 476 477- ``FCW`` 478 479- ``CRC`` 480 481- ``OK`` 482 483``OK`` means no errors are expected. Cannot be used with other values. 484 485.. parsed-literal:: 486 487 expected_status = 488 FCW, CRC 489 490 491Turbo encoder test vectors template 492~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 493 494For turbo encoder it has to be always set to ``RTE_BBDEV_OP_TURBO_ENC`` 495 496.. parsed-literal:: 497 498 op_type = 499 RTE_BBDEV_OP_TURBO_ENC 500 501Chain of uint32_t values 502 503.. parsed-literal:: 504 505 input0 = 506 0x11d2bcac, 0x4d 507 508Chain of uint32_t values 509 510.. parsed-literal:: 511 512 output0 = 513 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 514 0xa25b7f47, 0x2aa224ce, 0x79f2 515 516uint32_t value 517 518.. parsed-literal:: 519 520 e = 521 272 522 523uint16_t value 524 525.. parsed-literal:: 526 527 k = 528 40 529 530uint16_t value 531 532.. parsed-literal:: 533 534 ncb = 535 192 536 537uint8_t value 538 539.. parsed-literal:: 540 541 rv_index = 542 0 543 544Chain of flags for LDPC decoder operation based on the rte_bbdev_op_te_flag_bitmasks: 545 546``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` is used to indicate the parser to 547force the input data to be memory split and formed as a segmented mbuf. 548 549 550.. parsed-literal:: 551 552 op_flags = 553 RTE_BBDEV_TURBO_RATE_MATCH 554 555Chain of operation statuses that are expected after operation is performed. 556Following statuses can be used: 557 558- ``DMA`` 559 560- ``FCW`` 561 562- ``OK`` 563 564``OK`` means no errors are expected. Cannot be used with other values. 565 566.. parsed-literal:: 567 568 expected_status = 569 OK 570 571LDPC decoder test vectors template 572~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 573 574For LDPC decoder it has to be always set to ``RTE_BBDEV_OP_LDPC_DEC`` 575 576.. parsed-literal:: 577 578 op_type = 579 RTE_BBDEV_OP_LDPC_DEC 580 581Chain of uint32_t values. Note that it is possible to define more than one 582input/output entries which will result in chaining two or more data structures 583for *segmented Transport Blocks* 584 585.. parsed-literal:: 586 587 input0 = 588 0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100, 589 0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00 590 591.. parsed-literal:: 592 593 output0 = 594 0xa7d6732e 595 596uint8_t value 597 598.. parsed-literal:: 599 600 basegraph= 601 1 602 603uint16_t value 604 605.. parsed-literal:: 606 607 z_c= 608 224 609 610uint16_t value 611 612.. parsed-literal:: 613 614 n_cb= 615 14784 616 617uint8_t value 618 619.. parsed-literal:: 620 621 q_m= 622 1 623 624uint16_t value 625 626.. parsed-literal:: 627 628 n_filler= 629 40 630 631uint32_t value 632 633.. parsed-literal:: 634 635 e= 636 13072 637 638uint8_t value 639 640.. parsed-literal:: 641 642 rv_index= 643 2 644 645uint8_t value 646 647.. parsed-literal:: 648 code_block_mode= 649 1 650 651uint8_t value 652 653.. parsed-literal:: 654 655 iter_max= 656 20 657 658uint8_t value 659 660.. parsed-literal:: 661 662 expected_iter_count= 663 8 664 665 666Chain of flags for LDPC decoder operation based on the rte_bbdev_op_ldpcdec_flag_bitmasks: 667 668Example: 669 670 .. parsed-literal:: 671 672 op_flags = 673 RTE_BBDEV_LDPC_ITERATION_STOP_ENABLE, RTE_BBDEV_LDPC_HQ_COMBINE_OUT_ENABLE, 674 RTE_BBDEV_LDPC_HQ_COMBINE_IN_ENABLE, RTE_BBDEV_LDPC_HARQ_6BIT_COMPRESSION 675 676Chain of operation statuses that are expected after operation is performed. 677Following statuses can be used: 678 679- ``OK`` : No error reported. 680 681- ``SYN`` : LDPC syndrome parity check is failing. 682 683- ``CRC`` : CRC parity check is failing when CRC check operation is included. 684 685- ``SYNCRC`` : Both CRC and LDPC syndromes parity checks are failing. 686 687``OK`` means no errors are expected. Cannot be used with other values. 688 689.. parsed-literal:: 690 691 expected_status = 692 CRC 693 694 695LDPC encoder test vectors template 696~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 697 698For turbo encoder it has to be always set to ``RTE_BBDEV_OP_LDPC_ENC`` 699 700.. parsed-literal:: 701 702 op_type = 703 RTE_BBDEV_OP_LDPC_ENC 704 705Chain of uint32_t values 706 707.. parsed-literal:: 708 709 input0 = 710 0x11d2bcac, 0x4d 711 712Chain of uint32_t values 713 714.. parsed-literal:: 715 716 output0 = 717 0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29, 718 0xa25b7f47, 0x2aa224ce, 0x79f2 719 720 721uint8_t value 722 723.. parsed-literal:: 724 725 basegraph= 726 1 727 728uint16_t value 729 730.. parsed-literal:: 731 732 z_c= 733 52 734 735uint16_t value 736 737.. parsed-literal:: 738 739 n_cb= 740 3432 741 742uint8_t value 743 744.. parsed-literal:: 745 746 q_m= 747 6 748 749uint16_t value 750 751.. parsed-literal:: 752 753 n_filler= 754 0 755 756uint32_t value 757 758.. parsed-literal:: 759 760 e = 761 1380 762 763uint8_t value 764 765.. parsed-literal:: 766 767 rv_index = 768 1 769 770uint8_t value 771 772.. parsed-literal:: 773 774 code_block_mode = 775 1 776 777 778Chain of flags for LDPC encoder operation based on the 779rte_bbdev_op_ldpcenc_flag_bitmasks: 780 781.. parsed-literal:: 782 783 op_flags = 784 RTE_BBDEV_LDPC_RATE_MATCH 785 786Chain of operation statuses that are expected after operation is performed. 787Following statuses can be used: 788 789- ``DMA`` 790 791- ``FCW`` 792 793- ``OK`` 794 795``OK`` means no errors are expected. Cannot be used with other values. 796 797.. parsed-literal:: 798 799 expected_status = 800 OK 801