xref: /dpdk/doc/guides/tools/testbbdev.rst (revision f5057be340e44f3edc0fe90fa875eb89a4c49b4f)
1..  SPDX-License-Identifier: BSD-3-Clause
2    Copyright(c) 2017 Intel Corporation
3
4dpdk-test-bbdev Application
5===========================
6
7The ``dpdk-test-bbdev`` tool is a Data Plane Development Kit (DPDK) utility that
8allows measuring performance parameters of PMDs available in the bbdev framework.
9Tests available for execution are: latency, throughput, validation,
10bler and sanity tests. Execution of tests can be customized using various
11parameters passed to a python running script.
12
13Compiling the Application
14-------------------------
15
16**Step 1: PMD setting**
17
18The ``dpdk-test-bbdev`` tool depends on crypto device drivers PMD which
19are disabled by default in the build configuration file ``common_base``.
20The bbdevice drivers PMD which should be tested can be enabled by setting
21
22   ``CONFIG_RTE_LIBRTE_PMD_<name>=y``
23
24Setting example for (*baseband_turbo_sw*) PMD
25
26   ``CONFIG_RTE_LIBRTE_PMD_BBDEV_TURBO_SW=y``
27
28**Step 2: Build the application**
29
30Execute the ``dpdk-setup.sh`` script to build the DPDK library together with the
31``dpdk-test-bbdev`` application.
32
33Initially, the user must select a DPDK target to choose the correct target type
34and compiler options to use when building the libraries.
35The user must have all libraries, modules, updates and compilers installed
36in the system prior to this, as described in the earlier chapters in this
37Getting Started Guide.
38
39Running the Application
40-----------------------
41
42The tool application has a number of command line options:
43
44.. code-block:: console
45
46    test-bbdev.py [-h] [-p TESTAPP_PATH] [-e EAL_PARAMS] [-t TIMEOUT]
47                       [-c TEST_CASE [TEST_CASE ...]]
48                       [-v TEST_VECTOR [TEST_VECTOR...]] [-n NUM_OPS]
49                       [-b BURST_SIZE [BURST_SIZE ...]] [-l NUM_LCORES]
50                       [-t MAX_ITERS [MAX_ITERS ...]]
51                       [-s SNR [SNR ...]]
52
53command-line Options
54~~~~~~~~~~~~~~~~~~~~
55
56The following are the command-line options:
57
58``-h, --help``
59 Shows help message and exit.
60
61``-p TESTAPP_PATH, --testapp_path TESTAPP_PATH``
62 Indicates the path to the bbdev test app. If not specified path is set based
63 on *$RTE_SDK* environment variable concatenated with "*/build/app/testbbdev*".
64
65``-e EAL_PARAMS, --eal_params EAL_PARAMS``
66 Specifies EAL arguments which are passed to the test app. For more details,
67 refer to DPDK documentation at :doc:`../linux_gsg/linux_eal_parameters`.
68
69``-t TIMEOUT, --timeout TIMEOUT``
70 Specifies timeout in seconds. If not specified timeout is set to 300 seconds.
71
72``-c TEST_CASE [TEST_CASE ...], --test_cases TEST_CASE [TEST_CASE ...]``
73 Defines test cases to run. If not specified all available tests are run.
74
75 **Example usage:**
76
77 ``./test-bbdev.py -c validation``
78  Runs validation test suite
79
80 ``./test-bbdev.py -c latency throughput``
81  Runs latency and throughput test suites
82
83``-v TEST_VECTOR [TEST_VECTOR ...], --test_vector TEST_VECTOR [TEST_VECTOR ...]``
84 Specifies paths to the test vector files. If not specified path is set based
85 on *$RTE_SDK* environment variable concatenated with
86 "*/app/test-bbdev/test_vectors/bbdev_null.data*" and indicates default
87 data file.
88
89 **Example usage:**
90
91 ``./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_dec_test1.data``
92  Fills vector based on turbo_dec_test1.data file and runs all tests
93
94 ``./test-bbdev.py -v turbo_dec_test1.data turbo_enc_test2.data``
95  The bbdev test app is executed twice. First time vector is filled based on
96  *turbo_dec_test1.data* file and second time based on
97  *turb_enc_test2.data* file. For both executions all tests are run.
98
99``-n NUM_OPS, --num_ops NUM_OPS``
100 Specifies number of operations to process on device. If not specified num_ops
101 is set to 32 operations.
102
103``-l NUM_LCORES, --num_lcores NUM_LCORES``
104 Specifies number of lcores to run. If not specified num_lcores is set
105 according to value from RTE configuration (EAL coremask)
106
107``-b BURST_SIZE [BURST_SIZE ...], --burst-size BURST_SIZE [BURST_SIZE ...]``
108 Specifies operations enqueue/dequeue burst size. If not specified burst_size is
109 set to 32. Maximum is 512.
110
111``-t MAX_ITERS [MAX_ITERS ...], --iter_max MAX_ITERS [MAX_ITERS ...]``
112 Specifies LDPC decoder operations maximum number of iterations for throughput
113 and bler tests. If not specified iter_max is set to 6.
114
115``-s SNR [SNR ...], --snr SNR [SNR ...]``
116 Specifies for LDPC decoder operations the SNR in dB used when generating LLRs
117 for bler tests. If not specified snr is set to 0 dB.
118
119Test Cases
120~~~~~~~~~~
121
122There are 7 main test cases that can be executed using testbbdev tool:
123
124* Sanity checks [-c unittest]
125    - Performs sanity checks on BBDEV interface, validating basic functionality
126
127* Validation tests [-c validation]
128    - Performs full operation of enqueue and dequeue
129    - Compares the dequeued data buffer with a expected values in the test
130      vector (TV) being used
131    - Fails if any dequeued value does not match the data in the TV
132
133* Offload Cost measurement [-c offload]
134    - Measures the CPU cycles consumed from the receipt of a user enqueue
135      until it is put on the device queue
136    - The test measures 4 metrics
137        (a) *SW Enq Offload Cost*: Software only enqueue offload cost, the cycle
138            counts and time (us) from the point the enqueue API is called until
139            the point the operation is put on the accelerator queue.
140        (b) *Acc Enq Offload Cost*: The cycle count and time (us) from the
141            point the operation is put on the accelerator queue until the return
142            from enqueue.
143        (c) *SW Deq Offload Cost*: Software dequeue cost, the cycle counts and
144            time (us) consumed to dequeue one operation.
145        (d) *Empty Queue Enq Offload Cost*: The cycle count and time (us)
146            consumed to dequeue from an empty queue.
147
148* Latency measurement [-c latency]
149    - Measures the time consumed from the first enqueue until the first
150      appearance of a dequeued result
151    - This measurement represents the full latency of a bbdev operation
152      (encode or decode) to execute
153
154* Poll-mode Throughput measurement [-c throughput]
155    - Performs full operation of enqueue and dequeue
156    - Executes in poll mode
157    - Measures the achieved throughput on a subset or all available CPU cores
158    - Dequeued data is not validated against expected values stored in TV
159    - Results are printed in million operations per second and million bits
160      per second
161
162* BLER measurement [-c bler]
163    - Performs full operation of enqueue and dequeue
164    - Measures the achieved throughput on a subset or all available CPU cores
165    - Computed BLER (Block Error Rate, ratio of blocks not decoded at a given
166      SNR) in % based on the total number of operations.
167
168* Interrupt-mode Throughput [-c interrupt]
169    - Similar to Throughput test case, but using interrupts. No polling.
170
171
172Parameter Globbing
173~~~~~~~~~~~~~~~~~~
174
175Thanks to the globbing functionality in python test-bbdev.py script allows to
176run tests with different set of vector files without giving all of them explicitly.
177
178**Example usage for 4G:**
179
180.. code-block:: console
181
182  ./test-bbdev.py -v app/test-bbdev/test_vectors/turbo_<enc/dec>_c<c>_k<k>_r<r>_e<e>_<extra-info>.data
183
184It runs all tests with following vectors:
185
186- ``bbdev_null.data``
187
188- ``turbo_dec_c1_k6144_r0_e34560_sbd_negllr.data``
189
190- ``turbo_enc_c1_k40_r0_e1196_rm.data``
191
192- ``turbo_enc_c2_k5952_r0_e17868_crc24b.data``
193
194- ``turbo_dec_c1_k40_r0_e17280_sbd_negllr.data``
195
196- ``turbo_dec_c1_k6144_r0_e34560_sbd_posllr.data``
197
198- ``turbo_enc_c1_k40_r0_e272_rm.data``
199
200- ``turbo_enc_c3_k4800_r2_e14412_crc24b.data``
201
202- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data``
203
204- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr_crc24b.data``
205
206- ``turbo_enc_c1_k6144_r0_e120_rm_rvidx.data``
207
208- ``turbo_enc_c4_k4800_r2_e14412_crc24b.data``
209
210- ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_low_snr.data``
211
212- ``turbo_dec_c2_k3136_r0_e4920_sbd_negllr.data``
213
214- ``turbo_enc_c1_k6144_r0_e18444.data``
215
216- ``turbo_dec_c1_k6144_r0_e34560_negllr.data``
217
218- ``turbo_enc_c1_k40_r0_e1190_rm.data``
219
220- ``turbo_enc_c1_k6144_r0_e18448_crc24a.data``
221
222- ``turbo_dec_c1_k6144_r0_e34560_posllr.data``
223
224- ``turbo_enc_c1_k40_r0_e1194_rm.data``
225
226- ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data``
227
228.. code-block:: console
229
230  ./test-bbdev.py -v app/test-bbdev/turbo_*_default.data
231
232It runs all tests with "default" vectors.
233
234* ``turbo_dec_default.data`` is a soft link to
235  ``turbo_dec_c1_k6144_r0_e10376_crc24b_sbd_negllr_high_snr.data``
236
237* ``turbo_enc_default.data`` is a soft link to
238  ``turbo_enc_c1_k6144_r0_e32256_crc24b_rm.data``
239
240* ``ldpc_dec_default.data`` is a soft link to
241  ``ldpc_dec_v6563.data``
242
243* ``ldpc_enc_default.data`` is a soft link to
244  ``ldpc_enc_c1_k8148_r0_e9372_rm.data``
245
246Running Tests
247-------------
248
249All default reference test-vectors are stored in the test_vector
250directory below.
251The prefix trivially defines which type of operation is included :
252turbo_enc, turbo_dec, ldpc_enc, ldpc_dec.
253The details of the configuration are captured in the file but some
254vector name refer more explicitly processing specificity such as
255'HARQ' when HARQ retransmission is used, 'loopback' when the data
256is purely read/written for external DDR, lbrm when limited buffer
257rate matching is expected, or crc_fail when a CRC failure is expected.
258They are chosen to have a good coverage across sizes and processing
259parameters while still keeping their number limited as part of sanity
260regression.
261
262Shortened tree of isg_cid-wireless_dpdk_ae with dpdk compiled for
263x86_64-native-linux-icc target:
264
265::
266
267 |-- app
268     |-- test-bbdev
269         |-- test_vectors
270
271 |-- x86_64-native-linux-icc
272     |-- app
273         |-- testbbdev
274
275All bbdev devices
276~~~~~~~~~~~~~~~~~
277
278.. code-block:: console
279
280  ./test-bbdev.py -p ../../x86_64-native-linux-icc/app/testbbdev
281  -v turbo_dec_default.data
282
283It runs all available tests using the test vector filled based on
284*turbo_dec_default.data* file.
285By default number of operations to process on device is set to 32, timeout is
286set to 300s and operations enqueue/dequeue burst size is set to 32.
287Moreover a bbdev (*baseband_null*) device will be created.
288
289baseband turbo_sw device
290~~~~~~~~~~~~~~~~~~~~~~~~
291
292.. code-block:: console
293
294  ./test-bbdev.py -p ../../x86_64-native-linux-icc/app/testbbdev
295  -e="--vdev=baseband_turbo_sw" -t 120 -c validation
296  -v ./test_vectors/* -n 64 -b 8 32
297
298It runs **validation** test for each vector file that matches the given pattern.
299Number of operations to process on device is set to 64 and operations timeout is
300set to 120s and enqueue/dequeue burst size is set to 8 and to 32.
301Moreover a bbdev (*baseband_turbo_sw*) device will be created.
302
303
304bbdev null device
305~~~~~~~~~~~~~~~~~
306
307Executing bbdev null device with *bbdev_null.data* helps in measuring the
308overhead introduced by the bbdev framework.
309
310.. code-block:: console
311
312  ./test-bbdev.py -e="--vdev=baseband_null0"
313  -v ./test_vectors/bbdev_null.data
314
315**Note:**
316
317baseband_null device does not have to be defined explicitly as it is created by default.
318
319
320
321Test Vector files
322-----------------
323
324Test Vector files contain the data which is used to set turbo decoder/encoder
325parameters and buffers for validation purpose. New test vector files should be
326stored in ``app/test-bbdev/test_vectors/`` directory. Detailed description of
327the syntax of the test vector files is in the following section.
328
329
330Basic principles for test vector files
331~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
332Line started with ``#`` is treated as a comment and is ignored.
333
334If variable is a chain of values, values should be separated by a comma. If
335assignment is split into several lines, each line (except the last one) has to
336be ended with a comma.
337There is no comma after last value in last line. Correct assignment should
338look like the following:
339
340.. parsed-literal::
341
342 variable =
343 value, value, value, value,
344 value, value
345
346In case where variable is a single value correct assignment looks like the
347following:
348
349.. parsed-literal::
350
351 variable =
352 value
353
354Length of chain variable is calculated by parser. Can not be defined
355explicitly.
356
357Variable op_type has to be defined as a first variable in file. It specifies
358what type of operations will be executed. For 4G decoder op_type has to be set to
359``RTE_BBDEV_OP_TURBO_DEC`` and for 4G encoder to ``RTE_BBDEV_OP_TURBO_ENC``.
360
361Full details of the meaning and valid values for the below fields are
362documented in *rte_bbdev_op.h*
363
364
365Turbo decoder test vectors template
366~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
367
368For turbo decoder it has to be always set to ``RTE_BBDEV_OP_TURBO_DEC``
369
370.. parsed-literal::
371
372    op_type =
373    RTE_BBDEV_OP_TURBO_DEC
374
375Chain of uint32_t values. Note that it is possible to define more than one
376input/output entries which will result in chaining two or more data structures
377for *segmented Transport Blocks*
378
379.. parsed-literal::
380
381    input0 =
382    0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100,
383    0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00
384
385Chain of uint32_t values
386
387.. parsed-literal::
388
389    input1 =
390    0x7f7f0000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000,
391    0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000
392
393Chain of uint32_t values
394
395.. parsed-literal::
396
397    input2 =
398    0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000,
399    0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000
400
401Chain of uint32_t values
402
403.. parsed-literal::
404
405    hard_output0 =
406    0xa7d6732e
407
408Chain of uint32_t values
409
410.. parsed-literal::
411
412    hard_output1 =
413    0xa61
414
415Chain of uint32_t values
416
417.. parsed-literal::
418
419    soft_output0 =
420    0x817f817f, 0x7f817f7f, 0x81818181, 0x817f7f81, 0x7f818181, 0x8181817f, 0x817f817f, 0x8181817f
421
422Chain of uint32_t values
423
424.. parsed-literal::
425
426    soft_output1 =
427    0x817f7f81, 0x7f7f7f81, 0x7f7f8181
428
429uint32_t value
430
431.. parsed-literal::
432
433    e =
434    44
435
436uint16_t value
437
438.. parsed-literal::
439
440    k =
441    40
442
443uint8_t value
444
445.. parsed-literal::
446
447    rv_index =
448    0
449
450uint8_t value
451
452.. parsed-literal::
453
454    iter_max =
455    8
456
457uint8_t value
458
459.. parsed-literal::
460
461    iter_min =
462    4
463
464uint8_t value
465
466.. parsed-literal::
467
468    expected_iter_count =
469    8
470
471uint8_t value
472
473.. parsed-literal::
474
475    ext_scale =
476    15
477
478uint8_t value
479
480.. parsed-literal::
481
482    num_maps =
483    0
484
485Chain of flags for LDPC decoder operation based on the rte_bbdev_op_td_flag_bitmasks:
486
487Example:
488
489    .. parsed-literal::
490
491        op_flags =
492        RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE, RTE_BBDEV_TURBO_EQUALIZER,
493        RTE_BBDEV_TURBO_SOFT_OUTPUT
494
495Chain of operation statuses that are expected after operation is performed.
496Following statuses can be used:
497
498- ``DMA``
499
500- ``FCW``
501
502- ``CRC``
503
504- ``OK``
505
506``OK`` means no errors are expected. Cannot be used with other values.
507
508.. parsed-literal::
509
510    expected_status =
511    FCW, CRC
512
513
514Turbo encoder test vectors template
515~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
516
517For turbo encoder it has to be always set to ``RTE_BBDEV_OP_TURBO_ENC``
518
519.. parsed-literal::
520
521    op_type =
522    RTE_BBDEV_OP_TURBO_ENC
523
524Chain of uint32_t values
525
526.. parsed-literal::
527
528    input0 =
529    0x11d2bcac, 0x4d
530
531Chain of uint32_t values
532
533.. parsed-literal::
534
535    output0 =
536    0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29,
537    0xa25b7f47, 0x2aa224ce, 0x79f2
538
539uint32_t value
540
541.. parsed-literal::
542
543    e =
544    272
545
546uint16_t value
547
548.. parsed-literal::
549
550    k =
551    40
552
553uint16_t value
554
555.. parsed-literal::
556
557    ncb =
558    192
559
560uint8_t value
561
562.. parsed-literal::
563
564    rv_index =
565    0
566
567Chain of flags for LDPC decoder operation based on the rte_bbdev_op_te_flag_bitmasks:
568
569``RTE_BBDEV_TURBO_ENC_SCATTER_GATHER`` is used to indicate the parser to
570force the input data to be memory split and formed as a segmented mbuf.
571
572
573.. parsed-literal::
574
575    op_flags =
576    RTE_BBDEV_TURBO_RATE_MATCH
577
578Chain of operation statuses that are expected after operation is performed.
579Following statuses can be used:
580
581- ``DMA``
582
583- ``FCW``
584
585- ``OK``
586
587``OK`` means no errors are expected. Cannot be used with other values.
588
589.. parsed-literal::
590
591    expected_status =
592    OK
593
594LDPC decoder test vectors template
595~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
596
597For LDPC decoder it has to be always set to ``RTE_BBDEV_OP_LDPC_DEC``
598
599.. parsed-literal::
600
601    op_type =
602    RTE_BBDEV_OP_LDPC_DEC
603
604Chain of uint32_t values. Note that it is possible to define more than one
605input/output entries which will result in chaining two or more data structures
606for *segmented Transport Blocks*
607
608.. parsed-literal::
609
610    input0 =
611    0x00000000, 0x7f817f00, 0x7f7f8100, 0x817f8100, 0x81008100, 0x7f818100, 0x81817f00, 0x7f818100,
612    0x81007f00, 0x7f818100, 0x817f8100, 0x81817f00, 0x81008100, 0x817f7f00, 0x7f7f8100, 0x81817f00
613
614.. parsed-literal::
615
616    output0 =
617    0xa7d6732e
618
619uint8_t value
620
621.. parsed-literal::
622
623    basegraph=
624    1
625
626uint16_t value
627
628.. parsed-literal::
629
630    z_c=
631    224
632
633uint16_t value
634
635.. parsed-literal::
636
637    n_cb=
638    14784
639
640uint8_t value
641
642.. parsed-literal::
643
644    q_m=
645    1
646
647uint16_t value
648
649.. parsed-literal::
650
651    n_filler=
652    40
653
654uint32_t value
655
656.. parsed-literal::
657
658    e=
659    13072
660
661uint8_t value
662
663.. parsed-literal::
664
665    rv_index=
666    2
667
668uint8_t value
669
670.. parsed-literal::
671    code_block_mode=
672    1
673
674uint8_t value
675
676.. parsed-literal::
677
678    iter_max=
679    20
680
681uint8_t value
682
683.. parsed-literal::
684
685    expected_iter_count=
686    8
687
688
689Chain of flags for LDPC decoder operation based on the rte_bbdev_op_ldpcdec_flag_bitmasks:
690
691Example:
692
693    .. parsed-literal::
694
695        op_flags =
696        RTE_BBDEV_LDPC_ITERATION_STOP_ENABLE, RTE_BBDEV_LDPC_HQ_COMBINE_OUT_ENABLE,
697        RTE_BBDEV_LDPC_HQ_COMBINE_IN_ENABLE, RTE_BBDEV_LDPC_HARQ_6BIT_COMPRESSION
698
699Chain of operation statuses that are expected after operation is performed.
700Following statuses can be used:
701
702- ``OK`` : No error reported.
703
704- ``SYN`` : LDPC syndrome parity check is failing.
705
706- ``CRC`` : CRC parity check is failing when CRC check operation is included.
707
708- ``SYNCRC`` : Both CRC and LDPC syndromes parity checks are failing.
709
710``OK`` means no errors are expected. Cannot be used with other values.
711
712.. parsed-literal::
713
714    expected_status =
715    CRC
716
717
718LDPC encoder test vectors template
719~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
720
721For turbo encoder it has to be always set to ``RTE_BBDEV_OP_LDPC_ENC``
722
723.. parsed-literal::
724
725    op_type =
726    RTE_BBDEV_OP_LDPC_ENC
727
728Chain of uint32_t values
729
730.. parsed-literal::
731
732    input0 =
733    0x11d2bcac, 0x4d
734
735Chain of uint32_t values
736
737.. parsed-literal::
738
739    output0 =
740    0xd2399179, 0x640eb999, 0x2cbaf577, 0xaf224ae2, 0x9d139927, 0xe6909b29,
741    0xa25b7f47, 0x2aa224ce, 0x79f2
742
743
744uint8_t value
745
746.. parsed-literal::
747
748    basegraph=
749    1
750
751uint16_t value
752
753.. parsed-literal::
754
755    z_c=
756    52
757
758uint16_t value
759
760.. parsed-literal::
761
762    n_cb=
763    3432
764
765uint8_t value
766
767.. parsed-literal::
768
769    q_m=
770    6
771
772uint16_t value
773
774.. parsed-literal::
775
776    n_filler=
777    0
778
779uint32_t value
780
781.. parsed-literal::
782
783    e =
784    1380
785
786uint8_t value
787
788.. parsed-literal::
789
790    rv_index =
791    1
792
793uint8_t value
794
795.. parsed-literal::
796
797    code_block_mode =
798    1
799
800
801Chain of flags for LDPC encoder operation based on the
802rte_bbdev_op_ldpcenc_flag_bitmasks:
803
804.. parsed-literal::
805
806    op_flags =
807    RTE_BBDEV_LDPC_RATE_MATCH
808
809Chain of operation statuses that are expected after operation is performed.
810Following statuses can be used:
811
812- ``DMA``
813
814- ``FCW``
815
816- ``OK``
817
818``OK`` means no errors are expected. Cannot be used with other values.
819
820.. parsed-literal::
821
822    expected_status =
823    OK
824