xref: /dpdk/doc/guides/tools/cryptoperf.rst (revision e9fd1ebf981f361844aea9ec94e17f4bda5e1479)
1..  SPDX-License-Identifier: BSD-3-Clause
2    Copyright(c) 2016 Intel Corporation.
3
4dpdk-test-crypto-perf Application
5=================================
6
7The ``dpdk-test-crypto-perf`` tool is a Data Plane Development Kit (DPDK)
8utility that allows measuring performance parameters of PMDs available in the
9crypto tree. There are available two measurement types: throughput and latency.
10User can use multiply cores to run tests on but only
11one type of crypto PMD can be measured during single application
12execution. Cipher parameters, type of device, type of operation and
13chain mode have to be specified in the command line as application
14parameters. These parameters are checked using device capabilities
15structure.
16
17Limitations
18-----------
19On hardware devices the cycle-count doesn't always represent the actual offload
20cost. The cycle-count only represents the offload cost when the hardware
21accelerator is not fully loaded, when loaded the cpu cycles freed up by the
22offload are still consumed by the test tool and included in the cycle-count.
23These cycles are consumed by retries and inefficient API calls enqueuing and
24dequeuing smaller bursts than specified by the cmdline parameter. This results
25in a larger cycle-count measurement and should not be interpreted as an offload
26cost measurement. Using "pmd-cyclecount" mode will give a better idea of
27actual costs of hardware acceleration.
28
29On hardware devices the throughput measurement is not necessarily the maximum
30possible for the device, e.g. it may be necessary to use multiple cores to keep
31the hardware accelerator fully loaded and so measure maximum throughput.
32
33
34Linearization setting
35---------------------
36
37It is possible linearized input segmented packets just before crypto operation
38for devices which doesn't support scatter-gather, and allows to measure
39performance also for this use case.
40
41To set on the linearization options add below definition to the
42``cperf_ops.h`` file::
43
44   #define CPERF_LINEARIZATION_ENABLE
45
46
47Running the Application
48-----------------------
49
50The tool application has a number of command line options:
51
52.. code-block:: console
53
54   dpdk-test-crypto-perf [EAL Options] -- [Application Options]
55
56EAL Options
57~~~~~~~~~~~
58
59The following are the EAL command-line options that can be used in conjunction
60with the ``dpdk-test-crypto-perf`` application.
61See the DPDK Getting Started Guides for more information on these options.
62
63*   ``-c <COREMASK>`` or ``-l <CORELIST>``
64
65        Set the hexadecimal bitmask of the cores to run on. The corelist is a
66        list cores to use.
67
68*   ``-a <PCI>``
69
70        Add a PCI device in allow list.
71
72*   ``--vdev <driver><id>``
73
74        Add a virtual device.
75
76Application Options
77~~~~~~~~~~~~~~~~~~~
78
79The following are the application command-line options:
80
81* ``--ptest type``
82
83        Set test type, where ``type`` is one of the following::
84
85           throughput
86           latency
87           verify
88           pmd-cyclecount
89
90* ``--silent``
91
92        Disable options dump.
93
94* ``--pool-sz <n>``
95
96        Set the number of mbufs to be allocated in the mbuf pool.
97
98* ``--total-ops <n>``
99
100        Set the number of total operations performed.
101
102* ``--burst-sz <n>``
103
104        Set the number of packets per burst.
105
106        This can be set as:
107          * Single value (i.e. ``--burst-sz 16``)
108          * Range of values, using the following structure ``min:inc:max``,
109            where ``min`` is minimum size, ``inc`` is the increment size and ``max``
110            is the maximum size (i.e. ``--burst-sz 16:2:32``)
111          * List of values, up to 32 values, separated in commas (i.e. ``--burst-sz 16,24,32``)
112
113* ``--buffer-sz <n>``
114
115        Set the size of single packet (plaintext or ciphertext in it).
116
117        This can be set as:
118          * Single value (i.e. ``--buffer-sz 16``)
119          * Range of values, using the following structure ``min:inc:max``,
120            where ``min`` is minimum size, ``inc`` is the increment size and ``max``
121            is the maximum size (i.e. ``--buffer-sz 16:2:32``)
122          * List of values, up to 32 values, separated in commas (i.e. ``--buffer-sz 32,64,128``)
123
124* ``--imix <n>``
125
126        Set the distribution of packet sizes.
127
128        A list of weights must be passed, containing the same number of items than buffer-sz,
129        so each item in this list will be the weight of the packet size on the same position
130        in the buffer-sz parameter (a list have to be passed in that parameter).
131
132        Example:
133
134        To test a distribution of 20% packets of 64 bytes, 40% packets of 100 bytes and 40% packets
135        of 256 bytes, the command line would be: ``--buffer-sz 64,100,256 --imix 20,40,40``.
136        Note that the weights do not have to be percentages, so using ``--imix 1,2,2`` would result
137        in the same distribution
138
139* ``--segment-sz <n>``
140
141        Set the size of the segment to use, for Scatter Gather List testing.
142        By default, it is set to the size of the maximum buffer size, including the digest size,
143        so a single segment is created.
144
145* ``--devtype <name>``
146
147        Set device type, where ``name`` is one of the following::
148
149           crypto_aesni_gcm
150           crypto_aesni_mb
151           crypto_armv8
152           crypto_cn9k
153           crypto_cn10k
154           crypto_dpaa_sec
155           crypto_dpaa2_sec
156           crypto_kasumi
157           crypto_mvsam
158           crypto_null
159           crypto_octeontx
160           crypto_openssl
161           crypto_qat
162           crypto_scheduler
163           crypto_snow3g
164           crypto_zuc
165
166* ``--optype <name>``
167
168        Set operation type, where ``name`` is one of the following::
169
170           cipher-only
171           auth-only
172           cipher-then-auth
173           auth-then-cipher
174           aead
175           pdcp
176           docsis
177           modex
178           ipsec
179           tls-record
180
181        For GCM/CCM algorithms you should use aead flag.
182
183* ``--sessionless``
184
185        Enable session-less crypto operations mode.
186
187* ``--out-of-place``
188
189        Enable out-of-place crypto operations mode.
190
191* ``--test-file <name>``
192
193        Set test vector file path. See the Test Vector File chapter.
194
195* ``--test-name <name>``
196
197        Set specific test name section in the test vector file.
198
199* ``--cipher-algo <name>``
200
201        Set cipher algorithm name, where ``name`` is one of the following::
202
203           3des-cbc
204           3des-ecb
205           3des-ctr
206           aes-cbc
207           aes-ctr
208           aes-ecb
209           aes-f8
210           aes-xts
211           arc4
212           null
213           kasumi-f8
214           snow3g-uea2
215           zuc-eea3
216
217* ``--cipher-op <mode>``
218
219        Set cipher operation mode, where ``mode`` is one of the following::
220
221           encrypt
222           decrypt
223
224* ``--cipher-key-sz <n>``
225
226        Set the size of cipher key.
227
228* ``--cipher-iv-sz <n>``
229
230        Set the size of cipher iv.
231
232* ``--auth-algo <name>``
233
234        Set authentication algorithm name, where ``name`` is one
235        of the following::
236
237           aes-cbc-mac
238           aes-cmac
239           aes-gmac
240           aes-xcbc-mac
241           md5
242           md5-hmac
243           sha1
244           sha1-hmac
245           sha2-224
246           sha2-224-hmac
247           sha2-256
248           sha2-256-hmac
249           sha2-384
250           sha2-384-hmac
251           sha2-512
252           sha2-512-hmac
253           kasumi-f9
254           snow3g-uia2
255           zuc-eia3
256
257* ``--auth-op <mode>``
258
259        Set authentication operation mode, where ``mode`` is one of
260        the following::
261
262           verify
263           generate
264
265* ``--auth-key-sz <n>``
266
267        Set the size of authentication key.
268
269* ``--auth-iv-sz <n>``
270
271        Set the size of auth iv.
272
273* ``--aead-algo <name>``
274
275        Set AEAD algorithm name, where ``name`` is one
276        of the following::
277
278           aes-ccm
279           aes-gcm
280
281* ``--aead-op <mode>``
282
283        Set AEAD operation mode, where ``mode`` is one of
284        the following::
285
286           encrypt
287           decrypt
288
289* ``--aead-key-sz <n>``
290
291        Set the size of AEAD key.
292
293* ``--aead-iv-sz <n>``
294
295        Set the size of AEAD iv.
296
297* ``--aead-aad-sz <n>``
298
299        Set the size of AEAD aad.
300
301* ``--digest-sz <n>``
302
303        Set the size of digest.
304
305* ``--desc-nb <n>``
306
307        Set number of descriptors for each crypto device.
308
309* ``--pmd-cyclecount-delay-ms <n>``
310
311        Add a delay (in milliseconds) between enqueue and dequeue in
312        pmd-cyclecount benchmarking mode (useful when benchmarking
313        hardware acceleration).
314
315* ``--csv-friendly``
316
317        Enable test result output CSV friendly rather than human friendly.
318
319* ``--pdcp-sn-sz <n>``
320
321        Set PDCP sequence number size(n) in bits. Valid values of n will
322        be 5/7/12/15/18.
323
324* ``--pdcp-domain <control/user>``
325
326        Set PDCP domain to specify short_mac/control/user plane.
327
328* ``--docsis-hdr-sz <n>``
329
330        Set DOCSIS header size(n) in bytes.
331
332* ``--pdcp-ses-hfn-en``
333
334        Enable fixed session based HFN instead of per packet HFN.
335
336* ``--enable-sdap``
337
338        Enable Service Data Adaptation Protocol.
339
340* ``--modex-len <n>``
341
342        Set modex length for asymmetric crypto perf test.
343        Supported lengths are 60, 128, 255, 448. Default length is 128.
344
345* ``--tls-version <TLS1.2/TLS1.3/DTLS1.2>``
346
347        Set TLS/DTLS protocol version for perf test (default is TLS1.2).
348
349Test Vector File
350~~~~~~~~~~~~~~~~
351
352The test vector file is a text file contain information about test vectors.
353The file is made of the sections. The first section doesn't have header.
354It contain global information used in each test variant vectors -
355typically information about plaintext, ciphertext, cipher key, auth key,
356initial vector. All other sections begin header.
357The sections contain particular information typically digest.
358
359**Format of the file:**
360
361Each line beginning with sign '#' contain comment and it is ignored by parser::
362
363   # <comment>
364
365Header line is just name in square bracket::
366
367   [<section name>]
368
369Data line contain information token then sign '=' and
370a string of bytes in C byte array format::
371
372   <token> = <C byte array>
373
374**Tokens list:**
375
376* ``plaintext``
377
378        Original plaintext to be encrypted.
379
380* ``ciphertext``
381
382        Encrypted plaintext string.
383
384* ``cipher_key``
385
386        Key used in cipher operation.
387
388* ``auth_key``
389
390        Key used in auth operation.
391
392* ``cipher_iv``
393
394        Cipher Initial Vector.
395
396* ``auth_iv``
397
398        Auth Initial Vector.
399
400* ``aad``
401
402        Additional data.
403
404* ``digest``
405
406        Digest string.
407
408Examples
409--------
410
411Call application for performance throughput test of single Aesni MB PMD
412for cipher encryption aes-cbc and auth generation sha1-hmac,
413one million operations, burst size 32, packet size 64::
414
415   dpdk-test-crypto-perf -l 6-7 --vdev crypto_aesni_mb -a 0000:00:00.0 --
416   --ptest throughput --devtype crypto_aesni_mb --optype cipher-then-auth
417   --cipher-algo aes-cbc --cipher-op encrypt --cipher-key-sz 16 --auth-algo
418   sha1-hmac --auth-op generate --auth-key-sz 64 --digest-sz 12
419   --total-ops 10000000 --burst-sz 32 --buffer-sz 64
420
421Call application for performance latency test of two Aesni MB PMD executed
422on two cores for cipher encryption aes-cbc, ten operations in silent mode::
423
424   dpdk-test-crypto-perf -l 4-7 --vdev crypto_aesni_mb1
425   --vdev crypto_aesni_mb2 -a 0000:00:00.0 -- --devtype crypto_aesni_mb
426   --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16
427   --cipher-op encrypt --optype cipher-only --silent
428   --ptest latency --total-ops 10
429
430Call application for verification test of single open ssl PMD
431for cipher encryption aes-gcm and auth generation aes-gcm,ten operations
432in silent mode, test vector provide in file "test_aes_gcm.data"
433with packet verification::
434
435   dpdk-test-crypto-perf -l 4-7 --vdev crypto_openssl -a 0000:00:00.0 --
436   --devtype crypto_openssl --aead-algo aes-gcm --aead-key-sz 16
437   --aead-iv-sz 16 --aead-op encrypt --aead-aad-sz 16 --digest-sz 16
438   --optype aead --silent --ptest verify --total-ops 10
439   --test-file test_aes_gcm.data
440
441Test vector file for cipher algorithm aes cbc 256 with authorization sha::
442
443   # Global Section
444   plaintext =
445   0xff, 0xca, 0xfb, 0xf1, 0x38, 0x20, 0x2f, 0x7b, 0x24, 0x98, 0x26, 0x7d, 0x1d, 0x9f, 0xb3, 0x93,
446   0xd9, 0xef, 0xbd, 0xad, 0x4e, 0x40, 0xbd, 0x60, 0xe9, 0x48, 0x59, 0x90, 0x67, 0xd7, 0x2b, 0x7b,
447   0x8a, 0xe0, 0x4d, 0xb0, 0x70, 0x38, 0xcc, 0x48, 0x61, 0x7d, 0xee, 0xd6, 0x35, 0x49, 0xae, 0xb4,
448   0xaf, 0x6b, 0xdd, 0xe6, 0x21, 0xc0, 0x60, 0xce, 0x0a, 0xf4, 0x1c, 0x2e, 0x1c, 0x8d, 0xe8, 0x7b
449   ciphertext =
450   0x77, 0xF9, 0xF7, 0x7A, 0xA3, 0xCB, 0x68, 0x1A, 0x11, 0x70, 0xD8, 0x7A, 0xB6, 0xE2, 0x37, 0x7E,
451   0xD1, 0x57, 0x1C, 0x8E, 0x85, 0xD8, 0x08, 0xBF, 0x57, 0x1F, 0x21, 0x6C, 0xAD, 0xAD, 0x47, 0x1E,
452   0x0D, 0x6B, 0x79, 0x39, 0x15, 0x4E, 0x5B, 0x59, 0x2D, 0x76, 0x87, 0xA6, 0xD6, 0x47, 0x8F, 0x82,
453   0xB8, 0x51, 0x91, 0x32, 0x60, 0xCB, 0x97, 0xDE, 0xBE, 0xF0, 0xAD, 0xFC, 0x23, 0x2E, 0x22, 0x02
454   cipher_key =
455   0xE4, 0x23, 0x33, 0x8A, 0x35, 0x64, 0x61, 0xE2, 0x49, 0x03, 0xDD, 0xC6, 0xB8, 0xCA, 0x55, 0x7A,
456   0xd0, 0xe7, 0x4b, 0xfb, 0x5d, 0xe5, 0x0c, 0xe7, 0x6f, 0x21, 0xb5, 0x52, 0x2a, 0xbb, 0xc7, 0xf7
457   auth_key =
458   0xaf, 0x96, 0x42, 0xf1, 0x8c, 0x50, 0xdc, 0x67, 0x1a, 0x43, 0x47, 0x62, 0xc7, 0x04, 0xab, 0x05,
459   0xf5, 0x0c, 0xe7, 0xa2, 0xa6, 0x23, 0xd5, 0x3d, 0x95, 0xd8, 0xcd, 0x86, 0x79, 0xf5, 0x01, 0x47,
460   0x4f, 0xf9, 0x1d, 0x9d, 0x36, 0xf7, 0x68, 0x1a, 0x64, 0x44, 0x58, 0x5d, 0xe5, 0x81, 0x15, 0x2a,
461   0x41, 0xe4, 0x0e, 0xaa, 0x1f, 0x04, 0x21, 0xff, 0x2c, 0xf3, 0x73, 0x2b, 0x48, 0x1e, 0xd2, 0xf7
462   cipher_iv =
463   0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F
464   # Section sha 1 hmac buff 32
465   [sha1_hmac_buff_32]
466   digest =
467   0x36, 0xCA, 0x49, 0x6A, 0xE3, 0x54, 0xD8, 0x4F, 0x0B, 0x76, 0xD8, 0xAA, 0x78, 0xEB, 0x9D, 0x65,
468   0x2C, 0xCA, 0x1F, 0x97
469   # Section sha 256 hmac buff 32
470   [sha256_hmac_buff_32]
471   digest =
472   0x1C, 0xB2, 0x3D, 0xD1, 0xF9, 0xC7, 0x6C, 0x49, 0x2E, 0xDA, 0x94, 0x8B, 0xF1, 0xCF, 0x96, 0x43,
473   0x67, 0x50, 0x39, 0x76, 0xB5, 0xA1, 0xCE, 0xA1, 0xD7, 0x77, 0x10, 0x07, 0x43, 0x37, 0x05, 0xB4
474
475
476Graph Crypto Perf Results
477-------------------------
478
479The ``dpdk-graph-crypto-perf.py`` tool is a simple script to automate
480running crypto performance tests, and graphing the results.
481It can be found in the ``app/test-crypto-perf/`` directory.
482The output graphs include various grouped barcharts for throughput
483tests, and histogram and boxplot graphs for latency tests.
484These are output to PDF files, with one PDF per test suite graph type.
485
486
487Dependencies
488~~~~~~~~~~~~
489
490The following python modules must be installed to run the script:
491
492.. code-block:: console
493
494   pip3 install img2pdf plotly pandas psutil kaleido
495
496
497Test Configuration
498~~~~~~~~~~~~~~~~~~
499
500The test cases run by the script are defined by a JSON config file.
501Some config files can be found in ``app/test-crypto-perf/configs/``,
502or the user may create a new one following the same format as the config files provided.
503
504An example of this format is shown below for one test suite in the ``crypto-perf-aesni-mb.json`` file.
505This shows the required default config for the test suite, and one test case.
506The test case has additional app config that will be combined with
507the default config when running the test case.
508
509.. code-block:: c
510
511   "throughput": {
512       "default": {
513           "eal": {
514               "l": "1,2",
515               "vdev": "crypto_aesni_mb"
516           },
517           "app": {
518               "csv-friendly": true,
519               "buffer-sz": "64,128,256,512,768,1024,1408,2048",
520               "burst-sz": "1,4,8,16,32",
521               "ptest": "throughput",
522               "devtype": "crypto_aesni_mb"
523           }
524        },
525       "AES-CBC-128 SHA1-HMAC auth-then-cipher decrypt": {
526               "cipher-algo": "aes-cbc",
527               "cipher-key-sz": "16",
528               "auth-algo": "sha1-hmac",
529               "optype": "auth-then-cipher",
530               "cipher-op": "decrypt"
531        }
532   }
533
534.. note::
535   The specific test cases only allow modification of app parameters,
536   and not EAL parameters.
537   The default case is required for each test suite in the config file,
538   to specify EAL parameters.
539
540Currently, crypto_qat, crypto_aesni_mb, and crypto_aesni_gcm devices for
541both throughput and latency ptests are supported.
542
543
544Usage
545~~~~~
546
547.. code-block:: console
548
549   ./dpdk-graph-crypto-perf <config_file>
550
551The ``config_file`` positional argument is required to run the script.
552This points to a valid JSON config file containing test suites.
553
554.. code-block:: console
555
556   ./dpdk-graph-crypto-perf configs/crypto-perf-aesni-mb.json
557
558The following are the application optional command-line options:
559
560* ``-h, --help``
561
562  Display usage information and quit.
563
564* ``-f <file_path>, --file-path <file_path>``
565
566  Provide path to ``dpdk-test-crypto-perf`` application.
567  The script uses the installed app by default.
568
569  .. code-block:: console
570
571     ./dpdk-graph-crypto-perf <config_file> \
572         -f <build_dir>/app/dpdk-test-crypto-perf
573
574* ``-t <test_suite_list>, --test-suites <test_suite_list>``
575
576  Specify test suites to run. All test suites are run by default.
577
578  To run crypto-perf-qat latency test suite only:
579
580  .. code-block:: console
581
582     ./dpdk-graph-crypto-perf configs/crypto-perf-qat -t latency
583
584  To run both crypto-perf-aesni-mb throughput and latency test suites
585
586  .. code-block:: console
587
588     ./dpdk-graph-crypto-perf configs/crypto-perf-aesni-mb -t throughput latency
589
590* ``-o <output_path>, --output-path <output_path>``
591
592  Specify directory to use for output files.
593  The default is to use the script's directory.
594
595  .. code-block:: console
596
597     ./dpdk-graph-crypto-perf <config_file> -o <output_dir>
598
599* ``-v, --verbose``
600
601  Enable verbose output. This displays ``dpdk-test-crypto-perf`` app output in real-time.
602
603  .. code-block:: console
604
605     ./dpdk-graph-crypto-perf <config_file> -v
606
607  .. warning::
608     Latency performance tests have a large amount of output.
609     It is not recommended to use the verbose option for latency tests.
610