1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation. 3 4Cryptodev Scheduler Poll Mode Driver Library 5============================================ 6 7Scheduler PMD is a software crypto PMD, which has the capabilities of 8attaching hardware and/or software cryptodevs, and distributes ingress 9crypto ops among them in a certain manner. 10 11.. figure:: img/scheduler-overview.* 12 13 Cryptodev Scheduler Overview 14 15 16The Cryptodev Scheduler PMD library (**librte_crypto_scheduler**) acts as 17a software crypto PMD and shares the same API provided by librte_cryptodev. 18The PMD supports attaching multiple crypto PMDs, software or hardware, as 19workers, and distributes the crypto workload to them with certain behavior. 20The behaviors are categorizes as different "modes". Basically, a scheduling 21mode defines certain actions for scheduling crypto ops to its workers. 22 23The librte_crypto_scheduler library exports a C API which provides an API 24for attaching/detaching workers, set/get scheduling modes, and enable/disable 25crypto ops reordering. 26 27Limitations 28----------- 29 30* Sessionless crypto operation is not supported 31* OOP crypto operation is not supported when the crypto op reordering feature 32 is enabled. 33 34 35Initialization 36-------------- 37 38To use the PMD in an application, user must: 39 40* Call rte_vdev_init("crypto_scheduler") within the application. 41 42* Use --vdev="crypto_scheduler" in the EAL options, which will call 43 rte_vdev_init() internally. 44 45 46The following parameters (all optional) can be provided in the previous 47two calls: 48 49* socket_id: Specify the socket where the memory for the device is going 50 to be allocated (by default, socket_id will be the socket where the core 51 that is creating the PMD is running on). 52 53* max_nb_sessions: Specify the maximum number of sessions that can be 54 created. This value may be overwritten internally if there are too 55 many devices are attached. 56 57* worker: If a cryptodev has been initialized with specific name, it can be 58 attached to the scheduler using this parameter, simply filling the name 59 here. Multiple cryptodevs can be attached initially by presenting this 60 parameter multiple times. 61 62* mode: Specify the scheduling mode of the PMD. The supported scheduling 63 mode parameter values are specified in the "Cryptodev Scheduler Modes 64 Overview" section. 65 66* mode_param: Specify the mode-specific parameter. Some scheduling modes 67 may be initialized with specific parameters other than the default ones, 68 such as the **threshold** packet size of **packet-size-distr** mode. This 69 parameter fulfills the purpose. 70 71* ordering: Specify the status of the crypto operations ordering feature. 72 The value of this parameter can be "enable" or "disable". This feature 73 is disabled by default. 74 75Example: 76 77.. code-block:: console 78 79 ... --vdev "crypto_aesni_mb0,name=aesni_mb_1" --vdev "crypto_aesni_mb1,name=aesni_mb_2" --vdev "crypto_scheduler,worker=aesni_mb_1,worker=aesni_mb_2" ... 80 81.. note:: 82 83 * The scheduler cryptodev cannot be started unless the scheduling mode 84 is set and at least one worker is attached. Also, to configure the 85 scheduler in the run-time, like attach/detach worker(s), change 86 scheduling mode, or enable/disable crypto op ordering, one should stop 87 the scheduler first, otherwise an error will be returned. 88 89 * The crypto op reordering feature requires using the userdata field of 90 every mbuf to be processed to store temporary data. By the end of 91 processing, the field is set to pointing to NULL, any previously 92 stored value of this field will be lost. 93 94 95Cryptodev Scheduler Modes Overview 96---------------------------------- 97 98Currently the Crypto Scheduler PMD library supports following modes of 99operation: 100 101* **CDEV_SCHED_MODE_ROUNDROBIN:** 102 103 *Initialization mode parameter*: **round-robin** 104 105 Round-robin mode, which distributes the enqueued burst of crypto ops 106 among its workers in a round-robin manner. This mode may help to fill 107 the throughput gap between the physical core and the existing cryptodevs 108 to increase the overall performance. 109 110* **CDEV_SCHED_MODE_PKT_SIZE_DISTR:** 111 112 *Initialization mode parameter*: **packet-size-distr** 113 114 Packet-size based distribution mode, which works with 2 workers, the primary 115 worker and the secondary worker, and distributes the enqueued crypto 116 operations to them based on their data lengths. A crypto operation will be 117 distributed to the primary worker if its data length is equal to or bigger 118 than the designated threshold, otherwise it will be handled by the secondary 119 worker. 120 121 A typical use case in this mode is with the QAT cryptodev as the primary and 122 a software cryptodev as the secondary worker. This may help applications to 123 process additional crypto workload than what the QAT cryptodev can handle on 124 its own, by making use of the available CPU cycles to deal with smaller 125 crypto workloads. 126 127 The threshold is set to 128 bytes by default. It can be updated by calling 128 function **rte_cryptodev_scheduler_option_set**. The parameter of 129 **option_type** must be **CDEV_SCHED_OPTION_THRESHOLD** and **option** should 130 point to a rte_cryptodev_scheduler_threshold_option structure filled with 131 appropriate threshold value. Please NOTE this threshold has be a power-of-2 132 unsigned integer. It is possible to use **mode_param** initialization 133 parameter to achieve the same purpose. For example: 134 135 ... --vdev "crypto_scheduler,mode=packet-size-distr,mode_param=threshold:512" ... 136 137 The above parameter will overwrite the threshold value to 512. 138 139* **CDEV_SCHED_MODE_FAILOVER:** 140 141 *Initialization mode parameter*: **fail-over** 142 143 Fail-over mode, which works with 2 workers, the primary worker and the 144 secondary worker. In this mode, the scheduler will enqueue the incoming 145 crypto operation burst to the primary worker. When one or more crypto 146 operations fail to be enqueued, then they will be enqueued to the secondary 147 worker. 148 149* **CDEV_SCHED_MODE_MULTICORE:** 150 151 *Initialization mode parameter*: **multi-core** 152 153 Multi-core mode, which distributes the workload with several (up to eight) 154 worker cores. The enqueued bursts are distributed among the worker cores in a 155 round-robin manner. If scheduler cannot enqueue entire burst to the same worker, 156 it will enqueue the remaining operations to the next available worker. 157 For pure small packet size (64 bytes) traffic however the multi-core mode is not 158 an optimal solution, as it doesn't give significant per-core performance improvement. 159 For mixed traffic (IMIX) the optimal number of worker cores is around 2-3. 160 For large packets (1.5 kbytes) scheduler shows linear scaling in performance 161 up to eight cores. 162 Each worker uses its own cryptodev. Only software cryptodevs 163 are supported. Only the same type of cryptodevs should be used concurrently. 164 165 The multi-core mode uses one extra parameter: 166 167 * corelist: Semicolon-separated list of logical cores to be used as workers. 168 The number of worker cores should be equal to the number of worker cryptodevs. 169 These cores should be present in EAL core list parameter and 170 should not be used by the application or any other process. 171 172 Example: 173 ... --vdev "crypto_aesni_mb1,name=aesni_mb_1" --vdev "crypto_aesni_mb_pmd2,name=aesni_mb_2" \ 174 --vdev "crypto_scheduler,worker=aesni_mb_1,worker=aesni_mb_2,mode=multi-core,corelist=23;24" ... 175