1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation. 3 4Cryptodev Scheduler Poll Mode Driver Library 5============================================ 6 7Scheduler PMD is a software crypto PMD, which has the capabilities of 8attaching hardware and/or software cryptodevs, and distributes ingress 9crypto ops among them in a certain manner. 10 11.. figure:: img/scheduler-overview.* 12 13 Cryptodev Scheduler Overview 14 15 16The Cryptodev Scheduler PMD library (**librte_pmd_crypto_scheduler**) acts as 17a software crypto PMD and shares the same API provided by librte_cryptodev. 18The PMD supports attaching multiple crypto PMDs, software or hardware, as 19slaves, and distributes the crypto workload to them with certain behavior. 20The behaviors are categorizes as different "modes". Basically, a scheduling 21mode defines certain actions for scheduling crypto ops to its slaves. 22 23The librte_pmd_crypto_scheduler library exports a C API which provides an API 24for attaching/detaching slaves, set/get scheduling modes, and enable/disable 25crypto ops reordering. 26 27Limitations 28----------- 29 30* Sessionless crypto operation is not supported 31* OOP crypto operation is not supported when the crypto op reordering feature 32 is enabled. 33 34 35Installation 36------------ 37 38To build DPDK with CRYTPO_SCHEDULER_PMD the user is required to set 39CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y in config/common_base, and 40recompile DPDK 41 42 43Initialization 44-------------- 45 46To use the PMD in an application, user must: 47 48* Call rte_vdev_init("crypto_scheduler") within the application. 49 50* Use --vdev="crypto_scheduler" in the EAL options, which will call 51 rte_vdev_init() internally. 52 53 54The following parameters (all optional) can be provided in the previous 55two calls: 56 57* socket_id: Specify the socket where the memory for the device is going 58 to be allocated (by default, socket_id will be the socket where the core 59 that is creating the PMD is running on). 60 61* max_nb_sessions: Specify the maximum number of sessions that can be 62 created. This value may be overwritten internally if there are too 63 many devices are attached. 64 65* slave: If a cryptodev has been initialized with specific name, it can be 66 attached to the scheduler using this parameter, simply filling the name 67 here. Multiple cryptodevs can be attached initially by presenting this 68 parameter multiple times. 69 70* mode: Specify the scheduling mode of the PMD. The supported scheduling 71 mode parameter values are specified in the "Cryptodev Scheduler Modes 72 Overview" section. 73 74* ordering: Specify the status of the crypto operations ordering feature. 75 The value of this parameter can be "enable" or "disable". This feature 76 is disabled by default. 77 78Example: 79 80.. code-block:: console 81 82 ... --vdev "crypto_aesni_mb0,name=aesni_mb_1" --vdev "crypto_aesni_mb1,name=aesni_mb_2" --vdev "crypto_scheduler,slave=aesni_mb_1,slave=aesni_mb_2" ... 83 84.. note:: 85 86 * The scheduler cryptodev cannot be started unless the scheduling mode 87 is set and at least one slave is attached. Also, to configure the 88 scheduler in the run-time, like attach/detach slave(s), change 89 scheduling mode, or enable/disable crypto op ordering, one should stop 90 the scheduler first, otherwise an error will be returned. 91 92 * The crypto op reordering feature requires using the userdata field of 93 every mbuf to be processed to store temporary data. By the end of 94 processing, the field is set to pointing to NULL, any previously 95 stored value of this field will be lost. 96 97 98Cryptodev Scheduler Modes Overview 99---------------------------------- 100 101Currently the Crypto Scheduler PMD library supports following modes of 102operation: 103 104* **CDEV_SCHED_MODE_ROUNDROBIN:** 105 106 *Initialization mode parameter*: **round-robin** 107 108 Round-robin mode, which distributes the enqueued burst of crypto ops 109 among its slaves in a round-robin manner. This mode may help to fill 110 the throughput gap between the physical core and the existing cryptodevs 111 to increase the overall performance. 112 113* **CDEV_SCHED_MODE_PKT_SIZE_DISTR:** 114 115 *Initialization mode parameter*: **packet-size-distr** 116 117 Packet-size based distribution mode, which works with 2 slaves, the primary 118 slave and the secondary slave, and distributes the enqueued crypto 119 operations to them based on their data lengths. A crypto operation will be 120 distributed to the primary slave if its data length is equal to or bigger 121 than the designated threshold, otherwise it will be handled by the secondary 122 slave. 123 124 A typical usecase in this mode is with the QAT cryptodev as the primary and 125 a software cryptodev as the secondary slave. This may help applications to 126 process additional crypto workload than what the QAT cryptodev can handle on 127 its own, by making use of the available CPU cycles to deal with smaller 128 crypto workloads. 129 130 The threshold is set to 128 bytes by default. It can be updated by calling 131 function **rte_cryptodev_scheduler_option_set**. The parameter of 132 **option_type** must be **CDEV_SCHED_OPTION_THRESHOLD** and **option** should 133 point to a rte_cryptodev_scheduler_threshold_option structure filled with 134 appropriate threshold value. Please NOTE this threshold has be a power-of-2 135 unsigned integer. 136 137* **CDEV_SCHED_MODE_FAILOVER:** 138 139 *Initialization mode parameter*: **fail-over** 140 141 Fail-over mode, which works with 2 slaves, the primary slave and the 142 secondary slave. In this mode, the scheduler will enqueue the incoming 143 crypto operation burst to the primary slave. When one or more crypto 144 operations fail to be enqueued, then they will be enqueued to the secondary 145 slave. 146 147* **CDEV_SCHED_MODE_MULTICORE:** 148 149 *Initialization mode parameter*: **multi-core** 150 151 Multi-core mode, which distributes the workload with several (up to eight) 152 worker cores. The enqueued bursts are distributed among the worker cores in a 153 round-robin manner. If scheduler cannot enqueue entire burst to the same worker, 154 it will enqueue the remaining operations to the next available worker. 155 For pure small packet size (64 bytes) traffic however the multi-core mode is not 156 an optimal solution, as it doesn't give significant per-core performance improvement. 157 For mixed traffic (IMIX) the optimal number of worker cores is around 2-3. 158 For large packets (1.5 Kbytes) scheduler shows linear scaling in performance 159 up to eight cores. 160 Each worker uses its own slave cryptodev. Only software cryptodevs 161 are supported. Only the same type of cryptodevs should be used concurrently. 162 163 The multi-core mode uses one extra parameter: 164 165 * corelist: Semicolon-separated list of logical cores to be used as workers. 166 The number of worker cores should be equal to the number of slave cryptodevs. 167 These cores should be present in EAL core list parameter and 168 should not be used by the application or any other process. 169 170 Example: 171 ... --vdev "crypto_aesni_mb1,name=aesni_mb_1" --vdev "crypto_aesni_mb_pmd2,name=aesni_mb_2" \ 172 --vdev "crypto_scheduler,slave=aesni_mb_1,slave=aesni_mb_2,mode=multi-core,corelist=23;24" ... 173