1.. BSD LICENSE 2 Copyright(c) 2016 Intel Corporation. All rights reserved. 3 4 Redistribution and use in source and binary forms, with or without 5 modification, are permitted provided that the following conditions 6 are met: 7 8 * Redistributions of source code must retain the above copyright 9 notice, this list of conditions and the following disclaimer. 10 * Redistributions in binary form must reproduce the above copyright 11 notice, this list of conditions and the following disclaimer in 12 the documentation and/or other materials provided with the 13 distribution. 14 * Neither the name of Intel Corporation nor the names of its 15 contributors may be used to endorse or promote products derived 16 from this software without specific prior written permission. 17 18 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 19 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 20 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 21 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 22 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 23 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 24 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 25 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 26 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 28 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 30 31Cryptography Device Library 32=========================== 33 34The cryptodev library provides a Crypto device framework for management and 35provisioning of hardware and software Crypto poll mode drivers, defining generic 36APIs which support a number of different Crypto operations. The framework 37currently only supports cipher, authentication, chained cipher/authentication 38and AEAD symmetric Crypto operations. 39 40 41Design Principles 42----------------- 43 44The cryptodev library follows the same basic principles as those used in DPDKs 45Ethernet Device framework. The Crypto framework provides a generic Crypto device 46framework which supports both physical (hardware) and virtual (software) Crypto 47devices as well as a generic Crypto API which allows Crypto devices to be 48managed and configured and supports Crypto operations to be provisioned on 49Crypto poll mode driver. 50 51 52Device Management 53----------------- 54 55Device Creation 56~~~~~~~~~~~~~~~ 57 58Physical Crypto devices are discovered during the PCI probe/enumeration of the 59EAL function which is executed at DPDK initialization, based on 60their PCI device identifier, each unique PCI BDF (bus/bridge, device, 61function). Specific physical Crypto devices, like other physical devices in DPDK 62can be white-listed or black-listed using the EAL command line options. 63 64Virtual devices can be created by two mechanisms, either using the EAL command 65line options or from within the application using an EAL API directly. 66 67From the command line using the --vdev EAL option 68 69.. code-block:: console 70 71 --vdev 'cryptodev_aesni_mb_pmd0,max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0' 72 73Our using the rte_vdev_init API within the application code. 74 75.. code-block:: c 76 77 rte_vdev_init("cryptodev_aesni_mb_pmd", 78 "max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0") 79 80All virtual Crypto devices support the following initialization parameters: 81 82* ``max_nb_queue_pairs`` - maximum number of queue pairs supported by the device. 83* ``max_nb_sessions`` - maximum number of sessions supported by the device 84* ``socket_id`` - socket on which to allocate the device resources on. 85 86 87Device Identification 88~~~~~~~~~~~~~~~~~~~~~ 89 90Each device, whether virtual or physical is uniquely designated by two 91identifiers: 92 93- A unique device index used to designate the Crypto device in all functions 94 exported by the cryptodev API. 95 96- A device name used to designate the Crypto device in console messages, for 97 administration or debugging purposes. For ease of use, the port name includes 98 the port index. 99 100 101Device Configuration 102~~~~~~~~~~~~~~~~~~~~ 103 104The configuration of each Crypto device includes the following operations: 105 106- Allocation of resources, including hardware resources if a physical device. 107- Resetting the device into a well-known default state. 108- Initialization of statistics counters. 109 110The rte_cryptodev_configure API is used to configure a Crypto device. 111 112.. code-block:: c 113 114 int rte_cryptodev_configure(uint8_t dev_id, 115 struct rte_cryptodev_config *config) 116 117The ``rte_cryptodev_config`` structure is used to pass the configuration parameters. 118In contains parameter for socket selection, number of queue pairs and the 119session mempool configuration. 120 121.. code-block:: c 122 123 struct rte_cryptodev_config { 124 int socket_id; 125 /**< Socket to allocate resources on */ 126 uint16_t nb_queue_pairs; 127 /**< Number of queue pairs to configure on device */ 128 129 struct { 130 uint32_t nb_objs; 131 uint32_t cache_size; 132 } session_mp; 133 /**< Session mempool configuration */ 134 }; 135 136 137Configuration of Queue Pairs 138~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 139 140Each Crypto devices queue pair is individually configured through the 141``rte_cryptodev_queue_pair_setup`` API. 142Each queue pairs resources may be allocated on a specified socket. 143 144.. code-block:: c 145 146 int rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, 147 const struct rte_cryptodev_qp_conf *qp_conf, 148 int socket_id) 149 150 struct rte_cryptodev_qp_conf { 151 uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ 152 }; 153 154 155Logical Cores, Memory and Queues Pair Relationships 156~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 157 158The Crypto device Library as the Poll Mode Driver library support NUMA for when 159a processor’s logical cores and interfaces utilize its local memory. Therefore 160Crypto operations, and in the case of symmetric Crypto operations, the session 161and the mbuf being operated on, should be allocated from memory pools created 162in the local memory. The buffers should, if possible, remain on the local 163processor to obtain the best performance results and buffer descriptors should 164be populated with mbufs allocated from a mempool allocated from local memory. 165 166The run-to-completion model also performs better, especially in the case of 167virtual Crypto devices, if the Crypto operation and session and data buffer is 168in local memory instead of a remote processor's memory. This is also true for 169the pipe-line model provided all logical cores used are located on the same 170processor. 171 172Multiple logical cores should never share the same queue pair for enqueuing 173operations or dequeuing operations on the same Crypto device since this would 174require global locks and hinder performance. It is however possible to use a 175different logical core to dequeue an operation on a queue pair from the logical 176core which it was enqueued on. This means that a crypto burst enqueue/dequeue 177APIs are a logical place to transition from one logical core to another in a 178packet processing pipeline. 179 180 181Device Features and Capabilities 182--------------------------------- 183 184Crypto devices define their functionality through two mechanisms, global device 185features and algorithm capabilities. Global devices features identify device 186wide level features which are applicable to the whole device such as 187the device having hardware acceleration or supporting symmetric Crypto 188operations, 189 190The capabilities mechanism defines the individual algorithms/functions which 191the device supports, such as a specific symmetric Crypto cipher or 192authentication operation. 193 194 195Device Features 196~~~~~~~~~~~~~~~ 197 198Currently the following Crypto device features are defined: 199 200* Symmetric Crypto operations 201* Asymmetric Crypto operations 202* Chaining of symmetric Crypto operations 203* SSE accelerated SIMD vector operations 204* AVX accelerated SIMD vector operations 205* AVX2 accelerated SIMD vector operations 206* AESNI accelerated instructions 207* Hardware off-load processing 208 209 210Device Operation Capabilities 211~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 212 213Crypto capabilities which identify particular algorithm which the Crypto PMD 214supports are defined by the operation type, the operation transform, the 215transform identifier and then the particulars of the transform. For the full 216scope of the Crypto capability see the definition of the structure in the 217*DPDK API Reference*. 218 219.. code-block:: c 220 221 struct rte_cryptodev_capabilities; 222 223Each Crypto poll mode driver defines its own private array of capabilities 224for the operations it supports. Below is an example of the capabilities for a 225PMD which supports the authentication algorithm SHA1_HMAC and the cipher 226algorithm AES_CBC. 227 228.. code-block:: c 229 230 static const struct rte_cryptodev_capabilities pmd_capabilities[] = { 231 { /* SHA1 HMAC */ 232 .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, 233 .sym = { 234 .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, 235 .auth = { 236 .algo = RTE_CRYPTO_AUTH_SHA1_HMAC, 237 .block_size = 64, 238 .key_size = { 239 .min = 64, 240 .max = 64, 241 .increment = 0 242 }, 243 .digest_size = { 244 .min = 12, 245 .max = 12, 246 .increment = 0 247 }, 248 .aad_size = { 0 } 249 } 250 } 251 }, 252 { /* AES CBC */ 253 .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, 254 .sym = { 255 .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, 256 .cipher = { 257 .algo = RTE_CRYPTO_CIPHER_AES_CBC, 258 .block_size = 16, 259 .key_size = { 260 .min = 16, 261 .max = 32, 262 .increment = 8 263 }, 264 .iv_size = { 265 .min = 16, 266 .max = 16, 267 .increment = 0 268 } 269 } 270 } 271 } 272 } 273 274 275Capabilities Discovery 276~~~~~~~~~~~~~~~~~~~~~~ 277 278Discovering the features and capabilities of a Crypto device poll mode driver 279is achieved through the ``rte_cryptodev_info_get`` function. 280 281.. code-block:: c 282 283 void rte_cryptodev_info_get(uint8_t dev_id, 284 struct rte_cryptodev_info *dev_info); 285 286This allows the user to query a specific Crypto PMD and get all the device 287features and capabilities. The ``rte_cryptodev_info`` structure contains all the 288relevant information for the device. 289 290.. code-block:: c 291 292 struct rte_cryptodev_info { 293 const char *driver_name; 294 enum rte_cryptodev_type dev_type; 295 struct rte_pci_device *pci_dev; 296 297 uint64_t feature_flags; 298 299 const struct rte_cryptodev_capabilities *capabilities; 300 301 unsigned max_nb_queue_pairs; 302 303 struct { 304 unsigned max_nb_sessions; 305 } sym; 306 }; 307 308 309Operation Processing 310-------------------- 311 312Scheduling of Crypto operations on DPDK's application data path is 313performed using a burst oriented asynchronous API set. A queue pair on a Crypto 314device accepts a burst of Crypto operations using enqueue burst API. On physical 315Crypto devices the enqueue burst API will place the operations to be processed 316on the devices hardware input queue, for virtual devices the processing of the 317Crypto operations is usually completed during the enqueue call to the Crypto 318device. The dequeue burst API will retrieve any processed operations available 319from the queue pair on the Crypto device, from physical devices this is usually 320directly from the devices processed queue, and for virtual device's from a 321``rte_ring`` where processed operations are place after being processed on the 322enqueue call. 323 324 325Enqueue / Dequeue Burst APIs 326~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 327 328The burst enqueue API uses a Crypto device identifier and a queue pair 329identifier to specify the Crypto device queue pair to schedule the processing on. 330The ``nb_ops`` parameter is the number of operations to process which are 331supplied in the ``ops`` array of ``rte_crypto_op`` structures. 332The enqueue function returns the number of operations it actually enqueued for 333processing, a return value equal to ``nb_ops`` means that all packets have been 334enqueued. 335 336.. code-block:: c 337 338 uint16_t rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, 339 struct rte_crypto_op **ops, uint16_t nb_ops) 340 341The dequeue API uses the same format as the enqueue API of processed but 342the ``nb_ops`` and ``ops`` parameters are now used to specify the max processed 343operations the user wishes to retrieve and the location in which to store them. 344The API call returns the actual number of processed operations returned, this 345can never be larger than ``nb_ops``. 346 347.. code-block:: c 348 349 uint16_t rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, 350 struct rte_crypto_op **ops, uint16_t nb_ops) 351 352 353Operation Representation 354~~~~~~~~~~~~~~~~~~~~~~~~ 355 356An Crypto operation is represented by an rte_crypto_op structure, which is a 357generic metadata container for all necessary information required for the 358Crypto operation to be processed on a particular Crypto device poll mode driver. 359 360.. figure:: img/crypto_op.* 361 362The operation structure includes the operation type and the operation status, 363a reference to the operation specific data, which can vary in size and content 364depending on the operation being provisioned. It also contains the source 365mempool for the operation, if it allocate from a mempool. Finally an 366opaque pointer for user specific data is provided. 367 368If Crypto operations are allocated from a Crypto operation mempool, see next 369section, there is also the ability to allocate private memory with the 370operation for applications purposes. 371 372Application software is responsible for specifying all the operation specific 373fields in the ``rte_crypto_op`` structure which are then used by the Crypto PMD 374to process the requested operation. 375 376 377Operation Management and Allocation 378~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 379 380The cryptodev library provides an API set for managing Crypto operations which 381utilize the Mempool Library to allocate operation buffers. Therefore, it ensures 382that the crytpo operation is interleaved optimally across the channels and 383ranks for optimal processing. 384A ``rte_crypto_op`` contains a field indicating the pool that it originated from. 385When calling ``rte_crypto_op_free(op)``, the operation returns to its original pool. 386 387.. code-block:: c 388 389 extern struct rte_mempool * 390 rte_crypto_op_pool_create(const char *name, enum rte_crypto_op_type type, 391 unsigned nb_elts, unsigned cache_size, uint16_t priv_size, 392 int socket_id); 393 394During pool creation ``rte_crypto_op_init()`` is called as a constructor to 395initialize each Crypto operation which subsequently calls 396``__rte_crypto_op_reset()`` to configure any operation type specific fields based 397on the type parameter. 398 399 400``rte_crypto_op_alloc()`` and ``rte_crypto_op_bulk_alloc()`` are used to allocate 401Crypto operations of a specific type from a given Crypto operation mempool. 402``__rte_crypto_op_reset()`` is called on each operation before being returned to 403allocate to a user so the operation is always in a good known state before use 404by the application. 405 406.. code-block:: c 407 408 struct rte_crypto_op *rte_crypto_op_alloc(struct rte_mempool *mempool, 409 enum rte_crypto_op_type type) 410 411 unsigned rte_crypto_op_bulk_alloc(struct rte_mempool *mempool, 412 enum rte_crypto_op_type type, 413 struct rte_crypto_op **ops, uint16_t nb_ops) 414 415``rte_crypto_op_free()`` is called by the application to return an operation to 416its allocating pool. 417 418.. code-block:: c 419 420 void rte_crypto_op_free(struct rte_crypto_op *op) 421 422 423Symmetric Cryptography Support 424------------------------------ 425 426The cryptodev library currently provides support for the following symmetric 427Crypto operations; cipher, authentication, including chaining of these 428operations, as well as also supporting AEAD operations. 429 430 431Session and Session Management 432~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 433 434Session are used in symmetric cryptographic processing to store the immutable 435data defined in a cryptographic transform which is used in the operation 436processing of a packet flow. Sessions are used to manage information such as 437expand cipher keys and HMAC IPADs and OPADs, which need to be calculated for a 438particular Crypto operation, but are immutable on a packet to packet basis for 439a flow. Crypto sessions cache this immutable data in a optimal way for the 440underlying PMD and this allows further acceleration of the offload of 441Crypto workloads. 442 443.. figure:: img/cryptodev_sym_sess.* 444 445The Crypto device framework provides a set of session pool management APIs for 446the creation and freeing of the sessions, utilizing the Mempool Library. 447 448The framework also provides hooks so the PMDs can pass the amount of memory 449required for that PMDs private session parameters, as well as initialization 450functions for the configuration of the session parameters and freeing function 451so the PMD can managed the memory on destruction of a session. 452 453**Note**: Sessions created on a particular device can only be used on Crypto 454devices of the same type, and if you try to use a session on a device different 455to that on which it was created then the Crypto operation will fail. 456 457``rte_cryptodev_sym_session_create()`` is used to create a symmetric session on 458Crypto device. A symmetric transform chain is used to specify the particular 459operation and its parameters. See the section below for details on transforms. 460 461.. code-block:: c 462 463 struct rte_cryptodev_sym_session * rte_cryptodev_sym_session_create( 464 uint8_t dev_id, struct rte_crypto_sym_xform *xform); 465 466**Note**: For AEAD operations the algorithm selected for authentication and 467ciphering must aligned, eg AES_GCM. 468 469 470Transforms and Transform Chaining 471~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 472 473Symmetric Crypto transforms (``rte_crypto_sym_xform``) are the mechanism used 474to specify the details of the Crypto operation. For chaining of symmetric 475operations such as cipher encrypt and authentication generate, the next pointer 476allows transform to be chained together. Crypto devices which support chaining 477must publish the chaining of symmetric Crypto operations feature flag. 478 479Currently there are two transforms types cipher and authentication, to specify 480an AEAD operation it is required to chain a cipher and an authentication 481transform together. Also it is important to note that the order in which the 482transforms are passed indicates the order of the chaining. 483 484.. code-block:: c 485 486 struct rte_crypto_sym_xform { 487 struct rte_crypto_sym_xform *next; 488 /**< next xform in chain */ 489 enum rte_crypto_sym_xform_type type; 490 /**< xform type */ 491 union { 492 struct rte_crypto_auth_xform auth; 493 /**< Authentication / hash xform */ 494 struct rte_crypto_cipher_xform cipher; 495 /**< Cipher xform */ 496 }; 497 }; 498 499The API does not place a limit on the number of transforms that can be chained 500together but this will be limited by the underlying Crypto device poll mode 501driver which is processing the operation. 502 503.. figure:: img/crypto_xform_chain.* 504 505 506Symmetric Operations 507~~~~~~~~~~~~~~~~~~~~ 508 509The symmetric Crypto operation structure contains all the mutable data relating 510to performing symmetric cryptographic processing on a referenced mbuf data 511buffer. It is used for either cipher, authentication, AEAD and chained 512operations. 513 514As a minimum the symmetric operation must have a source data buffer (``m_src``), 515the session type (session-based/less), a valid session (or transform chain if in 516session-less mode) and the minimum authentication/ cipher parameters required 517depending on the type of operation specified in the session or the transform 518chain. 519 520.. code-block:: c 521 522 struct rte_crypto_sym_op { 523 struct rte_mbuf *m_src; 524 struct rte_mbuf *m_dst; 525 526 enum rte_crypto_sym_op_sess_type type; 527 528 union { 529 struct rte_cryptodev_sym_session *session; 530 /**< Handle for the initialised session context */ 531 struct rte_crypto_sym_xform *xform; 532 /**< Session-less API Crypto operation parameters */ 533 }; 534 535 struct { 536 struct { 537 uint32_t offset; 538 uint32_t length; 539 } data; /**< Data offsets and length for ciphering */ 540 541 struct { 542 uint8_t *data; 543 phys_addr_t phys_addr; 544 uint16_t length; 545 } iv; /**< Initialisation vector parameters */ 546 } cipher; 547 548 struct { 549 struct { 550 uint32_t offset; 551 uint32_t length; 552 } data; /**< Data offsets and length for authentication */ 553 554 struct { 555 uint8_t *data; 556 phys_addr_t phys_addr; 557 uint16_t length; 558 } digest; /**< Digest parameters */ 559 560 struct { 561 uint8_t *data; 562 phys_addr_t phys_addr; 563 uint16_t length; 564 } aad; /**< Additional authentication parameters */ 565 } auth; 566 } 567 568 569Asymmetric Cryptography 570----------------------- 571 572Asymmetric functionality is currently not supported by the cryptodev API. 573 574 575Crypto Device API 576~~~~~~~~~~~~~~~~~ 577 578The cryptodev Library API is described in the *DPDK API Reference* document. 579