1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2015-2019 Intel Corporation. 3 4Intel(R) QuickAssist (QAT) Crypto Poll Mode Driver 5================================================== 6 7QAT documentation consists of three parts: 8 9* Details of the symmetric and asymmetric crypto services below. 10* Details of the :doc:`compression service <../compressdevs/qat_comp>` 11 in the compressdev drivers section. 12* Details of building the common QAT infrastructure and the PMDs to support the 13 above services. See :ref:`building_qat` below. 14 15 16Symmetric Crypto Service on QAT 17------------------------------- 18 19The QAT symmetric crypto PMD (hereafter referred to as `QAT SYM [PMD]`) provides 20poll mode crypto driver support for the following hardware accelerator devices: 21 22* ``Intel QuickAssist Technology DH895xCC`` 23* ``Intel QuickAssist Technology C62x`` 24* ``Intel QuickAssist Technology C3xxx`` 25* ``Intel QuickAssist Technology 200xx`` 26* ``Intel QuickAssist Technology D15xx`` 27* ``Intel QuickAssist Technology C4xxx`` 28 29 30Features 31~~~~~~~~ 32 33The QAT SYM PMD has support for: 34 35Cipher algorithms: 36 37* ``RTE_CRYPTO_CIPHER_3DES_CBC`` 38* ``RTE_CRYPTO_CIPHER_3DES_CTR`` 39* ``RTE_CRYPTO_CIPHER_AES128_CBC`` 40* ``RTE_CRYPTO_CIPHER_AES192_CBC`` 41* ``RTE_CRYPTO_CIPHER_AES256_CBC`` 42* ``RTE_CRYPTO_CIPHER_AES128_CTR`` 43* ``RTE_CRYPTO_CIPHER_AES192_CTR`` 44* ``RTE_CRYPTO_CIPHER_AES256_CTR`` 45* ``RTE_CRYPTO_CIPHER_AES_XTS`` 46* ``RTE_CRYPTO_CIPHER_SNOW3G_UEA2`` 47* ``RTE_CRYPTO_CIPHER_NULL`` 48* ``RTE_CRYPTO_CIPHER_KASUMI_F8`` 49* ``RTE_CRYPTO_CIPHER_DES_CBC`` 50* ``RTE_CRYPTO_CIPHER_AES_DOCSISBPI`` 51* ``RTE_CRYPTO_CIPHER_DES_DOCSISBPI`` 52* ``RTE_CRYPTO_CIPHER_ZUC_EEA3`` 53 54Hash algorithms: 55 56* ``RTE_CRYPTO_AUTH_SHA1`` 57* ``RTE_CRYPTO_AUTH_SHA1_HMAC`` 58* ``RTE_CRYPTO_AUTH_SHA224`` 59* ``RTE_CRYPTO_AUTH_SHA224_HMAC`` 60* ``RTE_CRYPTO_AUTH_SHA256`` 61* ``RTE_CRYPTO_AUTH_SHA256_HMAC`` 62* ``RTE_CRYPTO_AUTH_SHA384`` 63* ``RTE_CRYPTO_AUTH_SHA384_HMAC`` 64* ``RTE_CRYPTO_AUTH_SHA512`` 65* ``RTE_CRYPTO_AUTH_SHA512_HMAC`` 66* ``RTE_CRYPTO_AUTH_AES_XCBC_MAC`` 67* ``RTE_CRYPTO_AUTH_SNOW3G_UIA2`` 68* ``RTE_CRYPTO_AUTH_MD5_HMAC`` 69* ``RTE_CRYPTO_AUTH_NULL`` 70* ``RTE_CRYPTO_AUTH_KASUMI_F9`` 71* ``RTE_CRYPTO_AUTH_AES_GMAC`` 72* ``RTE_CRYPTO_AUTH_ZUC_EIA3`` 73* ``RTE_CRYPTO_AUTH_AES_CMAC`` 74 75Supported AEAD algorithms: 76 77* ``RTE_CRYPTO_AEAD_AES_GCM`` 78* ``RTE_CRYPTO_AEAD_AES_CCM`` 79* ``RTE_CRYPTO_AEAD_CHACHA20_POLY1305`` 80 81Protocol offloads: 82 83* ``RTE_SECURITY_PROTOCOL_DOCSIS`` 84 85Supported Chains 86~~~~~~~~~~~~~~~~ 87 88All the usual chains are supported and also some mixed chains: 89 90.. table:: Supported hash-cipher chains for wireless digest-encrypted cases 91 92 +------------------+-----------+-------------+----------+----------+ 93 | Cipher algorithm | NULL AUTH | SNOW3G UIA2 | ZUC EIA3 | AES CMAC | 94 +==================+===========+=============+==========+==========+ 95 | NULL CIPHER | Y | 2&3 | 2&3 | Y | 96 +------------------+-----------+-------------+----------+----------+ 97 | SNOW3G UEA2 | 2&3 | Y | 2&3 | 2&3 | 98 +------------------+-----------+-------------+----------+----------+ 99 | ZUC EEA3 | 2&3 | 2&3 | 2&3 | 2&3 | 100 +------------------+-----------+-------------+----------+----------+ 101 | AES CTR | Y | 2&3 | 2&3 | Y | 102 +------------------+-----------+-------------+----------+----------+ 103 104* The combinations marked as "Y" are supported on all QAT hardware versions. 105* The combinations marked as "2&3" are supported on GEN2/GEN3 QAT hardware only. 106 107 108Limitations 109~~~~~~~~~~~ 110 111* Only supports the session-oriented API implementation (session-less APIs are not supported). 112* SNOW 3G (UEA2), KASUMI (F8) and ZUC (EEA3) supported only if cipher length and offset fields are byte-multiple. 113* SNOW 3G (UIA2) and ZUC (EIA3) supported only if hash length and offset fields are byte-multiple. 114* No BSD support as BSD QAT kernel driver not available. 115* ZUC EEA3/EIA3 is not supported by dh895xcc devices 116* Maximum additional authenticated data (AAD) for GCM is 240 bytes long and must be passed to the device in a buffer rounded up to the nearest block-size multiple (x16) and padded with zeros. 117* Queue-pairs are thread-safe on Intel CPUs but Queues are not (that is, within a single 118 queue-pair all enqueues to the TX queue must be done from one thread and all dequeues 119 from the RX queue must be done from one thread, but enqueues and dequeues may be done 120 in different threads.) 121* A GCM limitation exists, but only in the case where there are multiple 122 generations of QAT devices on a single platform. 123 To optimise performance, the GCM crypto session should be initialised for the 124 device generation to which the ops will be enqueued. Specifically if a GCM 125 session is initialised on a GEN2 device, but then attached to an op enqueued 126 to a GEN3 device, it will work but cannot take advantage of hardware 127 optimisations in the GEN3 device. And if a GCM session is initialised on a 128 GEN3 device, then attached to an op sent to a GEN1/GEN2 device, it will not be 129 enqueued to the device and will be marked as failed. The simplest way to 130 mitigate this is to use the bdf whitelist to avoid mixing devices of different 131 generations in the same process if planning to use for GCM. 132* The mixed algo feature on GEN2 is not supported by all kernel drivers. Check 133 the notes under the Available Kernel Drivers table below for specific details. 134* Out-of-place is not supported for combined Crypto-CRC DOCSIS security 135 protocol. 136* ``RTE_CRYPTO_CIPHER_DES_DOCSISBPI`` is not supported for combined Crypto-CRC 137 DOCSIS security protocol. 138* Multi-segment buffers are not supported for combined Crypto-CRC DOCSIS 139 security protocol. 140 141Extra notes on KASUMI F9 142~~~~~~~~~~~~~~~~~~~~~~~~ 143 144When using KASUMI F9 authentication algorithm, the input buffer must be 145constructed according to the 146`3GPP KASUMI specification <http://cryptome.org/3gpp/35201-900.pdf>`_ 147(section 4.4, page 13). The input buffer has to have COUNT (4 bytes), 148FRESH (4 bytes), MESSAGE and DIRECTION (1 bit) concatenated. After the DIRECTION 149bit, a single '1' bit is appended, followed by between 0 and 7 '0' bits, so that 150the total length of the buffer is multiple of 8 bits. Note that the actual 151message can be any length, specified in bits. 152 153Once this buffer is passed this way, when creating the crypto operation, 154length of data to authenticate "op.sym.auth.data.length" must be the length 155of all the items described above, including the padding at the end. 156Also, offset of data to authenticate "op.sym.auth.data.offset" 157must be such that points at the start of the COUNT bytes. 158 159Asymmetric Crypto Service on QAT 160-------------------------------- 161 162The QAT asymmetric crypto PMD (hereafter referred to as `QAT ASYM [PMD]`) provides 163poll mode crypto driver support for the following hardware accelerator devices: 164 165* ``Intel QuickAssist Technology DH895xCC`` 166* ``Intel QuickAssist Technology C62x`` 167* ``Intel QuickAssist Technology C3xxx`` 168* ``Intel QuickAssist Technology D15xx`` 169* ``Intel QuickAssist Technology C4xxx`` 170 171The QAT ASYM PMD has support for: 172 173* ``RTE_CRYPTO_ASYM_XFORM_MODEX`` 174* ``RTE_CRYPTO_ASYM_XFORM_MODINV`` 175 176Limitations 177~~~~~~~~~~~ 178 179* Big integers longer than 4096 bits are not supported. 180* Queue-pairs are thread-safe on Intel CPUs but Queues are not (that is, within a single 181 queue-pair all enqueues to the TX queue must be done from one thread and all dequeues 182 from the RX queue must be done from one thread, but enqueues and dequeues may be done 183 in different threads.) 184* RSA-2560, RSA-3584 are not supported 185 186.. _building_qat: 187 188Building PMDs on QAT 189-------------------- 190 191A QAT device can host multiple acceleration services: 192 193* symmetric cryptography 194* data compression 195* asymmetric cryptography 196 197These services are provided to DPDK applications via PMDs which register to 198implement the corresponding cryptodev and compressdev APIs. The PMDs use 199common QAT driver code which manages the QAT PCI device. They also depend on a 200QAT kernel driver being installed on the platform, see :ref:`qat_kernel` below. 201 202 203Configuring and Building the DPDK QAT PMDs 204~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 205 206 207Further information on configuring, building and installing DPDK is described 208:doc:`here <../linux_gsg/build_dpdk>`. 209 210.. _building_qat_config: 211 212Build Configuration 213~~~~~~~~~~~~~~~~~~~ 214 215These are the build configuration options affecting QAT, and their default values: 216 217.. code-block:: console 218 219 RTE_PMD_QAT_MAX_PCI_DEVICES=48 220 RTE_PMD_QAT_COMP_IM_BUFFER_SIZE=65536 221 222Both QAT SYM PMD and QAT ASYM PMD have an external dependency on libcrypto, so are not 223built by default. 224 225The QAT compressdev PMD has no external dependencies, so is built by default. 226 227The number of VFs per PF varies - see table below. If multiple QAT packages are 228installed on a platform then RTE_PMD_QAT_MAX_PCI_DEVICES should be 229adjusted to the number of VFs which the QAT common code will need to handle. 230 231.. Note:: 232 233 There are separate config items (not QAT-specific) for max cryptodevs 234 RTE_CRYPTO_MAX_DEVS and max compressdevs RTE_COMPRESS_MAX_DEVS, 235 if necessary these should be adjusted to handle the total of QAT and other 236 devices which the process will use. In particular for crypto, where each 237 QAT VF may expose two crypto devices, sym and asym, it may happen that the 238 number of devices will be bigger than MAX_DEVS and the process will show an error 239 during PMD initialisation. To avoid this problem RTE_CRYPTO_MAX_DEVS may be 240 increased or -w, pci-whitelist domain:bus:devid:func option may be used. 241 242 243QAT compression PMD needs intermediate buffers to support Deflate compression 244with Dynamic Huffman encoding. RTE_PMD_QAT_COMP_IM_BUFFER_SIZE 245specifies the size of a single buffer, the PMD will allocate a multiple of these, 246plus some extra space for associated meta-data. For GEN2 devices, 20 buffers are 247allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes overhead. 248 249.. Note:: 250 251 If the compressed output of a Deflate operation using Dynamic Huffman 252 Encoding is too big to fit in an intermediate buffer, then the 253 operation will be split into smaller operations and their results will 254 be merged afterwards. 255 This is not possible if any checksum calculation was requested - in such 256 case the code falls back to fixed compression. 257 To avoid this less performant case, applications should configure 258 the intermediate buffer size to be larger than the expected input data size 259 (compressed output size is usually unknown, so the only option is to make 260 larger than the input size). 261 262 263Running QAT PMD with minimum threshold for burst size 264~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 265 266If only a small number or packets can be enqueued. Each enqueue causes an expensive MMIO write. 267These MMIO write occurrences can be optimised by setting any of the following parameters: 268 269- qat_sym_enq_threshold 270- qat_asym_enq_threshold 271- qat_comp_enq_threshold 272 273When any of these parameters is set rte_cryptodev_enqueue_burst function will 274return 0 (thereby avoiding an MMIO) if the device is congested and number of packets 275possible to enqueue is smaller. 276To use this feature the user must set the parameter on process start as a device additional parameter:: 277 278 -w 03:01.1,qat_sym_enq_threshold=32,qat_comp_enq_threshold=16 279 280All parameters can be used with the same device regardless of order. Parameters are separated 281by comma. When the same parameter is used more than once first occurrence of the parameter 282is used. 283Maximum threshold that can be set is 32. 284 285 286Device and driver naming 287~~~~~~~~~~~~~~~~~~~~~~~~ 288 289* The qat cryptodev symmetric crypto driver name is "crypto_qat". 290* The qat cryptodev asymmetric crypto driver name is "crypto_qat_asym". 291 292The "rte_cryptodev_devices_get()" returns the devices exposed by either of these drivers. 293 294* Each qat sym crypto device has a unique name, in format 295 "<pci bdf>_<service>", e.g. "0000:41:01.0_qat_sym". 296* Each qat asym crypto device has a unique name, in format 297 "<pci bdf>_<service>", e.g. "0000:41:01.0_qat_asym". 298 This name can be passed to "rte_cryptodev_get_dev_id()" to get the device_id. 299 300.. Note:: 301 302 The cryptodev driver name is passed to the dpdk-test-crypto-perf tool in the "-devtype" parameter. 303 304 The qat crypto device name is in the format of the worker parameter passed to the crypto scheduler. 305 306* The qat compressdev driver name is "compress_qat". 307 The rte_compressdev_devices_get() returns the devices exposed by this driver. 308 309* Each qat compression device has a unique name, in format 310 <pci bdf>_<service>, e.g. "0000:41:01.0_qat_comp". 311 This name can be passed to rte_compressdev_get_dev_id() to get the device_id. 312 313.. _qat_kernel: 314 315Dependency on the QAT kernel driver 316~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 317 318To use QAT an SRIOV-enabled QAT kernel driver is required. The VF 319devices created and initialised by this driver will be used by the QAT PMDs. 320 321Instructions for installation are below, but first an explanation of the 322relationships between the PF/VF devices and the PMDs visible to 323DPDK applications. 324 325Each QuickAssist PF device exposes a number of VF devices. Each VF device can 326enable one symmetric cryptodev PMD and/or one asymmetric cryptodev PMD and/or 327one compressdev PMD. 328These QAT PMDs share the same underlying device and pci-mgmt code, but are 329enumerated independently on their respective APIs and appear as independent 330devices to applications. 331 332.. Note:: 333 334 Each VF can only be used by one DPDK process. It is not possible to share 335 the same VF across multiple processes, even if these processes are using 336 different acceleration services. 337 338 Conversely one DPDK process can use one or more QAT VFs and can expose both 339 cryptodev and compressdev instances on each of those VFs. 340 341 342Available kernel drivers 343~~~~~~~~~~~~~~~~~~~~~~~~ 344 345Kernel drivers for each device for each service are listed in the following table. (Scroll right 346to see the full table) 347 348 349.. _table_qat_pmds_drivers: 350 351.. table:: QAT device generations, devices and drivers 352 353 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 354 | S | A | C | Gen | Device | Driver/ver | Kernel Module | Pci Driver | PF Did | #PFs | VF Did | VFs/PF | 355 +=====+=====+=====+=====+==========+===============+===============+============+========+======+========+========+ 356 | Yes | No | No | 1 | DH895xCC | linux/4.4+ | qat_dh895xcc | dh895xcc | 435 | 1 | 443 | 32 | 357 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 358 | Yes | Yes | No | " | " | 01.org/4.2.0+ | " | " | " | " | " | " | 359 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 360 | Yes | Yes | Yes | " | " | 01.org/4.3.0+ | " | " | " | " | " | " | 361 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 362 | Yes | No | No | 2 | C62x | linux/4.5+ | qat_c62x | c6xx | 37c8 | 3 | 37c9 | 16 | 363 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 364 | Yes | Yes | Yes | " | " | 01.org/4.2.0+ | " | " | " | " | " | " | 365 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 366 | Yes | No | No | 2 | C3xxx | linux/4.5+ | qat_c3xxx | c3xxx | 19e2 | 1 | 19e3 | 16 | 367 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 368 | Yes | Yes | Yes | " | " | 01.org/4.2.0+ | " | " | " | " | " | " | 369 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 370 | Yes | No | No | 2 | 200xx | p | qat_200xx | 200xx | 18ee | 1 | 18ef | 16 | 371 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 372 | Yes | No | No | 2 | D15xx | 01.org/4.2.0+ | qat_d15xx | d15xx | 6f54 | 1 | 6f55 | 16 | 373 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 374 | Yes | No | No | 3 | C4xxx | p | qat_c4xxx | c4xxx | 18a0 | 1 | 18a1 | 128 | 375 +-----+-----+-----+-----+----------+---------------+---------------+------------+--------+------+--------+--------+ 376 377* Note: Symmetric mixed crypto algorithms feature on Gen 2 works only with 01.org driver version 4.9.0+ 378 379The first 3 columns indicate the service: 380 381* S = Symmetric crypto service (via cryptodev API) 382* A = Asymmetric crypto service (via cryptodev API) 383* C = Compression service (via compressdev API) 384 385The ``Driver`` column indicates either the Linux kernel version in which 386support for this device was introduced or a driver available on Intel's 01.org 387website. There are both linux in-tree and 01.org kernel drivers available for some 388devices. p = release pending. 389 390If you are running on a kernel which includes a driver for your device, see 391`Installation using kernel.org driver`_ below. Otherwise see 392`Installation using 01.org QAT driver`_. 393 394 395Installation using kernel.org driver 396~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 397 398The examples below are based on the C62x device, if you have a different device 399use the corresponding values in the above table. 400 401In BIOS ensure that SRIOV is enabled and either: 402 403* Disable VT-d or 404* Enable VT-d and set ``"intel_iommu=on iommu=pt"`` in the grub file. 405 406Check that the QAT driver is loaded on your system, by executing:: 407 408 lsmod | grep qa 409 410You should see the kernel module for your device listed, e.g.:: 411 412 qat_c62x 5626 0 413 intel_qat 82336 1 qat_c62x 414 415Next, you need to expose the Virtual Functions (VFs) using the sysfs file system. 416 417First find the BDFs (Bus-Device-Function) of the physical functions (PFs) of 418your device, e.g.:: 419 420 lspci -d:37c8 421 422You should see output similar to:: 423 424 1a:00.0 Co-processor: Intel Corporation Device 37c8 425 3d:00.0 Co-processor: Intel Corporation Device 37c8 426 3f:00.0 Co-processor: Intel Corporation Device 37c8 427 428Enable the VFs for each PF by echoing the number of VFs per PF to the pci driver:: 429 430 echo 16 > /sys/bus/pci/drivers/c6xx/0000:1a:00.0/sriov_numvfs 431 echo 16 > /sys/bus/pci/drivers/c6xx/0000:3d:00.0/sriov_numvfs 432 echo 16 > /sys/bus/pci/drivers/c6xx/0000:3f:00.0/sriov_numvfs 433 434Check that the VFs are available for use. For example ``lspci -d:37c9`` should 435list 48 VF devices available for a ``C62x`` device. 436 437To complete the installation follow the instructions in 438`Binding the available VFs to the vfio-pci driver`_. 439 440.. Note:: 441 442 If the QAT kernel modules are not loaded and you see an error like ``Failed 443 to load MMP firmware qat_895xcc_mmp.bin`` in kernel logs, this may be as a 444 result of not using a distribution, but just updating the kernel directly. 445 446 Download firmware from the `kernel firmware repo 447 <http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/>`_. 448 449 Copy qat binaries to ``/lib/firmware``:: 450 451 cp qat_895xcc.bin /lib/firmware 452 cp qat_895xcc_mmp.bin /lib/firmware 453 454 Change to your linux source root directory and start the qat kernel modules:: 455 456 insmod ./drivers/crypto/qat/qat_common/intel_qat.ko 457 insmod ./drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 458 459.. Note:: 460 461 If you see the following warning in ``/var/log/messages`` it can be ignored: 462 ``IOMMU should be enabled for SR-IOV to work correctly``. 463 464 465Installation using 01.org QAT driver 466~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 467 468Download the latest QuickAssist Technology Driver from `01.org 469<https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches>`_. 470Consult the *Getting Started Guide* at the same URL for further information. 471 472The steps below assume you are: 473 474* Building on a platform with one ``C62x`` device. 475* Using package ``qat1.7.l.4.2.0-000xx.tar.gz``. 476* On Fedora26 kernel ``4.11.11-300.fc26.x86_64``. 477 478In the BIOS ensure that SRIOV is enabled and VT-d is disabled. 479 480Uninstall any existing QAT driver, for example by running: 481 482* ``./installer.sh uninstall`` in the directory where originally installed. 483 484 485Build and install the SRIOV-enabled QAT driver:: 486 487 mkdir /QAT 488 cd /QAT 489 490 # Copy the package to this location and unpack 491 tar zxof qat1.7.l.4.2.0-000xx.tar.gz 492 493 ./configure --enable-icp-sriov=host 494 make install 495 496You can use ``cat /sys/kernel/debug/qat<your device type and bdf>/version/fw`` to confirm the driver is correctly installed and is using firmware version 4.2.0. 497You can use ``lspci -d:37c9`` to confirm the presence of the 16 VF devices available per ``C62x`` PF. 498 499Confirm the driver is correctly installed and is using firmware version 4.2.0:: 500 501 cat /sys/kernel/debug/qat<your device type and bdf>/version/fw 502 503 504Confirm the presence of 48 VF devices - 16 per PF:: 505 506 lspci -d:37c9 507 508 509To complete the installation - follow instructions in 510`Binding the available VFs to the vfio-pci driver`_. 511 512.. Note:: 513 514 If using a later kernel and the build fails with an error relating to 515 ``strict_stroul`` not being available apply the following patch: 516 517 .. code-block:: diff 518 519 /QAT/QAT1.6/quickassist/utilities/downloader/Target_CoreLibs/uclo/include/linux/uclo_platform.h 520 + #if LINUX_VERSION_CODE >= KERNEL_VERSION(3,18,5) 521 + #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (kstrtoul((str), (base), (num))) printk("Error strtoull convert %s\n", str); } 522 + #else 523 #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,38) 524 #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (strict_strtoull((str), (base), (num))) printk("Error strtoull convert %s\n", str); } 525 #else 526 #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,25) 527 #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; strict_strtoll((str), (base), (num));} 528 #else 529 #define STR_TO_64(str, base, num, endPtr) \ 530 do { \ 531 if (str[0] == '-') \ 532 { \ 533 *(num) = -(simple_strtoull((str+1), &(endPtr), (base))); \ 534 }else { \ 535 *(num) = simple_strtoull((str), &(endPtr), (base)); \ 536 } \ 537 } while(0) 538 + #endif 539 #endif 540 #endif 541 542 543.. Note:: 544 545 If the build fails due to missing header files you may need to do following:: 546 547 sudo yum install zlib-devel 548 sudo yum install openssl-devel 549 sudo yum install libudev-devel 550 551.. Note:: 552 553 If the build or install fails due to mismatching kernel sources you may need to do the following:: 554 555 sudo yum install kernel-headers-`uname -r` 556 sudo yum install kernel-src-`uname -r` 557 sudo yum install kernel-devel-`uname -r` 558 559 560Binding the available VFs to the vfio-pci driver 561~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 562 563Note: 564 565* Please note that due to security issues, the usage of older DPDK igb-uio 566 driver is not recommended. This document shows how to use the more secure 567 vfio-pci driver. 568* If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the 569 QATE-39220 and QATE-7495 issues in 570 `01.org doc <https://01.org/sites/default/files/downloads/336211-015-qatsoftwareforlinux-rn-hwv1.7-final.pdf>`_ 571 which details the constraint about trusted guests and add `disable_denylist=1` 572 to the vfio-pci params to use QAT. See also `this patch description <https://lkml.org/lkml/2020/7/23/1155>`_. 573 574Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver. 575 576For an Intel(R) QuickAssist Technology DH895xCC device 577^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 578 579The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your 580VFs are different adjust the unbind command below:: 581 582 cd to the top-level DPDK directory 583 for device in $(seq 1 4); do \ 584 for fn in $(seq 0 7); do \ 585 usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \ 586 done; \ 587 done 588 589For an Intel(R) QuickAssist Technology C62x device 590^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 591 592The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``, 593``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different 594adjust the unbind command below:: 595 596 cd to the top-level DPDK directory 597 for device in $(seq 1 2); do \ 598 for fn in $(seq 0 7); do \ 599 usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \ 600 usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \ 601 usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \ 602 done; \ 603 done 604 605For Intel(R) QuickAssist Technology C3xxx or 200xx or D15xx device 606^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 607 608The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your 609VFs are different adjust the unbind command below:: 610 611 cd to the top-level DPDK directory 612 for device in $(seq 1 2); do \ 613 for fn in $(seq 0 7); do \ 614 usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \ 615 done; \ 616 done 617 618Bind to the vfio-pci driver 619^^^^^^^^^^^^^^^^^^^^^^^^^^^ 620 621Load the vfio-pci driver, bind the VF PCI Device id to it using the 622``dpdk-devbind.py`` script then use the ``--status`` option 623to confirm the VF devices are now in use by vfio-pci kernel driver, 624e.g. for the C62x device:: 625 626 cd to the top-level DPDK directory 627 modprobe vfio-pci 628 usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1 629 usertools/dpdk-devbind.py --status 630 631Use ``modprobe vfio-pci disable_denylist=1`` from kernel 5.9 onwards. 632See note in the section `Binding the available VFs to the vfio-pci driver`_ 633above. 634 635Testing 636~~~~~~~ 637 638QAT SYM crypto PMD can be tested by running the test application:: 639 640 cd ./<build_dir>/app/test 641 ./dpdk-test -l1 -n1 -w <your qat bdf> 642 RTE>>cryptodev_qat_autotest 643 644QAT ASYM crypto PMD can be tested by running the test application:: 645 646 cd ./<build_dir>/app/test 647 ./dpdk-test -l1 -n1 -w <your qat bdf> 648 RTE>>cryptodev_qat_asym_autotest 649 650QAT compression PMD can be tested by running the test application:: 651 652 cd ./<build_dir>/app/test 653 ./dpdk-test -l1 -n1 -w <your qat bdf> 654 RTE>>compressdev_autotest 655 656 657Debugging 658~~~~~~~~~ 659 660There are 2 sets of trace available via the dynamic logging feature: 661 662* pmd.qat_dp exposes trace on the data-path. 663* pmd.qat_general exposes all other trace. 664 665pmd.qat exposes both sets of traces. 666They can be enabled using the log-level option (where 8=maximum log level) on 667the process cmdline, e.g. using any of the following:: 668 669 --log-level="pmd.qat_general,8" 670 --log-level="pmd.qat_dp,8" 671 --log-level="pmd.qat,8" 672 673.. Note:: 674 675 The global RTE_LOG_DP_LEVEL overrides data-path trace so must be set to 676 RTE_LOG_DEBUG to see all the trace. This variable is in config/rte_config.h 677 for meson build. 678 Also the dynamic global log level overrides both sets of trace, so e.g. no 679 QAT trace would display in this case:: 680 681 --log-level="7" --log-level="pmd.qat_general,8" 682