1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright 2019 Mellanox Technologies, Ltd 3 4.. include:: <isonum.txt> 5 6MLX5 vDPA driver 7================ 8 9The MLX5 vDPA (vhost data path acceleration) driver library 10(**librte_pmd_mlx5_vdpa**) provides support for **Mellanox ConnectX-6**, 11**Mellanox ConnectX-6 Dx** and **Mellanox BlueField** families of 1210/25/40/50/100/200 Gb/s adapters as well as their virtual functions (VF) in 13SR-IOV context. 14 15.. note:: 16 17 Due to external dependencies, this driver is disabled in default 18 configuration of the "make" build. It can be enabled with 19 ``CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD=y`` or by using "meson" build system which 20 will detect dependencies. 21 22 23Design 24------ 25 26For security reasons and robustness, this driver only deals with virtual 27memory addresses. The way resources allocations are handled by the kernel, 28combined with hardware specifications that allow to handle virtual memory 29addresses directly, ensure that DPDK applications cannot access random 30physical memory (or memory that does not belong to the current process). 31 32The PMD can use libibverbs and libmlx5 to access the device firmware 33or directly the hardware components. 34There are different levels of objects and bypassing abilities 35to get the best performances: 36 37- Verbs is a complete high-level generic API 38- Direct Verbs is a device-specific API 39- DevX allows to access firmware objects 40- Direct Rules manages flow steering at low-level hardware layer 41 42Enabling librte_pmd_mlx5_vdpa causes DPDK applications to be linked against 43libibverbs. 44 45A Mellanox mlx5 PCI device can be probed by either net/mlx5 driver or vdpa/mlx5 46driver but not in parallel. Hence, the user should decide the driver by the 47``class`` parameter in the device argument list. 48By default, the mlx5 device will be probed by the net/mlx5 driver. 49 50Supported NICs 51-------------- 52 53* Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) 54* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 25G MCX621102AN-ADAT (2x25G) 55* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 100G MCX623106AN-CDAT (2x100G) 56* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 200G MCX623105AN-VDAT (1x200G) 57* Mellanox\ |reg| BlueField SmartNIC 25G MBF1M332A-ASCAT (2x25G) 58 59Prerequisites 60------------- 61 62- Mellanox OFED version: **5.0** 63 see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. 64 65Compilation options 66~~~~~~~~~~~~~~~~~~~ 67 68These options can be modified in the ``.config`` file. 69 70- ``CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD`` (default **n**) 71 72 Toggle compilation of librte_pmd_mlx5 itself. 73 74- ``CONFIG_RTE_IBVERBS_LINK_DLOPEN`` (default **n**) 75 76 Build PMD with additional code to make it loadable without hard 77 dependencies on **libibverbs** nor **libmlx5**, which may not be installed 78 on the target system. 79 80 In this mode, their presence is still required for it to run properly, 81 however their absence won't prevent a DPDK application from starting (with 82 ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up as 83 missing with ``ldd(1)``. 84 85 It works by moving these dependencies to a purpose-built rdma-core "glue" 86 plug-in which must either be installed in a directory whose name is based 87 on ``CONFIG_RTE_EAL_PMD_PATH`` suffixed with ``-glue`` if set, or in a 88 standard location for the dynamic linker (e.g. ``/lib``) if left to the 89 default empty string (``""``). 90 91 This option has no performance impact. 92 93- ``CONFIG_RTE_IBVERBS_LINK_STATIC`` (default **n**) 94 95 Embed static flavor of the dependencies **libibverbs** and **libmlx5** 96 in the PMD shared library or the executable static binary. 97 98.. note:: 99 100 For BlueField, target should be set to ``arm64-bluefield-linux-gcc``. This 101 will enable ``CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD`` and set 102 ``RTE_CACHE_LINE_SIZE`` to 64. Default armv8a configuration of make build and 103 meson build set it to 128 then brings performance degradation. 104 105Run-time configuration 106~~~~~~~~~~~~~~~~~~~~~~ 107 108- **ethtool** operations on related kernel interfaces also affect the PMD. 109 110Driver options 111^^^^^^^^^^^^^^ 112 113- ``class`` parameter [string] 114 115 Select the class of the driver that should probe the device. 116 `vdpa` for the mlx5 vDPA driver. 117 118- ``event_mode`` parameter [int] 119 120 - 0, Completion queue scheduling will be managed by a timer thread which 121 automatically adjusts its delays to the coming traffic rate. 122 123 - 1, Completion queue scheduling will be managed by a timer thread with fixed 124 delay time. 125 126 - 2, Completion queue scheduling will be managed by interrupts. Each CQ burst 127 arms the CQ in order to get an interrupt event in the next traffic burst. 128 129 - Default mode is 0. 130 131- ``event_us`` parameter [int] 132 133 Per mode micro-seconds parameter - relevant only for event mode 0 and 1: 134 135 - 0, A nonzero value to set timer step in micro-seconds. The timer thread 136 dynamic delay change steps according to this value. Default value is 1us. 137 138 - 1, A nonzero value to set fixed timer delay in micro-seconds. Default value 139 is 100us. 140 141- ``no_traffic_time`` parameter [int] 142 143 A nonzero value defines the traffic off time, in seconds, that moves the 144 driver to no-traffic mode. In this mode the timer events are stopped and 145 interrupts are configured to the device in order to notify traffic for the 146 driver. Default value is 2s. 147