1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright 2019 Mellanox Technologies, Ltd 3 4.. include:: <isonum.txt> 5 6MLX5 vDPA driver 7================ 8 9The MLX5 vDPA (vhost data path acceleration) driver library 10(**librte_vdpa_mlx5**) provides support for **Mellanox ConnectX-6**, 11**Mellanox ConnectX-6 Dx** and **Mellanox BlueField** families of 1210/25/40/50/100/200 Gb/s adapters as well as their virtual functions (VF) in 13SR-IOV context. 14 15.. note:: 16 17 This driver is enabled automatically when using "meson" build system which 18 will detect dependencies. 19 20 21Design 22------ 23 24For security reasons and robustness, this driver only deals with virtual 25memory addresses. The way resources allocations are handled by the kernel, 26combined with hardware specifications that allow to handle virtual memory 27addresses directly, ensure that DPDK applications cannot access random 28physical memory (or memory that does not belong to the current process). 29 30The PMD can use libibverbs and libmlx5 to access the device firmware 31or directly the hardware components. 32There are different levels of objects and bypassing abilities 33to get the best performances: 34 35- Verbs is a complete high-level generic API 36- Direct Verbs is a device-specific API 37- DevX allows to access firmware objects 38- Direct Rules manages flow steering at low-level hardware layer 39 40Enabling librte_vdpa_mlx5 causes DPDK applications to be linked against 41libibverbs. 42 43A Mellanox mlx5 PCI device can be probed by either net/mlx5 driver or vdpa/mlx5 44driver but not in parallel. Hence, the user should decide the driver by the 45``class`` parameter in the device argument list. 46By default, the mlx5 device will be probed by the net/mlx5 driver. 47 48Supported NICs 49-------------- 50 51* Mellanox\ |reg| ConnectX\ |reg|-6 200G MCX654106A-HCAT (2x200G) 52* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 25G MCX621102AN-ADAT (2x25G) 53* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 100G MCX623106AN-CDAT (2x100G) 54* Mellanox\ |reg| ConnectX\ |reg|-6 Dx EN 200G MCX623105AN-VDAT (1x200G) 55* Mellanox\ |reg| BlueField SmartNIC 25G MBF1M332A-ASCAT (2x25G) 56 57Prerequisites 58------------- 59 60- Mellanox OFED version: **5.0** 61 see :doc:`../../nics/mlx5` guide for more Mellanox OFED details. 62 63Compilation option 64~~~~~~~~~~~~~~~~~~ 65 66The meson option ``ibverbs_link`` is **shared** by default, 67but can be configured to have the following values: 68 69- ``dlopen`` 70 71 Build PMD with additional code to make it loadable without hard 72 dependencies on **libibverbs** nor **libmlx5**, which may not be installed 73 on the target system. 74 75 In this mode, their presence is still required for it to run properly, 76 however their absence won't prevent a DPDK application from starting (with 77 DPDK shared build disabled) and they won't show up as missing with ``ldd(1)``. 78 79 It works by moving these dependencies to a purpose-built rdma-core "glue" 80 plug-in which must be installed in a directory whose name is based 81 on ``RTE_EAL_PMD_PATH`` suffixed with ``-glue``. 82 83 This option has no performance impact. 84 85- ``static`` 86 87 Embed static flavor of the dependencies **libibverbs** and **libmlx5** 88 in the PMD shared library or the executable static binary. 89 90.. note:: 91 92 Default armv8a configuration of meson build sets ``RTE_CACHE_LINE_SIZE`` 93 to 128 then brings performance degradation. 94 95Run-time configuration 96~~~~~~~~~~~~~~~~~~~~~~ 97 98- **ethtool** operations on related kernel interfaces also affect the PMD. 99 100Driver options 101^^^^^^^^^^^^^^ 102 103- ``class`` parameter [string] 104 105 Select the class of the driver that should probe the device. 106 `vdpa` for the mlx5 vDPA driver. 107 108- ``event_mode`` parameter [int] 109 110 - 0, Completion queue scheduling will be managed by a timer thread which 111 automatically adjusts its delays to the coming traffic rate. 112 113 - 1, Completion queue scheduling will be managed by a timer thread with fixed 114 delay time. 115 116 - 2, Completion queue scheduling will be managed by interrupts. Each CQ burst 117 arms the CQ in order to get an interrupt event in the next traffic burst. 118 119 - Default mode is 1. 120 121- ``event_us`` parameter [int] 122 123 Per mode micro-seconds parameter - relevant only for event mode 0 and 1: 124 125 - 0, A nonzero value to set timer step in micro-seconds. The timer thread 126 dynamic delay change steps according to this value. Default value is 1us. 127 128 - 1, A value to set fixed timer delay in micro-seconds. Default value is 0us. 129 130- ``no_traffic_time`` parameter [int] 131 132 A nonzero value defines the traffic off time, in seconds, that moves the 133 driver to no-traffic mode. In this mode the timer events are stopped and 134 interrupts are configured to the device in order to notify traffic for the 135 driver. Default value is 2s. 136 137- ``event_core`` parameter [int] 138 139 CPU core number to set polling thread affinity to, default to control plane 140 cpu. 141 142- ``hw_latency_mode`` parameter [int] 143 144 The completion queue moderation mode: 145 146 - 0, HW default. 147 148 - 1, Latency is counted from the first packet completion report. 149 150 - 2, Latency is counted from the last packet completion. 151 152- ``hw_max_latency_us`` parameter [int] 153 154 - 1 - 4095, The maximum time in microseconds that packet completion report 155 can be delayed. 156 157 - 0, HW default. 158 159- ``hw_max_pending_comp`` parameter [int] 160 161 - 1 - 65535, The maximum number of pending packets completions in an HW queue. 162 163 - 0, HW default. 164 165 166Error handling 167^^^^^^^^^^^^^^ 168 169Upon potential hardware errors, mlx5 PMD try to recover, give up if failed 3 170times in 3 seconds, virtq will be put in disable state. User should check log 171to get error information, or query vdpa statistics counter to know error type 172and count report. 173