1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019-2021 Xilinx, Inc. 3 Copyright(c) 2016-2019 Solarflare Communications Inc. 4 5 This software was jointly developed between OKTET Labs (under contract 6 for Solarflare) and Solarflare Communications, Inc. 7 8Solarflare libefx-based Poll Mode Driver 9======================================== 10 11The SFC EFX PMD (**librte_net_sfc_efx**) provides poll mode driver support 12for **Solarflare SFN7xxx and SFN8xxx** family of 10/40 Gbps adapters, 13**Solarflare XtremeScale X2xxx** family of 10/25/40/50/100 Gbps adapters and 14**Alveo SN1000 SmartNICs** family of 10/25/40/50/100 Gbps adapters. 15SFC EFX PMD has support for the latest Linux and FreeBSD operating systems. 16 17More information can be found at `Solarflare Communications website 18<http://solarflare.com>`_. 19 20 21Features 22-------- 23 24SFC EFX PMD has support for: 25 26- Multiple transmit and receive queues 27 28- Link state information including link status change interrupt 29 30- IPv4/IPv6 TCP/UDP transmit checksum offload 31 32- Inner IPv4/IPv6 TCP/UDP transmit checksum offload 33 34- Port hardware statistics 35 36- Extended statistics (see Solarflare Server Adapter User's Guide for 37 the statistics description) 38 39- Basic flow control 40 41- MTU update 42 43- Jumbo frames up to 9K 44 45- Promiscuous mode 46 47- Allmulticast mode 48 49- TCP segmentation offload (TSO) including VXLAN and GENEVE encapsulated 50 51- Multicast MAC filter 52 53- IPv4/IPv6 TCP/UDP receive checksum offload 54 55- Inner IPv4/IPv6 TCP/UDP receive checksum offload 56 57- Received packet type information 58 59- Receive side scaling (RSS) 60 61- RSS hash 62 63- Scattered Rx DMA for packet that are larger that a single Rx descriptor 64 65- Receive queue interrupts 66 67- Deferred receive and transmit queue start 68 69- Transmit VLAN insertion (if running firmware variant supports it) 70 71- Flow API 72 73- Loopback 74 75- SR-IOV PF 76 77 78Non-supported Features 79---------------------- 80 81The features not yet supported include: 82 83- Priority-based flow control 84 85- Configurable RX CRC stripping (always stripped) 86 87- Header split on receive 88 89- VLAN filtering 90 91- VLAN stripping 92 93- LRO 94 95 96Limitations 97----------- 98 99Due to requirements on receive buffer alignment and usage of the receive 100buffer for the auxiliary packet information provided by the NIC up to 101extra 269 (14 bytes prefix plus up to 255 bytes for end padding) bytes may be 102required in the receive buffer. 103It should be taken into account when mbuf pool for receive is created. 104 105 106Equal stride super-buffer mode 107~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 108 109When the receive queue uses equal stride super-buffer DMA mode, one HW Rx 110descriptor carries many Rx buffers which contiguously follow each other 111with some stride (equal to total size of rte_mbuf as mempool object). 112Each Rx buffer is an independent rte_mbuf. 113However dedicated mempool manager must be used when mempool for the Rx 114queue is created. The manager must support dequeue of the contiguous 115block of objects and provide mempool info API to get the block size. 116 117Another limitation of a equal stride super-buffer mode, imposed by the 118firmware, is that it allows for a single RSS context. 119 120 121Tunnels support 122--------------- 123 124NVGRE, VXLAN and GENEVE tunnels are supported on SFN8xxx and X2xxx family 125adapters with full-feature firmware variant running. 126**sfboot** should be used to configure NIC to run full-feature firmware variant. 127See Solarflare Server Adapter User's Guide for details. 128 129SFN8xxx and X2xxx family adapters provide either inner or outer packet classes. 130If adapter firmware advertises support for tunnels then the PMD 131configures the hardware to report inner classes, and outer classes are 132not reported in received packets. 133However, for VXLAN and GENEVE tunnels the PMD does report UDP as the 134outer layer 4 packet type. 135 136SFN8xxx and X2xxx family adapters report GENEVE packets as VXLAN. 137If UDP ports are configured for only one tunnel type then it is safe to 138treat VXLAN packet type indication as the corresponding UDP tunnel type. 139 140 141Flow API support 142---------------- 143 144Supported attributes: 145 146- Ingress 147- Transfer 148 149Supported pattern items (***non-transfer*** rules): 150 151- VOID 152 153- ETH (exact match of source/destination addresses, individual/group match 154 of destination address, EtherType in the outer frame and exact match of 155 destination addresses, individual/group match of destination address in 156 the inner frame) 157 158- VLAN (exact match of VID, double-tagging is supported) 159 160- IPV4 (exact match of source/destination addresses, 161 IP transport protocol) 162 163- IPV6 (exact match of source/destination addresses, 164 IP transport protocol) 165 166- TCP (exact match of source/destination ports) 167 168- UDP (exact match of source/destination ports) 169 170- VXLAN (exact match of VXLAN network identifier) 171 172- GENEVE (exact match of virtual network identifier, only Ethernet (0x6558) 173 protocol type is supported) 174 175- NVGRE (exact match of virtual subnet ID) 176 177Supported actions (***non-transfer*** rules): 178 179- VOID 180 181- QUEUE 182 183- RSS 184 185- DROP 186 187- FLAG (supported only with ef10_essb Rx datapath) 188 189- MARK (supported only with ef10_essb Rx datapath) 190 191Supported pattern items (***transfer*** rules): 192 193- PORT_ID (cannot repeat; conflicts with other traffic source items) 194 195- PHY_PORT (cannot repeat; conflicts with other traffic source items) 196 197- PF (cannot repeat; conflicts with other traffic source items) 198 199- VF (cannot repeat; conflicts with other traffic source items) 200 201- ETH 202 203- VLAN (double-tagging is supported) 204 205- IPV4 (source/destination addresses, IP transport protocol, 206 type of service, time to live) 207 208- IPV6 (source/destination addresses, IP transport protocol, 209 traffic class, hop limit) 210 211- TCP (source/destination ports, TCP header length + TCP flags) 212 213- UDP (source/destination ports) 214 215- VXLAN (exact match of VXLAN network identifier) 216 217- GENEVE (exact match of virtual network identifier) 218 219- NVGRE (exact match of virtual subnet ID) 220 221Supported actions (***transfer*** rules): 222 223- OF_POP_VLAN 224 225- OF_PUSH_VLAN 226 227- OF_VLAN_SET_VID 228 229- OF_VLAN_SET_PCP 230 231- FLAG 232 233- MARK 234 235- PHY_PORT 236 237- PF 238 239- VF 240 241- PORT_ID 242 243- DROP 244 245Validating flow rules depends on the firmware variant. 246 247The :ref:`flow_isolated_mode` is supported. 248 249The implementation is natively thread-safe. 250 251Ethernet destination individual/group match 252~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 253 254Ethernet item supports I/G matching, if only the corresponding bit is set 255in the mask of destination address. If destination address in the spec is 256multicast, it matches all multicast (and broadcast) packets, otherwise it 257matches unicast packets that are not filtered by other flow rules. 258 259Exceptions to flow rules 260~~~~~~~~~~~~~~~~~~~~~~~~ 261 262There is a list of exceptional flow rule patterns which will not be 263accepted by the PMD. A pattern will be rejected if at least one of the 264conditions is met: 265 266- Filtering by IPv4 or IPv6 EtherType without pattern items of internet 267 layer and above. 268 269- The last item is IPV4 or IPV6, and it's empty. 270 271- Filtering by TCP or UDP IP transport protocol without pattern items of 272 transport layer and above. 273 274- The last item is TCP or UDP, and it's empty. 275 276 277Supported NICs 278-------------- 279 280- Xilinx Adapters: 281 282 - Alveo SN1022 SmartNIC 283 284- Solarflare XtremeScale Adapters: 285 286 - Solarflare X2522 Dual Port SFP28 10/25GbE Adapter 287 288 - Solarflare X2541 Single Port QSFP28 10/25G/100G Adapter 289 290 - Solarflare X2542 Dual Port QSFP28 10/25G/100G Adapter 291 292- Solarflare Flareon [Ultra] Server Adapters: 293 294 - Solarflare SFN8522 Dual Port SFP+ Server Adapter 295 296 - Solarflare SFN8522M Dual Port SFP+ Server Adapter 297 298 - Solarflare SFN8042 Dual Port QSFP+ Server Adapter 299 300 - Solarflare SFN8542 Dual Port QSFP+ Server Adapter 301 302 - Solarflare SFN8722 Dual Port SFP+ OCP Server Adapter 303 304 - Solarflare SFN7002F Dual Port SFP+ Server Adapter 305 306 - Solarflare SFN7004F Quad Port SFP+ Server Adapter 307 308 - Solarflare SFN7042Q Dual Port QSFP+ Server Adapter 309 310 - Solarflare SFN7122F Dual Port SFP+ Server Adapter 311 312 - Solarflare SFN7124F Quad Port SFP+ Server Adapter 313 314 - Solarflare SFN7142Q Dual Port QSFP+ Server Adapter 315 316 - Solarflare SFN7322F Precision Time Synchronization Server Adapter 317 318 319Prerequisites 320------------- 321 322- Requires firmware version: 323 324 - SFN7xxx: **4.7.1.1001** or higher 325 326 - SFN8xxx: **6.0.2.1004** or higher 327 328Visit `Solarflare Support Downloads <https://support.solarflare.com>`_ to get 329Solarflare Utilities (either Linux or FreeBSD) with the latest firmware. 330Follow instructions from Solarflare Server Adapter User's Guide to 331update firmware and configure the adapter. 332 333 334Pre-Installation Configuration 335------------------------------ 336 337 338Build Options 339~~~~~~~~~~~~~ 340 341The following build-time options may be enabled on build time using 342``-Dc_args=`` meson argument (e.g. ``-Dc_args=-DRTE_LIBRTE_SFC_EFX_DEBUG``). 343 344Please note that enabling debugging options may affect system performance. 345 346- ``RTE_LIBRTE_SFC_EFX_DEBUG`` (undefined by default) 347 348 Enable compilation of the extra run-time consistency checks. 349 350 351Per-Device Parameters 352~~~~~~~~~~~~~~~~~~~~~ 353 354The following per-device parameters can be passed via EAL PCI device 355allow option like "-a 02:00.0,arg1=value1,...". 356 357Case-insensitive 1/y/yes/on or 0/n/no/off may be used to specify 358boolean parameters value. 359 360- ``class`` [net|vdpa] (default **net**) 361 362 Choose the mode of operation of ef100 device. 363 **net** device will work as network device and will be probed by net/sfc driver. 364 **vdpa** device will work as vdpa device and will be probed by vdpa/sfc driver. 365 If this parameter is not specified then ef100 device will operate as 366 network device. 367 368- ``rx_datapath`` [auto|efx|ef10|ef10_essb] (default **auto**) 369 370 Choose receive datapath implementation. 371 **auto** allows the driver itself to make a choice based on firmware 372 features available and required by the datapath implementation. 373 **efx** chooses libefx-based datapath which supports Rx scatter. 374 Supported for SFN7xxx, SFN8xxx and X2xxx family adapters only. 375 **ef10** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which is 376 more efficient than libefx-based and provides richer packet type 377 classification. 378 **ef10_essb** chooses SFNX2xxx equal stride super-buffer datapath 379 which may be used on DPDK firmware variant only 380 (see notes about its limitations above). 381 **ef100** chooses EF100 native datapath which is the only supported 382 Rx datapath for EF100 architecture based NICs. 383 384- ``tx_datapath`` [auto|efx|ef10|ef10_simple|ef100] (default **auto**) 385 386 Choose transmit datapath implementation. 387 **auto** allows the driver itself to make a choice based on firmware 388 features available and required by the datapath implementation. 389 **efx** chooses libefx-based datapath which supports VLAN insertion 390 (full-feature firmware variant only), TSO and multi-segment mbufs. 391 Mbuf segments may come from different mempools, and mbuf reference 392 counters are treated responsibly. 393 Supported for SFN7xxx, SFN8xxx and X2xxx family adapters only. 394 **ef10** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which is 395 more efficient than libefx-based but has no VLAN insertion support yet. 396 Mbuf segments may come from different mempools, and mbuf reference 397 counters are treated responsibly. 398 **ef10_simple** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which 399 is even more faster then **ef10** but does not support multi-segment 400 mbufs, disallows multiple mempools and neglects mbuf reference counters. 401 **ef100** chooses EF100 native datapath which supports multi-segment 402 mbufs, VLAN insertion, inner/outer IPv4 and TCP/UDP checksum and TCP 403 segmentation offloads including VXLAN and GENEVE IPv4/IPv6 tunnels. 404 405- ``perf_profile`` [auto|throughput|low-latency] (default **throughput**) 406 407 Choose hardware tuning to be optimized for either throughput or 408 low-latency. 409 **auto** allows NIC firmware to make a choice based on 410 installed licenses and firmware variant configured using **sfboot**. 411 412- ``stats_update_period_ms`` [long] (default **1000**) 413 414 Adjust period in milliseconds to update port hardware statistics. 415 The accepted range is 0 to 65535. The value of **0** may be used 416 to disable periodic statistics update. One should note that it's 417 only possible to set an arbitrary value on SFN8xxx and X2xxx provided that 418 firmware version is 6.2.1.1033 or higher, otherwise any positive 419 value will select a fixed update period of **1000** milliseconds 420 421- ``fw_variant`` [dont-care|full-feature|ultra-low-latency| 422 capture-packed-stream|dpdk] (default **dont-care**) 423 424 Choose the preferred firmware variant to use. 425 The parameter is supported for SFN7xxX, SFN8xxx and X2xxx families only. 426 In order for the selected option to have an effect, the **sfboot** utility 427 must be configured with the **auto** firmware-variant option. 428 The preferred firmware variant applies to all ports on the NIC. 429 **dont-care** ensures that the driver can attach to an unprivileged function. 430 The datapath firmware type to use is controlled by the **sfboot** 431 utility. 432 **full-feature** chooses full featured firmware. 433 **ultra-low-latency** chooses firmware with fewer features but lower latency. 434 **capture-packed-stream** chooses firmware for SolarCapture packed stream 435 mode. 436 **dpdk** chooses DPDK firmware with equal stride super-buffer Rx mode 437 for higher Rx packet rate and packet marks support and firmware subvariant 438 without checksumming on transmit for higher Tx packet rate if 439 checksumming is not required. 440 441- ``rxd_wait_timeout_ns`` [long] (default **200 us**) 442 443 Adjust timeout in nanoseconds to head-of-line block to wait for 444 Rx descriptors. 445 The accepted range is 0 to 400 ms. 446 Flow control should be enabled to make it work. 447 The value of **0** disables it and packets are dropped immediately. 448 When a packet is dropped because of no Rx descriptors, 449 ``rx_nodesc_drop_cnt`` counter grows. 450 The feature is supported only by the DPDK firmware variant when equal 451 stride super-buffer Rx mode is used. 452 453 454Dynamic Logging Parameters 455~~~~~~~~~~~~~~~~~~~~~~~~~~ 456 457One may leverage EAL option "--log-level" to change default levels 458for the log types supported by the driver. The option is used with 459an argument typically consisting of two parts separated by a colon. 460 461Level value is the last part which takes a symbolic name (or integer). 462Log type is the former part which may shell match syntax. 463Depending on the choice of the expression, the given log level may 464be used either for some specific log type or for a subset of types. 465 466SFC EFX PMD provides the following log types available for control: 467 468- ``pmd.net.sfc.driver`` (default level is **notice**) 469 470 Affects driver-wide messages unrelated to any particular devices. 471 472- ``pmd.net.sfc.main`` (default level is **notice**) 473 474 Matches a subset of per-port log types registered during runtime. 475 A full name for a particular type may be obtained by appending a 476 dot and a PCI device identifier (``XXXX:XX:XX.X``) to the prefix. 477 478- ``pmd.net.sfc.mcdi`` (default level is **notice**) 479 480 Extra logging of the communication with the NIC's management CPU. 481 The format of the log is consumed by the Solarflare netlogdecode 482 cross-platform tool. May be managed per-port, as explained above. 483