1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2019-2021 Xilinx, Inc. 3 Copyright(c) 2016-2019 Solarflare Communications Inc. 4 5 This software was jointly developed between OKTET Labs (under contract 6 for Solarflare) and Solarflare Communications, Inc. 7 8Solarflare libefx-based Poll Mode Driver 9======================================== 10 11The SFC EFX PMD (**librte_net_sfc_efx**) provides poll mode driver support 12for **Solarflare SFN7xxx and SFN8xxx** family of 10/40 Gbps adapters, 13**Solarflare XtremeScale X2xxx** family of 10/25/40/50/100 Gbps adapters and 14**Alveo SN1000 SmartNICs** family of 10/25/40/50/100 Gbps adapters. 15SFC EFX PMD has support for the latest Linux and FreeBSD operating systems. 16 17More information can be found at `Solarflare Communications website 18<http://solarflare.com>`_. 19 20 21Supported NICs 22-------------- 23 24- Xilinx Adapters: 25 26 - Alveo SN1022 SmartNIC 27 28- Solarflare XtremeScale Adapters: 29 30 - Solarflare X2522 Dual Port SFP28 10/25GbE Adapter 31 32 - Solarflare X2541 Single Port QSFP28 10/25G/100G Adapter 33 34 - Solarflare X2542 Dual Port QSFP28 10/25G/100G Adapter 35 36- Solarflare Flareon [Ultra] Server Adapters: 37 38 - Solarflare SFN8522 Dual Port SFP+ Server Adapter 39 40 - Solarflare SFN8522M Dual Port SFP+ Server Adapter 41 42 - Solarflare SFN8042 Dual Port QSFP+ Server Adapter 43 44 - Solarflare SFN8542 Dual Port QSFP+ Server Adapter 45 46 - Solarflare SFN8722 Dual Port SFP+ OCP Server Adapter 47 48 - Solarflare SFN7002F Dual Port SFP+ Server Adapter 49 50 - Solarflare SFN7004F Quad Port SFP+ Server Adapter 51 52 - Solarflare SFN7042Q Dual Port QSFP+ Server Adapter 53 54 - Solarflare SFN7122F Dual Port SFP+ Server Adapter 55 56 - Solarflare SFN7124F Quad Port SFP+ Server Adapter 57 58 - Solarflare SFN7142Q Dual Port QSFP+ Server Adapter 59 60 - Solarflare SFN7322F Precision Time Synchronization Server Adapter 61 62 63Features 64-------- 65 66SFC EFX PMD has support for: 67 68- Multiple transmit and receive queues 69 70- Link state information including link status change interrupt 71 72- IPv4/IPv6 TCP/UDP transmit checksum offload 73 74- Inner IPv4/IPv6 TCP/UDP transmit checksum offload 75 76- Port hardware statistics 77 78- Extended statistics (see Solarflare Server Adapter User's Guide for 79 the statistics description) 80 81- Basic flow control 82 83- MTU update 84 85- Jumbo frames up to 9K 86 87- Promiscuous mode 88 89- Allmulticast mode 90 91- TCP segmentation offload (TSO) including VXLAN and GENEVE encapsulated 92 93- Multicast MAC filter 94 95- IPv4/IPv6 TCP/UDP receive checksum offload 96 97- Inner IPv4/IPv6 TCP/UDP receive checksum offload 98 99- Received packet type information 100 101- Receive side scaling (RSS) 102 103- RSS hash 104 105- Scattered Rx DMA for packet that are larger that a single Rx descriptor 106 107- Receive queue interrupts 108 109- Deferred receive and transmit queue start 110 111- Transmit VLAN insertion (if running firmware variant supports it) 112 113- Flow API 114 115- Loopback 116 117- Configurable Rx CRC stripping (if running firmware variant supports it and 118 if NIC is configured with single PF per port and without VFs, otherwise 119 always stripped) 120 121- SR-IOV PF 122 123- Port representors (see :ref: switch_representation) 124 125- VLAN stripping (if running firmware variant supports it) 126 127 128Non-supported Features 129---------------------- 130 131The features not yet supported include: 132 133- Priority-based flow control 134 135- Header split on receive 136 137- VLAN filtering 138 139- LRO 140 141 142Limitations 143----------- 144 145Due to requirements on receive buffer alignment and usage of the receive 146buffer for the auxiliary packet information provided by the NIC up to 147extra 269 (14 bytes prefix plus up to 255 bytes for end padding) bytes may be 148required in the receive buffer. 149It should be taken into account when mbuf pool for receive is created. 150 151 152Equal stride super-buffer mode 153~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 154 155When the receive queue uses equal stride super-buffer DMA mode, one HW Rx 156descriptor carries many Rx buffers which contiguously follow each other 157with some stride (equal to total size of rte_mbuf as mempool object). 158Each Rx buffer is an independent rte_mbuf. 159However dedicated mempool manager must be used when mempool for the Rx 160queue is created. The manager must support dequeue of the contiguous 161block of objects and provide mempool info API to get the block size. 162 163Another limitation of a equal stride super-buffer mode, imposed by the 164firmware, is that it allows for a single RSS context. 165 166 167Tunnels support 168--------------- 169 170NVGRE, VXLAN and GENEVE tunnels are supported on SFN8xxx and X2xxx family 171adapters with full-feature firmware variant running. 172**sfboot** should be used to configure NIC to run full-feature firmware variant. 173See Solarflare Server Adapter User's Guide for details. 174 175SFN8xxx and X2xxx family adapters provide either inner or outer packet classes. 176If adapter firmware advertises support for tunnels then the PMD 177configures the hardware to report inner classes, and outer classes are 178not reported in received packets. 179However, for VXLAN and GENEVE tunnels the PMD does report UDP as the 180outer layer 4 packet type. 181 182SFN8xxx and X2xxx family adapters report GENEVE packets as VXLAN. 183If UDP ports are configured for only one tunnel type then it is safe to 184treat VXLAN packet type indication as the corresponding UDP tunnel type. 185 186 187Flow API support 188---------------- 189 190Supported attributes: 191 192- Ingress 193- Transfer 194 195Supported pattern items (***non-transfer*** rules): 196 197- VOID 198 199- ETH (exact match of source/destination addresses, individual/group match 200 of destination address, EtherType in the outer frame and exact match of 201 destination addresses, individual/group match of destination address in 202 the inner frame) 203 204- VLAN (exact match of VID, double-tagging is supported) 205 206- IPV4 (exact match of source/destination addresses, 207 IP transport protocol) 208 209- IPV6 (exact match of source/destination addresses, 210 IP transport protocol) 211 212- TCP (exact match of source/destination ports) 213 214- UDP (exact match of source/destination ports) 215 216- VXLAN (exact match of VXLAN network identifier) 217 218- GENEVE (exact match of virtual network identifier, only Ethernet (0x6558) 219 protocol type is supported) 220 221- NVGRE (exact match of virtual subnet ID) 222 223Supported actions (***non-transfer*** rules): 224 225- VOID 226 227- QUEUE 228 229- RSS 230 231- DROP 232 233- FLAG (supported only with ef10_essb Rx datapath) 234 235- MARK (supported only with ef10_essb Rx datapath) 236 237Supported pattern items (***transfer*** rules): 238 239- PORT_REPRESENTOR (cannot repeat; conflicts with other traffic source items) 240 241- REPRESENTED_PORT (cannot repeat; conflicts with other traffic source items) 242 243- PORT_ID (cannot repeat; conflicts with other traffic source items) 244 245- ETH 246 247- VLAN (double-tagging is supported) 248 249- IPV4 (source/destination addresses, IP transport protocol, 250 type of service, time to live) 251 252- IPV6 (source/destination addresses, IP transport protocol, 253 traffic class, hop limit) 254 255- TCP (source/destination ports, TCP header length + TCP flags) 256 257- UDP (source/destination ports) 258 259- VXLAN (exact match of VXLAN network identifier) 260 261- GENEVE (exact match of virtual network identifier) 262 263- NVGRE (exact match of virtual subnet ID) 264 265Supported actions (***transfer*** rules): 266 267- OF_POP_VLAN 268 269- OF_PUSH_VLAN 270 271- OF_VLAN_SET_VID 272 273- OF_VLAN_SET_PCP 274 275- SET_IPV4_DST 276 277- SET_IPV4_SRC 278 279- SET_MAC_DST 280 281- SET_MAC_SRC 282 283- SET_TP_DST 284 285- SET_TP_SRC 286 287- OF_DEC_NW_TTL 288 289- DEC_TTL 290 291- VXLAN_DECAP 292 293- VXLAN_ENCAP 294 295- FLAG 296 297- MARK 298 299- PF 300 301- VF 302 303- PORT_REPRESENTOR 304 305- REPRESENTED_PORT 306 307- PORT_ID 308 309- COUNT 310 311- INDIRECT 312 313- DROP 314 315Validating flow rules depends on the firmware variant. 316 317The :ref:`flow_isolated_mode` is supported. 318 319The implementation is natively thread-safe. 320 321Ethernet destination individual/group match 322~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 323 324Ethernet item supports I/G matching, if only the corresponding bit is set 325in the mask of destination address. If destination address in the spec is 326multicast, it matches all multicast (and broadcast) packets, otherwise it 327matches unicast packets that are not filtered by other flow rules. 328 329Exceptions to flow rules 330~~~~~~~~~~~~~~~~~~~~~~~~ 331 332There is a list of exceptional flow rule patterns which will not be 333accepted by the PMD. A pattern will be rejected if at least one of the 334conditions is met: 335 336- Filtering by IPv4 or IPv6 EtherType without pattern items of internet 337 layer and above. 338 339- The last item is IPV4 or IPV6, and it's empty. 340 341- Filtering by TCP or UDP IP transport protocol without pattern items of 342 transport layer and above. 343 344- The last item is TCP or UDP, and it's empty. 345 346 347Prerequisites 348------------- 349 350- Requires firmware version: 351 352 - SFN7xxx: **4.7.1.1001** or higher 353 354 - SFN8xxx: **6.0.2.1004** or higher 355 356Visit `Solarflare Support Downloads <https://support.solarflare.com>`_ to get 357Solarflare Utilities (either Linux or FreeBSD) with the latest firmware. 358Follow instructions from Solarflare Server Adapter User's Guide to 359update firmware and configure the adapter. 360 361 362Configuration 363------------- 364 365 366Compilation Options 367~~~~~~~~~~~~~~~~~~~ 368 369The following build-time options may be enabled on build time using 370``-Dc_args=`` meson argument (e.g. ``-Dc_args=-DRTE_LIBRTE_SFC_EFX_DEBUG``). 371 372Please note that enabling debugging options may affect system performance. 373 374- ``RTE_LIBRTE_SFC_EFX_DEBUG`` (undefined by default) 375 376 Enable compilation of the extra run-time consistency checks. 377 378 379Runtime Configuration 380~~~~~~~~~~~~~~~~~~~~~ 381 382The following per-device parameters can be passed via EAL PCI device 383allow option like "-a 02:00.0,arg1=value1,...". 384 385Case-insensitive 1/y/yes/on or 0/n/no/off may be used to specify 386boolean parameters value. 387 388- ``class`` [net|vdpa] (default **net**) 389 390 Choose the mode of operation of ef100 device. 391 **net** device will work as network device and will be probed by net/sfc driver. 392 **vdpa** device will work as vdpa device and will be probed by vdpa/sfc driver. 393 If this parameter is not specified then ef100 device will operate as 394 network device. 395 396- ``switch_mode`` [legacy|switchdev] (see below for default) 397 398 In legacy mode, NIC firmware provides Ethernet virtual bridging (EVB) API 399 to configure switching inside NIC to deliver traffic to physical (PF) and 400 virtual (VF) PCI functions. PF driver is responsible to build the 401 infrastructure for VFs, and traffic goes to/from VF by default in accordance 402 with MAC address assigned, permissions and filters installed by VF drivers. 403 In switchdev mode VF traffic goes via port representor (if any) on PF, and 404 software virtual switch (for example, Open vSwitch) makes the decision. 405 Software virtual switch may install MAE rules to pass established traffic 406 flows via hardware and offload software datapath as the result. 407 Default is legacy, unless representors are specified, in which case switchdev 408 is chosen. 409 410- ``representor`` parameter [list] 411 412 Instantiate port representor Ethernet devices for specified Virtual 413 Functions list. 414 415 It is a standard parameter whose format is described in 416 :ref:`ethernet_device_standard_device_arguments`. 417 418- ``rx_datapath`` [auto|efx|ef10|ef10_essb] (default **auto**) 419 420 Choose receive datapath implementation. 421 **auto** allows the driver itself to make a choice based on firmware 422 features available and required by the datapath implementation. 423 **efx** chooses libefx-based datapath which supports Rx scatter. 424 Supported for SFN7xxx, SFN8xxx and X2xxx family adapters only. 425 **ef10** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which is 426 more efficient than libefx-based and provides richer packet type 427 classification. 428 **ef10_essb** chooses SFNX2xxx equal stride super-buffer datapath 429 which may be used on DPDK firmware variant only 430 (see notes about its limitations above). 431 **ef100** chooses EF100 native datapath which is the only supported 432 Rx datapath for EF100 architecture based NICs. 433 434- ``tx_datapath`` [auto|efx|ef10|ef10_simple|ef100] (default **auto**) 435 436 Choose transmit datapath implementation. 437 **auto** allows the driver itself to make a choice based on firmware 438 features available and required by the datapath implementation. 439 **efx** chooses libefx-based datapath which supports VLAN insertion 440 (full-feature firmware variant only), TSO and multi-segment mbufs. 441 Mbuf segments may come from different mempools, and mbuf reference 442 counters are treated responsibly. 443 Supported for SFN7xxx, SFN8xxx and X2xxx family adapters only. 444 **ef10** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which is 445 more efficient than libefx-based but has no VLAN insertion support yet. 446 Mbuf segments may come from different mempools, and mbuf reference 447 counters are treated responsibly. 448 **ef10_simple** chooses EF10 (SFN7xxx, SFN8xxx, X2xxx) native datapath which 449 is even more faster then **ef10** but does not support multi-segment 450 mbufs, disallows multiple mempools and neglects mbuf reference counters. 451 **ef100** chooses EF100 native datapath which supports multi-segment 452 mbufs, VLAN insertion, inner/outer IPv4 and TCP/UDP checksum and TCP 453 segmentation offloads including VXLAN and GENEVE IPv4/IPv6 tunnels. 454 455- ``perf_profile`` [auto|throughput|low-latency] (default **throughput**) 456 457 Choose hardware tuning to be optimized for either throughput or 458 low-latency. 459 **auto** allows NIC firmware to make a choice based on 460 installed licenses and firmware variant configured using **sfboot**. 461 462- ``stats_update_period_ms`` [long] (default **1000**) 463 464 Adjust period in milliseconds to update port hardware statistics. 465 The accepted range is 0 to 65535. The value of **0** may be used 466 to disable periodic statistics update. One should note that it's 467 only possible to set an arbitrary value on SFN8xxx and X2xxx provided that 468 firmware version is 6.2.1.1033 or higher, otherwise any positive 469 value will select a fixed update period of **1000** milliseconds 470 471- ``fw_variant`` [dont-care|full-feature|ultra-low-latency| 472 capture-packed-stream|dpdk] (default **dont-care**) 473 474 Choose the preferred firmware variant to use. 475 The parameter is supported for SFN7xxX, SFN8xxx and X2xxx families only. 476 In order for the selected option to have an effect, the **sfboot** utility 477 must be configured with the **auto** firmware-variant option. 478 The preferred firmware variant applies to all ports on the NIC. 479 **dont-care** ensures that the driver can attach to an unprivileged function. 480 The datapath firmware type to use is controlled by the **sfboot** 481 utility. 482 **full-feature** chooses full featured firmware. 483 **ultra-low-latency** chooses firmware with fewer features but lower latency. 484 **capture-packed-stream** chooses firmware for SolarCapture packed stream 485 mode. 486 **dpdk** chooses DPDK firmware with equal stride super-buffer Rx mode 487 for higher Rx packet rate and packet marks support and firmware subvariant 488 without checksumming on transmit for higher Tx packet rate if 489 checksumming is not required. 490 491- ``rxd_wait_timeout_ns`` [long] (default **200 us**) 492 493 Adjust timeout in nanoseconds to head-of-line block to wait for 494 Rx descriptors. 495 The accepted range is 0 to 400 ms. 496 Flow control should be enabled to make it work. 497 The value of **0** disables it and packets are dropped immediately. 498 When a packet is dropped because of no Rx descriptors, 499 ``rx_nodesc_drop_cnt`` counter grows. 500 The feature is supported only by the DPDK firmware variant when equal 501 stride super-buffer Rx mode is used. 502 503 504Dynamic Logging Parameters 505~~~~~~~~~~~~~~~~~~~~~~~~~~ 506 507One may leverage EAL option "--log-level" to change default levels 508for the log types supported by the driver. The option is used with 509an argument typically consisting of two parts separated by a colon. 510 511Level value is the last part which takes a symbolic name (or integer). 512Log type is the former part which may shell match syntax. 513Depending on the choice of the expression, the given log level may 514be used either for some specific log type or for a subset of types. 515 516SFC EFX PMD provides the following log types available for control: 517 518- ``pmd.net.sfc.driver`` (default level is **notice**) 519 520 Affects driver-wide messages unrelated to any particular devices. 521 522- ``pmd.net.sfc.main`` (default level is **notice**) 523 524 Matches a subset of per-port log types registered during runtime. 525 A full name for a particular type may be obtained by appending a 526 dot and a PCI device identifier (``XXXX:XX:XX.X``) to the prefix. 527 528- ``pmd.net.sfc.mcdi`` (default level is **notice**) 529 530 Extra logging of the communication with the NIC's management CPU. 531 The format of the log is consumed by the Solarflare netlogdecode 532 cross-platform tool. May be managed per-port, as explained above. 533