145a49cc7SRasesh Mody.. SPDX-License-Identifier: BSD-3-Clause 247a298e7SRasesh Mody Copyright (c) 2015 QLogic Corporation 347a298e7SRasesh Mody 447a298e7SRasesh ModyBNX2X Poll Mode Driver 547a298e7SRasesh Mody====================== 647a298e7SRasesh Mody 78809f78cSBruce RichardsonThe BNX2X poll mode driver library (**librte_net_bnx2x**) implements support 847a298e7SRasesh Modyfor **QLogic 578xx** 10/20 Gbps family of adapters as well as their virtual 947a298e7SRasesh Modyfunctions (VF) in SR-IOV context. It is supported on several standard Linux 102efba79bSRasesh Modydistros like RHEL and SLES. It is compile-tested under FreeBSD OS. 1147a298e7SRasesh Mody 1247a298e7SRasesh ModyMore information can be found at `QLogic Corporation's Official Website 1347a298e7SRasesh Mody<http://www.qlogic.com>`_. 1447a298e7SRasesh Mody 1547a298e7SRasesh ModySupported Features 1647a298e7SRasesh Mody------------------ 1747a298e7SRasesh Mody 1847a298e7SRasesh ModyBNX2X PMD has support for: 1947a298e7SRasesh Mody 2047a298e7SRasesh Mody- Base L2 features 2147a298e7SRasesh Mody- Unicast/multicast filtering 2247a298e7SRasesh Mody- Promiscuous mode 2347a298e7SRasesh Mody- Port hardware statistics 2447a298e7SRasesh Mody- SR-IOV VF 25*048ca647SJulien Aube- Experimental 2.5Gbps support 2647a298e7SRasesh Mody 2747a298e7SRasesh ModyNon-supported Features 2847a298e7SRasesh Mody---------------------- 2947a298e7SRasesh Mody 3047a298e7SRasesh ModyThe features not yet supported include: 3147a298e7SRasesh Mody 3247a298e7SRasesh Mody- TSS (Transmit Side Scaling) 3347a298e7SRasesh Mody- RSS (Receive Side Scaling) 3447a298e7SRasesh Mody- LRO/TSO offload 3547a298e7SRasesh Mody- Checksum offload 3647a298e7SRasesh Mody- SR-IOV PF 378ed4d808SRasesh Mody- Rx TX scatter gather 3847a298e7SRasesh Mody 3947a298e7SRasesh ModyCo-existence considerations 4047a298e7SRasesh Mody--------------------------- 4147a298e7SRasesh Mody 422efba79bSRasesh Mody- QLogic 578xx CNAs support Ethernet, iSCSI and FCoE functionalities. 432efba79bSRasesh Mody These functionalities are supported using QLogic Linux kernel 442efba79bSRasesh Mody drivers bnx2x, cnic, bnx2i and bnx2fc. DPDK is supported on these 452efba79bSRasesh Mody adapters using bnx2x PMD. 4647a298e7SRasesh Mody 472efba79bSRasesh Mody- When SR-IOV is not enabled on the adapter, 482efba79bSRasesh Mody QLogic Linux kernel drivers (bnx2x, cnic, bnx2i and bnx2fc) and bnx2x 492efba79bSRasesh Mody PMD can’t be attached to different PFs on a given QLogic 578xx 502efba79bSRasesh Mody adapter. 512efba79bSRasesh Mody A given adapter needs to be completely used by DPDK or Linux drivers. 522efba79bSRasesh Mody Before binding DPDK driver to one or more PFs on the adapter, 532efba79bSRasesh Mody please make sure to unbind Linux drivers from all PFs of the adapter. 542efba79bSRasesh Mody If there are multiple adapters on the system, one or more adapters 552efba79bSRasesh Mody can be used by DPDK driver completely and other adapters can be used 562efba79bSRasesh Mody by Linux drivers completely. 5747a298e7SRasesh Mody 582efba79bSRasesh Mody- When SR-IOV is enabled on the adapter, 592efba79bSRasesh Mody Linux kernel drivers (bnx2x, cnic, bnx2i and bnx2fc) can be bound 602efba79bSRasesh Mody to the PFs of a given adapter and either bnx2x PMD or Linux drivers 612efba79bSRasesh Mody bnx2x can be bound to the VFs of the adapter. 6247a298e7SRasesh Mody 6347a298e7SRasesh ModySupported QLogic NICs 6447a298e7SRasesh Mody--------------------- 6547a298e7SRasesh Mody 6647a298e7SRasesh Mody- 578xx 6747a298e7SRasesh Mody 6847a298e7SRasesh ModyPrerequisites 6947a298e7SRasesh Mody------------- 7047a298e7SRasesh Mody 710cb4150fSRasesh Mody- Requires firmware version **7.13.11.0**. It is included in most of the 7247a298e7SRasesh Mody standard Linux distros. If it is not available visit 730cb4150fSRasesh Mody `linux-firmware git repository <https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/bnx2x/bnx2x-e2-7.13.11.0.fw>`_ 7447a298e7SRasesh Mody to get the required firmware. 7547a298e7SRasesh Mody 76*048ca647SJulien Aube- 2.5Gbps speed currently requires that the firmware's NVM configuration 77*048ca647SJulien Aube number 7 / 35 (first port) and 36 (second port) are set to 0x70 78*048ca647SJulien Aube for 1G/2.5G/10G support. 79*048ca647SJulien Aube This can be done for EFI or DOS using EDIAG tool from Broadcom. 80*048ca647SJulien Aube 8147a298e7SRasesh ModyPre-Installation Configuration 8247a298e7SRasesh Mody------------------------------ 8347a298e7SRasesh Mody 8468d99d00SCiara PowerConfig Options 8568d99d00SCiara Power~~~~~~~~~~~~~~ 8647a298e7SRasesh Mody 8768d99d00SCiara PowerThe following options can be enabled with Meson flags. Please note that 8847a298e7SRasesh Modyenabling debugging options may affect system performance. 8947a298e7SRasesh Mody 9068d99d00SCiara Power- ``RTE_LIBRTE_BNX2X_DEBUG_TX`` (default **disabled**) 9147a298e7SRasesh Mody 9247a298e7SRasesh Mody Toggle display of transmit fast path run-time messages. 9347a298e7SRasesh Mody 9468d99d00SCiara Power- ``RTE_LIBRTE_BNX2X_DEBUG_RX`` (default **disabled**) 9547a298e7SRasesh Mody 9647a298e7SRasesh Mody Toggle display of receive fast path run-time messages. 9747a298e7SRasesh Mody 9868d99d00SCiara Power- ``RTE_LIBRTE_BNX2X_DEBUG_PERIODIC`` (default **disabled**) 9947a298e7SRasesh Mody 10047a298e7SRasesh Mody Toggle display of register reads and writes. 10147a298e7SRasesh Mody 10247a298e7SRasesh Mody.. _bnx2x_driver-compilation: 10347a298e7SRasesh Mody 10446161756SShijith ThottonDriver compilation and testing 10546161756SShijith Thotton------------------------------ 10647a298e7SRasesh Mody 10746161756SShijith ThottonRefer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` 10846161756SShijith Thottonfor details. 10947a298e7SRasesh Mody 11080a73194SRasesh ModyJumbo: Limitation 11180a73194SRasesh Mody----------------- 11280a73194SRasesh Mody 11380a73194SRasesh ModyRx descriptor limit for number of segments per MTU is set to 1. 1149c30a6f3SHenry NadeauPMD doesn't support Jumbo Rx scatter gather. Some applications can 11580a73194SRasesh Modyadjust mbuf_size based on this param and max_pkt_len. 11680a73194SRasesh Mody 11780a73194SRasesh ModyFor others, PMD detects the condition where Rx packet length cannot 11880a73194SRasesh Modybe held by configured mbuf size and logs the message. 11980a73194SRasesh Mody 12080a73194SRasesh ModyExample output: 12180a73194SRasesh Mody 12280a73194SRasesh Mody .. code-block:: console 12380a73194SRasesh Mody 12480a73194SRasesh Mody [...] 12580a73194SRasesh Mody [bnx2x_recv_pkts:397(04:00.0:dpdk-port-0)] mbuf size 2048 is not enough to hold Rx packet length more than 2046 12680a73194SRasesh Mody 12747a298e7SRasesh ModySR-IOV: Prerequisites and sample Application Notes 12846161756SShijith Thotton-------------------------------------------------- 12947a298e7SRasesh Mody 13047a298e7SRasesh ModyThis section provides instructions to configure SR-IOV with Linux OS. 13147a298e7SRasesh Mody 13247a298e7SRasesh Mody#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``: 13347a298e7SRasesh Mody 13447a298e7SRasesh Mody .. code-block:: console 13547a298e7SRasesh Mody 13647a298e7SRasesh Mody lspci -s <slot> -vvv 13747a298e7SRasesh Mody 13847a298e7SRasesh Mody Example output: 13947a298e7SRasesh Mody 14047a298e7SRasesh Mody .. code-block:: console 14147a298e7SRasesh Mody 14247a298e7SRasesh Mody [...] 14347a298e7SRasesh Mody Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI) 14447a298e7SRasesh Mody [...] 14547a298e7SRasesh Mody Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV) 14647a298e7SRasesh Mody [...] 14747a298e7SRasesh Mody Kernel driver in use: igb_uio 14847a298e7SRasesh Mody 14947a298e7SRasesh Mody#. Load the kernel module: 15047a298e7SRasesh Mody 15147a298e7SRasesh Mody .. code-block:: console 15247a298e7SRasesh Mody 15347a298e7SRasesh Mody modprobe bnx2x 15447a298e7SRasesh Mody 15547a298e7SRasesh Mody Example output: 15647a298e7SRasesh Mody 15747a298e7SRasesh Mody .. code-block:: console 15847a298e7SRasesh Mody 15947a298e7SRasesh Mody systemd-udevd[4848]: renamed network interface eth0 to ens5f0 16047a298e7SRasesh Mody systemd-udevd[4848]: renamed network interface eth1 to ens5f1 16147a298e7SRasesh Mody 16247a298e7SRasesh Mody#. Bring up the PF ports: 16347a298e7SRasesh Mody 16447a298e7SRasesh Mody .. code-block:: console 16547a298e7SRasesh Mody 16647a298e7SRasesh Mody ifconfig ens5f0 up 16747a298e7SRasesh Mody ifconfig ens5f1 up 16847a298e7SRasesh Mody 16947a298e7SRasesh Mody#. Create VF device(s): 17047a298e7SRasesh Mody 17147a298e7SRasesh Mody Echo the number of VFs to be created into "sriov_numvfs" sysfs entry 17247a298e7SRasesh Mody of the parent PF. 17347a298e7SRasesh Mody 17447a298e7SRasesh Mody Example output: 17547a298e7SRasesh Mody 17647a298e7SRasesh Mody .. code-block:: console 17747a298e7SRasesh Mody 17847a298e7SRasesh Mody echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs 17947a298e7SRasesh Mody 18047a298e7SRasesh Mody#. Assign VF MAC address: 18147a298e7SRasesh Mody 1822fe68f32SJohn McNamara Assign MAC address to the VF using iproute2 utility. The syntax is: 18347a298e7SRasesh Mody ip link set <PF iface> vf <VF id> mac <macaddr> 18447a298e7SRasesh Mody 18547a298e7SRasesh Mody Example output: 18647a298e7SRasesh Mody 18747a298e7SRasesh Mody .. code-block:: console 18847a298e7SRasesh Mody 18947a298e7SRasesh Mody ip link set ens5f0 vf 0 mac 52:54:00:2f:9d:e8 19047a298e7SRasesh Mody 1912fe68f32SJohn McNamara#. PCI Passthrough: 19247a298e7SRasesh Mody 19347a298e7SRasesh Mody The VF devices may be passed through to the guest VM using virt-manager or 19447a298e7SRasesh Mody virsh etc. bnx2x PMD should be used to bind the VF devices in the guest VM 19547a298e7SRasesh Mody using the instructions outlined in the Application notes below. 19646161756SShijith Thotton 19746161756SShijith Thotton#. Running testpmd: 19884785018SRasesh Mody (Supply ``--log-level="pmd.net.bnx2x.driver",7`` to view informational messages): 19946161756SShijith Thotton 20046161756SShijith Thotton Follow instructions available in the document 20146161756SShijith Thotton :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` 20246161756SShijith Thotton to run testpmd. 20346161756SShijith Thotton 20446161756SShijith Thotton Example output: 20546161756SShijith Thotton 20646161756SShijith Thotton .. code-block:: console 20746161756SShijith Thotton 20846161756SShijith Thotton [...] 20946161756SShijith Thotton EAL: PCI device 0000:84:00.0 on NUMA socket 1 21046161756SShijith Thotton EAL: probe driver: 14e4:168e rte_bnx2x_pmd 21146161756SShijith Thotton EAL: PCI memory mapped at 0x7f14f6fe5000 21246161756SShijith Thotton EAL: PCI memory mapped at 0x7f14f67e5000 21346161756SShijith Thotton EAL: PCI memory mapped at 0x7f15fbd9b000 21446161756SShijith Thotton EAL: PCI device 0000:84:00.1 on NUMA socket 1 21546161756SShijith Thotton EAL: probe driver: 14e4:168e rte_bnx2x_pmd 21646161756SShijith Thotton EAL: PCI memory mapped at 0x7f14f5fe5000 21746161756SShijith Thotton EAL: PCI memory mapped at 0x7f14f57e5000 21846161756SShijith Thotton EAL: PCI memory mapped at 0x7f15fbd4f000 21946161756SShijith Thotton Interactive-mode selected 22046161756SShijith Thotton Configuring Port 0 (socket 0) 22146161756SShijith Thotton PMD: bnx2x_dev_tx_queue_setup(): fp[00] req_bd=512, thresh=512, 22246161756SShijith Thotton usable_bd=1020, total_bd=1024, 22346161756SShijith Thotton tx_pages=4 22446161756SShijith Thotton PMD: bnx2x_dev_rx_queue_setup(): fp[00] req_bd=128, thresh=0, 22546161756SShijith Thotton usable_bd=510, total_bd=512, 22646161756SShijith Thotton rx_pages=1, cq_pages=8 22746161756SShijith Thotton PMD: bnx2x_print_adapter_info(): 22846161756SShijith Thotton [...] 22946161756SShijith Thotton Checking link statuses... 23046161756SShijith Thotton Port 0 Link Up - speed 10000 Mbps - full-duplex 23146161756SShijith Thotton Port 1 Link Up - speed 10000 Mbps - full-duplex 23246161756SShijith Thotton Done 23346161756SShijith Thotton testpmd> 234