1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright 2020 Broadcom Inc. 3 4.. include:: <isonum.txt> 5 6BNXT Poll Mode Driver 7===================== 8 9The Broadcom BNXT PMD (**librte_net_bnxt**) implements support for adapters 10based on Ethernet controllers belonging to the Broadcom 11BCM5741X/BCM575XX NetXtreme-E\ |reg| Family of Ethernet Network Controllers. 12 13A complete list is in the Supported Network Adapters section. 14 15CPU Support 16----------- 17 18BNXT PMD supports multiple CPU architectures, including x86-32, x86-64, and ARMv8. 19 20Kernel Dependency 21----------------- 22 23BNXT PMD requires a kernel module (VFIO or UIO) for setting up a device, mapping 24device memory to userspace, registering interrupts, etc. 25VFIO is more secure than UIO, relying on IOMMU protection. 26UIO requires the IOMMU disabled or configured to pass-through mode. 27 28The BNXT PMD supports operating with: 29 30* Linux vfio-pci 31* Linux uio_pci_generic 32* Linux igb_uio 33* BSD nic_uio 34 35Running BNXT PMD 36---------------- 37 38Bind the device to one of the kernel modules listed above 39 40.. code-block:: console 41 42 ./dpdk-devbind.py -b vfio-pci|igb_uio|uio_pci_generic bus_id:device_id.function_id 43 44The BNXT PMD can run on PF or VF. 45 46PCI-SIG Single Root I/O Virtualization (SR-IOV) involves the direct assignment 47of part of the network port resources to guest operating systems using the 48SR-IOV standard. 49NIC is logically distributed among multiple virtual machines (VMs), while still 50having global data in common to share with the PF and other VFs. 51 52Sysadmin can create and configure VFs: 53 54.. code-block:: console 55 56 echo num_vfs > /sys/bus/pci/devices/domain_id:bus_id:device_id:function_id/sriov_numvfs 57 (ex) echo 4 > /sys/bus/pci/devices/0000:82:00:0/sriov_numvfs 58 59Sysadmin also can change the VF property such as MAC address, transparent VLAN, 60TX rate limit, and trusted VF: 61 62.. code-block:: console 63 64 ip link set pf_id vf vf_id mac (mac_address) vlan (vlan_id) txrate (rate_value) trust (enable|disable) 65 (ex) ip link set 0 vf 0 mac 00:11:22:33:44:55 vlan 0x100 txrate 100 trust disable 66 67Running on VF 68~~~~~~~~~~~~~ 69 70Flow Bifurcation 71^^^^^^^^^^^^^^^^ 72 73The Flow Bifurcation splits the incoming data traffic to user space applications 74(such as DPDK applications) and/or kernel space programs (such as the Linux 75kernel stack). 76It can direct some traffic, for example data plane traffic, to DPDK. 77Rest of the traffic, for example control plane traffic, would be redirected to 78the traditional Linux networking stack. 79 80Refer to :doc:`../howto/flow_bifurcation` 81 82Benefits of the flow bifurcation include: 83 84* Better performance with less CPU overhead, as user application can directly 85 access the NIC for data path 86* NIC is still being controlled by the kernel, as control traffic is forwarded 87 only to the kernel driver 88* Control commands, e.g. ethtool, will work as usual 89 90Running on a VF, the BXNT PMD supports the flow bifurcation with a combination 91of SR-IOV and packet classification and/or forwarding capability. 92In the simplest case of flow bifurcation, a PF driver configures a NIC to 93forward all user traffic directly to VFs with matching destination MAC address, 94while the rest of the traffic is forwarded to a PF. 95Note that the broadcast packets will be forwarded to both PF and VF. 96 97.. code-block:: console 98 99 (ex) ethtool --config-ntuple ens2f0 flow-type ether dst 00:01:02:03:00:01 vlan 10 vlan-mask 0xf000 action 0x100000000 100 101Trusted VF 102^^^^^^^^^^ 103 104By default, VFs are *not* allowed to perform privileged operations, such as 105modifying the VF’s MAC address in the guest. These security measures are 106designed to prevent possible attacks. 107However, when a DPDK application can be trusted (e.g., OVS-DPDK, 108`here <https://docs.openvswitch.org/en/latest/intro/install/dpdk/>`_), 109these operations performed by a VF would be legitimate and better be allowed. 110 111To enable VF to request "trusted mode," a new trusted VF concept was introduced 112in Linux kernel 4.4 and allowed VFs to become “trusted” and perform some 113privileged operations. 114 115The BNXT PMD supports the trusted VF mode of operation. Only a PF can enable the 116trusted attribute on the VF. It is preferable to enable the Trusted setting on a 117VF before starting applications. 118However, the BNXT PMD handles dynamic changes in trusted settings as well. 119 120Note that control commands, e.g., ethtool, will work via the kernel PF driver, 121*not* via the trusted VF driver. 122 123Operations supported by trusted VF: 124 125* MAC address configuration 126* Promiscuous mode setting 127* Flow rule creation 128 129Operations *not* supported by trusted VF: 130 131* Firmware upgrade 132 133Running on PF 134~~~~~~~~~~~~~ 135 136Unlike the VF when BNXT PMD runs on a PF there are no restrictions placed on the 137features which the PF can enable or request. In a multiport NIC, each port will 138have a corresponding PF. Also depending on the configuration of the NIC there 139can be more than one PF associated per port. 140A sysadmin can load the kernel driver on one PF, and run BNXT PMD on the other 141PF or run the PMD on both the PFs. In such cases, the firmware picks one of the 142PFs as a master PF. 143 144Much like in the trusted VF, the DPDK application must be *trusted* and expected 145to be *well-behaved*. 146 147Features 148-------- 149 150The BNXT PMD supports the following features: 151 152* Port Control 153 * Port MTU 154 * LED 155 * Flow Control and Autoneg 156* Packet Filtering 157 * Unicast MAC Filter 158 * Multicast MAC Filter 159 * VLAN Filtering 160 * Allmulticast Mode 161 * Promiscuous Mode 162* Stateless Offloads 163 * CRC Offload 164 * Checksum Offload (IPv4, TCP, and UDP) 165 * Multi-Queue (TSS and RSS) 166 * Segmentation and Reassembly (TSO and LRO) 167* VLAN insert strip 168* Stats Collection 169* Generic Flow Offload 170 171Port Control 172~~~~~~~~~~~~ 173 174**Port MTU**: BNXT PMD supports the MTU (Maximum Transmission Unit) up to 9,574 175bytes: 176 177.. code-block:: console 178 179 testpmd> port config mtu (port_id) mtu_value 180 testpmd> show port info (port_id) 181 182**LED**: Application tunes on (or off) a port LED, typically for a port 183identification: 184 185.. code-block:: console 186 187 int rte_eth_led_on (uint16_t port_id) 188 int rte_eth_led_off (uint16_t port_id) 189 190**Flow Control and Autoneg**: Application tunes on (or off) flow control and/or 191auto-negotiation on a port: 192 193.. code-block:: console 194 195 testpmd> set flow_ctrl rx (on|off) (port_id) 196 testpmd> set flow_ctrl tx (on|off) (port_id) 197 testpmd> set flow_ctrl autoneg (on|off) (port_id) 198 199Note that the BNXT PMD does *not* support some options and ignores them when 200requested: 201 202* high_water 203* low_water 204* pause_time 205* mac_ctrl_frame_fwd 206* send_xon 207 208Packet Filtering 209~~~~~~~~~~~~~~~~ 210 211Applications control the packet-forwarding behaviors with packet filters. 212 213The BNXT PMD supports hardware-based packet filtering: 214 215* UC (Unicast) MAC Filters 216 * No unicast packets are forwarded to an application except the one with 217 DMAC address added to the port 218 * At initialization, the station MAC address is added to the port 219* MC (Multicast) MAC Filters 220 * No multicast packets are forwarded to an application except the one with 221 MC address added to the port 222 * When the application listens to a multicast group, it adds the MC address 223 to the port 224* VLAN Filtering Mode 225 * When enabled, no packets are forwarded to an application except the ones 226 with the VLAN tag assigned to the port 227* Allmulticast Mode 228 * When enabled, every multicast packet received on the port is forwarded to 229 the application 230 * Typical usage is routing applications 231* Promiscuous Mode 232 * When enabled, every packet received on the port is forwarded to the 233 application 234 235Unicast MAC Filter 236^^^^^^^^^^^^^^^^^^ 237 238The application can add (or remove) MAC addresses to enable (or disable) 239filtering on MAC address used to accept packets. 240 241.. code-block:: console 242 243 testpmd> show port (port_id) macs 244 testpmd> mac_addr add port_id XX:XX:XX:XX:XX:XX 245 testpmd> mac_addr remove port_id XX:XX:XX:XX:XX:XX 246 247Multicast MAC Filter 248^^^^^^^^^^^^^^^^^^^^ 249 250The application can add (or remove) Multicast addresses that enable (or disable) 251filtering on multicast MAC address used to accept packets. 252 253.. code-block:: console 254 255 testpmd> show port (port_id) mcast_macs 256 testpmd> mcast_addr add port_id XX:XX:XX:XX:XX:XX 257 testpmd> mcast_addr remove port_id XX:XX:XX:XX:XX:XX 258 259Application adds (or removes) Multicast addresses to enable (or disable) 260allowlist filtering to accept packets. 261 262Note that the BNXT PMD supports up to 16 MC MAC filters. if the user adds more 263than 16 MC MACs, the BNXT PMD puts the port into the Allmulticast mode. 264 265VLAN Filtering 266^^^^^^^^^^^^^^ 267 268The application enables (or disables) VLAN filtering mode. When the mode is 269enabled, no packets are forwarded to an application except ones with VLAN tag 270assigned for the application. 271 272.. code-block:: console 273 274 testpmd> vlan set filter (on|off) (port_id) 275 testpmd> rx_vlan (add|rm) (vlan_id) (port_id) 276 277Allmulticast Mode 278^^^^^^^^^^^^^^^^^ 279 280The application enables (or disables) the allmulticast mode. When the mode is 281enabled, every multicast packet received is forwarded to the application. 282 283.. code-block:: console 284 285 testpmd> show port info (port_id) 286 testpmd> set allmulti (port_id) (on|off) 287 288Promiscuous Mode 289^^^^^^^^^^^^^^^^ 290 291The application enables (or disables) the promiscuous mode. When the mode is 292enabled on a port, every packet received on the port is forwarded to the 293application. 294 295.. code-block:: console 296 297 testpmd> show port info (port_id) 298 testpmd> set promisc port_id (on|off) 299 300Stateless Offloads 301~~~~~~~~~~~~~~~~~~ 302 303Like Linux, DPDK provides enabling hardware offload of some stateless processing 304(such as checksum calculation) of the stack, alleviating the CPU from having to 305burn cycles on every packet. 306 307Listed below are the stateless offloads supported by the BNXT PMD: 308 309* CRC offload (for both TX and RX packets) 310* Checksum Offload (for both TX and RX packets) 311 * IPv4 Checksum Offload 312 * TCP Checksum Offload 313 * UDP Checksum Offload 314* Segmentation/Reassembly Offloads 315 * TCP Segmentation Offload (TSO) 316 * Large Receive Offload (LRO) 317* Multi-Queue 318 * Transmit Side Scaling (TSS) 319 * Receive Side Scaling (RSS) 320 321Also, the BNXT PMD supports stateless offloads on inner frames for tunneled 322packets. Listed below are the tunneling protocols supported by the BNXT PMD: 323 324* VXLAN 325* GRE 326* NVGRE 327 328Note that enabling (or disabling) stateless offloads requires applications to 329stop DPDK before changing configuration. 330 331CRC Offload 332^^^^^^^^^^^ 333 334The FCS (Frame Check Sequence) in the Ethernet frame is a four-octet CRC (Cyclic 335Redundancy Check) that allows detection of corrupted data within the entire 336frame as received on the receiver side. 337 338The BNXT PMD supports hardware-based CRC offload: 339 340* TX: calculate and insert CRC 341* RX: check and remove CRC, notify the application on CRC error 342 343Note that the CRC offload is always turned on. 344 345Checksum Offload 346^^^^^^^^^^^^^^^^ 347 348The application enables hardware checksum calculation for IPv4, TCP, and UDP. 349 350.. code-block:: console 351 352 testpmd> port stop (port_id) 353 testpmd> csum set (ip|tcp|udp|outer-ip|outer-udp) (sw|hw) (port_id) 354 testpmd> set fwd csum 355 356Multi-Queue 357^^^^^^^^^^^ 358 359Multi-Queue, also known as TSS (Transmit Side Scaling) or RSS (Receive Side 360Scaling), is a common networking technique that allows for more efficient load 361balancing across multiple CPU cores. 362 363The application enables multiple TX and RX queues when it is started. 364 365.. code-block:: console 366 367 dpdk-testpmd -l 1,3,5 --main-lcore 1 --txq=2 –rxq=2 --nb-cores=2 368 369**TSS** 370 371TSS distributes network transmit processing across several hardware-based 372transmit queues, allowing outbound network traffic to be processed by multiple 373CPU cores. 374 375**RSS** 376 377RSS distributes network receive processing across several hardware-based receive 378queues, allowing inbound network traffic to be processed by multiple CPU cores. 379 380The application can select the RSS mode, i.e. select the header fields that are 381included for hash calculation. The BNXT PMD supports the RSS mode of 382``default|ip|tcp|udp|none``, where default mode is L3 and L4. 383 384For tunneled packets, RSS hash is calculated over inner frame header fields. 385Applications may want to select the tunnel header fields for hash calculation, 386and it will be supported in 20.08 using RSS level. 387 388.. code-block:: console 389 390 testpmd> port config (port_id) rss (all|default|ip|tcp|udp|none) 391 392 // note that the testpmd defaults the RSS mode to ip 393 // ensure to issue the command below to enable L4 header (TCP or UDP) along with IPv4 header 394 testpmd> port config (port_id) rss default 395 396 // to check the current RSS configuration, such as RSS function and RSS key 397 testpmd> show port (port_id) rss-hash key 398 399 // RSS is enabled by default. However, application can disable RSS as follows 400 testpmd> port config (port_id) rss none 401 402Application can change the flow distribution, i.e. remap the received traffic to 403CPU cores, using RSS RETA (Redirection Table). 404 405.. code-block:: console 406 407 // application queries the current RSS RETA configuration 408 testpmd> show port (port_id) rss reta size (mask0, mask1) 409 410 // application changes the RSS RETA configuration 411 testpmd> port config (port_id) rss reta (hash, queue) [, (hash, queue)] 412 413TSO 414^^^ 415 416TSO (TCP Segmentation Offload), also known as LSO (Large Send Offload), enables 417the TCP/IP stack to pass to the NIC a larger datagram than the MTU (Maximum 418Transmit Unit). NIC breaks it into multiple segments before sending it to the 419network. 420 421The BNXT PMD supports hardware-based TSO. 422 423.. code-block:: console 424 425 // display the status of TSO 426 testpmd> tso show (port_id) 427 428 // enable/disable TSO 429 testpmd> port config (port_id) tx_offload tcp_tso (on|off) 430 431 // set TSO segment size 432 testpmd> tso set segment_size (port_id) 433 434The BNXT PMD also supports hardware-based tunneled TSO. 435 436.. code-block:: console 437 438 // display the status of tunneled TSO 439 testpmd> tunnel_tso show (port_id) 440 441 // enable/disable tunneled TSO 442 testpmd> port config (port_id) tx_offload vxlan_tnl_tso|gre_tnl_tso (on|off) 443 444 // set tunneled TSO segment size 445 testpmd> tunnel_tso set segment_size (port_id) 446 447Note that the checksum offload is always assumed to be enabled for TSO. 448 449LRO 450^^^ 451 452LRO (Large Receive Offload) enables NIC to aggregate multiple incoming TCP/IP 453packets from a single stream into a larger buffer, before passing to the 454networking stack. 455 456The BNXT PMD supports hardware-based LRO. 457 458.. code-block:: console 459 460 // display the status of LRO 461 testpmd> show port (port_id) rx_offload capabilities 462 testpmd> show port (port_id) rx_offload configuration 463 464 // enable/disable LRO 465 testpmd> port config (port_id) rx_offload tcp_lro (on|off) 466 467 // set max LRO packet (datagram) size 468 testpmd> port config (port_id) max-lro-pkt-size (max_size) 469 470The BNXT PMD also supports tunneled LRO. 471 472Some applications, such as routing, should *not* change the packet headers as 473they pass through (i.e. received from and sent back to the network). In such a 474case, GRO (Generic Receive Offload) should be used instead of LRO. 475 476VLAN Insert/Strip 477~~~~~~~~~~~~~~~~~ 478 479DPDK application offloads VLAN insert/strip to improve performance. The BNXT PMD 480supports hardware-based VLAN insert/strip offload for both single and double 481VLAN packets. 482 483 484VLAN Insert 485^^^^^^^^^^^ 486 487Application configures the VLAN TPID (Tag Protocol ID). By default, the TPID is 4880x8100. 489 490.. code-block:: console 491 492 // configure outer TPID value for a port 493 testpmd> vlan set outer tpid (tpid_value) (port_id) 494 495The inner TPID set will be rejected as the BNXT PMD supports inserting only an 496outer VLAN. Note that when a packet has a single VLAN, the tag is considered as 497outer, i.e. the inner VLAN is relevant only when a packet is double-tagged. 498 499The BNXT PMD supports various TPID values shown below. Any other values will be 500rejected. 501 502* ``0x8100`` 503* ``0x88a8`` 504* ``0x9100`` 505* ``0x9200`` 506* ``0x9300`` 507 508The BNXT PMD supports the VLAN insert offload per-packet basis. The application 509provides the TCI (Tag Control Info) for a packet via mbuf. In turn, the BNXT PMD 510inserts the VLAN tag (via hardware) using the provided TCI along with the 511configured TPID. 512 513.. code-block:: console 514 515 // enable VLAN insert offload 516 testpmd> port config (port_id) rx_offload vlan_insert|qinq_insert (on|off) 517 518 if (mbuf->ol_flags && RTE_MBUF_F_TX_QINQ) // case-1: insert VLAN to single-tagged packet 519 tci_value = mbuf->vlan_tci_outer 520 else if (mbuf->ol_flags && RTE_MBUF_F_TX_VLAN) // case-2: insert VLAN to untagged packet 521 tci_value = mbuf->vlan_tci 522 523VLAN Strip 524^^^^^^^^^^ 525 526The application configures the per-port VLAN strip offload. 527 528.. code-block:: console 529 530 // enable VLAN strip on a port 531 testpmd> port config (port_id) tx_offload vlan_strip (on|off) 532 533 // notify application VLAN strip via mbuf 534 mbuf->ol_flags |= RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_STRIPPED // outer VLAN is found and stripped 535 mbuf->vlan_tci = tci_value // TCI of the stripped VLAN 536 537Time Synchronization 538~~~~~~~~~~~~~~~~~~~~ 539 540System operators may run a PTP (Precision Time Protocol) client application to 541synchronize the time on the NIC (and optionally, on the system) to a PTP time transmitter. 542 543The BNXT PMD supports a PTP client application to communicate with a PTP time transmitter 544using DPDK IEEE1588 APIs. 545Note that the PTP client application needs to 546run on PF and vector mode needs to be disabled. 547 548.. code-block:: console 549 550 testpmd> set fwd ieee1588 // enable IEEE 1588 mode 551 552When enabled, the BNXT PMD configures hardware to insert IEEE 1588 timestamps to 553the outgoing PTP packets and reports IEEE 1588 timestamps from the incoming PTP 554packets to application via mbuf. 555 556.. code-block:: console 557 558 // RX packet completion will indicate whether the packet is PTP 559 mbuf->ol_flags |= RTE_MBUF_F_RX_IEEE1588_PTP 560 561Statistics Collection 562~~~~~~~~~~~~~~~~~~~~~ 563 564In Linux, the *ethtool -S* enables us to query the NIC stats. DPDK provides the 565similar functionalities via rte_eth_stats and rte_eth_xstats. 566 567The BNXT PMD supports both basic and extended stats collection: 568 569* Basic stats 570* Extended stats 571 572Basic Stats 573^^^^^^^^^^^ 574 575The application collects per-port and per-queue stats using rte_eth_stats APIs. 576 577.. code-block:: console 578 579 testpmd> show port stats (port_id) 580 581Basic stats include: 582 583* ipackets 584* ibytes 585* opackets 586* obytes 587* imissed 588* ierrors 589* oerrors 590 591By default, per-queue stats for 16 queues are supported. For more than 16 592queues, BNXT PMD should be compiled with ``RTE_ETHDEV_QUEUE_STAT_CNTRS`` 593set to the desired number of queues. 594 595Extended Stats 596^^^^^^^^^^^^^^ 597 598Unlike basic stats, the extended stats are vendor-specific, i.e. each vendor 599provides its own set of counters. 600 601The BNXT PMD provides a rich set of counters, including per-flow counters, 602per-cos counters, per-priority counters, etc. 603 604.. code-block:: console 605 606 testpmd> show port xstats (port_id) 607 608Shown below is the elaborated sequence to retrieve extended stats: 609 610.. code-block:: console 611 612 // application queries the number of xstats 613 len = rte_eth_xstats_get(port_id, NULL, 0); 614 // BNXT PMD returns the size of xstats array (i.e. the number of entries) 615 // BNXT PMD returns 0, if the feature is compiled out or disabled 616 617 // application allocates memory for xstats 618 struct rte_eth_xstats_name *names; // name is 64 character or less 619 struct rte_eth_xstats *xstats; 620 names = calloc(len, sizeof(*names)); 621 xstats = calloc(len, sizeof(*xstats)); 622 623 // application retrieves xstats // names and values 624 ret = rte_eth_xstats_get_names(port_id, *names, len); 625 ret = rte_eth_xstats_get(port_id, *xstats, len); 626 627 // application checks the xstats 628 // application may repeat the below: 629 len = rte_eth_xstats_reset(port_id); // reset the xstats 630 631 // reset can be skipped, if application wants to see accumulated stats 632 // run traffic 633 // probably stop the traffic 634 // retrieve xstats // no need to retrieve xstats names again 635 // check xstats 636 637Generic Flow Offload 638~~~~~~~~~~~~~~~~~~~~ 639 640Applications can get benefit by offloading all or part of flow processing to 641hardware. For example, applications can offload packet classification only 642(partial offload) or whole match-action (full offload). 643 644DPDK offers the Generic Flow API (rte_flow API) to configure hardware to 645perform flow processing. 646 647TruFlow\ |reg| 648^^^^^^^^^^^^^^ 649 650To fully support the generic flow offload, TruFlow was introduced in BNXT PMD. 651Before TruFlow, hardware flow processing resources were 652mapped to and handled by firmware. 653With TruFlow, hardware flow processing resources are 654mapped to and handled by driver. 655 656Alleviating the limitations of firmware-based feature development, 657TruFlow not only increases the flow offload feature velocity 658but also improves the control plane performance (i.e., higher flow update rate). 659 660Flow APIs, Items, and Actions 661----------------------------- 662 663BNXT PMD supports thread-safe rte_flow operations for rte_flow APIs 664and rich set of flow items (i.e., matching patterns) and actions. 665Refer to the "Supported APIs" section for the list of rte_flow APIs 666as well as flow items and actions. 667 668Flow Persistency 669---------------- 670 671On stopping a device port, all the flows created on a port 672by the application will be flushed from the hardware 673and any tables maintained by the PMD. 674After stopping the device port, all flows on the port become invalid 675and are not represented in the system anymore. 676Instead of destroying or flushing such flows an application should discard 677all references to these flows and re-create the flows as required 678after the port is restarted. 679 680Note: A VNIC represents a virtual interface in the hardware. It is a resource 681in the RX path of the chip and is used to setup various target actions such as 682RSS, MAC filtering etc. for the physical function in use. 683 684Virtual Function Port Representors 685---------------------------------- 686 687The BNXT PMD supports the creation of VF port representors for the control 688and monitoring of BNXT virtual function devices. Each port representor 689corresponds to a single virtual function of that device that is connected to a 690VF. When there is no hardware flow offload, each packet transmitted by the VF 691will be received by the corresponding representor. Similarly each packet that is 692sent to a representor will be received by the VF. Applications can take 693advantage of this feature when SRIOV is enabled. The representor will allow the 694first packet that is transmitted by the VF to be received by the DPDK 695application which can then decide if the flow should be offloaded to the 696hardware. Once the flow is offloaded in the hardware, any packet matching the 697flow will be received by the VF while the DPDK application will not receive it 698any more. The BNXT PMD supports creation and handling of the port representors 699when the PMD is initialized on a PF or trusted-VF. The user can specify the list 700of VF IDs of the VFs for which the representors are needed by using the 701``devargs`` option ``representor``.:: 702 703 -a DBDF,representor=[0,1,4] 704 705Note that currently hot-plugging of representor ports is not supported so all 706the required representors must be specified on the creation of the PF or the 707trusted VF. 708 709Application Support 710------------------- 711 712Firmware 713~~~~~~~~ 714 715The BNXT PMD supports the application to retrieve the firmware version. 716 717.. code-block:: console 718 719 testpmd> show port info (port_id) 720 721Note that the applications cannot update the firmware using BNXT PMD. 722 723Multiple Processes 724~~~~~~~~~~~~~~~~~~ 725 726When two or more DPDK applications (e.g., testpmd and dpdk-pdump) share a single 727instance of DPDK, the BNXT PMD supports a single primary application and one or 728more secondary applications. Note that the DPDK-layer (not the PMD) ensures 729there is only one primary application. 730 731There are two modes: 732 733Manual mode 734 735* Application notifies whether it is primary or secondary using *proc-type* flag 736* 1st process should be spawned with ``--proc-type=primary`` 737* All subsequent processes should be spawned with ``--proc-type=secondary`` 738 739Auto detection mode 740 741* Application is using ``proc-type=auto`` flag 742* A process is spawned as a secondary if a primary is already running 743 744The BNXT PMD uses the info to skip a device initialization, i.e. performs a 745device initialization only when being brought up by a primary application. 746 747Runtime Queue Setup 748~~~~~~~~~~~~~~~~~~~ 749 750Typically, a DPDK application allocates TX and RX queues statically: i.e. queues 751are allocated at start. However, an application may want to increase (or 752decrease) the number of queues dynamically for various reasons, e.g. power 753savings. 754 755The BNXT PMD supports applications to increase or decrease queues at runtime. 756 757.. code-block:: console 758 759 testpmd> port config all (rxq|txq) (num_queues) 760 761Note that a DPDK application must allocate default queues (one for TX and one 762for RX at minimum) at initialization. 763 764Descriptor Status 765~~~~~~~~~~~~~~~~~ 766 767Applications may use the descriptor status for various reasons, e.g. for power 768savings. For example, an application may stop polling and change to interrupt 769mode when the descriptor status shows no packets to service for a while. 770 771The BNXT PMD supports the application to retrieve both TX and RX descriptor 772status. 773 774.. code-block:: console 775 776 testpmd> show port (port_id) (rxq|txq) (queue_id) desc (desc_id) status 777 778Bonding 779~~~~~~~ 780 781DPDK implements a light-weight library to allow PMDs to be bonded together and provide a single logical PMD to the application. 782 783.. code-block:: console 784 785 dpdk-testpmd -l 0-3 -n4 --vdev 'net_bonding0,mode=0,member=<PCI B:D.F device 1>,member=<PCI B:D.F device 2>,mac=XX:XX:XX:XX:XX:XX’ – --socket_num=1 – -i --port-topology=chained 786 (ex) dpdk-testpmd -l 1,3,5,7,9 -n4 --vdev 'net_bonding0,mode=0,member=0000:82:00.0,member=0000:82:00.1,mac=00:1e:67:1d:fd:1d' – --socket-num=1 – -i --port-topology=chained 787 788Vector Processing 789----------------- 790 791Vector mode provides significantly improved performance over scalar mode, 792using SIMD (Single Instruction Multiple Data) instructions 793to operate on multiple packets in parallel. 794 795The BNXT PMD provides vectorized burst transmit/receive function implementations 796on x86-based platforms and ARM-based platforms. 797The BNXT PMD supports SSE (Streaming SIMD Extensions) 798and AVX2 (Advanced Vector Extensions 2) instructions for x86-based platforms, 799and NEON instructions for ARM-based platforms. 800 801The BNXT Vector PMD is enabled in DPDK builds by default. 802However, the vector mode is disabled when applying SIMD instructions 803does not improve the performance due to non-uniform packet handling. 804TX and RX vector mode can be enabled independently from each other, 805and the decision to disable vector mode is made at run-time 806when the port is started. 807 808The vector mode is disabled with TX and RX offloads. 809However, a limited set of offloads can be enabled in a vector mode. 810Listed below are the TX and RX offloads with which the vector mode can be enabled: 811 812 * TX offloads supported in vector mode 813 814 RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE 815 816 * RX offloads supported in vector mode 817 818 RTE_ETH_RX_OFFLOAD_VLAN_STRIP 819 RTE_ETH_RX_OFFLOAD_KEEP_CRC 820 RTE_ETH_RX_OFFLOAD_IPV4_CKSUM 821 RTE_ETH_RX_OFFLOAD_UDP_CKSUM 822 RTE_ETH_RX_OFFLOAD_TCP_CKSUM 823 RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM 824 RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM 825 RTE_ETH_RX_OFFLOAD_RSS_HASH 826 RTE_ETH_RX_OFFLOAD_VLAN_FILTER 827 828Note that the offload configuration changes impacting the vector mode enablement 829are allowed only when the port is stopped. 830 831Performance Report 832------------------ 833 834Broadcom DPDK performance has been reported since 19.08 release. 835The reports provide not only the performance results 836but also test scenarios including test topology and detailed configurations. 837See the reports at `DPDK performance link <https://core.dpdk.org/perf-reports/>`. 838 839Supported Network Adapters 840-------------------------- 841 842Listed below are BCM57400 and BCM57500 NetXtreme-E\ |reg| family 843of Ethernet network adapters. 844More information can be found in the `NetXtreme\ |reg| Brand section 845<https://www.broadcom.com/products/ethernet-connectivity/network-adapters/>`_ 846of the `Broadcom website <http://www.broadcom.com/>`_. 847 848BCM57400 NetXtreme-E\ |reg| Family of Ethernet Network Controllers 849~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 850 851PCIe NICs 852^^^^^^^^^ 853 854* ``P210P .... Dual-port 10 Gigabit Ethernet Adapter`` 855* ``P210TP ... Dual-port 10 Gigabit Ethernet Adapter`` 856* ``P225P .... Dual-port 25 Gigabit Ethernet Adapter`` 857* ``P150P .... Single-port 50 Gigabit Ethernet Adapter`` 858 859OCP 2.0 NICs 860^^^^^^^^^^^^ 861 862* ``M210P .... Dual-port 10 Gigabit Ethernet Adapter`` 863* ``M210TP ... Dual-port 10 Gigabit Ethernet Adapter`` 864* ``M125P .... Single-port 25 Gigabit Ethernet Adapter`` 865* ``M225P .... Dual-port 25 Gigabit Ethernet Adapter`` 866* ``M150P .... Single-port 50 Gigabit Ethernet Adapter`` 867* ``M150PM ... Single-port Multi-Host 50 Gigabit Ethernet Adapter`` 868 869OCP 3.0 NICs 870^^^^^^^^^^^^ 871 872* ``N210P .... Dual-port 10 Gigabit Ethernet Adapter`` 873* ``N210TP ... Dual-port 10 Gigabit Ethernet Adapter`` 874* ``N225P ... Dual-port 10 Gigabit Ethernet Adapter`` 875 876BCM57500 NetXtreme-E\ |reg| Family of Ethernet Network Controllers 877~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 878 879PCIe NICs 880^^^^^^^^^ 881 882* ``P410SG ... Quad-port 10 Gigabit Ethernet Adapter`` 883* ``P410SGBT . Quad-port 10 Gigabit Ethernet Adapter`` 884* ``P425G .... Quad-port 25 Gigabit Ethernet Adapter`` 885* ``P1100G ... Single-port 100 Gigabit Ethernet Adapter`` 886* ``P2100G ... Dual-port 100 Gigabit Ethernet Adapter`` 887* ``P2200G ... Dual-port 200 Gigabit Ethernet Adapter`` 888 889OCP 2.0 NICs 890^^^^^^^^^^^^ 891 892* ``M1100G ... Single-port OCP 2.0 10/25/50/100 Gigabit Ethernet Adapter`` 893 894OCP 3.0 NICs 895^^^^^^^^^^^^ 896 897* ``N410SG ... Quad-port 10 Gigabit Ethernet Adapter`` 898* ``N410SGBT . Quad-port 10 Gigabit Ethernet Adapter`` 899* ``N425G .... Quad-port 25 Gigabit Ethernet Adapter`` 900* ``N150G .... Single-port 50 Gigabit Ethernet Adapter`` 901* ``N250G .... Dual-port 50 Gigabit Ethernet Adapter`` 902* ``N1100G ... Single-port 100 Gigabit Ethernet Adapter`` 903* ``N2100G ... Dual-port 100 Gigabit Ethernet Adapter`` 904* ``N2200G ... Dual-port 200 Gigabit Ethernet Adapter`` 905 906Supported Firmware Versions 907--------------------------- 908 909Shown below are Ethernet Network Adapters and their supported firmware versions 910(refer to section “Supported Network Adapters” for the list of adapters): 911 912* ``BCM57400 NetXtreme-E\ |reg| Family`` ... Firmware 219.0.0 or later 913* ``BCM57500 NetXtreme-E\ |reg| Family`` ... Firmware 219.0.0 or later 914 915Shown below are DPDK LTS releases and their supported firmware versions: 916 917* ``DPDK Release 19.11`` ... Firmware 219.0.103 or later 918* ``DPDK Release 20.11`` ... Firmware 219.0.103 or later 919* ``DPDK Release 21.11`` ... Firmware 221.0.101 or later 920* ``DPDK Release 22.11`` ... Firmware 226.0.131 or later 921 922Supported APIs 923-------------- 924 925rte_eth APIs 926~~~~~~~~~~~~ 927 928Listed below are the rte_eth functions supported: 929* ``rte_eth_allmulticast_disable`` 930* ``rte_eth_allmulticast_enable`` 931* ``rte_eth_allmulticast_get`` 932* ``rte_eth_dev_close`` 933* ``rte_eth_dev_conf_get`` 934* ``rte_eth_dev_configure`` 935* ``rte_eth_dev_default_mac_addr_set`` 936* ``rte_eth_dev_flow_ctrl_get`` 937* ``rte_eth_dev_flow_ctrl_set`` 938* ``rte_eth_dev_fw_version_get`` 939* ``rte_eth_dev_get_eeprom`` 940* ``rte_eth_dev_get_eeprom`` 941* ``rte_eth_dev_get_eeprom_length`` 942* ``rte_eth_dev_get_eeprom_length`` 943* ``rte_eth_dev_get_module_eeprom`` 944* ``rte_eth_dev_get_module_info`` 945* ``rte_eth_dev_get_mtu`` 946* ``rte_eth_dev_get_name_by_port rte_eth_dev_get_port_by_name`` 947* ``rte_eth_dev_get_supported_ptypes`` 948* ``rte_eth_dev_get_vlan_offload`` 949* ``rte_eth_dev_info_get`` 950* ``rte_eth_dev_infos_get`` 951* ``rte_eth_dev_mac_addr_add`` 952* ``rte_eth_dev_mac_addr_remove`` 953* ``rte_eth_dev_macaddr_get`` 954* ``rte_eth_dev_macaddrs_get`` 955* ``rte_eth_dev_owner_get`` 956* ``rte_eth_dev_rss_hash_conf_get`` 957* ``rte_eth_dev_rss_hash_update`` 958* ``rte_eth_dev_rss_reta_query`` 959* ``rte_eth_dev_rss_reta_update`` 960* ``rte_eth_dev_rx_intr_disable`` 961* ``rte_eth_dev_rx_intr_enable`` 962* ``rte_eth_dev_rx_queue_start`` 963* ``rte_eth_dev_rx_queue_stop`` 964* ``rte_eth_dev_set_eeprom`` 965* ``rte_eth_dev_set_link_down`` 966* ``rte_eth_dev_set_link_up`` 967* ``rte_eth_dev_set_mc_addr_list`` 968* ``rte_eth_dev_set_mtu`` 969* ``rte_eth_dev_set_vlan_ether_type`` 970* ``rte_eth_dev_set_vlan_offload`` 971* ``rte_eth_dev_set_vlan_pvid`` 972* ``rte_eth_dev_start`` 973* ``rte_eth_dev_stop`` 974* ``rte_eth_dev_supported_ptypes_get`` 975* ``rte_eth_dev_tx_queue_start`` 976* ``rte_eth_dev_tx_queue_stop`` 977* ``rte_eth_dev_udp_tunnel_port_add`` 978* ``rte_eth_dev_udp_tunnel_port_delete`` 979* ``rte_eth_dev_vlan_filter`` 980* ``rte_eth_led_off`` 981* ``rte_eth_led_on`` 982* ``rte_eth_link_get`` 983* ``rte_eth_link_get_nowait`` 984* ``rte_eth_macaddr_get`` 985* ``rte_eth_macaddrs_get`` 986* ``rte_eth_promiscuous_disable`` 987* ``rte_eth_promiscuous_enable`` 988* ``rte_eth_promiscuous_get`` 989* ``rte_eth_rx_burst_mode_get`` 990* ``rte_eth_rx_queue_info_get`` 991* ``rte_eth_rx_queue_setup`` 992* ``rte_eth_stats_get`` 993* ``rte_eth_stats_reset`` 994* ``rte_eth_timesync_adjust_time`` 995* ``rte_eth_timesync_disable`` 996* ``rte_eth_timesync_enable`` 997* ``rte_eth_timesync_read_rx_timestamp`` 998* ``rte_eth_timesync_read_time`` 999* ``rte_eth_timesync_read_tx_timestamp`` 1000* ``rte_eth_timesync_write_time`` 1001* ``rte_eth_tx_burst_mode_get`` 1002* ``rte_eth_tx_queue_info_get`` 1003* ``rte_eth_tx_queue_setup`` 1004* ``rte_eth_xstats_get`` 1005* ``rte_eth_xstats_get_names`` 1006 1007rte_flow APIs 1008~~~~~~~~~~~~~ 1009 1010Listed below are the rte_flow functions supported: 1011* ``rte_flow_ops_get`` 1012* ``rte_flow_validate`` 1013* ``rte_flow_create`` 1014* ``rte_flow_destroy`` 1015* ``rte_flow_flush`` 1016* ``rte_flow_tunnel_action_decap_release`` 1017* ``rte_flow_tunnel_decap_set`` 1018* ``rte_flow_tunnel_item_release`` 1019* ``rte_flow_tunnel_match`` 1020 1021rte_flow Items 1022~~~~~~~~~~~~~~ 1023 1024Refer to :ref:`rte_flow items availability in networking drivers`. 1025 1026Listed below are the rte_flow items supported: 1027 1028* ``any`` 1029* ``ecpri`` 1030* ``eth`` 1031* ``geneve`` 1032* ``gre`` 1033* ``icmp`` 1034* ``icmp6`` 1035* ``ipv4`` 1036* ``ipv6`` 1037* ``port_id`` 1038* ``port_representor`` 1039* ``represented_port`` 1040* ``tcp`` 1041* ``udp`` 1042* ``vlan`` 1043* ``vxlan`` 1044* ``vxlan_gpe`` 1045 1046rte_flow Actions 1047~~~~~~~~~~~~~~~~ 1048 1049Refer to :ref:`rte_flow actions availability in networking drivers`. 1050 1051Listed below are the rte_flow actions supported: 1052 1053* ``count`` 1054* ``dec_ttl`` 1055* ``drop`` 1056* ``jump`` 1057* ``mark`` 1058* ``meter`` 1059* ``of_pop_vlan`` 1060* ``of_push_vlan`` 1061* ``of_set_vlan_pcp`` 1062* ``of_set_vlan_vid`` 1063* ``pf`` 1064* ``phy_port`` 1065* ``port_id`` 1066* ``port_representor`` 1067* ``represented_port`` 1068* ``rss`` 1069* ``sample`` 1070* ``set_ipv4_dst`` 1071* ``set_ipv4_src`` 1072* ``set_ipv6_dst`` 1073* ``set_ipv6_src`` 1074* ``set_tp_dst`` 1075* ``set_tp_src`` 1076* ``set_ttl`` 1077* ``vf`` 1078* ``vxlan_decap`` 1079* ``vxlan_encap`` 1080