1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2014-2018 Chelsio Communications. 3 All rights reserved. 4 5CXGBE Poll Mode Driver 6====================== 7 8The CXGBE PMD (**librte_net_cxgbe**) provides poll mode driver support 9for **Chelsio Terminator** 10/25/40/100 Gbps family of adapters. CXGBE PMD 10has support for the latest Linux and FreeBSD operating systems. 11 12CXGBEVF PMD provides poll mode driver support for SR-IOV Virtual functions 13and has support for the latest Linux operating systems. 14 15More information can be found at `Chelsio Communications Official Website 16<http://www.chelsio.com>`_. 17 18.. _t5-nics: 19 20Supported Chelsio T5 NICs 21------------------------- 22 23- 1G NICs: T502-BT 24- 10G NICs: T520-BT, T520-CR, T520-LL-CR, T520-SO-CR, T540-CR 25- 40G NICs: T580-CR, T580-LP-CR, T580-SO-CR 26- Other T5 NICs: T522-CR 27 28.. _t6-nics: 29 30Supported Chelsio T6 NICs 31------------------------- 32 33- 25G NICs: T6425-CR, T6225-CR, T6225-LL-CR, T6225-SO-CR 34- 100G NICs: T62100-CR, T62100-LP-CR, T62100-SO-CR 35 36Supported SR-IOV Chelsio NICs 37----------------------------- 38 39SR-IOV virtual functions are supported on all the Chelsio NICs listed 40in :ref:`t5-nics` and :ref:`t6-nics`. 41 42Features 43-------- 44 45CXGBE and CXGBEVF PMD has support for: 46 47- Multiple queues for TX and RX 48- Receiver Side Steering (RSS) 49 Receiver Side Steering (RSS) on IPv4, IPv6, IPv4-TCP/UDP, IPv6-TCP/UDP. 50 For 4-tuple, enabling 'RSS on TCP' and 'RSS on TCP + UDP' is supported. 51- VLAN filtering 52- Checksum offload 53- Promiscuous mode 54- All multicast mode 55- Port hardware statistics 56- Jumbo frames 57- Flow API - Support for both Wildcard (LE-TCAM) and Exact (HASH) match filters. 58 59Prerequisites 60------------- 61 62- Requires firmware version **1.25.6.0** and higher. Visit 63 `Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware 64 bundled with the latest Chelsio Unified Wire package. 65 66 For Linux, installing and loading the latest cxgb4 kernel driver from the 67 Chelsio Unified Wire package should get you the latest firmware. More 68 information can be obtained from the User Guide that is bundled with the 69 Chelsio Unified Wire package. 70 71 For FreeBSD, the latest firmware obtained from the Chelsio Unified Wire 72 package must be manually flashed via cxgbetool available in FreeBSD source 73 repository. 74 75 Instructions on how to manually flash the firmware are given in section 76 :ref:`linux-installation` for Linux and section :ref:`freebsd-installation` 77 for FreeBSD. 78 79 80Runtime Configuration 81--------------------- 82 83The following ``devargs`` options can be enabled at runtime. They must 84be passed as part of EAL arguments. For example, 85 86.. code-block:: console 87 88 dpdk-testpmd -a 02:00.4,keep_ovlan=1 -- -i 89 90Common Runtime Options 91~~~~~~~~~~~~~~~~~~~~~~ 92 93- ``keep_ovlan`` (default **0**) 94 95 Toggle behavior to keep/strip outer VLAN in Q-in-Q packets. If 96 enabled, the outer VLAN tag is preserved in Q-in-Q packets. Otherwise, 97 the outer VLAN tag is stripped in Q-in-Q packets. 98 99- ``tx_mode_latency`` (default **0**) 100 101 When set to 1, Tx doesn't wait for max number of packets to get 102 coalesced and sends the packets immediately at the end of the 103 current Tx burst. When set to 0, Tx waits across multiple Tx bursts 104 until the max number of packets have been coalesced. In this case, 105 Tx only sends the coalesced packets to hardware once the max 106 coalesce limit has been reached. 107 108CXGBE VF Only Runtime Options 109~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 110 111- ``force_link_up`` (default **0**) 112 113 When set to 1, CXGBEVF PMD always forces link as up for all VFs on 114 underlying Chelsio NICs. This enables multiple VFs on the same NIC 115 to send traffic to each other even when the physical link is down. 116 117CXGBE PF Only Runtime Options 118~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 119 120- ``filtermode`` (default **0**) 121 122 Apart from the 4-tuple (IP src/dst addresses and TCP/UDP src/dst port 123 addresses), there are only 40-bits available to match other fields in 124 packet headers. So, ``filtermode`` devarg allows user to dynamically 125 select a 40-bit supported match field combination for LETCAM (wildcard) 126 filters. 127 128 Default value of **0** makes driver pick the combination configured in 129 the firmware configuration file on the adapter. 130 131 The supported flags and their corresponding values are shown in table below. 132 These flags can be OR'd to create 1 of the multiple supported combinations 133 for LETCAM filters. 134 135 ================== ====== 136 FLAG VALUE 137 ================== ====== 138 Physical Port 0x1 139 PFVF 0x2 140 Destination MAC 0x4 141 Ethertype 0x8 142 Inner VLAN 0x10 143 Outer VLAN 0x20 144 IP TOS 0x40 145 IP Protocol 0x80 146 ================== ====== 147 148 The supported ``filtermode`` combinations and their corresponding OR'd 149 values are shown in table below. 150 151 +-----------------------------------+-----------+ 152 | FILTERMODE COMBINATIONS | VALUE | 153 +===================================+===========+ 154 | Protocol, TOS, Outer VLAN, Port | 0xE1 | 155 +-----------------------------------+-----------+ 156 | Protocol, TOS, Outer VLAN | 0xE0 | 157 +-----------------------------------+-----------+ 158 | Protocol, TOS, Inner VLAN, Port | 0xD1 | 159 +-----------------------------------+-----------+ 160 | Protocol, TOS, Inner VLAN | 0xD0 | 161 +-----------------------------------+-----------+ 162 | Protocol, TOS, PFVF, Port | 0xC3 | 163 +-----------------------------------+-----------+ 164 | Protocol, TOS, PFVF | 0xC2 | 165 +-----------------------------------+-----------+ 166 | Protocol, TOS, Port | 0xC1 | 167 +-----------------------------------+-----------+ 168 | Protocol, TOS | 0xC0 | 169 +-----------------------------------+-----------+ 170 | Protocol, Outer VLAN, Port | 0xA1 | 171 +-----------------------------------+-----------+ 172 | Protocol, Outer VLAN | 0xA0 | 173 +-----------------------------------+-----------+ 174 | Protocol, Inner VLAN, Port | 0x91 | 175 +-----------------------------------+-----------+ 176 | Protocol, Inner VLAN | 0x90 | 177 +-----------------------------------+-----------+ 178 | Protocol, Ethertype, DstMAC, Port | 0x8D | 179 +-----------------------------------+-----------+ 180 | Protocol, Ethertype, DstMAC | 0x8C | 181 +-----------------------------------+-----------+ 182 | Protocol, Ethertype, Port | 0x89 | 183 +-----------------------------------+-----------+ 184 | Protocol, Ethertype | 0x88 | 185 +-----------------------------------+-----------+ 186 | Protocol, DstMAC, PFVF, Port | 0x87 | 187 +-----------------------------------+-----------+ 188 | Protocol, DstMAC, PFVF | 0x86 | 189 +-----------------------------------+-----------+ 190 | Protocol, DstMAC, Port | 0x85 | 191 +-----------------------------------+-----------+ 192 | Protocol, DstMAC | 0x84 | 193 +-----------------------------------+-----------+ 194 | Protocol, PFVF, Port | 0x83 | 195 +-----------------------------------+-----------+ 196 | Protocol, PFVF | 0x82 | 197 +-----------------------------------+-----------+ 198 | Protocol, Port | 0x81 | 199 +-----------------------------------+-----------+ 200 | Protocol | 0x80 | 201 +-----------------------------------+-----------+ 202 | TOS, Outer VLAN, Port | 0x61 | 203 +-----------------------------------+-----------+ 204 | TOS, Outer VLAN | 0x60 | 205 +-----------------------------------+-----------+ 206 | TOS, Inner VLAN, Port | 0x51 | 207 +-----------------------------------+-----------+ 208 | TOS, Inner VLAN | 0x50 | 209 +-----------------------------------+-----------+ 210 | TOS, Ethertype, DstMAC, Port | 0x4D | 211 +-----------------------------------+-----------+ 212 | TOS, Ethertype, DstMAC | 0x4C | 213 +-----------------------------------+-----------+ 214 | TOS, Ethertype, Port | 0x49 | 215 +-----------------------------------+-----------+ 216 | TOS, Ethertype | 0x48 | 217 +-----------------------------------+-----------+ 218 | TOS, DstMAC, PFVF, Port | 0x47 | 219 +-----------------------------------+-----------+ 220 | TOS, DstMAC, PFVF | 0x46 | 221 +-----------------------------------+-----------+ 222 | TOS, DstMAC, Port | 0x45 | 223 +-----------------------------------+-----------+ 224 | TOS, DstMAC | 0x44 | 225 +-----------------------------------+-----------+ 226 | TOS, PFVF, Port | 0x43 | 227 +-----------------------------------+-----------+ 228 | TOS, PFVF | 0x42 | 229 +-----------------------------------+-----------+ 230 | TOS, Port | 0x41 | 231 +-----------------------------------+-----------+ 232 | TOS | 0x40 | 233 +-----------------------------------+-----------+ 234 | Outer VLAN, Inner VLAN, Port | 0x31 | 235 +-----------------------------------+-----------+ 236 | Outer VLAN, Ethertype, Port | 0x29 | 237 +-----------------------------------+-----------+ 238 | Outer VLAN, Ethertype | 0x28 | 239 +-----------------------------------+-----------+ 240 | Outer VLAN, DstMAC, Port | 0x25 | 241 +-----------------------------------+-----------+ 242 | Outer VLAN, DstMAC | 0x24 | 243 +-----------------------------------+-----------+ 244 | Outer VLAN, Port | 0x21 | 245 +-----------------------------------+-----------+ 246 | Outer VLAN | 0x20 | 247 +-----------------------------------+-----------+ 248 | Inner VLAN, Ethertype, Port | 0x19 | 249 +-----------------------------------+-----------+ 250 | Inner VLAN, Ethertype | 0x18 | 251 +-----------------------------------+-----------+ 252 | Inner VLAN, DstMAC, Port | 0x15 | 253 +-----------------------------------+-----------+ 254 | Inner VLAN, DstMAC | 0x14 | 255 +-----------------------------------+-----------+ 256 | Inner VLAN, Port | 0x11 | 257 +-----------------------------------+-----------+ 258 | Inner VLAN | 0x10 | 259 +-----------------------------------+-----------+ 260 | Ethertype, DstMAC, Port | 0xD | 261 +-----------------------------------+-----------+ 262 | Ethertype, DstMAC | 0xC | 263 +-----------------------------------+-----------+ 264 | Ethertype, PFVF, Port | 0xB | 265 +-----------------------------------+-----------+ 266 | Ethertype, PFVF | 0xA | 267 +-----------------------------------+-----------+ 268 | Ethertype, Port | 0x9 | 269 +-----------------------------------+-----------+ 270 | Ethertype | 0x8 | 271 +-----------------------------------+-----------+ 272 | DstMAC, PFVF, Port | 0x7 | 273 +-----------------------------------+-----------+ 274 | DstMAC, PFVF | 0x6 | 275 +-----------------------------------+-----------+ 276 | DstMAC, Port | 0x5 | 277 +-----------------------------------+-----------+ 278 | Destination MAC | 0x4 | 279 +-----------------------------------+-----------+ 280 | PFVF, Port | 0x3 | 281 +-----------------------------------+-----------+ 282 | PFVF | 0x2 | 283 +-----------------------------------+-----------+ 284 | Physical Port | 0x1 + 285 +-----------------------------------+-----------+ 286 287 For example, to enable matching ``ethertype`` field in Ethernet 288 header, and ``protocol`` field in IPv4 header, the ``filtermode`` 289 combination must be given as: 290 291 .. code-block:: console 292 293 dpdk-testpmd -a 02:00.4,filtermode=0x88 -- -i 294 295- ``filtermask`` (default **0**) 296 297 ``filtermask`` devarg works similar to ``filtermode``, but is used 298 to configure a filter mode combination for HASH (exact-match) filters. 299 300 .. note:: 301 302 The combination chosen for ``filtermask`` devarg **must be a subset** of 303 the combination chosen for ``filtermode`` devarg. 304 305 Default value of **0** makes driver pick the combination configured in 306 the firmware configuration file on the adapter. 307 308 Note that the filter rule will only be inserted in HASH region, if the 309 rule contains **all** the fields specified in the ``filtermask`` combination. 310 Otherwise, the filter rule will get inserted in LETCAM region. 311 312 The same combination list explained in the tables in ``filtermode`` devarg 313 section earlier applies for ``filtermask`` devarg, as well. 314 315 For example, to enable matching only protocol field in IPv4 header, the 316 ``filtermask`` combination must be given as: 317 318 .. code-block:: console 319 320 dpdk-testpmd -a 02:00.4,filtermode=0x88,filtermask=0x80 -- -i 321 322.. _driver-compilation: 323 324Driver compilation and testing 325------------------------------ 326 327Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` 328for details. 329 330Linux 331----- 332 333.. _linux-installation: 334 335Linux Installation 336~~~~~~~~~~~~~~~~~~ 337 338Steps to manually install the latest firmware from the downloaded Chelsio 339Unified Wire package for Linux operating system are as follows: 340 341#. Load the kernel module: 342 343 .. code-block:: console 344 345 modprobe cxgb4 346 347#. Use ifconfig to get the interface name assigned to Chelsio card: 348 349 .. code-block:: console 350 351 ifconfig -a | grep "00:07:43" 352 353 Example output: 354 355 .. code-block:: console 356 357 p1p1 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C0 358 p1p2 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C8 359 360#. Install cxgbtool: 361 362 .. code-block:: console 363 364 cd <path_to_uwire>/tools/cxgbtool 365 make install 366 367#. Use cxgbtool to load the firmware config file onto the card: 368 369 .. code-block:: console 370 371 cxgbtool p1p1 loadcfg <path_to_uwire>/src/network/firmware/t5-config.txt 372 373#. Use cxgbtool to load the firmware image onto the card: 374 375 .. code-block:: console 376 377 cxgbtool p1p1 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin 378 379#. Unload and reload the kernel module: 380 381 .. code-block:: console 382 383 modprobe -r cxgb4 384 modprobe cxgb4 385 386#. Verify with ethtool: 387 388 .. code-block:: console 389 390 ethtool -i p1p1 | grep "firmware" 391 392 Example output: 393 394 .. code-block:: console 395 396 firmware-version: 1.25.6.0, TP 0.1.23.2 397 398Running testpmd 399~~~~~~~~~~~~~~~ 400 401This section demonstrates how to launch **testpmd** with Chelsio 402devices managed by librte_net_cxgbe in Linux operating system. 403 404#. Load the kernel module: 405 406 .. code-block:: console 407 408 modprobe cxgb4 409 410#. Get the PCI bus addresses of the interfaces bound to cxgb4 driver: 411 412 .. code-block:: console 413 414 dmesg | tail -2 415 416 Example output: 417 418 .. code-block:: console 419 420 cxgb4 0000:02:00.4 p1p1: renamed from eth0 421 cxgb4 0000:02:00.4 p1p2: renamed from eth1 422 423 .. note:: 424 425 Both the interfaces of a Chelsio 2-port adapter are bound to the 426 same PCI bus address. 427 428#. Unload the kernel module: 429 430 .. code-block:: console 431 432 modprobe -ar cxgb4 csiostor 433 434#. Running testpmd 435 436 Follow instructions available in the document 437 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` 438 to run testpmd. 439 440 .. note:: 441 442 Currently, CXGBE PMD only supports the binding of PF4 for Chelsio NICs. 443 444 Example output: 445 446 .. code-block:: console 447 448 [...] 449 EAL: PCI device 0000:02:00.4 on NUMA socket -1 450 EAL: probe driver: 1425:5401 rte_cxgbe_pmd 451 EAL: PCI memory mapped at 0x7fd7c0200000 452 EAL: PCI memory mapped at 0x7fd77cdfd000 453 EAL: PCI memory mapped at 0x7fd7c10b7000 454 PMD: rte_cxgbe_pmd: fw: 1.25.6.0, TP: 0.1.23.2 455 PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter 456 Interactive-mode selected 457 Configuring Port 0 (socket 0) 458 Port 0: 00:07:43:2D:EA:C0 459 Configuring Port 1 (socket 0) 460 Port 1: 00:07:43:2D:EA:C8 461 Checking link statuses... 462 PMD: rte_cxgbe_pmd: Port0: passive DA port module inserted 463 PMD: rte_cxgbe_pmd: Port1: passive DA port module inserted 464 Port 0 Link Up - speed 10000 Mbps - full-duplex 465 Port 1 Link Up - speed 10000 Mbps - full-duplex 466 Done 467 testpmd> 468 469 .. note:: 470 471 Flow control pause TX/RX is disabled by default and can be enabled via 472 testpmd. Refer section :ref:`flow-control` for more details. 473 474Configuring SR-IOV Virtual Functions 475~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 476 477This section demonstrates how to enable SR-IOV virtual functions 478on Chelsio NICs and demonstrates how to run testpmd with SR-IOV 479virtual functions. 480 481#. Load the kernel module: 482 483 .. code-block:: console 484 485 modprobe cxgb4 486 487#. Get the PCI bus addresses of the interfaces bound to cxgb4 driver: 488 489 .. code-block:: console 490 491 dmesg | tail -2 492 493 Example output: 494 495 .. code-block:: console 496 497 cxgb4 0000:02:00.4 p1p1: renamed from eth0 498 cxgb4 0000:02:00.4 p1p2: renamed from eth1 499 500 .. note:: 501 502 Both the interfaces of a Chelsio 2-port adapter are bound to the 503 same PCI bus address. 504 505#. Use ifconfig to get the interface name assigned to Chelsio card: 506 507 .. code-block:: console 508 509 ifconfig -a | grep "00:07:43" 510 511 Example output: 512 513 .. code-block:: console 514 515 p1p1 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C0 516 p1p2 Link encap:Ethernet HWaddr 00:07:43:2D:EA:C8 517 518#. Bring up the interfaces: 519 520 .. code-block:: console 521 522 ifconfig p1p1 up 523 ifconfig p1p2 up 524 525#. Instantiate SR-IOV Virtual Functions. PF0..3 can be used for 526 SR-IOV VFs. Multiple VFs can be instantiated on each of PF0..3. 527 To instantiate one SR-IOV VF on each PF0 and PF1: 528 529 .. code-block:: console 530 531 echo 1 > /sys/bus/pci/devices/0000\:02\:00.0/sriov_numvfs 532 echo 1 > /sys/bus/pci/devices/0000\:02\:00.1/sriov_numvfs 533 534#. Get the PCI bus addresses of the virtual functions: 535 536 .. code-block:: console 537 538 lspci | grep -i "Chelsio" | grep -i "VF" 539 540 Example output: 541 542 .. code-block:: console 543 544 02:01.0 Ethernet controller: Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller [VF] 545 02:01.1 Ethernet controller: Chelsio Communications Inc T540-CR Unified Wire Ethernet Controller [VF] 546 547#. Running testpmd 548 549 Follow instructions available in the document 550 :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` 551 to bind virtual functions and run testpmd. 552 553 Example output: 554 555 .. code-block:: console 556 557 [...] 558 EAL: PCI device 0000:02:01.0 on NUMA socket 0 559 EAL: probe driver: 1425:5803 net_cxgbevf 560 PMD: rte_cxgbe_pmd: Firmware version: 1.25.6.0 561 PMD: rte_cxgbe_pmd: TP Microcode version: 0.1.23.2 562 PMD: rte_cxgbe_pmd: Chelsio rev 0 563 PMD: rte_cxgbe_pmd: No bootstrap loaded 564 PMD: rte_cxgbe_pmd: No Expansion ROM loaded 565 PMD: rte_cxgbe_pmd: 0000:02:01.0 Chelsio rev 0 1G/10GBASE-SFP 566 EAL: PCI device 0000:02:01.1 on NUMA socket 0 567 EAL: probe driver: 1425:5803 net_cxgbevf 568 PMD: rte_cxgbe_pmd: Firmware version: 1.25.6.0 569 PMD: rte_cxgbe_pmd: TP Microcode version: 0.1.23.2 570 PMD: rte_cxgbe_pmd: Chelsio rev 0 571 PMD: rte_cxgbe_pmd: No bootstrap loaded 572 PMD: rte_cxgbe_pmd: No Expansion ROM loaded 573 PMD: rte_cxgbe_pmd: 0000:02:01.1 Chelsio rev 0 1G/10GBASE-SFP 574 Configuring Port 0 (socket 0) 575 Port 0: 06:44:29:44:40:00 576 Configuring Port 1 (socket 0) 577 Port 1: 06:44:29:44:40:10 578 Checking link statuses... 579 Done 580 testpmd> 581 582FreeBSD 583------- 584 585.. _freebsd-installation: 586 587FreeBSD Installation 588~~~~~~~~~~~~~~~~~~~~ 589 590Steps to manually install the latest firmware from the downloaded Chelsio 591Unified Wire package for FreeBSD operating system are as follows: 592 593#. Load the kernel module: 594 595 .. code-block:: console 596 597 kldload if_cxgbe 598 599#. Use dmesg to get the t5nex instance assigned to the Chelsio card: 600 601 .. code-block:: console 602 603 dmesg | grep "t5nex" 604 605 Example output: 606 607 .. code-block:: console 608 609 t5nex0: <Chelsio T520-CR> irq 16 at device 0.4 on pci2 610 cxl0: <port 0> on t5nex0 611 cxl1: <port 1> on t5nex0 612 t5nex0: PCIe x8, 2 ports, 14 MSI-X interrupts, 31 eq, 13 iq 613 614 In the example above, a Chelsio T520-CR card is bound to a t5nex0 instance. 615 616#. Install cxgbetool from FreeBSD source repository: 617 618 .. code-block:: console 619 620 cd <path_to_FreeBSD_source>/tools/tools/cxgbetool/ 621 make && make install 622 623#. Use cxgbetool to load the firmware image onto the card: 624 625 .. code-block:: console 626 627 cxgbetool t5nex0 loadfw <path_to_uwire>/src/network/firmware/t5fw-*.bin 628 629#. Unload and reload the kernel module: 630 631 .. code-block:: console 632 633 kldunload if_cxgbe 634 kldload if_cxgbe 635 636#. Verify with sysctl: 637 638 .. code-block:: console 639 640 sysctl -a | grep "t5nex" | grep "firmware" 641 642 Example output: 643 644 .. code-block:: console 645 646 dev.t5nex.0.firmware_version: 1.25.6.0 647 648Running testpmd 649~~~~~~~~~~~~~~~ 650 651This section demonstrates how to launch **testpmd** with Chelsio 652devices managed by librte_net_cxgbe in FreeBSD operating system. 653 654#. Change to DPDK source directory where the target has been compiled in 655 section :ref:`driver-compilation`: 656 657 .. code-block:: console 658 659 cd <DPDK-source-directory> 660 661#. Copy the contigmem kernel module to /boot/kernel directory: 662 663 .. code-block:: console 664 665 cp <build_dir>/kernel/freebsd/contigmem.ko /boot/kernel/ 666 667#. Add the following lines to /boot/loader.conf: 668 669 .. code-block:: console 670 671 # reserve 2 x 1G blocks of contiguous memory using contigmem driver 672 hw.contigmem.num_buffers=2 673 hw.contigmem.buffer_size=1073741824 674 # load contigmem module during boot process 675 contigmem_load="YES" 676 677 The above lines load the contigmem kernel module during boot process and 678 allocate 2 x 1G blocks of contiguous memory to be used for DPDK later on. 679 This is to avoid issues with potential memory fragmentation during later 680 system up time, which may result in failure of allocating the contiguous 681 memory required for the contigmem kernel module. 682 683#. Restart the system and ensure the contigmem module is loaded successfully: 684 685 .. code-block:: console 686 687 reboot 688 kldstat | grep "contigmem" 689 690 Example output: 691 692 .. code-block:: console 693 694 2 1 0xffffffff817f1000 3118 contigmem.ko 695 696#. Repeat step 1 to ensure that you are in the DPDK source directory. 697 698#. Load the cxgbe kernel module: 699 700 .. code-block:: console 701 702 kldload if_cxgbe 703 704#. Get the PCI bus addresses of the interfaces bound to t5nex driver: 705 706 .. code-block:: console 707 708 pciconf -l | grep "t5nex" 709 710 Example output: 711 712 .. code-block:: console 713 714 t5nex0@pci0:2:0:4: class=0x020000 card=0x00001425 chip=0x54011425 rev=0x00 715 716 In the above example, the t5nex0 is bound to 2:0:4 bus address. 717 718 .. note:: 719 720 Both the interfaces of a Chelsio 2-port adapter are bound to the 721 same PCI bus address. 722 723#. Unload the kernel module: 724 725 .. code-block:: console 726 727 kldunload if_cxgbe 728 729#. Set the PCI bus addresses to hw.nic_uio.bdfs kernel environment parameter: 730 731 .. code-block:: console 732 733 kenv hw.nic_uio.bdfs="2:0:4" 734 735 This automatically binds 2:0:4 to nic_uio kernel driver when it is loaded in 736 the next step. 737 738 .. note:: 739 740 Currently, CXGBE PMD only supports the binding of PF4 for Chelsio NICs. 741 742#. Load nic_uio kernel driver: 743 744 .. code-block:: console 745 746 kldload <build_dir>/kernel/freebsd/nic_uio.ko 747 748#. Start testpmd with basic parameters: 749 750 .. code-block:: console 751 752 ./<build_dir>/app/dpdk-testpmd -l 0-3 -n 4 -a 0000:02:00.4 -- -i 753 754 Example output: 755 756 .. code-block:: console 757 758 [...] 759 EAL: PCI device 0000:02:00.4 on NUMA socket 0 760 EAL: probe driver: 1425:5401 rte_cxgbe_pmd 761 EAL: PCI memory mapped at 0x8007ec000 762 EAL: PCI memory mapped at 0x842800000 763 EAL: PCI memory mapped at 0x80086c000 764 PMD: rte_cxgbe_pmd: fw: 1.25.6.0, TP: 0.1.23.2 765 PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter 766 Interactive-mode selected 767 Configuring Port 0 (socket 0) 768 Port 0: 00:07:43:2D:EA:C0 769 Configuring Port 1 (socket 0) 770 Port 1: 00:07:43:2D:EA:C8 771 Checking link statuses... 772 PMD: rte_cxgbe_pmd: Port0: passive DA port module inserted 773 PMD: rte_cxgbe_pmd: Port1: passive DA port module inserted 774 Port 0 Link Up - speed 10000 Mbps - full-duplex 775 Port 1 Link Up - speed 10000 Mbps - full-duplex 776 Done 777 testpmd> 778 779.. note:: 780 781 Flow control pause TX/RX is disabled by default and can be enabled via 782 testpmd. Refer section :ref:`flow-control` for more details. 783 784Sample Application Notes 785------------------------ 786 787.. _flow-control: 788 789Enable/Disable Flow Control 790~~~~~~~~~~~~~~~~~~~~~~~~~~~ 791 792Flow control pause TX/RX is disabled by default and can be enabled via 793testpmd as follows: 794 795.. code-block:: console 796 797 testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 0 798 testpmd> set flow_ctrl rx on tx on 0 0 0 0 mac_ctrl_frame_fwd off autoneg on 1 799 800To disable again, run: 801 802.. code-block:: console 803 804 testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 0 805 testpmd> set flow_ctrl rx off tx off 0 0 0 0 mac_ctrl_frame_fwd off autoneg off 1 806 807Jumbo Mode 808~~~~~~~~~~ 809 810There are two ways to enable sending and receiving of jumbo frames via testpmd. 811One method involves using the **mtu** command, which changes the mtu of an 812individual port without having to stop the selected port. Another method 813involves stopping all the ports first and then running **max-pkt-len** command 814to configure the mtu of all the ports with a single command. 815 816- To configure each port individually, run the mtu command as follows: 817 818 .. code-block:: console 819 820 testpmd> port config mtu 0 9000 821 testpmd> port config mtu 1 9000 822 823- To configure all the ports at once, stop all the ports first and run the 824 max-pkt-len command as follows: 825 826 .. code-block:: console 827 828 testpmd> port stop all 829 testpmd> port config all max-pkt-len 9000 830 831Hardware Configuration and Debugging 832------------------------------------ 833 834Firmware Configuration File 835~~~~~~~~~~~~~~~~~~~~~~~~~~~ 836 837To enable or disable Chelsio NIC features before firmware initialization, 838the Chelsio firmware configuration file can be placed in following 839directory. 840 841.. code-block:: console 842 843 # For Chelsio T5 NIC series 844 cp <path_to_config_file>/t5-config.txt /lib/firmware/cxgb4/t5-config.txt 845 846 # For Chelsio T6 NIC series 847 cp <path_to_config_file>/t6-config.txt /lib/firmware/cxgb4/t6-config.txt 848 849The firmware configuration file is mainly intended to be used to debug 850firmware initialization failures. It can also be used to redistribute 851NIC resources from disabled physical functions (PFs) to main PF before 852initializing firmware. 853 854The CXGBE PMD will search and pick up the firmware configuration file 855during the Chelsio NIC probe, in following order. 856 857#. If the firmware configuration file is present in /lib/firmware/cxgb4/ 858 directory, then this file is downloaded to temporary location in 859 NIC's on-chip RAM. When firmware is initializing, it picks up the 860 file from the temporary on-chip RAM location. 861 862#. Otherwise, if the firmware configuration file has been written 863 onto the NIC persistent flash area using cxgbtool, then this 864 file is picked up from the persistent flash area during 865 firmware initialization. 866 867#. If no firmware configuration file is found at /lib/firmware/cxgb4/ 868 directory or on the NIC persistent flash area, then the firmware 869 will initialize with the default configuration file embedded inside 870 the firmware binary. 871 872.. warning:: 873 874 Note that the Chelsio firmware configuration file contains very low 875 level details that is specific to the Chelsio NIC. Hence, the 876 firmware configuration file must not be modified without expert 877 guidance from Chelsio support team. 878 879Limitations 880----------- 881 882The Chelsio Terminator series of devices provide two/four ports but 883expose a single PCI bus address, thus, librte_net_cxgbe registers 884itself as a PCI driver that allocates one Ethernet device per detected 885port. 886 887For this reason, one cannot allow/block a single port without 888allowing/blocking the other ports on the same device. 889 890