xref: /dpdk/doc/guides/nics/mvpp2.rst (revision 68a03efeed657e6e05f281479b33b51102797e15)
1..  SPDX-License-Identifier: BSD-3-Clause
2    Copyright(c) 2017 Marvell International Ltd.
3    Copyright(c) 2017 Semihalf.
4
5.. _mvpp2_poll_mode_driver:
6
7MVPP2 Poll Mode Driver
8======================
9
10The MVPP2 PMD (**librte_net_mvpp2**) provides poll mode driver support
11for the Marvell PPv2 (Packet Processor v2) 1/10 Gbps adapter.
12
13Detailed information about SoCs that use PPv2 can be obtained here:
14
15* https://www.marvell.com/embedded-processors/armada-70xx/
16* https://www.marvell.com/embedded-processors/armada-80xx/
17
18
19Features
20--------
21
22Features of the MVPP2 PMD are:
23
24- Speed capabilities
25- Link status
26- Tx Queue start/stop
27- MTU update
28- Jumbo frame
29- Promiscuous mode
30- Allmulticast mode
31- Unicast MAC filter
32- Multicast MAC filter
33- RSS hash
34- VLAN filter
35- CRC offload
36- L3 checksum offload
37- L4 checksum offload
38- Packet type parsing
39- Basic stats
40- :ref:`Extended stats <extstats>`
41- RX flow control
42- Scattered TX frames
43- :ref:`QoS <extconf>`
44- :ref:`Flow API <flowapi>`
45- :ref:`Traffic metering and policing <mtrapi>`
46- :ref:`Traffic Management API <tmapi>`
47
48Limitations
49-----------
50
51- Number of lcores is limited to 9 by MUSDK internal design. If more lcores
52  need to be allocated, locking will have to be considered. Number of available
53  lcores can be changed via ``MRVL_MUSDK_HIFS_RESERVED`` define in
54  ``mrvl_ethdev.c`` source file.
55
56- Flushing vlans added for filtering is not possible due to MUSDK missing
57  functionality. Current workaround is to reset board so that PPv2 has a
58  chance to start in a sane state.
59
60- MUSDK architecture does not support changing configuration in run time.
61  All necessary configurations should be done before first dev_start().
62
63- RX queue start/stop is not supported.
64
65- Current implementation does not support replacement of buffers in the HW buffer pool
66  at run time, so it is responsibility of the application to ensure that MTU does not exceed the configured buffer size.
67
68- Configuring TX flow control currently is not supported.
69
70- In current implementation, mechanism for acknowledging transmitted packets (``tx_done_cleanup``) is not supported.
71
72- Running more than one DPDK-MUSDK application simultaneously is not supported.
73
74
75Prerequisites
76-------------
77
78- Custom Linux Kernel sources
79
80  .. code-block:: console
81
82     git clone https://github.com/MarvellEmbeddedProcessors/linux-marvell.git -b linux-4.4.120-armada-18.09
83
84- Out of tree `mvpp2x_sysfs` kernel module sources
85
86  .. code-block:: console
87
88     git clone https://github.com/MarvellEmbeddedProcessors/mvpp2x-marvell.git -b mvpp2x-armada-18.09
89
90- MUSDK (Marvell User-Space SDK) sources
91
92  .. code-block:: console
93
94     git clone https://github.com/MarvellEmbeddedProcessors/musdk-marvell.git -b musdk-release-SDK-10.3.5.0-PR2
95
96  MUSDK is a light-weight library that provides direct access to Marvell's
97  PPv2 (Packet Processor v2). Alternatively prebuilt MUSDK library can be
98  requested from `Marvell Extranet <https://extranet.marvell.com>`_. Once
99  approval has been granted, library can be found by typing ``musdk`` in
100  the search box.
101
102  To get better understanding of the library one can consult documentation
103  available in the ``doc`` top level directory of the MUSDK sources.
104
105- DPDK environment
106
107  Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup
108  DPDK environment.
109
110
111Building MUSDK
112--------------
113
114Driver needs precompiled MUSDK library during compilation.
115
116.. code-block:: console
117
118   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
119   ./bootstrap
120   ./configure --host=aarch64-linux-gnu
121   make install
122
123MUSDK will be installed to `usr/local` under current directory.
124For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
125
126
127Building DPDK
128-------------
129
130Add path to libmusdk.pc in PKG_CONFIG_PATH environment variable.
131
132.. code-block:: console
133
134   export PKG_CONFIG_PATH=$<musdk_install_dir>/lib/pkgconfig/:$PKG_CONFIG_PATH
135
136   meson build --cross-file config/arm/arm64_armada_linux_gcc
137   ninja -C build
138
139
140Usage Example
141-------------
142
143MVPP2 PMD requires extra out of tree kernel modules to function properly.
144`musdk_cma` sources are part of the MUSDK. Please consult
145``doc/musdk_get_started.txt`` for the detailed build instructions.
146For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
147detailed build instructions.
148
149.. code-block:: console
150
151   insmod musdk_cma.ko
152   insmod mvpp2x_sysfs.ko
153
154Additionally interfaces used by DPDK application need to be put up:
155
156.. code-block:: console
157
158   ip link set eth0 up
159   ip link set eth2 up
160
161In order to run testpmd example application following command can be used:
162
163.. code-block:: console
164
165   ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
166     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
167     -i -a --rss-udp
168
169.. _extstats:
170
171Extended stats
172--------------
173
174MVPP2 PMD supports the following extended statistics:
175
176	- ``rx_bytes``:	number of RX bytes
177	- ``rx_packets``: number of RX packets
178	- ``rx_unicast_packets``: number of RX unicast packets
179	- ``rx_errors``: number of RX MAC errors
180	- ``rx_fullq_dropped``: number of RX packets dropped due to full RX queue
181	- ``rx_bm_dropped``: number of RX packets dropped due to no available buffers in the HW pool
182	- ``rx_early_dropped``: number of RX packets that were early dropped
183	- ``rx_fifo_dropped``: number of RX packets dropped due to RX fifo overrun
184	- ``rx_cls_dropped``: number of RX packets dropped by classifier
185	- ``tx_bytes``: number of TX bytes
186	- ``tx_packets``: number of TX packets
187	- ``tx_unicast_packets``: number of TX unicast packets
188	- ``tx_errors``: number of TX MAC errors
189
190
191.. _extconf:
192
193External Configuration
194----------------------
195
196Several driver configuration (e.g. QoS) can be done through external configuration file. Path to the
197file must be given as `cfg` in driver's vdev parameter list.
198
199Configuration syntax
200~~~~~~~~~~~~~~~~~~~~
201
202.. code-block:: console
203
204   [policer <policer_id>]
205   token_unit = <token_unit>
206   color = <color_mode>
207   cir = <cir>
208   ebs = <ebs>
209   cbs = <cbs>
210
211   [parser udf <udf_id>]
212   proto = <proto>
213   field = <field>
214   key = <key>
215   mask = <mask>
216   offset = <offset>
217
218   [port <portnum> default]
219   start_hdr = <start_hdr>
220   forward_bad_frames = <forward_bad_frames>
221   fill_bpool_buffs = <fill_bpool_buffs>
222   default_tc = <default_tc>
223   mapping_priority = <mapping_priority>
224
225   rate_limit_enable = <rate_limit_enable>
226   rate_limit = <rate_limit>
227   burst_size = <burst_size>
228
229   default_policer = <policer_id>
230
231   [port <portnum> tc <traffic_class>]
232   rxq = <rx_queue_list>
233   pcp = <pcp_list>
234   dscp = <dscp_list>
235   default_color = <default_color>
236
237   [port <portnum> tc <traffic_class>]
238   rxq = <rx_queue_list>
239   pcp = <pcp_list>
240   dscp = <dscp_list>
241
242   [port <portnum> txq <txqnum>]
243   sched_mode = <sched_mode>
244   wrr_weight = <wrr_weight>
245
246   rate_limit_enable = <rate_limit_enable>
247   rate_limit = <rate_limit>
248   burst_size = <burst_size>
249
250Where:
251
252- ``<udf_id>``: Logical UDF id.
253
254- ``<proto>``: Indicate the preceding hdr before the UDF header (`eth` or `udp`).
255
256- ``<field>``: Indicate the field of the <proto> hdr (`type` (eth) or `dport` (udp).
257
258- ``<key>``: UDF key in string format starting with '0x'.
259
260- ``<mask>``: UDF mask in string format starting with '0x'.
261
262- ``<offset>``: Starting UDF offset from the <proto> hdr.
263
264- ``<portnum>``: DPDK Port number (0..n).
265
266- ``<start_hdr>``: Indicate what is the start header mode (`none` (eth), `dsa`, `ext_dsa` or `custom`).
267
268- ``<forward_bad_frames>``: Indicate whether to forward or drop l2 bad packets (0 or 1).
269
270- ``<fill_bpool_buffs>``: Control the amount of refill buffers (default is 64).
271
272- ``<default_tc>``: Default traffic class (e.g. 0)
273
274- ``<mapping_priority>``: QoS priority for mapping (`ip`, `vlan`, `ip/vlan` or `vlan/ip`).
275
276- ``<traffic_class>``: Traffic Class to be configured.
277
278- ``<rx_queue_list>``: List of DPDK RX queues (e.g. 0 1 3-4)
279
280- ``<pcp_list>``: List of PCP values to handle in particular TC (e.g. 0 1 3-4 7).
281
282- ``<dscp_list>``: List of DSCP values to handle in particular TC (e.g. 0-12 32-48 63).
283
284- ``<default_policer>``: Id of the policer configuration section to be used as default.
285
286- ``<policer_id>``: Id of the policer configuration section (0..31).
287
288- ``<token_unit>``: Policer token unit (`bytes` or `packets`).
289
290- ``<color_mode>``: Policer color mode (`aware` or `blind`).
291
292- ``<cir>``: Committed information rate in unit of kilo bits per second (data rate) or packets per second.
293
294- ``<cbs>``: Committed burst size in unit of kilo bytes or number of packets.
295
296- ``<ebs>``: Excess burst size in unit of kilo bytes or number of packets.
297
298- ``<default_color>``: Default color for specific tc.
299
300- ``<rate_limit_enable>``: Enables per port or per txq rate limiting (`0`/`1` to disable/enable).
301
302- ``<rate_limit>``: Committed information rate, in kilo bits per second.
303
304- ``<burst_size>``: Committed burst size, in kilo bytes.
305
306- ``<sched_mode>``: Egress scheduler mode (`wrr` or `sp`).
307
308- ``<wrr_weight>``: Txq weight.
309
310Setting PCP/DSCP values for the default TC is not required. All PCP/DSCP
311values not assigned explicitly to particular TC will be handled by the
312default TC.
313
314Configuration file example
315^^^^^^^^^^^^^^^^^^^^^^^^^^
316
317.. code-block:: console
318
319   [policer 0]
320   token_unit = bytes
321   color = blind
322   cir = 100000
323   ebs = 64
324   cbs = 64
325
326   [port 0 default]
327   default_tc = 0
328   mapping_priority = ip
329
330   rate_limit_enable = 1
331   rate_limit = 1000
332   burst_size = 2000
333
334   [port 0 tc 0]
335   rxq = 0 1
336
337   [port 0 txq 0]
338   sched_mode = wrr
339   wrr_weight = 10
340
341   [port 0 txq 1]
342   sched_mode = wrr
343   wrr_weight = 100
344
345   [port 0 txq 2]
346   sched_mode = sp
347
348   [port 0 tc 1]
349   rxq = 2
350   pcp = 5 6 7
351   dscp = 26-38
352
353   [port 1 default]
354   default_tc = 0
355   mapping_priority = vlan/ip
356
357   default_policer = 0
358
359   [port 1 tc 0]
360   rxq = 0
361   dscp = 10
362
363   [port 1 tc 1]
364   rxq = 1
365   dscp = 11-20
366
367   [port 1 tc 2]
368   rxq = 2
369   dscp = 30
370
371   [port 1 txq 0]
372   rate_limit_enable = 1
373   rate_limit = 10000
374   burst_size = 2000
375
376Configuration file example with UDF
377^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
378
379.. code-block:: console
380
381   [parser udf 0]
382   proto = eth
383   field = type
384   key = 0x8842
385   mask = 0xffff
386   offset = 6
387
388Usage example
389^^^^^^^^^^^^^
390
391.. code-block:: console
392
393   ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \
394     -c 7 -- -i -a --disable-hw-vlan-strip --rxq=3 --txq=3
395
396.. _flowapi:
397
398Flow API
399--------
400
401PPv2 offers packet classification capabilities via classifier engine which
402can be configured via generic flow API offered by DPDK.
403
404The :ref:`flow_isolated_mode` is supported.
405
406For an additional description please refer to DPDK :doc:`../prog_guide/rte_flow`.
407
408Supported flow actions
409~~~~~~~~~~~~~~~~~~~~~~
410
411Following flow action items are supported by the driver:
412
413* DROP
414* QUEUE
415* METER
416
417Supported flow items
418~~~~~~~~~~~~~~~~~~~~
419
420Following flow items and their respective fields are supported by the driver:
421
422* ETH
423
424  * source MAC
425  * destination MAC
426  * ethertype
427
428* VLAN
429
430  * PCP
431  * VID
432
433* IPV4
434
435  * DSCP
436  * protocol
437  * source address
438  * destination address
439
440* IPV6
441
442  * flow label
443  * next header
444  * source address
445  * destination address
446
447* UDP
448
449  * source port
450  * destination port
451
452* TCP
453
454  * source port
455  * destination port
456
457Classifier match engine
458~~~~~~~~~~~~~~~~~~~~~~~
459
460Classifier has an internal match engine which can be configured to
461operate in either exact or maskable mode.
462
463Mode is selected upon creation of the first unique flow rule as follows:
464
465* maskable, if key size is up to 8 bytes.
466* exact, otherwise, i.e for keys bigger than 8 bytes.
467
468Where the key size equals the number of bytes of all fields specified
469in the flow items.
470
471.. table:: Examples of key size calculation
472
473   +----------------------------------------------------------------------------+-------------------+-------------+
474   | Flow pattern                                                               | Key size in bytes | Used engine |
475   +============================================================================+===================+=============+
476   | ETH (destination MAC) / VLAN (VID)                                         | 6 + 2 = 8         | Maskable    |
477   +----------------------------------------------------------------------------+-------------------+-------------+
478   | VLAN (VID) / IPV4 (source address)                                         | 2 + 4 = 6         | Maskable    |
479   +----------------------------------------------------------------------------+-------------------+-------------+
480   | TCP (source port, destination port)                                        | 2 + 2 = 4         | Maskable    |
481   +----------------------------------------------------------------------------+-------------------+-------------+
482   | VLAN (priority) / IPV4 (source address)                                    | 1 + 4 = 5         | Maskable    |
483   +----------------------------------------------------------------------------+-------------------+-------------+
484   | IPV4 (destination address) / UDP (source port, destination port)           | 6 + 2 + 2 = 10    | Exact       |
485   +----------------------------------------------------------------------------+-------------------+-------------+
486   | VLAN (VID) / IPV6 (flow label, destination address)                        | 2 + 3 + 16 = 21   | Exact       |
487   +----------------------------------------------------------------------------+-------------------+-------------+
488   | IPV4 (DSCP, source address, destination address)                           | 1 + 4 + 4 = 9     | Exact       |
489   +----------------------------------------------------------------------------+-------------------+-------------+
490   | IPV6 (flow label, source address, destination address)                     | 3 + 16 + 16 = 35  | Exact       |
491   +----------------------------------------------------------------------------+-------------------+-------------+
492
493From the user perspective maskable mode means that masks specified
494via flow rules are respected. In case of exact match mode, masks
495which do not provide exact matching (all bits masked) are ignored.
496
497If the flow matches more than one classifier rule the first
498(with the lowest index) matched takes precedence.
499
500Flow rules usage example
501~~~~~~~~~~~~~~~~~~~~~~~~
502
503Before proceeding run testpmd user application:
504
505.. code-block:: console
506
507   ./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 3 -- -i --p 3 -a --disable-hw-vlan-strip
508
509Example #1
510^^^^^^^^^^
511
512.. code-block:: console
513
514   testpmd> flow create 0 ingress pattern eth src is 10:11:12:13:14:15 / end actions drop / end
515
516In this case key size is 6 bytes thus maskable type is selected. Testpmd
517will set mask to ff:ff:ff:ff:ff:ff i.e traffic explicitly matching
518above rule will be dropped.
519
520Example #2
521^^^^^^^^^^
522
523.. code-block:: console
524
525   testpmd> flow create 0 ingress pattern ipv4 src spec 10.10.10.0 src mask 255.255.255.0 / tcp src spec 0x10 src mask 0x10 / end action drop / end
526
527In this case key size is 8 bytes thus maskable type is selected.
528Flows which have IPv4 source addresses ranging from 10.10.10.0 to 10.10.10.255
529and tcp source port set to 16 will be dropped.
530
531Example #3
532^^^^^^^^^^
533
534.. code-block:: console
535
536   testpmd> flow create 0 ingress pattern vlan vid spec 0x10 vid mask 0x10 / ipv4 src spec 10.10.1.1 src mask 255.255.0.0 dst spec 11.11.11.1 dst mask 255.255.255.0 / end actions drop / end
537
538In this case key size is 10 bytes thus exact type is selected.
539Even though each item has partial mask set, masks will be ignored.
540As a result only flows with VID set to 16 and IPv4 source and destination
541addresses set to 10.10.1.1 and 11.11.11.1 respectively will be dropped.
542
543Limitations
544~~~~~~~~~~~
545
546Following limitations need to be taken into account while creating flow rules:
547
548* For IPv4 exact match type the key size must be up to 12 bytes.
549* For IPv6 exact match type the key size must be up to 36 bytes.
550* Following fields cannot be partially masked (all masks are treated as
551  if they were exact):
552
553  * ETH: ethertype
554  * VLAN: PCP, VID
555  * IPv4: protocol
556  * IPv6: next header
557  * TCP/UDP: source port, destination port
558
559* Only one classifier table can be created thus all rules in the table
560  have to match table format. Table format is set during creation of
561  the first unique flow rule.
562* Up to 5 fields can be specified per flow rule.
563* Up to 20 flow rules can be added.
564
565For additional information about classifier please consult
566``doc/musdk_cls_user_guide.txt``.
567
568.. _mtrapi:
569
570Traffic metering and policing
571-----------------------------
572
573MVPP2 PMD supports DPDK traffic metering and policing that allows the following:
574
5751. Meter ingress traffic.
5762. Do policing.
5773. Gather statistics.
578
579For an additional description please refer to DPDK :doc:`Traffic Metering and Policing API <../prog_guide/traffic_metering_and_policing>`.
580
581The policer objects defined by this feature can work with the default policer defined via config file as described in :ref:`QoS Support <extconf>`.
582
583Limitations
584~~~~~~~~~~~
585
586The following capabilities are not supported:
587
588- MTR object meter DSCP table update
589- MTR object policer action update
590- MTR object enabled statistics
591
592Usage example
593~~~~~~~~~~~~~
594
5951. Run testpmd user app:
596
597   .. code-block:: console
598
599		./dpdk-testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 6 -- -i -p 3 -a --txd 1024 --rxd 1024
600
6012. Create meter profile:
602
603   .. code-block:: console
604
605		testpmd> add port meter profile 0 0 srtcm_rfc2697 2000 256 256
606
6073. Create meter:
608
609   .. code-block:: console
610
611		testpmd> create port meter 0 0 0 yes d d d 0 1 0
612
6134. Create flow rule witch meter attached:
614
615   .. code-block:: console
616
617		testpmd> flow create 0 ingress pattern ipv4 src is 10.10.10.1 / end actions meter mtr_id 0 / end
618
619For a detailed usage description please refer to "Traffic Metering and Policing" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
620
621
622
623.. _tmapi:
624
625Traffic Management API
626----------------------
627
628MVPP2 PMD supports generic DPDK Traffic Management API which allows to
629configure the following features:
630
6311. Hierarchical scheduling
6322. Traffic shaping
6333. Congestion management
6344. Packet marking
635
636Internally TM is represented by a hierarchy (tree) of nodes.
637Node which has a parent is called a leaf whereas node without
638parent is called a non-leaf (root).
639MVPP2 PMD supports two level hierarchy where level 0 represents ports and level 1 represents tx queues of a given port.
640
641.. figure:: img/mvpp2_tm.*
642
643Nodes hold following types of settings:
644
645- for egress scheduler configuration: weight
646- for egress rate limiter: private shaper
647- bitmask indicating which statistics counters will be read
648
649Hierarchy is always constructed from the top, i.e first a root node is added
650then some number of leaf nodes. Number of leaf nodes cannot exceed number
651of configured tx queues.
652
653After hierarchy is complete it can be committed.
654
655
656For an additional description please refer to DPDK :doc:`Traffic Management API <../prog_guide/traffic_management>`.
657
658Limitations
659~~~~~~~~~~~
660
661The following capabilities are not supported:
662
663- Traffic manager WRED profile and WRED context
664- Traffic manager shared shaper update
665- Traffic manager packet marking
666- Maximum number of levels in hierarchy is 2
667- Currently dynamic change of a hierarchy is not supported
668
669Usage example
670~~~~~~~~~~~~~
671
672For a detailed usage description please refer to "Traffic Management" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
673
6741. Run testpmd as follows:
675
676   .. code-block:: console
677
678		./dpdk-testpmd --vdev=net_mrvl,iface=eth0,iface=eth2,cfg=./qos_config -c 7 -- \
679		-i -p 3 --disable-hw-vlan-strip --rxq 3 --txq 3 --txd 1024 --rxd 1024
680
6812. Stop all ports:
682
683   .. code-block:: console
684
685		testpmd> port stop all
686
6873. Add shaper profile:
688
689   .. code-block:: console
690
691		testpmd> add port tm node shaper profile 0 0 900000 70000 0
692
693   Parameters have following meaning::
694
695		0       - Id of a port.
696		0       - Id of a new shaper profile.
697		900000  - Shaper rate in bytes/s.
698		70000   - Bucket size in bytes.
699		0       - Packet length adjustment - ignored.
700
7014. Add non-leaf node for port 0:
702
703   .. code-block:: console
704
705		testpmd> add port tm nonleaf node 0 3 -1 0 0 0 0 0 1 3 0
706
707   Parameters have following meaning::
708
709		 0  - Id of a port
710		 3  - Id of a new node.
711		-1  - Indicate that root does not have a parent.
712		 0  - Priority of the node.
713		 0  - Weight of the node.
714		 0  - Id of a level. Since this is a root 0 is passed.
715		 0  - Id of the shaper profile.
716		 0  - Number of SP priorities.
717		 3  - Enable statistics for both number of transmitted packets and bytes.
718		 0  - Number of shared shapers.
719
7205. Add leaf node for tx queue 0:
721
722   .. code-block:: console
723
724		testpmd> add port tm leaf node 0 0 3 0 30 1 -1 0 0 1 0
725
726   Parameters have following meaning::
727
728		 0  - Id of a port.
729		 0  - Id of a new node.
730		 3  - Id of the parent node.
731		 0  - Priority of a node.
732		 30 - WRR weight.
733		 1  - Id of a level. Since this is a leaf node 1 is passed.
734		-1  - Id of a shaper. -1 indicates that shaper is not attached.
735		 0  - Congestion management is not supported.
736		 0  - Congestion management is not supported.
737		 1  - Enable statistics counter for number of transmitted packets.
738		 0  - Number of shared shapers.
739
7406. Add leaf node for tx queue 1:
741
742   .. code-block:: console
743
744	testpmd> add port tm leaf node 0 1 3 0 60 1 -1 0 0 1 0
745
746   Parameters have following meaning::
747
748		 0  - Id of a port.
749		 1  - Id of a new node.
750		 3  - Id of the parent node.
751		 0  - Priority of a node.
752		 60 - WRR weight.
753		 1  - Id of a level. Since this is a leaf node 1 is passed.
754		-1  - Id of a shaper. -1 indicates that shaper is not attached.
755		 0  - Congestion management is not supported.
756		 0  - Congestion management is not supported.
757		 1  - Enable statistics counter for number of transmitted packets.
758		 0  - Number of shared shapers.
759
7607. Add leaf node for tx queue 2:
761
762   .. code-block:: console
763
764		testpmd> add port tm leaf node 0 2 3 0 99 1 -1 0 0 1 0
765
766   Parameters have following meaning::
767
768		 0  - Id of a port.
769		 2  - Id of a new node.
770		 3  - Id of the parent node.
771		 0  - Priority of a node.
772		 99 - WRR weight.
773		 1  - Id of a level. Since this is a leaf node 1 is passed.
774		-1  - Id of a shaper. -1 indicates that shaper is not attached.
775		 0  - Congestion management is not supported.
776		 0  - Congestion management is not supported.
777		 1  - Enable statistics counter for number of transmitted packets.
778		 0  - Number of shared shapers.
779
7808. Commit hierarchy:
781
782   .. code-block:: console
783
784		testpmd> port tm hierarchy commit 0 no
785
786  Parameters have following meaning::
787
788		0  - Id of a port.
789		no - Do not flush TM hierarchy if commit fails.
790
7919. Start all ports
792
793   .. code-block:: console
794
795		testpmd> port start all
796
797
798
79910. Enable forwarding
800
801   .. code-block:: console
802
803		testpmd> start
804