xref: /dpdk/doc/guides/nics/mvpp2.rst (revision 089e5ed727a15da2729cfee9b63533dd120bd04c)
1..  SPDX-License-Identifier: BSD-3-Clause
2    Copyright(c) 2017 Marvell International Ltd.
3    Copyright(c) 2017 Semihalf.
4
5.. _mvpp2_poll_mode_driver:
6
7MVPP2 Poll Mode Driver
8======================
9
10The MVPP2 PMD (librte_pmd_mvpp2) provides poll mode driver support
11for the Marvell PPv2 (Packet Processor v2) 1/10 Gbps adapter.
12
13Detailed information about SoCs that use PPv2 can be obtained here:
14
15* https://www.marvell.com/embedded-processors/armada-70xx/
16* https://www.marvell.com/embedded-processors/armada-80xx/
17
18.. Note::
19
20   Due to external dependencies, this driver is disabled by default. It must
21   be enabled manually by setting relevant configuration option manually.
22   Please refer to `Config File Options`_ section for further details.
23
24
25Features
26--------
27
28Features of the MVPP2 PMD are:
29
30- Speed capabilities
31- Link status
32- Tx Queue start/stop
33- MTU update
34- Jumbo frame
35- Promiscuous mode
36- Allmulticast mode
37- Unicast MAC filter
38- Multicast MAC filter
39- RSS hash
40- VLAN filter
41- CRC offload
42- L3 checksum offload
43- L4 checksum offload
44- Packet type parsing
45- Basic stats
46- :ref:`Extended stats <extstats>`
47- RX flow control
48- Scattered TX frames
49- :ref:`QoS <qossupport>`
50- :ref:`Flow API <flowapi>`
51- :ref:`Traffic metering and policing <mtrapi>`
52- :ref:`Traffic Management API <tmapi>`
53
54Limitations
55-----------
56
57- Number of lcores is limited to 9 by MUSDK internal design. If more lcores
58  need to be allocated, locking will have to be considered. Number of available
59  lcores can be changed via ``MRVL_MUSDK_HIFS_RESERVED`` define in
60  ``mrvl_ethdev.c`` source file.
61
62- Flushing vlans added for filtering is not possible due to MUSDK missing
63  functionality. Current workaround is to reset board so that PPv2 has a
64  chance to start in a sane state.
65
66- MUSDK architecture does not support changing configuration in run time.
67  All necessary configurations should be done before first dev_start().
68
69- RX queue start/stop is not supported.
70
71- Current implementation does not support replacement of buffers in the HW buffer pool
72  at run time, so it is responsibility of the application to ensure that MTU does not exceed the configured buffer size.
73
74- Configuring TX flow control currently is not supported.
75
76- In current implementation, mechanism for acknowledging transmitted packets (``tx_done_cleanup``) is not supported.
77
78- Running more than one DPDK-MUSDK application simultaneously is not supported.
79
80
81Prerequisites
82-------------
83
84- Custom Linux Kernel sources
85
86  .. code-block:: console
87
88     git clone https://github.com/MarvellEmbeddedProcessors/linux-marvell.git -b linux-4.4.120-armada-18.09
89
90- Out of tree `mvpp2x_sysfs` kernel module sources
91
92  .. code-block:: console
93
94     git clone https://github.com/MarvellEmbeddedProcessors/mvpp2x-marvell.git -b mvpp2x-armada-18.09
95
96- MUSDK (Marvell User-Space SDK) sources
97
98  .. code-block:: console
99
100     git clone https://github.com/MarvellEmbeddedProcessors/musdk-marvell.git -b musdk-armada-18.09
101
102  MUSDK is a light-weight library that provides direct access to Marvell's
103  PPv2 (Packet Processor v2). Alternatively prebuilt MUSDK library can be
104  requested from `Marvell Extranet <https://extranet.marvell.com>`_. Once
105  approval has been granted, library can be found by typing ``musdk`` in
106  the search box.
107
108  To get better understanding of the library one can consult documentation
109  available in the ``doc`` top level directory of the MUSDK sources.
110
111- DPDK environment
112
113  Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup
114  DPDK environment.
115
116
117Config File Options
118-------------------
119
120The following options can be modified in the ``config`` file.
121
122- ``CONFIG_RTE_LIBRTE_MVPP2_PMD`` (default ``n``)
123
124    Toggle compilation of the librte mvpp2 driver.
125
126    .. Note::
127
128       When MVPP2 PMD is enabled ``CONFIG_RTE_LIBRTE_MVNETA_PMD`` must be disabled
129
130
131Building DPDK
132-------------
133
134Driver needs precompiled MUSDK library during compilation.
135
136.. code-block:: console
137
138   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
139   ./bootstrap
140   ./configure --host=aarch64-linux-gnu
141   make install
142
143MUSDK will be installed to `usr/local` under current directory.
144For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
145
146Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with
147the path to the MUSDK installation directory needs to be exported.
148
149For additional instructions regarding DPDK cross compilation please refer to :doc:`Cross compile DPDK for ARM64 <../linux_gsg/cross_build_dpdk_for_arm64>`.
150
151.. code-block:: console
152
153   export LIBMUSDK_PATH=<musdk>/usr/local
154   export CROSS=<toolchain>/bin/aarch64-linux-gnu-
155   export RTE_KERNELDIR=<kernel-dir>
156   export RTE_TARGET=arm64-armv8a-linux-gcc
157
158   make config T=arm64-armv8a-linux-gcc
159   sed -i "s/MVNETA_PMD=y/MVNETA_PMD=n/" build/.config
160   sed -i "s/MVPP2_PMD=n/MVPP2_PMD=y/" build/.config
161   make
162
163Usage Example
164-------------
165
166MVPP2 PMD requires extra out of tree kernel modules to function properly.
167`musdk_cma` sources are part of the MUSDK. Please consult
168``doc/musdk_get_started.txt`` for the detailed build instructions.
169For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
170detailed build instructions.
171
172.. code-block:: console
173
174   insmod musdk_cma.ko
175   insmod mvpp2x_sysfs.ko
176
177Additionally interfaces used by DPDK application need to be put up:
178
179.. code-block:: console
180
181   ip link set eth0 up
182   ip link set eth2 up
183
184In order to run testpmd example application following command can be used:
185
186.. code-block:: console
187
188   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
189     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
190     -i -a --rss-udp
191
192.. _extstats:
193
194Extended stats
195--------------
196
197MVPP2 PMD supports the following extended statistics:
198
199	- ``rx_bytes``:	number of RX bytes
200	- ``rx_packets``: number of RX packets
201	- ``rx_unicast_packets``: number of RX unicast packets
202	- ``rx_errors``: number of RX MAC errors
203	- ``rx_fullq_dropped``: number of RX packets dropped due to full RX queue
204	- ``rx_bm_dropped``: number of RX packets dropped due to no available buffers in the HW pool
205	- ``rx_early_dropped``: number of RX packets that were early dropped
206	- ``rx_fifo_dropped``: number of RX packets dropped due to RX fifo overrun
207	- ``rx_cls_dropped``: number of RX packets dropped by classifier
208	- ``tx_bytes``: number of TX bytes
209	- ``tx_packets``: number of TX packets
210	- ``tx_unicast_packets``: number of TX unicast packets
211	- ``tx_errors``: number of TX MAC errors
212
213
214.. _qossupport:
215
216QoS Configuration
217-----------------
218
219QoS configuration is done through external configuration file. Path to the
220file must be given as `cfg` in driver's vdev parameter list.
221
222Configuration syntax
223~~~~~~~~~~~~~~~~~~~~
224
225.. code-block:: console
226
227   [policer <policer_id>]
228   token_unit = <token_unit>
229   color = <color_mode>
230   cir = <cir>
231   ebs = <ebs>
232   cbs = <cbs>
233
234   [port <portnum> default]
235   default_tc = <default_tc>
236   mapping_priority = <mapping_priority>
237
238   rate_limit_enable = <rate_limit_enable>
239   rate_limit = <rate_limit>
240   burst_size = <burst_size>
241
242   default_policer = <policer_id>
243
244   [port <portnum> tc <traffic_class>]
245   rxq = <rx_queue_list>
246   pcp = <pcp_list>
247   dscp = <dscp_list>
248   default_color = <default_color>
249
250   [port <portnum> tc <traffic_class>]
251   rxq = <rx_queue_list>
252   pcp = <pcp_list>
253   dscp = <dscp_list>
254
255   [port <portnum> txq <txqnum>]
256   sched_mode = <sched_mode>
257   wrr_weight = <wrr_weight>
258
259   rate_limit_enable = <rate_limit_enable>
260   rate_limit = <rate_limit>
261   burst_size = <burst_size>
262
263Where:
264
265- ``<portnum>``: DPDK Port number (0..n).
266
267- ``<default_tc>``: Default traffic class (e.g. 0)
268
269- ``<mapping_priority>``: QoS priority for mapping (`ip`, `vlan`, `ip/vlan` or `vlan/ip`).
270
271- ``<traffic_class>``: Traffic Class to be configured.
272
273- ``<rx_queue_list>``: List of DPDK RX queues (e.g. 0 1 3-4)
274
275- ``<pcp_list>``: List of PCP values to handle in particular TC (e.g. 0 1 3-4 7).
276
277- ``<dscp_list>``: List of DSCP values to handle in particular TC (e.g. 0-12 32-48 63).
278
279- ``<default_policer>``: Id of the policer configuration section to be used as default.
280
281- ``<policer_id>``: Id of the policer configuration section (0..31).
282
283- ``<token_unit>``: Policer token unit (`bytes` or `packets`).
284
285- ``<color_mode>``: Policer color mode (`aware` or `blind`).
286
287- ``<cir>``: Committed information rate in unit of kilo bits per second (data rate) or packets per second.
288
289- ``<cbs>``: Committed burst size in unit of kilo bytes or number of packets.
290
291- ``<ebs>``: Excess burst size in unit of kilo bytes or number of packets.
292
293- ``<default_color>``: Default color for specific tc.
294
295- ``<rate_limit_enable>``: Enables per port or per txq rate limiting (`0`/`1` to disable/enable).
296
297- ``<rate_limit>``: Committed information rate, in kilo bits per second.
298
299- ``<burst_size>``: Committed burst size, in kilo bytes.
300
301- ``<sched_mode>``: Egress scheduler mode (`wrr` or `sp`).
302
303- ``<wrr_weight>``: Txq weight.
304
305Setting PCP/DSCP values for the default TC is not required. All PCP/DSCP
306values not assigned explicitly to particular TC will be handled by the
307default TC.
308
309Configuration file example
310^^^^^^^^^^^^^^^^^^^^^^^^^^
311
312.. code-block:: console
313
314   [policer 0]
315   token_unit = bytes
316   color = blind
317   cir = 100000
318   ebs = 64
319   cbs = 64
320
321   [port 0 default]
322   default_tc = 0
323   mapping_priority = ip
324
325   rate_limit_enable = 1
326   rate_limit = 1000
327   burst_size = 2000
328
329   [port 0 tc 0]
330   rxq = 0 1
331
332   [port 0 txq 0]
333   sched_mode = wrr
334   wrr_weight = 10
335
336   [port 0 txq 1]
337   sched_mode = wrr
338   wrr_weight = 100
339
340   [port 0 txq 2]
341   sched_mode = sp
342
343   [port 0 tc 1]
344   rxq = 2
345   pcp = 5 6 7
346   dscp = 26-38
347
348   [port 1 default]
349   default_tc = 0
350   mapping_priority = vlan/ip
351
352   default_policer = 0
353
354   [port 1 tc 0]
355   rxq = 0
356   dscp = 10
357
358   [port 1 tc 1]
359   rxq = 1
360   dscp = 11-20
361
362   [port 1 tc 2]
363   rxq = 2
364   dscp = 30
365
366   [port 1 txq 0]
367   rate_limit_enable = 1
368   rate_limit = 10000
369   burst_size = 2000
370
371Usage example
372^^^^^^^^^^^^^
373
374.. code-block:: console
375
376   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \
377     -c 7 -- -i -a --disable-hw-vlan-strip --rxq=3 --txq=3
378
379.. _flowapi:
380
381Flow API
382--------
383
384PPv2 offers packet classification capabilities via classifier engine which
385can be configured via generic flow API offered by DPDK.
386
387The :ref:`flow_isolated_mode` is supported.
388
389For an additional description please refer to DPDK :doc:`../prog_guide/rte_flow`.
390
391Supported flow actions
392~~~~~~~~~~~~~~~~~~~~~~
393
394Following flow action items are supported by the driver:
395
396* DROP
397* QUEUE
398
399Supported flow items
400~~~~~~~~~~~~~~~~~~~~
401
402Following flow items and their respective fields are supported by the driver:
403
404* ETH
405
406  * source MAC
407  * destination MAC
408  * ethertype
409
410* VLAN
411
412  * PCP
413  * VID
414
415* IPV4
416
417  * DSCP
418  * protocol
419  * source address
420  * destination address
421
422* IPV6
423
424  * flow label
425  * next header
426  * source address
427  * destination address
428
429* UDP
430
431  * source port
432  * destination port
433
434* TCP
435
436  * source port
437  * destination port
438
439Classifier match engine
440~~~~~~~~~~~~~~~~~~~~~~~
441
442Classifier has an internal match engine which can be configured to
443operate in either exact or maskable mode.
444
445Mode is selected upon creation of the first unique flow rule as follows:
446
447* maskable, if key size is up to 8 bytes.
448* exact, otherwise, i.e for keys bigger than 8 bytes.
449
450Where the key size equals the number of bytes of all fields specified
451in the flow items.
452
453.. table:: Examples of key size calculation
454
455   +----------------------------------------------------------------------------+-------------------+-------------+
456   | Flow pattern                                                               | Key size in bytes | Used engine |
457   +============================================================================+===================+=============+
458   | ETH (destination MAC) / VLAN (VID)                                         | 6 + 2 = 8         | Maskable    |
459   +----------------------------------------------------------------------------+-------------------+-------------+
460   | VLAN (VID) / IPV4 (source address)                                         | 2 + 4 = 6         | Maskable    |
461   +----------------------------------------------------------------------------+-------------------+-------------+
462   | TCP (source port, destination port)                                        | 2 + 2 = 4         | Maskable    |
463   +----------------------------------------------------------------------------+-------------------+-------------+
464   | VLAN (priority) / IPV4 (source address)                                    | 1 + 4 = 5         | Maskable    |
465   +----------------------------------------------------------------------------+-------------------+-------------+
466   | IPV4 (destination address) / UDP (source port, destination port)           | 6 + 2 + 2 = 10    | Exact       |
467   +----------------------------------------------------------------------------+-------------------+-------------+
468   | VLAN (VID) / IPV6 (flow label, destination address)                        | 2 + 3 + 16 = 21   | Exact       |
469   +----------------------------------------------------------------------------+-------------------+-------------+
470   | IPV4 (DSCP, source address, destination address)                           | 1 + 4 + 4 = 9     | Exact       |
471   +----------------------------------------------------------------------------+-------------------+-------------+
472   | IPV6 (flow label, source address, destination address)                     | 3 + 16 + 16 = 35  | Exact       |
473   +----------------------------------------------------------------------------+-------------------+-------------+
474
475From the user perspective maskable mode means that masks specified
476via flow rules are respected. In case of exact match mode, masks
477which do not provide exact matching (all bits masked) are ignored.
478
479If the flow matches more than one classifier rule the first
480(with the lowest index) matched takes precedence.
481
482Flow rules usage example
483~~~~~~~~~~~~~~~~~~~~~~~~
484
485Before proceeding run testpmd user application:
486
487.. code-block:: console
488
489   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 3 -- -i --p 3 -a --disable-hw-vlan-strip
490
491Example #1
492^^^^^^^^^^
493
494.. code-block:: console
495
496   testpmd> flow create 0 ingress pattern eth src is 10:11:12:13:14:15 / end actions drop / end
497
498In this case key size is 6 bytes thus maskable type is selected. Testpmd
499will set mask to ff:ff:ff:ff:ff:ff i.e traffic explicitly matching
500above rule will be dropped.
501
502Example #2
503^^^^^^^^^^
504
505.. code-block:: console
506
507   testpmd> flow create 0 ingress pattern ipv4 src spec 10.10.10.0 src mask 255.255.255.0 / tcp src spec 0x10 src mask 0x10 / end action drop / end
508
509In this case key size is 8 bytes thus maskable type is selected.
510Flows which have IPv4 source addresses ranging from 10.10.10.0 to 10.10.10.255
511and tcp source port set to 16 will be dropped.
512
513Example #3
514^^^^^^^^^^
515
516.. code-block:: console
517
518   testpmd> flow create 0 ingress pattern vlan vid spec 0x10 vid mask 0x10 / ipv4 src spec 10.10.1.1 src mask 255.255.0.0 dst spec 11.11.11.1 dst mask 255.255.255.0 / end actions drop / end
519
520In this case key size is 10 bytes thus exact type is selected.
521Even though each item has partial mask set, masks will be ignored.
522As a result only flows with VID set to 16 and IPv4 source and destination
523addresses set to 10.10.1.1 and 11.11.11.1 respectively will be dropped.
524
525Limitations
526~~~~~~~~~~~
527
528Following limitations need to be taken into account while creating flow rules:
529
530* For IPv4 exact match type the key size must be up to 12 bytes.
531* For IPv6 exact match type the key size must be up to 36 bytes.
532* Following fields cannot be partially masked (all masks are treated as
533  if they were exact):
534
535  * ETH: ethertype
536  * VLAN: PCP, VID
537  * IPv4: protocol
538  * IPv6: next header
539  * TCP/UDP: source port, destination port
540
541* Only one classifier table can be created thus all rules in the table
542  have to match table format. Table format is set during creation of
543  the first unique flow rule.
544* Up to 5 fields can be specified per flow rule.
545* Up to 20 flow rules can be added.
546
547For additional information about classifier please consult
548``doc/musdk_cls_user_guide.txt``.
549
550.. _mtrapi:
551
552Traffic metering and policing
553-----------------------------
554
555MVPP2 PMD supports DPDK traffic metering and policing that allows the following:
556
5571. Meter ingress traffic.
5582. Do policing.
5593. Gather statistics.
560
561For an additional description please refer to DPDK :doc:`Traffic Metering and Policing API <../prog_guide/traffic_metering_and_policing>`.
562
563The policer objects defined by this feature can work with the default policer defined via config file as described in :ref:`QoS Support <qossupport>`.
564
565Limitations
566~~~~~~~~~~~
567
568The following capabilities are not supported:
569
570- MTR object meter DSCP table update
571- MTR object policer action update
572- MTR object enabled statistics
573
574Usage example
575~~~~~~~~~~~~~
576
5771. Run testpmd user app:
578
579   .. code-block:: console
580
581		./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 6 -- -i -p 3 -a --txd 1024 --rxd 1024
582
5832. Create meter profile:
584
585   .. code-block:: console
586
587		testpmd> add port meter profile 0 0 srtcm_rfc2697 2000 256 256
588
5893. Create meter:
590
591   .. code-block:: console
592
593		testpmd> create port meter 0 0 0 yes d d d 0 1 0
594
5954. Create flow rule witch meter attached:
596
597   .. code-block:: console
598
599		testpmd> flow create 0 ingress pattern ipv4 src is 10.10.10.1 / end actions meter mtr_id 0 / end
600
601For a detailed usage description please refer to "Traffic Metering and Policing" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
602
603
604
605.. _tmapi:
606
607Traffic Management API
608----------------------
609
610MVPP2 PMD supports generic DPDK Traffic Management API which allows to
611configure the following features:
612
6131. Hierarchical scheduling
6142. Traffic shaping
6153. Congestion management
6164. Packet marking
617
618Internally TM is represented by a hierarchy (tree) of nodes.
619Node which has a parent is called a leaf whereas node without
620parent is called a non-leaf (root).
621MVPP2 PMD supports two level hierarchy where level 0 represents ports and level 1 represents tx queues of a given port.
622
623.. figure:: img/mvpp2_tm.*
624
625Nodes hold following types of settings:
626
627- for egress scheduler configuration: weight
628- for egress rate limiter: private shaper
629- bitmask indicating which statistics counters will be read
630
631Hierarchy is always constructed from the top, i.e first a root node is added
632then some number of leaf nodes. Number of leaf nodes cannot exceed number
633of configured tx queues.
634
635After hierarchy is complete it can be committed.
636
637
638For an additional description please refer to DPDK :doc:`Traffic Management API <../prog_guide/traffic_management>`.
639
640Limitations
641~~~~~~~~~~~
642
643The following capabilities are not supported:
644
645- Traffic manager WRED profile and WRED context
646- Traffic manager shared shaper update
647- Traffic manager packet marking
648- Maximum number of levels in hierarchy is 2
649- Currently dynamic change of a hierarchy is not supported
650
651Usage example
652~~~~~~~~~~~~~
653
654For a detailed usage description please refer to "Traffic Management" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
655
6561. Run testpmd as follows:
657
658   .. code-block:: console
659
660		./testpmd --vdev=net_mrvl,iface=eth0,iface=eth2,cfg=./qos_config -c 7 -- \
661		-i -p 3 --disable-hw-vlan-strip --rxq 3 --txq 3 --txd 1024 --rxd 1024
662
6632. Stop all ports:
664
665   .. code-block:: console
666
667		testpmd> port stop all
668
6693. Add shaper profile:
670
671   .. code-block:: console
672
673		testpmd> add port tm node shaper profile 0 0 900000 70000 0
674
675   Parameters have following meaning::
676
677		0       - Id of a port.
678		0       - Id of a new shaper profile.
679		900000  - Shaper rate in bytes/s.
680		70000   - Bucket size in bytes.
681		0       - Packet length adjustment - ignored.
682
6834. Add non-leaf node for port 0:
684
685   .. code-block:: console
686
687		testpmd> add port tm nonleaf node 0 3 -1 0 0 0 0 0 1 3 0
688
689   Parameters have following meaning::
690
691		 0  - Id of a port
692		 3  - Id of a new node.
693		-1  - Indicate that root does not have a parent.
694		 0  - Priority of the node.
695		 0  - Weight of the node.
696		 0  - Id of a level. Since this is a root 0 is passed.
697		 0  - Id of the shaper profile.
698		 0  - Number of SP priorities.
699		 3  - Enable statistics for both number of transmitted packets and bytes.
700		 0  - Number of shared shapers.
701
7025. Add leaf node for tx queue 0:
703
704   .. code-block:: console
705
706		testpmd> add port tm leaf node 0 0 3 0 30 1 -1 0 0 1 0
707
708   Parameters have following meaning::
709
710		 0  - Id of a port.
711		 0  - Id of a new node.
712		 3  - Id of the parent node.
713		 0  - Priority of a node.
714		 30 - WRR weight.
715		 1  - Id of a level. Since this is a leaf node 1 is passed.
716		-1  - Id of a shaper. -1 indicates that shaper is not attached.
717		 0  - Congestion management is not supported.
718		 0  - Congestion management is not supported.
719		 1  - Enable statistics counter for number of transmitted packets.
720		 0  - Number of shared shapers.
721
7226. Add leaf node for tx queue 1:
723
724   .. code-block:: console
725
726	testpmd> add port tm leaf node 0 1 3 0 60 1 -1 0 0 1 0
727
728   Parameters have following meaning::
729
730		 0  - Id of a port.
731		 1  - Id of a new node.
732		 3  - Id of the parent node.
733		 0  - Priority of a node.
734		 60 - WRR weight.
735		 1  - Id of a level. Since this is a leaf node 1 is passed.
736		-1  - Id of a shaper. -1 indicates that shaper is not attached.
737		 0  - Congestion management is not supported.
738		 0  - Congestion management is not supported.
739		 1  - Enable statistics counter for number of transmitted packets.
740		 0  - Number of shared shapers.
741
7427. Add leaf node for tx queue 2:
743
744   .. code-block:: console
745
746		testpmd> add port tm leaf node 0 2 3 0 99 1 -1 0 0 1 0
747
748   Parameters have following meaning::
749
750		 0  - Id of a port.
751		 2  - Id of a new node.
752		 3  - Id of the parent node.
753		 0  - Priority of a node.
754		 99 - WRR weight.
755		 1  - Id of a level. Since this is a leaf node 1 is passed.
756		-1  - Id of a shaper. -1 indicates that shaper is not attached.
757		 0  - Congestion management is not supported.
758		 0  - Congestion management is not supported.
759		 1  - Enable statistics counter for number of transmitted packets.
760		 0  - Number of shared shapers.
761
7628. Commit hierarchy:
763
764   .. code-block:: console
765
766		testpmd> port tm hierarchy commit 0 no
767
768  Parameters have following meaning::
769
770		0  - Id of a port.
771		no - Do not flush TM hierarchy if commit fails.
772
7739. Start all ports
774
775   .. code-block:: console
776
777		testpmd> port start all
778
779
780
78110. Enable forwarding
782
783   .. code-block:: console
784
785		testpmd> start
786