1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(C) 2020 Marvell International Ltd. 3 4L3 Forwarding Graph Sample Application 5====================================== 6 7The L3 Forwarding Graph application is a simple example of packet processing 8using the DPDK Graph framework. The application performs L3 forwarding using 9Graph framework and nodes written for graph framework. 10 11Overview 12-------- 13 14The application demonstrates the use of the graph framework and graph nodes 15``ethdev_rx``, ``ip4_lookup``, ``ip4_rewrite``, ``ethdev_tx`` and ``pkt_drop`` in DPDK to 16implement packet forwarding. 17 18The initialization is very similar to those of the :doc:`l3_forward`. 19There is also additional initialization of graph for graph object creation 20and configuration per lcore. 21Run-time path is main thing that differs from L3 forwarding sample application. 22Difference is that forwarding logic starting from Rx, followed by LPM lookup, 23TTL update and finally Tx is implemented inside graph nodes. These nodes are 24interconnected in graph framework. Application main loop needs to walk over 25graph using ``rte_graph_walk()`` with graph objects created one per worker lcore. 26 27The lookup method is as per implementation of ``ip4_lookup`` graph node. 28The ID of the output interface for the input packet is the next hop returned by 29the LPM lookup. The set of LPM rules used by the application is statically 30configured and provided to ``ip4_lookup`` graph node and ``ip4_rewrite`` graph node 31using node control API ``rte_node_ip4_route_add()`` and ``rte_node_ip4_rewrite_add()``. 32 33In the sample application, only IPv4 forwarding is supported as of now. 34 35Compiling the Application 36------------------------- 37 38To compile the sample application see :doc:`compiling`. 39 40The application is located in the ``l3fwd-graph`` sub-directory. 41 42Running the Application 43----------------------- 44 45The application has a number of command line options similar to l3fwd:: 46 47 ./dpdk-l3fwd-graph [EAL options] -- -p PORTMASK 48 [-P] 49 --config(port,queue,lcore)[,(port,queue,lcore)] 50 [--eth-dest=X,MM:MM:MM:MM:MM:MM] 51 [--max-pkt-len PKTLEN] 52 [--no-numa] 53 [--per-port-pool] 54 55Where, 56 57* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure 58 59* ``-P:`` Optional, sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. 60 Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. 61 62* ``--config (port,queue,lcore)[,(port,queue,lcore)]:`` Determines which queues from which ports are mapped to which cores. 63 64* ``--eth-dest=X,MM:MM:MM:MM:MM:MM:`` Optional, ethernet destination for port X. 65 66* ``--max-pkt-len:`` Optional, maximum packet length in decimal (64-9600). 67 68* ``--no-numa:`` Optional, disables numa awareness. 69 70* ``--per-port-pool:`` Optional, set to use independent buffer pools per port. Without this option, single buffer pool is used for all ports. 71 72For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, 73while cores 8-15 and 24-31 appear on socket 1. 74 75To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2, 76(which are in the same socket too), use the following command: 77 78.. code-block:: console 79 80 ./<build_dir>/examples/dpdk-l3fwd-graph -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)" 81 82In this command: 83 84* The -l option enables cores 1, 2 85 86* The -p option enables ports 0 and 1 87 88* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core. 89 The following table shows the mapping in this example: 90 91+----------+-----------+-----------+-------------------------------------+ 92| **Port** | **Queue** | **lcore** | **Description** | 93| | | | | 94+----------+-----------+-----------+-------------------------------------+ 95| 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. | 96| | | | | 97+----------+-----------+-----------+-------------------------------------+ 98| 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. | 99| | | | | 100+----------+-----------+-----------+-------------------------------------+ 101 102Refer to the *DPDK Getting Started Guide* for general information on running applications and 103the Environment Abstraction Layer (EAL) options. 104 105.. _l3_fwd_graph_explanation: 106 107Explanation 108----------- 109 110The following sections provide some explanation of the sample application code. 111As mentioned in the overview section, the initialization is similar to that of 112the :doc:`l3_forward`. Run-time path though similar in functionality to that of 113:doc:`l3_forward`, major part of the implementation is in graph nodes via used 114via ``librte_node`` library. 115The following sections describe aspects that are specific to the L3 Forwarding 116Graph sample application. 117 118Graph Node Pre-Init Configuration 119~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 120 121After device configuration and device Rx, Tx queue setup is complete, 122a minimal config of port id, num_rx_queues, num_tx_queues, mempools etc will 123be passed to *ethdev_** node ctrl API ``rte_node_eth_config()``. This will be 124lead to the clone of ``ethdev_rx`` and ``ethdev_tx`` nodes as ``ethdev_rx-X-Y`` and 125``ethdev_tx-X`` where X, Y represent port id and queue id associated with them. 126In case of ``ethdev_tx-X`` nodes, tx queue id assigned per instance of the node 127is same as graph id. 128 129These cloned nodes along with existing static nodes such as ``ip4_lookup`` and 130``ip4_rewrite`` will be used in graph creation to associate node's to lcore 131specific graph object. 132 133.. literalinclude:: ../../../examples/l3fwd-graph/main.c 134 :language: c 135 :start-after: Initialize all ports. 8< 136 :end-before: >8 End of graph creation. 137 :dedent: 1 138 139Graph Initialization 140~~~~~~~~~~~~~~~~~~~~ 141 142Now a graph needs to be created with a specific set of nodes for every lcore. 143A graph object returned after graph creation is a per lcore object and 144cannot be shared between lcores. Since ``ethdev_tx-X`` node is per port node, 145it can be associated with all the graphs created as all the lcores should have 146Tx capability for every port. But ``ethdev_rx-X-Y`` node is created per 147(port, rx_queue_id), so they should be associated with a graph based on 148the application argument ``--config`` specifying rx queue mapping to lcore. 149 150.. note:: 151 152 The Graph creation will fail if the passed set of shell node pattern's 153 are not sufficient to meet their inter-dependency or even one node is not 154 found with a given regex node pattern. 155 156.. literalinclude:: ../../../examples/l3fwd-graph/main.c 157 :language: c 158 :start-after: Graph initialization. 8< 159 :end-before: >8 End of graph initialization. 160 :dedent: 1 161 162Forwarding data(Route, Next-Hop) addition 163~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 164 165Once graph objects are created, node specific info like routes and rewrite 166headers will be provided run-time using ``rte_node_ip4_route_add()`` and 167``rte_node_ip4_rewrite_add()`` API. 168 169.. note:: 170 171 Since currently ``ip4_lookup`` and ``ip4_rewrite`` nodes don't support 172 lock-less mechanisms(RCU, etc) to add run-time forwarding data like route and 173 rewrite data, forwarding data is added before packet processing loop is 174 launched on worker lcore. 175 176.. literalinclude:: ../../../examples/l3fwd-graph/main.c 177 :language: c 178 :start-after: Add route to ip4 graph infra. 8< 179 :end-before: >8 End of adding route to ip4 graph infa. 180 :dedent: 1 181 182Packet Forwarding using Graph Walk 183~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 184 185Now that all the device configurations are done, graph creations are done and 186forwarding data is updated with nodes, worker lcores will be launched with graph 187main loop. Graph main loop is very simple in the sense that it needs to 188continuously call a non-blocking API ``rte_graph_walk()`` with it's lcore 189specific graph object that was already created. 190 191.. note:: 192 193 rte_graph_walk() will walk over all the sources nodes i.e ``ethdev_rx-X-Y`` 194 associated with a given graph and Rx the available packets and enqueue them 195 to the following node ``ip4_lookup`` which then will enqueue them to ``ip4_rewrite`` 196 node if LPM lookup succeeds. ``ip4_rewrite`` node then will update Ethernet header 197 as per next-hop data and transmit the packet via port 'Z' by enqueuing 198 to ``ethdev_tx-Z`` node instance in its graph object. 199 200.. literalinclude:: ../../../examples/l3fwd-graph/main.c 201 :language: c 202 :start-after: Main processing loop. 8< 203 :end-before: >8 End of main processing loop. 204