1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(C) 2020 Marvell International Ltd. 3 4L3 Forwarding Graph Sample Application 5====================================== 6 7The L3 Forwarding Graph application is a simple example of packet processing 8using the DPDK Graph framework. The application performs L3 forwarding using 9Graph framework and nodes written for graph framework. 10 11Overview 12-------- 13 14The application demonstrates the use of the graph framework and graph nodes 15``ethdev_rx``, ``ip4_lookup``, ``ip4_rewrite``, ``ethdev_tx`` and ``pkt_drop`` in DPDK to 16implement packet forwarding. 17 18The initialization is very similar to those of the :doc:`l3_forward`. 19There is also additional initialization of graph for graph object creation 20and configuration per lcore. 21Run-time path is main thing that differs from L3 forwarding sample application. 22Difference is that forwarding logic starting from Rx, followed by LPM lookup, 23TTL update and finally Tx is implemented inside graph nodes. These nodes are 24interconnected in graph framework. Application main loop needs to walk over 25graph using ``rte_graph_walk()`` with graph objects created one per worker lcore. 26 27The lookup method is as per implementation of ``ip4_lookup`` graph node. 28The ID of the output interface for the input packet is the next hop returned by 29the LPM lookup. The set of LPM rules used by the application is statically 30configured and provided to ``ip4_lookup`` graph node and ``ip4_rewrite`` graph node 31using node control API ``rte_node_ip4_route_add()`` and ``rte_node_ip4_rewrite_add()``. 32 33In the sample application, only IPv4 forwarding is supported as of now. 34 35Compiling the Application 36------------------------- 37 38To compile the sample application see :doc:`compiling`. 39 40The application is located in the ``l3fwd-graph`` sub-directory. 41 42Running the Application 43----------------------- 44 45The application has a number of command line options similar to l3fwd:: 46 47 ./dpdk-l3fwd-graph [EAL options] -- -p PORTMASK 48 [-P] 49 --config(port,queue,lcore)[,(port,queue,lcore)] 50 [--eth-dest=X,MM:MM:MM:MM:MM:MM] 51 [--enable-jumbo [--max-pkt-len PKTLEN]] 52 [--no-numa] 53 [--per-port-pool] 54 55Where, 56 57* ``-p PORTMASK:`` Hexadecimal bitmask of ports to configure 58 59* ``-P:`` Optional, sets all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. 60 Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. 61 62* ``--config (port,queue,lcore)[,(port,queue,lcore)]:`` Determines which queues from which ports are mapped to which cores. 63 64* ``--eth-dest=X,MM:MM:MM:MM:MM:MM:`` Optional, ethernet destination for port X. 65 66* ``--enable-jumbo:`` Optional, enables jumbo frames. 67 68* ``--max-pkt-len:`` Optional, under the premise of enabling jumbo, maximum packet length in decimal (64-9600). 69 70* ``--no-numa:`` Optional, disables numa awareness. 71 72* ``--per-port-pool:`` Optional, set to use independent buffer pools per port. Without this option, single buffer pool is used for all ports. 73 74For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, 75while cores 8-15 and 24-31 appear on socket 1. 76 77To enable L3 forwarding between two ports, assuming that both ports are in the same socket, using two cores, cores 1 and 2, 78(which are in the same socket too), use the following command: 79 80.. code-block:: console 81 82 ./<build_dir>/examples/dpdk-l3fwd-graph -l 1,2 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2)" 83 84In this command: 85 86* The -l option enables cores 1, 2 87 88* The -p option enables ports 0 and 1 89 90* The --config option enables one queue on each port and maps each (port,queue) pair to a specific core. 91 The following table shows the mapping in this example: 92 93+----------+-----------+-----------+-------------------------------------+ 94| **Port** | **Queue** | **lcore** | **Description** | 95| | | | | 96+----------+-----------+-----------+-------------------------------------+ 97| 0 | 0 | 1 | Map queue 0 from port 0 to lcore 1. | 98| | | | | 99+----------+-----------+-----------+-------------------------------------+ 100| 1 | 0 | 2 | Map queue 0 from port 1 to lcore 2. | 101| | | | | 102+----------+-----------+-----------+-------------------------------------+ 103 104Refer to the *DPDK Getting Started Guide* for general information on running applications and 105the Environment Abstraction Layer (EAL) options. 106 107.. _l3_fwd_graph_explanation: 108 109Explanation 110----------- 111 112The following sections provide some explanation of the sample application code. 113As mentioned in the overview section, the initialization is similar to that of 114the :doc:`l3_forward`. Run-time path though similar in functionality to that of 115:doc:`l3_forward`, major part of the implementation is in graph nodes via used 116via ``librte_node`` library. 117The following sections describe aspects that are specific to the L3 Forwarding 118Graph sample application. 119 120Graph Node Pre-Init Configuration 121~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 122 123After device configuration and device Rx, Tx queue setup is complete, 124a minimal config of port id, num_rx_queues, num_tx_queues, mempools etc will 125be passed to *ethdev_** node ctrl API ``rte_node_eth_config()``. This will be 126lead to the clone of ``ethdev_rx`` and ``ethdev_tx`` nodes as ``ethdev_rx-X-Y`` and 127``ethdev_tx-X`` where X, Y represent port id and queue id associated with them. 128In case of ``ethdev_tx-X`` nodes, tx queue id assigned per instance of the node 129is same as graph id. 130 131These cloned nodes along with existing static nodes such as ``ip4_lookup`` and 132``ip4_rewrite`` will be used in graph creation to associate node's to lcore 133specific graph object. 134 135.. literalinclude:: ../../../examples/l3fwd-graph/main.c 136 :language: c 137 :start-after: Initialize all ports. 8< 138 :end-before: >8 End of graph creation. 139 :dedent: 1 140 141Graph Initialization 142~~~~~~~~~~~~~~~~~~~~ 143 144Now a graph needs to be created with a specific set of nodes for every lcore. 145A graph object returned after graph creation is a per lcore object and 146cannot be shared between lcores. Since ``ethdev_tx-X`` node is per port node, 147it can be associated with all the graphs created as all the lcores should have 148Tx capability for every port. But ``ethdev_rx-X-Y`` node is created per 149(port, rx_queue_id), so they should be associated with a graph based on 150the application argument ``--config`` specifying rx queue mapping to lcore. 151 152.. note:: 153 154 The Graph creation will fail if the passed set of shell node pattern's 155 are not sufficient to meet their inter-dependency or even one node is not 156 found with a given regex node pattern. 157 158.. literalinclude:: ../../../examples/l3fwd-graph/main.c 159 :language: c 160 :start-after: Graph initialization. 8< 161 :end-before: >8 End of graph initialization. 162 :dedent: 1 163 164Forwarding data(Route, Next-Hop) addition 165~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 166 167Once graph objects are created, node specific info like routes and rewrite 168headers will be provided run-time using ``rte_node_ip4_route_add()`` and 169``rte_node_ip4_rewrite_add()`` API. 170 171.. note:: 172 173 Since currently ``ip4_lookup`` and ``ip4_rewrite`` nodes don't support 174 lock-less mechanisms(RCU, etc) to add run-time forwarding data like route and 175 rewrite data, forwarding data is added before packet processing loop is 176 launched on worker lcore. 177 178.. literalinclude:: ../../../examples/l3fwd-graph/main.c 179 :language: c 180 :start-after: Add route to ip4 graph infra. 8< 181 :end-before: >8 End of adding route to ip4 graph infa. 182 :dedent: 1 183 184Packet Forwarding using Graph Walk 185~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 186 187Now that all the device configurations are done, graph creations are done and 188forwarding data is updated with nodes, worker lcores will be launched with graph 189main loop. Graph main loop is very simple in the sense that it needs to 190continuously call a non-blocking API ``rte_graph_walk()`` with it's lcore 191specific graph object that was already created. 192 193.. note:: 194 195 rte_graph_walk() will walk over all the sources nodes i.e ``ethdev_rx-X-Y`` 196 associated with a given graph and Rx the available packets and enqueue them 197 to the following node ``ip4_lookup`` which then will enqueue them to ``ip4_rewrite`` 198 node if LPM lookup succeeds. ``ip4_rewrite`` node then will update Ethernet header 199 as per next-hop data and transmit the packet via port 'Z' by enqueuing 200 to ``ethdev_tx-Z`` node instance in its graph object. 201 202.. literalinclude:: ../../../examples/l3fwd-graph/main.c 203 :language: c 204 :start-after: Main processing loop. 8< 205 :end-before: >8 End of main processing loop. 206