1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4Packet Distributor Library 5========================== 6 7The DPDK Packet Distributor library is a library designed to be used for dynamic load balancing of traffic 8while supporting single packet at a time operation. 9When using this library, the logical cores in use are to be considered in two roles: firstly a distributor lcore, 10which is responsible for load balancing or distributing packets, 11and a set of worker lcores which are responsible for receiving the packets from the distributor and operating on them. 12The model of operation is shown in the diagram below. 13 14.. figure:: img/packet_distributor1.* 15 16 Packet Distributor mode of operation 17 18There are two modes of operation of the API in the distributor library, 19one which sends one packet at a time to workers using 32-bits for flow_id, 20and an optimized mode which sends bursts of up to 8 packets at a time to workers, using 15 bits of flow_id. 21The mode is selected by the type field in the ``rte_distributor_create()`` function. 22 23Distributor Core Operation 24-------------------------- 25 26The distributor core does the majority of the processing for ensuring that packets are fairly shared among workers. 27The operation of the distributor is as follows: 28 29#. Packets are passed to the distributor component by having the distributor lcore thread call the "rte_distributor_process()" API 30 31#. The worker lcores all share a single cache line with the distributor core in order to pass messages and packets to and from the worker. 32 The process API call will poll all the worker cache lines to see what workers are requesting packets. 33 34#. As workers request packets, the distributor takes packets from the set of packets passed in and distributes them to the workers. 35 As it does so, it examines the "tag" -- stored in the RSS hash field in the mbuf -- for each packet 36 and records what tags are being processed by each worker. 37 38#. If the next packet in the input set has a tag which is already being processed by a worker, 39 then that packet will be queued up for processing by that worker 40 and given to it in preference to other packets when that work next makes a request for work. 41 This ensures that no two packets with the same tag are processed in parallel, 42 and that all packets with the same tag are processed in input order. 43 44#. Once all input packets passed to the process API have either been distributed to workers 45 or been queued up for a worker which is processing a given tag, 46 then the process API returns to the caller. 47 48Other functions which are available to the distributor lcore are: 49 50* rte_distributor_returned_pkts() 51 52* rte_distributor_flush() 53 54* rte_distributor_clear_returns() 55 56Of these the most important API call is "rte_distributor_returned_pkts()" 57which should only be called on the lcore which also calls the process API. 58It returns to the caller all packets which have finished processing by all worker cores. 59Within this set of returned packets, all packets sharing the same tag will be returned in their original order. 60 61**NOTE:** 62If worker lcores buffer up packets internally for transmission in bulk afterwards, 63the packets sharing a tag will likely get out of order. 64Once a worker lcore requests a new packet, the distributor assumes that it has completely finished with the previous packet and 65therefore that additional packets with the same tag can safely be distributed to other workers -- 66who may then flush their buffered packets sooner and cause packets to get out of order. 67 68**NOTE:** 69No packet ordering guarantees are made about packets which do not share a common packet tag. 70 71Using the process and returned_pkts API, the following application workflow can be used, 72while allowing packet order within a packet flow -- identified by a tag -- to be maintained. 73 74 75.. figure:: img/packet_distributor2.* 76 77 Application workflow 78 79 80The flush and clear_returns API calls, mentioned previously, 81are likely of less use that the process and returned_pkts APIS, and are principally provided to aid in unit testing of the library. 82Descriptions of these functions and their use can be found in the DPDK API Reference document. 83 84Worker Operation 85---------------- 86 87Worker cores are the cores which do the actual manipulation of the packets distributed by the packet distributor. 88Each worker calls "rte_distributor_get_pkt()" API to request a new packet when it has finished processing the previous one. 89[The previous packet should be returned to the distributor component by passing it as the final parameter to this API call.] 90 91Since it may be desirable to vary the number of worker cores, depending on the traffic load 92i.e. to save power at times of lighter load, 93it is possible to have a worker stop processing packets by calling "rte_distributor_return_pkt()" to indicate that 94it has finished the current packet and does not want a new one. 95