1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright (c) 2023 Marvell. 3 4Event DMA Adapter Library 5========================= 6 7DPDK :doc:`eventdev library <eventdev>` provides event driven programming model 8with features to schedule events. 9:doc:`../dmadev` provides an interface to DMA poll mode drivers 10that support DMA operations. 11Event DMA adapter is intended to bridge between the event device and the DMA device. 12 13Packet flow from DMA device to the event device can be accomplished 14using software and hardware based transfer mechanisms. 15The adapter queries an eventdev PMD to determine which mechanism to be used. 16The adapter uses an EAL service core function for software-based packet transfer 17and uses the eventdev PMD functions to configure hardware-based packet transfer 18between DMA device and the event device. 19DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` 20to indicate the source of event. 21 22Application can choose to submit a DMA operation directly to a DMA device 23or send it to a DMA adapter via eventdev 24based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. 25The first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode 26and the second as the event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. 27Choice of mode can be specified while creating the adapter. 28In the former mode, it is the application's responsibility to enable ingress packet ordering. 29In the latter mode, it is the adapter's responsibility to enable ingress packet ordering. 30 31 32Adapter Modes 33------------- 34 35RTE_EVENT_DMA_ADAPTER_OP_NEW mode 36~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 37 38In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, 39application submits DMA operations directly to an DMA device. 40The adapter then dequeues DMA completions from the DMA device 41and enqueues them as events to the event device. 42This mode does not ensure ingress ordering 43as the application directly enqueues to the dmadev without going through DMA/atomic stage. 44In this mode, events dequeued from the adapter are treated as new events. 45The application has to specify event information (response information) 46which is needed to enqueue an event after the DMA operation is completed. 47 48.. _figure_event_dma_adapter_op_new: 49 50.. figure:: ../img/event_dma_adapter_op_new.* 51 52 Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode 53 54 55RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode 56~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 57 58In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, 59if the event PMD and DMA PMD supports internal event port 60(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), 61the application should use ``rte_event_dma_adapter_enqueue()`` API 62to enqueue DMA operations as events to DMA adapter. 63If not, application retrieves DMA adapter's event port 64using ``rte_event_dma_adapter_event_port_get()`` API, 65links its event queue to this port 66and starts enqueuing DMA operations as events to eventdev 67using ``rte_event_enqueue_burst()``. 68The adapter then dequeues the events 69and submits the DMA operations to the dmadev. 70After the DMA operation is complete, 71the adapter enqueues events to the event device. 72 73Applications can use this mode when ingress packet ordering is needed. 74In this mode, events dequeued from the adapter will be treated as forwarded events. 75Application has to specify event information (response information) 76needed to enqueue the event after the DMA operation has completed. 77 78.. _figure_event_dma_adapter_op_forward: 79 80.. figure:: ../img/event_dma_adapter_op_forward.* 81 82 Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode 83 84 85API Overview 86------------ 87 88This section has a brief introduction to the event DMA adapter APIs. 89The application is expected to create an adapter 90which is associated with a single eventdev, 91then add dmadev and vchan to the adapter instance. 92 93 94Create an adapter instance 95~~~~~~~~~~~~~~~~~~~~~~~~~~ 96 97An adapter instance is created using ``rte_event_dma_adapter_create()``. 98This function is called with event device 99to be associated with the adapter and port configuration 100for the adapter to setup an event port (if the adapter needs to use a service function). 101 102Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` 103or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode. 104 105.. code-block:: c 106 107 enum rte_event_dma_adapter_mode mode; 108 struct rte_event_dev_info dev_info; 109 struct rte_event_port_conf conf; 110 uint8_t evdev_id; 111 uint8_t dma_id; 112 int ret; 113 114 ret = rte_event_dev_info_get(dma_id, &dev_info); 115 116 conf.new_event_threshold = dev_info.max_num_events; 117 conf.dequeue_depth = dev_info.max_event_port_dequeue_depth; 118 conf.enqueue_depth = dev_info.max_event_port_enqueue_depth; 119 mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD; 120 ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode); 121 122 123``rte_event_dma_adapter_create_ext()`` function can be used by the application 124to have a finer control on eventdev port allocation and setup. 125The ``rte_event_dma_adapter_create_ext()`` function is passed a callback function. 126The callback function is invoked if the adapter creates a service function 127and uses an event port for it. 128The callback is expected to fill the ``struct rte_event_dma_adapter_conf`` passed to it. 129 130In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, 131if the event PMD and DMA PMD supports internal event port 132(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), 133events with DMA operations should be enqueued to the DMA adapter 134using ``rte_event_dma_adapter_enqueue()`` API. 135If not, the event port created by the adapter can be retrieved 136using ``rte_event_dma_adapter_event_port_get()`` API. 137An application can use this event port to link with an event queue, 138on which it enqueues events towards the DMA adapter using ``rte_event_enqueue_burst()``. 139 140.. code-block:: c 141 142 uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid; 143 struct rte_event ev; 144 uint32_t cap; 145 int ret; 146 147 /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */ 148 memset(&ev, 0, sizeof(ev)); 149 . 150 . 151 ev.event_ptr = op; 152 153 ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); 154 if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) { 155 ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events); 156 } else { 157 ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id); 158 ret = rte_event_queue_setup(evdev_id, app_qid, NULL); 159 ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1); 160 ev.queue_id = app_qid; 161 ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events); 162 } 163 164 165Event device configuration for service based adapter 166~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 167 168When ``rte_event_dma_adapter_create()`` is used for creating adapter instance, 169``rte_event_dev_config::nb_event_ports`` is automatically incremented, 170and event device is reconfigured with additional event port during service initialization. 171This event device reconfigure logic also 172increments the ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter 173if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``. 174 175Applications using this mode of adapter creation need not configure the event device 176with ``rte_event_dev_config::nb_event_ports`` and 177``rte_event_dev_config::nb_single_link_event_port_queues`` parameters 178required for DMA adapter when the adapter is created using the above-mentioned API. 179 180 181Querying adapter capabilities 182~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 183 184The ``rte_event_dma_adapter_caps_get()`` function allows the application 185to query the adapter capabilities for an eventdev and dmadev combination. 186This API provides whether dmadev and eventdev are connected using internal HW port or not. 187 188.. code-block:: c 189 190 rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap); 191 192 193Adding vchan to the adapter instance 194~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 195 196dmadev device ID and vchan are configured using dmadev APIs. 197For more information, see :doc:`dmadev <../dmadev>`. 198 199.. code-block:: c 200 201 struct rte_dma_vchan_conf vchan_conf; 202 struct rte_dma_conf dev_conf; 203 uint8_t dev_id = 0; 204 uint16_t vchan = 0; 205 206 rte_dma_configure(dev_id, &dev_conf); 207 rte_dma_vchan_setup(dev_id, vchan, &vchan_conf); 208 209These dmadev ID and vchan are added to the instance 210using the ``rte_event_dma_adapter_vchan_add()`` API. 211The same is removed using ``rte_event_dma_adapter_vchan_del()`` API. 212If hardware supports ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, 213event information must be passed to the add API. 214 215.. code-block:: c 216 217 uint32_t cap; 218 int ret; 219 220 ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); 221 if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) { 222 struct rte_event event; 223 224 rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf); 225 } else 226 rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL); 227 228 229Configuring service function 230~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 231 232If the adapter uses a service function, the application is required 233to assign a service core to the service function as show below. 234 235.. code-block:: c 236 237 uint32_t service_id; 238 239 if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0) 240 rte_service_map_lcore_set(service_id, CORE_ID); 241 242 243Set event response information 244~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 245 246In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` / ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, 247the application specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` 248and the event information (response information) 249needed to enqueue an event after the DMA operation has completed. 250The response information is specified in ``struct rte_event`` 251and appended to the ``struct rte_event_dma_adapter_op``. 252 253 254Start the adapter instance 255~~~~~~~~~~~~~~~~~~~~~~~~~~ 256 257The application calls ``rte_event_dma_adapter_start()`` to start the adapter. 258This function calls the start callbacks of the eventdev PMDs 259for hardware-based eventdev-dmadev connections 260and ``rte_service_run_state_set()`` to enable the service function if one exists. 261 262.. code-block:: c 263 264 rte_event_dma_adapter_start(id); 265 266.. note:: 267 268 The eventdev to which the event_dma_adapter is connected should be started 269 before calling ``rte_event_dma_adapter_start()``. 270 271 272Get adapter statistics 273~~~~~~~~~~~~~~~~~~~~~~ 274 275The ``rte_event_dma_adapter_stats_get()`` function reports counters 276defined in ``struct rte_event_dma_adapter_stats``. 277The received packet and enqueued event counts are a sum of the counts 278from the eventdev PMD callbacks if the callback is supported, 279and the counts maintained by the service function, if one exists. 280 281 282Set/Get adapter runtime configuration parameters 283~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 284 285The runtime configuration parameters of adapter can be set/get using 286``rte_event_dma_adapter_runtime_params_set()`` and 287``rte_event_dma_adapter_runtime_params_get()`` respectively. 288The parameters that can be set/get are defined in 289``struct rte_event_dma_adapter_runtime_params``. 290