xref: /dpdk/doc/guides/prog_guide/eventdev/event_dma_adapter.rst (revision 41dd9a6bc2d9c6e20e139ad713cc9d172572dd43)

.. SPDX-License-Identifier: BSD-3-Clause
   Copyright (c) 2023 Marvell.

Event DMA Adapter Library
=========================

DPDK :doc:`eventdev library <eventdev>` provides event driven programming model
with features to schedule events.
:doc:`../dmadev` provides an interface to DMA poll mode drivers
that support DMA operations.
Event DMA adapter is intended to bridge between the event device and the DMA device.

Packet flow from DMA device to the event device can be accomplished
using software and hardware based transfer mechanisms.
The adapter queries an eventdev PMD to determine which mechanism to be used.
The adapter uses an EAL service core function for software-based packet transfer
and uses the eventdev PMD functions to configure hardware-based packet transfer
between DMA device and the event device.
DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV``
to indicate the source of event.

Application can choose to submit a DMA operation directly to a DMA device
or send it to a DMA adapter via eventdev
based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability.
The first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode
and the second as the event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode.
Choice of mode can be specified while creating the adapter.
In the former mode, it is the application's responsibility to enable ingress packet ordering.
In the latter mode, it is the adapter's responsibility to enable ingress packet ordering.


Adapter Modes
-------------

RTE_EVENT_DMA_ADAPTER_OP_NEW mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode,
application submits DMA operations directly to an DMA device.
The adapter then dequeues DMA completions from the DMA device
and enqueues them as events to the event device.
This mode does not ensure ingress ordering
as the application directly enqueues to the dmadev without going through DMA/atomic stage.
In this mode, events dequeued from the adapter are treated as new events.
The application has to specify event information (response information)
which is needed to enqueue an event after the DMA operation is completed.

.. _figure_event_dma_adapter_op_new:

.. figure:: ../img/event_dma_adapter_op_new.*

   Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode


RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode,
if the event PMD and DMA PMD supports internal event port
(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``),
the application should use ``rte_event_dma_adapter_enqueue()`` API
to enqueue DMA operations as events to DMA adapter.
If not, application retrieves DMA adapter's event port
using ``rte_event_dma_adapter_event_port_get()`` API,
links its event queue to this port
and starts enqueuing DMA operations as events to eventdev
using ``rte_event_enqueue_burst()``.
The adapter then dequeues the events
and submits the DMA operations to the dmadev.
After the DMA operation is complete,
the adapter enqueues events to the event device.

Applications can use this mode when ingress packet ordering is needed.
In this mode, events dequeued from the adapter will be treated as forwarded events.
Application has to specify event information (response information)
needed to enqueue the event after the DMA operation has completed.

.. _figure_event_dma_adapter_op_forward:

.. figure:: ../img/event_dma_adapter_op_forward.*

   Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode


API Overview
------------

This section has a brief introduction to the event DMA adapter APIs.
The application is expected to create an adapter
which is associated with a single eventdev,
then add dmadev and vchan to the adapter instance.


Create an adapter instance
~~~~~~~~~~~~~~~~~~~~~~~~~~

An adapter instance is created using ``rte_event_dma_adapter_create()``.
This function is called with event device
to be associated with the adapter and port configuration
for the adapter to setup an event port (if the adapter needs to use a service function).

Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW``
or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode.

.. code-block:: c

   enum rte_event_dma_adapter_mode mode;
   struct rte_event_dev_info dev_info;
   struct rte_event_port_conf conf;
   uint8_t evdev_id;
   uint8_t dma_id;
   int ret;

   ret = rte_event_dev_info_get(dma_id, &dev_info);

   conf.new_event_threshold = dev_info.max_num_events;
   conf.dequeue_depth = dev_info.max_event_port_dequeue_depth;
   conf.enqueue_depth = dev_info.max_event_port_enqueue_depth;
   mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD;
   ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode);


``rte_event_dma_adapter_create_ext()`` function can be used by the application
to have a finer control on eventdev port allocation and setup.
The ``rte_event_dma_adapter_create_ext()`` function is passed a callback function.
The callback function is invoked if the adapter creates a service function
and uses an event port for it.
The callback is expected to fill the ``struct rte_event_dma_adapter_conf`` passed to it.

In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode,
if the event PMD and DMA PMD supports internal event port
(``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``),
events with DMA operations should be enqueued to the DMA adapter
using ``rte_event_dma_adapter_enqueue()`` API.
If not, the event port created by the adapter can be retrieved
using ``rte_event_dma_adapter_event_port_get()`` API.
An application can use this event port to link with an event queue,
on which it enqueues events towards the DMA adapter using ``rte_event_enqueue_burst()``.

.. code-block:: c

   uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid;
   struct rte_event ev;
   uint32_t cap;
   int ret;

   /* Fill in event info and update event_ptr with rte_event_dma_adapter_op */
   memset(&ev, 0, sizeof(ev));
   .
   .
   ev.event_ptr = op;

   ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
   if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) {
           ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events);
   } else {
           ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id);
           ret = rte_event_queue_setup(evdev_id, app_qid, NULL);
           ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1);
           ev.queue_id = app_qid;
           ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events);
   }


Event device configuration for service based adapter
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When ``rte_event_dma_adapter_create()`` is used for creating adapter instance,
``rte_event_dev_config::nb_event_ports`` is automatically incremented,
and event device is reconfigured with additional event port during service initialization.
This event device reconfigure logic also
increments the ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``.

Applications using this mode of adapter creation need not configure the event device
with ``rte_event_dev_config::nb_event_ports`` and
``rte_event_dev_config::nb_single_link_event_port_queues`` parameters
required for DMA adapter when the adapter is created using the above-mentioned API.


Querying adapter capabilities
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``rte_event_dma_adapter_caps_get()`` function allows the application
to query the adapter capabilities for an eventdev and dmadev combination.
This API provides whether dmadev and eventdev are connected using internal HW port or not.

.. code-block:: c

   rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap);


Adding vchan to the adapter instance
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

dmadev device ID and vchan are configured using dmadev APIs.
For more information, see :doc:`dmadev <../dmadev>`.

.. code-block:: c

   struct rte_dma_vchan_conf vchan_conf;
   struct rte_dma_conf dev_conf;
   uint8_t dev_id = 0;
   uint16_t vchan = 0;

   rte_dma_configure(dev_id, &dev_conf);
   rte_dma_vchan_setup(dev_id, vchan, &vchan_conf);

These dmadev ID and vchan are added to the instance
using the ``rte_event_dma_adapter_vchan_add()`` API.
The same is removed using ``rte_event_dma_adapter_vchan_del()`` API.
If hardware supports ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability,
event information must be passed to the add API.

.. code-block:: c

   uint32_t cap;
   int ret;

   ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap);
   if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) {
           struct rte_event event;

           rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf);
   } else
           rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL);


Configuring service function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the adapter uses a service function, the application is required
to assign a service core to the service function as show below.

.. code-block:: c

   uint32_t service_id;

   if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0)
           rte_service_map_lcore_set(service_id, CORE_ID);


Set event response information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` / ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode,
the application specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op``
and the event information (response information)
needed to enqueue an event after the DMA operation has completed.
The response information is specified in ``struct rte_event``
and appended to the ``struct rte_event_dma_adapter_op``.


Start the adapter instance
~~~~~~~~~~~~~~~~~~~~~~~~~~

The application calls ``rte_event_dma_adapter_start()`` to start the adapter.
This function calls the start callbacks of the eventdev PMDs
for hardware-based eventdev-dmadev connections
and ``rte_service_run_state_set()`` to enable the service function if one exists.

.. code-block:: c

   rte_event_dma_adapter_start(id);

.. note::

   The eventdev to which the event_dma_adapter is connected should be started
   before calling ``rte_event_dma_adapter_start()``.


Get adapter statistics
~~~~~~~~~~~~~~~~~~~~~~

The  ``rte_event_dma_adapter_stats_get()`` function reports counters
defined in ``struct rte_event_dma_adapter_stats``.
The received packet and enqueued event counts are a sum of the counts
from the eventdev PMD callbacks if the callback is supported,
and the counts maintained by the service function, if one exists.


Set/Get adapter runtime configuration parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The runtime configuration parameters of adapter can be set/get using
``rte_event_dma_adapter_runtime_params_set()`` and
``rte_event_dma_adapter_runtime_params_get()`` respectively.
The parameters that can be set/get are defined in
``struct rte_event_dma_adapter_runtime_params``.