1 /* SPDX-License-Identifier: BSD-3-Clause 2 * Copyright (c) 2023 Marvell. 3 */ 4 5 #ifndef RTE_EVENT_DMA_ADAPTER 6 #define RTE_EVENT_DMA_ADAPTER 7 8 /** 9 * @file rte_event_dma_adapter.h 10 * 11 * @warning 12 * @b EXPERIMENTAL: 13 * All functions in this file may be changed or removed without prior notice. 14 * 15 * DMA Event Adapter API. 16 * 17 * Eventdev library provides adapters to bridge between various components for providing new 18 * event source. The event DMA adapter is one of those adapters which is intended to bridge 19 * between event devices and DMA devices. 20 * 21 * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The 22 * packet flow between DMA device and the event device can be accomplished using both SW and HW 23 * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet 24 * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the 25 * DMA device and the event device. 26 * 27 * The application can choose to submit a DMA operation directly to an DMA device or send it to the 28 * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The 29 * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the 30 * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while 31 * creating the adapter. In the former mode, it is an application responsibility to enable ingress 32 * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress 33 * packet ordering. 34 * 35 * 36 * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode: 37 * 38 * +--------------+ +--------------+ 39 * | | | DMA stage | 40 * | Application |---[2]-->| + enqueue to | 41 * | | | dmadev | 42 * +--------------+ +--------------+ 43 * ^ ^ | 44 * | | [3] 45 * [6] [1] | 46 * | | | 47 * +--------------+ | 48 * | | | 49 * | Event device | | 50 * | | | 51 * +--------------+ | 52 * ^ | 53 * | | 54 * [5] | 55 * | v 56 * +--------------+ +--------------+ 57 * | | | | 58 * | DMA adapter |<--[4]---| dmadev | 59 * | | | | 60 * +--------------+ +--------------+ 61 * 62 * 63 * [1] Application dequeues events from the previous stage. 64 * [2] Application prepares the DMA operations. 65 * [3] DMA operations are submitted to dmadev by application. 66 * [4] DMA adapter dequeues DMA completions from dmadev. 67 * [5] DMA adapter enqueues events to the eventdev. 68 * [6] Application dequeues from eventdev for further processing. 69 * 70 * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA 71 * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the 72 * event device. This mode does not ensure ingress ordering, if the application directly enqueues 73 * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2]. 74 * 75 * Events dequeued from the adapter will be treated as new events. In this mode, application needs 76 * to specify event information (response information) which is needed to enqueue an event after the 77 * DMA operation is completed. 78 * 79 * 80 * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode: 81 * 82 * +--------------+ +--------------+ 83 * --[1]-->| |---[2]-->| Application | 84 * | Event device | | in | 85 * <--[8]--| |<--[3]---| Ordered stage| 86 * +--------------+ +--------------+ 87 * ^ | 88 * | [4] 89 * [7] | 90 * | v 91 * +----------------+ +--------------+ 92 * | |--[5]->| | 93 * | DMA adapter | | dmadev | 94 * | |<-[6]--| | 95 * +----------------+ +--------------+ 96 * 97 * 98 * [1] Events from the previous stage. 99 * [2] Application in ordered stage dequeues events from eventdev. 100 * [3] Application enqueues DMA operations as events to eventdev. 101 * [4] DMA adapter dequeues event from eventdev. 102 * [5] DMA adapter submits DMA operations to dmadev (Atomic stage). 103 * [6] DMA adapter dequeues DMA completions from dmadev 104 * [7] DMA adapter enqueues events to the eventdev 105 * [8] Events to the next stage 106 * 107 * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability 108 * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA 109 * operations to the dmadev. If not, application retrieves the event port of the DMA adapter 110 * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this 111 * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues 112 * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter 113 * enqueues events to the event device. 114 * 115 * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the 116 * adapter will be treated as forwarded events. In this mode, the application needs to specify the 117 * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition 118 * to the event information (response information) needed to enqueue an event after the DMA 119 * operation has completed. 120 * 121 * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to 122 * event devices for both SW and HW based transfers. The DMA event adapter's functions are: 123 * 124 * - rte_event_dma_adapter_create_ext() 125 * - rte_event_dma_adapter_create() 126 * - rte_event_dma_adapter_free() 127 * - rte_event_dma_adapter_vchan_add() 128 * - rte_event_dma_adapter_vchan_del() 129 * - rte_event_dma_adapter_start() 130 * - rte_event_dma_adapter_stop() 131 * - rte_event_dma_adapter_stats_get() 132 * - rte_event_dma_adapter_stats_reset() 133 * 134 * The application creates an instance using rte_event_dma_adapter_create() or 135 * rte_event_dma_adapter_create_ext(). 136 * 137 * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() / 138 * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability 139 * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the 140 * add API. 141 * 142 */ 143 144 #include <stdint.h> 145 146 #include <rte_common.h> 147 #include <rte_dmadev_pmd.h> 148 #include <rte_eventdev.h> 149 150 #ifdef __cplusplus 151 extern "C" { 152 #endif 153 154 /** 155 * A structure used to hold event based DMA operation entry. All the information 156 * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op" 157 * instance. 158 */ 159 struct rte_event_dma_adapter_op { 160 struct rte_dma_sge *src_seg; 161 /**< Source segments. */ 162 struct rte_dma_sge *dst_seg; 163 /**< Destination segments. */ 164 uint16_t nb_src; 165 /**< Number of source segments. */ 166 uint16_t nb_dst; 167 /**< Number of destination segments. */ 168 uint64_t flags; 169 /**< Flags related to the operation. 170 * @see RTE_DMA_OP_FLAG_* 171 */ 172 int16_t dma_dev_id; 173 /**< DMA device ID to be used */ 174 uint16_t vchan; 175 /**< DMA vchan ID to be used */ 176 struct rte_mempool *op_mp; 177 /**< Mempool from which op is allocated. */ 178 }; 179 180 /** 181 * DMA event adapter mode 182 */ 183 enum rte_event_dma_adapter_mode { 184 RTE_EVENT_DMA_ADAPTER_OP_NEW, 185 /**< Start the DMA adapter in event new mode. 186 * @see RTE_EVENT_OP_NEW. 187 * 188 * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA 189 * completions from dmadev and enqueue events to the eventdev. 190 */ 191 192 RTE_EVENT_DMA_ADAPTER_OP_FORWARD, 193 /**< Start the DMA adapter in event forward mode. 194 * @see RTE_EVENT_OP_FORWARD. 195 * 196 * Application submits DMA requests as events to the DMA adapter or DMA device based on 197 * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued 198 * back to the eventdev by DMA adapter. 199 */ 200 }; 201 202 /** 203 * Adapter configuration structure that the adapter configuration callback function is expected to 204 * fill out. 205 * 206 * @see rte_event_dma_adapter_conf_cb 207 */ 208 struct rte_event_dma_adapter_conf { 209 uint8_t event_port_id; 210 /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA 211 * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. 212 */ 213 214 uint32_t max_nb; 215 /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't 216 * treated as a requirement; batching may cause the adapter to process more than max_nb DMA 217 * ops. 218 */ 219 }; 220 221 /** 222 * Adapter runtime configuration parameters 223 */ 224 struct rte_event_dma_adapter_runtime_params { 225 uint32_t max_nb; 226 /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't 227 * treated as a requirement; batching may cause the adapter to process more than max_nb DMA 228 * ops. 229 * 230 * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter 231 * with default value of max_nb. 232 * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime 233 * (after adding at least one queue pair) 234 * 235 * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or 236 * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability. 237 */ 238 239 uint32_t rsvd[15]; 240 /**< Reserved fields for future expansion */ 241 }; 242 243 /** 244 * Function type used for adapter configuration callback. The callback is used to fill in members of 245 * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for 246 * packet transfer from dmadev vchan to the event device. The SW service is created within the 247 * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan 248 * to the event device are required. 249 * 250 * @param id 251 * Adapter identifier. 252 * @param evdev_id 253 * Event device identifier. 254 * @param conf 255 * Structure that needs to be populated by this callback. 256 * @param arg 257 * Argument to the callback. This is the same as the conf_arg passed to the 258 * rte_event_dma_adapter_create_ext(). 259 */ 260 typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id, 261 struct rte_event_dma_adapter_conf *conf, void *arg); 262 263 /** 264 * A structure used to retrieve statistics for an event DMA adapter instance. 265 */ 266 struct rte_event_dma_adapter_stats { 267 uint64_t event_poll_count; 268 /**< Event port poll count */ 269 270 uint64_t event_deq_count; 271 /**< Event dequeue count */ 272 273 uint64_t dma_enq_count; 274 /**< dmadev enqueue count */ 275 276 uint64_t dma_enq_fail_count; 277 /**< dmadev enqueue failed count */ 278 279 uint64_t dma_deq_count; 280 /**< dmadev dequeue count */ 281 282 uint64_t event_enq_count; 283 /**< Event enqueue count */ 284 285 uint64_t event_enq_retry_count; 286 /**< Event enqueue retry count */ 287 288 uint64_t event_enq_fail_count; 289 /**< Event enqueue fail count */ 290 }; 291 292 /** 293 * Create a new event DMA adapter with the specified identifier. 294 * 295 * @param id 296 * Adapter identifier. 297 * @param evdev_id 298 * Event device identifier. 299 * @param conf_cb 300 * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed 301 * into it. 302 * @param mode 303 * Flag to indicate the mode of the adapter. 304 * @see rte_event_dma_adapter_mode 305 * @param conf_arg 306 * Argument that is passed to the conf_cb function. 307 * 308 * @return 309 * - 0: Success 310 * - <0: Error code on failure 311 */ 312 __rte_experimental 313 int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id, 314 rte_event_dma_adapter_conf_cb conf_cb, 315 enum rte_event_dma_adapter_mode mode, void *conf_arg); 316 317 /** 318 * Create a new event DMA adapter with the specified identifier. This function uses an internal 319 * configuration function that creates an event port. This default function reconfigures the event 320 * device with an additional event port and set up the event port using the port_config parameter 321 * passed into this function. In case the application needs more control in configuration of the 322 * service, it should use the rte_event_dma_adapter_create_ext() version. 323 * 324 * @param id 325 * Adapter identifier. 326 * @param evdev_id 327 * Event device identifier. 328 * @param port_config 329 * Argument of type *rte_event_port_conf* that is passed to the conf_cb function. 330 * @param mode 331 * Flag to indicate the mode of the adapter. 332 * @see rte_event_dma_adapter_mode 333 * 334 * @return 335 * - 0: Success 336 * - <0: Error code on failure 337 */ 338 __rte_experimental 339 int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, 340 struct rte_event_port_conf *port_config, 341 enum rte_event_dma_adapter_mode mode); 342 343 /** 344 * Free an event DMA adapter 345 * 346 * @param id 347 * Adapter identifier. 348 * @return 349 * - 0: Success 350 * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function 351 * returns -EBUSY. 352 */ 353 __rte_experimental 354 int rte_event_dma_adapter_free(uint8_t id); 355 356 /** 357 * Retrieve the event port of an adapter. 358 * 359 * @param id 360 * Adapter identifier. 361 * 362 * @param [out] event_port_id 363 * Application links its event queue to this adapter port which is used in 364 * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. 365 * 366 * @return 367 * - 0: Success 368 * - <0: Error code on failure. 369 */ 370 __rte_experimental 371 int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id); 372 373 /** 374 * Add a vchan to an event DMA adapter. 375 * 376 * @param id 377 * Adapter identifier. 378 * @param dmadev_id 379 * dmadev identifier. 380 * @param vchan 381 * DMA device vchan identifier. If vchan is set -1, adapter adds all the 382 * preconfigured vchan to the instance. 383 * @param event 384 * If HW supports dmadev vchan to event queue binding, application is expected to fill in 385 * event information, else it will be NULL. 386 * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 387 * 388 * @return 389 * - 0: Success, vchan added correctly. 390 * - <0: Error code on failure. 391 */ 392 __rte_experimental 393 int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan, 394 const struct rte_event *event); 395 396 /** 397 * Delete a vchan from an event DMA adapter. 398 * 399 * @param id 400 * Adapter identifier. 401 * @param dmadev_id 402 * DMA device identifier. 403 * @param vchan 404 * DMA device vchan identifier. 405 * 406 * @return 407 * - 0: Success, vchan deleted successfully. 408 * - <0: Error code on failure. 409 */ 410 __rte_experimental 411 int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan); 412 413 /** 414 * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this 415 * function returns -ESRCH. 416 * 417 * @param id 418 * Adapter identifier. 419 * @param [out] service_id 420 * A pointer to a uint32_t, to be filled in with the service id. 421 * 422 * @return 423 * - 0: Success 424 * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function 425 * returns -ESRCH. 426 */ 427 __rte_experimental 428 int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id); 429 430 /** 431 * Start event DMA adapter 432 * 433 * @param id 434 * Adapter identifier. 435 * 436 * @return 437 * - 0: Success, adapter started successfully. 438 * - <0: Error code on failure. 439 * 440 * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started 441 * before calling rte_event_dma_adapter_start(). 442 */ 443 __rte_experimental 444 int rte_event_dma_adapter_start(uint8_t id); 445 446 /** 447 * Stop event DMA adapter 448 * 449 * @param id 450 * Adapter identifier. 451 * 452 * @return 453 * - 0: Success, adapter stopped successfully. 454 * - <0: Error code on failure. 455 */ 456 __rte_experimental 457 int rte_event_dma_adapter_stop(uint8_t id); 458 459 /** 460 * Initialize the adapter runtime configuration parameters 461 * 462 * @param params 463 * A pointer to structure of type struct rte_event_dma_adapter_runtime_params 464 * 465 * @return 466 * - 0: Success 467 * - <0: Error code on failure 468 */ 469 __rte_experimental 470 int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params); 471 472 /** 473 * Set the adapter runtime configuration parameters 474 * 475 * @param id 476 * Adapter identifier 477 * 478 * @param params 479 * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration 480 * parameter values. The reserved fields of this structure must be initialized to zero and the valid 481 * fields need to be set appropriately. This struct can be initialized using 482 * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this 483 * struct and update required fields. 484 * 485 * @return 486 * - 0: Success 487 * - <0: Error code on failure 488 */ 489 __rte_experimental 490 int rte_event_dma_adapter_runtime_params_set(uint8_t id, 491 struct rte_event_dma_adapter_runtime_params *params); 492 493 /** 494 * Get the adapter runtime configuration parameters 495 * 496 * @param id 497 * Adapter identifier 498 * 499 * @param[out] params 500 * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid 501 * adapter parameters when return value is 0. 502 * 503 * @return 504 * - 0: Success 505 * - <0: Error code on failure 506 */ 507 __rte_experimental 508 int rte_event_dma_adapter_runtime_params_get(uint8_t id, 509 struct rte_event_dma_adapter_runtime_params *params); 510 511 /** 512 * Retrieve statistics for an adapter 513 * 514 * @param id 515 * Adapter identifier. 516 * @param [out] stats 517 * A pointer to structure used to retrieve statistics for an adapter. 518 * 519 * @return 520 * - 0: Success, retrieved successfully. 521 * - <0: Error code on failure. 522 */ 523 __rte_experimental 524 int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats); 525 526 /** 527 * Reset statistics for an adapter. 528 * 529 * @param id 530 * Adapter identifier. 531 * 532 * @return 533 * - 0: Success, statistics reset successfully. 534 * - <0: Error code on failure. 535 */ 536 __rte_experimental 537 int rte_event_dma_adapter_stats_reset(uint8_t id); 538 539 /** 540 * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event 541 * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*. 542 * This function is supported if the eventdev PMD has the 543 * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set. 544 * 545 * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the 546 * *ev* array of *rte_event* structure. 547 * 548 * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually 549 * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued. 550 * 551 * @param evdev_id 552 * The identifier of the device. 553 * @param port_id 554 * The identifier of the event port. 555 * @param ev 556 * Points to an array of *nb_events* objects of type *rte_event* structure which contain the 557 * event object enqueue operations to be processed. 558 * @param nb_events 559 * The number of event objects to enqueue, typically number of 560 * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port. 561 * 562 * @return 563 * The number of event objects actually enqueued on the event device. The return value can be 564 * less than the value of the *nb_events* parameter when the event devices queue is full or if 565 * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*, 566 * the remaining events at the end of ev[] are not consumed and the caller has to take care of them, 567 * and rte_errno is set accordingly. Possible errno values include: 568 * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an 569 * event's sched type doesn't match the capabilities of the destination queue. 570 * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This 571 * error code is only applicable to closed systems. 572 */ 573 __rte_experimental 574 uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[], 575 uint16_t nb_events); 576 577 #ifdef __cplusplus 578 } 579 #endif 580 581 #endif /* RTE_EVENT_DMA_ADAPTER */ 582