1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2017 Intel Corporation 3 4Wireless Baseband Device Library 5================================ 6 7The Wireless Baseband library provides a common programming framework that 8abstracts HW accelerators based on FPGA and/or Fixed Function Accelerators that 9assist with 3gpp Physical Layer processing. Furthermore, it decouples the 10application from the compute-intensive wireless functions by abstracting their 11optimized libraries to appear as virtual bbdev devices. 12 13The functional scope of the BBDEV library are those functions in relation to 14the 3gpp Layer 1 signal processing (channel coding, modulation, ...). 15 16The framework currently only supports Turbo Code FEC function. 17 18 19Design Principles 20----------------- 21 22The Wireless Baseband library follows the same ideology of DPDK's Ethernet 23Device and Crypto Device frameworks. Wireless Baseband provides a generic 24acceleration abstraction framework which supports both physical (hardware) and 25virtual (software) wireless acceleration functions. 26 27Device Management 28----------------- 29 30Device Creation 31~~~~~~~~~~~~~~~ 32 33Physical bbdev devices are discovered during the PCI probe/enumeration of the 34EAL function which is executed at DPDK initialization, based on 35their PCI device identifier, each unique PCI BDF (bus/bridge, device, 36function). 37 38Virtual devices can be created by two mechanisms, either using the EAL command 39line options or from within the application using an EAL API directly. 40 41From the command line using the --vdev EAL option 42 43.. code-block:: console 44 45 --vdev 'turbo_sw,max_nb_queues=8,socket_id=0' 46 47Our using the rte_vdev_init API within the application code. 48 49.. code-block:: c 50 51 rte_vdev_init("turbo_sw", "max_nb_queues=2,socket_id=0") 52 53All virtual bbdev devices support the following initialization parameters: 54 55- ``max_nb_queues`` - maximum number of queues supported by the device. 56 57- ``socket_id`` - socket on which to allocate the device resources on. 58 59 60Device Identification 61~~~~~~~~~~~~~~~~~~~~~ 62 63Each device, whether virtual or physical is uniquely designated by two 64identifiers: 65 66- A unique device index used to designate the bbdev device in all functions 67 exported by the bbdev API. 68 69- A device name used to designate the bbdev device in console messages, for 70 administration or debugging purposes. For ease of use, the port name includes 71 the port index. 72 73 74Device Configuration 75~~~~~~~~~~~~~~~~~~~~ 76 77From the application point of view, each instance of a bbdev device consists of 78one or more queues identified by queue IDs. While different devices may have 79different capabilities (e.g. support different operation types), all queues on 80a device support identical configuration possibilities. A queue is configured 81for only one type of operation and is configured at initializations time. 82When an operation is enqueued to a specific queue ID, the result is dequeued 83from the same queue ID. 84 85Configuration of a device has two different levels: configuration that applies 86to the whole device, and configuration that applies to a single queue. 87 88Device configuration is applied with 89``rte_bbdev_setup_queues(dev_id,num_queues,socket_id)`` 90and queue configuration is applied with 91``rte_bbdev_queue_configure(dev_id,queue_id,conf)``. Note that, although all 92queues on a device support same capabilities, they can be configured differently 93and will then behave differently. 94Devices supporting interrupts can enable them by using 95``rte_bbdev_intr_enable(dev_id)``. 96 97The configuration of each bbdev device includes the following operations: 98 99- Allocation of resources, including hardware resources if a physical device. 100- Resetting the device into a well-known default state. 101- Initialization of statistics counters. 102 103The ``rte_bbdev_setup_queues`` API is used to setup queues for a bbdev device. 104 105.. code-block:: c 106 107 int rte_bbdev_setup_queues(uint16_t dev_id, uint16_t num_queues, 108 int socket_id); 109 110- ``num_queues`` argument identifies the total number of queues to setup for 111 this device. 112 113- ``socket_id`` specifies which socket will be used to allocate the memory. 114 115 116The ``rte_bbdev_intr_enable`` API is used to enable interrupts for a bbdev 117device, if supported by the driver. Should be called before starting the device. 118 119.. code-block:: c 120 121 int rte_bbdev_intr_enable(uint16_t dev_id); 122 123 124Queues Configuration 125~~~~~~~~~~~~~~~~~~~~ 126 127Each bbdev devices queue is individually configured through the 128``rte_bbdev_queue_configure()`` API. 129Each queue resources may be allocated on a specified socket. 130 131.. code-block:: c 132 133 struct rte_bbdev_queue_conf { 134 int socket; 135 uint32_t queue_size; 136 uint8_t priority; 137 bool deferred_start; 138 enum rte_bbdev_op_type op_type; 139 }; 140 141Device & Queues Management 142~~~~~~~~~~~~~~~~~~~~~~~~~~ 143 144After initialization, devices are in a stopped state, so must be started by the 145application. If an application is finished using a device it can close the 146device. Once closed, it cannot be restarted. 147 148.. code-block:: c 149 150 int rte_bbdev_start(uint16_t dev_id) 151 int rte_bbdev_stop(uint16_t dev_id) 152 int rte_bbdev_close(uint16_t dev_id) 153 int rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id) 154 int rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id) 155 156 157By default, all queues are started when the device is started, but they can be 158stopped individually. 159 160.. code-block:: c 161 162 int rte_bbdev_queue_start(uint16_t dev_id, uint16_t queue_id) 163 int rte_bbdev_queue_stop(uint16_t dev_id, uint16_t queue_id) 164 165 166Logical Cores, Memory and Queues Relationships 167~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 168 169The bbdev device Library as the Poll Mode Driver library support NUMA for when 170a processor’s logical cores and interfaces utilize its local memory. Therefore 171baseband operations, the mbuf being operated on should be allocated from memory 172pools created in the local memory. The buffers should, if possible, remain on 173the local processor to obtain the best performance results and buffer 174descriptors should be populated with mbufs allocated from a mempool allocated 175from local memory. 176 177The run-to-completion model also performs better, especially in the case of 178virtual bbdev devices, if the baseband operation and data buffers are in local 179memory instead of a remote processor's memory. This is also true for the 180pipe-line model provided all logical cores used are located on the same processor. 181 182Multiple logical cores should never share the same queue for enqueuing 183operations or dequeuing operations on the same bbdev device since this would 184require global locks and hinder performance. It is however possible to use a 185different logical core to dequeue an operation on a queue pair from the logical 186core which it was enqueued on. This means that a baseband burst enqueue/dequeue 187APIs are a logical place to transition from one logical core to another in a 188packet processing pipeline. 189 190 191Device Operation Capabilities 192----------------------------- 193 194Capabilities (in terms of operations supported, max number of queues, etc.) 195identify what a bbdev is capable of performing that differs from one device to 196another. For the full scope of the bbdev capability see the definition of the 197structure in the *DPDK API Reference*. 198 199.. code-block:: c 200 201 struct rte_bbdev_op_cap; 202 203A device reports its capabilities when registering itself in the bbdev framework. 204With the aid of this capabilities mechanism, an application can query devices to 205discover which operations within the 3gpp physical layer they are capable of 206performing. Below is an example of the capabilities for a PMD it supports in 207relation to Turbo Encoding and Decoding operations. 208 209.. code-block:: c 210 211 static const struct rte_bbdev_op_cap bbdev_capabilities[] = { 212 { 213 .type = RTE_BBDEV_OP_TURBO_DEC, 214 .cap.turbo_dec = { 215 .capability_flags = 216 RTE_BBDEV_TURBO_SUBBLOCK_DEINTERLEAVE | 217 RTE_BBDEV_TURBO_POS_LLR_1_BIT_IN | 218 RTE_BBDEV_TURBO_NEG_LLR_1_BIT_IN | 219 RTE_BBDEV_TURBO_CRC_TYPE_24B, 220 .num_buffers_src = RTE_BBDEV_MAX_CODE_BLOCKS, 221 .num_buffers_hard_out = 222 RTE_BBDEV_MAX_CODE_BLOCKS, 223 .num_buffers_soft_out = 0, 224 } 225 }, 226 { 227 .type = RTE_BBDEV_OP_TURBO_ENC, 228 .cap.turbo_enc = { 229 .capability_flags = 230 RTE_BBDEV_TURBO_CRC_24B_ATTACH | 231 RTE_BBDEV_TURBO_RATE_MATCH | 232 RTE_BBDEV_TURBO_RV_INDEX_BYPASS, 233 .num_buffers_src = RTE_BBDEV_MAX_CODE_BLOCKS, 234 .num_buffers_dst = RTE_BBDEV_MAX_CODE_BLOCKS, 235 } 236 }, 237 RTE_BBDEV_END_OF_CAPABILITIES_LIST() 238 }; 239 240Capabilities Discovery 241~~~~~~~~~~~~~~~~~~~~~~ 242 243Discovering the features and capabilities of a bbdev device poll mode driver 244is achieved through the ``rte_bbdev_info_get()`` function. 245 246.. code-block:: c 247 248 int rte_bbdev_info_get(uint16_t dev_id, struct rte_bbdev_info *dev_info) 249 250This allows the user to query a specific bbdev PMD and get all the device 251capabilities. The ``rte_bbdev_info`` structure provides two levels of 252information: 253 254- Device relevant information, like: name and related rte_bus. 255 256- Driver specific information, as defined by the ``struct rte_bbdev_driver_info`` 257 structure, this is where capabilities reside along with other specifics like: 258 maximum queue sizes and priority level. 259 260.. code-block:: c 261 262 struct rte_bbdev_info { 263 int socket_id; 264 const char *dev_name; 265 const struct rte_bus *bus; 266 uint16_t num_queues; 267 bool started; 268 struct rte_bbdev_driver_info drv; 269 }; 270 271Operation Processing 272-------------------- 273 274Scheduling of baseband operations on DPDK's application data path is 275performed using a burst oriented asynchronous API set. A queue on a bbdev 276device accepts a burst of baseband operations using enqueue burst API. On physical 277bbdev devices the enqueue burst API will place the operations to be processed 278on the device's hardware input queue, for virtual devices the processing of the 279baseband operations is usually completed during the enqueue call to the bbdev 280device. The dequeue burst API will retrieve any processed operations available 281from the queue on the bbdev device, from physical devices this is usually 282directly from the device's processed queue, and for virtual device's from a 283``rte_ring`` where processed operations are place after being processed on the 284enqueue call. 285 286 287Enqueue / Dequeue Burst APIs 288~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 289 290The burst enqueue API uses a bbdev device identifier and a queue 291identifier to specify the bbdev device queue to schedule the processing on. 292The ``num_ops`` parameter is the number of operations to process which are 293supplied in the ``ops`` array of ``rte_bbdev_*_op`` structures. 294The enqueue function returns the number of operations it actually enqueued for 295processing, a return value equal to ``num_ops`` means that all packets have been 296enqueued. 297 298.. code-block:: c 299 300 uint16_t rte_bbdev_enqueue_enc_ops(uint16_t dev_id, uint16_t queue_id, 301 struct rte_bbdev_enc_op **ops, uint16_t num_ops) 302 303 uint16_t rte_bbdev_enqueue_dec_ops(uint16_t dev_id, uint16_t queue_id, 304 struct rte_bbdev_dec_op **ops, uint16_t num_ops) 305 306The dequeue API uses the same format as the enqueue API of processed but 307the ``num_ops`` and ``ops`` parameters are now used to specify the max processed 308operations the user wishes to retrieve and the location in which to store them. 309The API call returns the actual number of processed operations returned, this 310can never be larger than ``num_ops``. 311 312.. code-block:: c 313 314 uint16_t rte_bbdev_dequeue_enc_ops(uint16_t dev_id, uint16_t queue_id, 315 struct rte_bbdev_enc_op **ops, uint16_t num_ops) 316 317 uint16_t rte_bbdev_dequeue_dec_ops(uint16_t dev_id, uint16_t queue_id, 318 struct rte_bbdev_dec_op **ops, uint16_t num_ops) 319 320Operation Representation 321~~~~~~~~~~~~~~~~~~~~~~~~ 322 323An encode bbdev operation is represented by ``rte_bbdev_enc_op`` structure, 324and by ``rte_bbdev_dec_op`` for decode. These structures act as metadata 325containers for all necessary information required for the bbdev operation to be 326processed on a particular bbdev device poll mode driver. 327 328.. code-block:: c 329 330 struct rte_bbdev_enc_op { 331 int status; 332 struct rte_mempool *mempool; 333 void *opaque_data; 334 struct rte_bbdev_op_turbo_enc turbo_enc; 335 }; 336 337 struct rte_bbdev_dec_op { 338 int status; 339 struct rte_mempool *mempool; 340 void *opaque_data; 341 struct rte_bbdev_op_turbo_dec turbo_dec; 342 }; 343 344The operation structure by itself defines the operation type. It includes an 345operation status, a reference to the operation specific data, which can vary in 346size and content depending on the operation being provisioned. It also contains 347the source mempool for the operation, if it is allocated from a mempool. 348 349If bbdev operations are allocated from a bbdev operation mempool, see next 350section, there is also the ability to allocate private memory with the 351operation for applications purposes. 352 353Application software is responsible for specifying all the operation specific 354fields in the ``rte_bbdev_*_op`` structure which are then used by the bbdev PMD 355to process the requested operation. 356 357 358Operation Management and Allocation 359~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 360 361The bbdev library provides an API set for managing bbdev operations which 362utilize the Mempool Library to allocate operation buffers. Therefore, it ensures 363that the bbdev operation is interleaved optimally across the channels and 364ranks for optimal processing. 365 366.. code-block:: c 367 368 struct rte_mempool * 369 rte_bbdev_op_pool_create(const char *name, enum rte_bbdev_op_type type, 370 unsigned int num_elements, unsigned int cache_size, 371 int socket_id) 372 373``rte_bbdev_*_op_alloc_bulk()`` and ``rte_bbdev_*_op_free_bulk()`` are used to 374allocate bbdev operations of a specific type from a given bbdev operation mempool. 375 376.. code-block:: c 377 378 int rte_bbdev_enc_op_alloc_bulk(struct rte_mempool *mempool, 379 struct rte_bbdev_enc_op **ops, uint16_t num_ops) 380 381 int rte_bbdev_dec_op_alloc_bulk(struct rte_mempool *mempool, 382 struct rte_bbdev_dec_op **ops, uint16_t num_ops) 383 384``rte_bbdev_*_op_free_bulk()`` is called by the application to return an 385operation to its allocating pool. 386 387.. code-block:: c 388 389 void rte_bbdev_dec_op_free_bulk(struct rte_bbdev_dec_op **ops, 390 unsigned int num_ops) 391 void rte_bbdev_enc_op_free_bulk(struct rte_bbdev_enc_op **ops, 392 unsigned int num_ops) 393 394BBDEV Operations 395~~~~~~~~~~~~~~~~ 396 397The bbdev operation structure contains all the mutable data relating to 398performing Turbo code processing on a referenced mbuf data buffer. It is used 399for either encode or decode operations. 400 401Turbo Encode operation accepts one input and one output. 402 403Turbo Decode operation accepts one input and two outputs, called *hard-decision* 404and *soft-decision* outputs. *Soft-decision* output is optional. 405 406It is expected that the application provides input and output ``mbuf`` pointers 407allocated and ready to use. The baseband framework supports turbo coding on 408Code Blocks (CB) and Transport Blocks (TB). 409 410For the output buffer(s), the application needs only to provide an allocated and 411free mbuf (containing only one mbuf segment), so that bbdev can write the 412operation outcome. 413 414**Turbo Encode Op structure** 415 416.. code-block:: c 417 418 struct rte_bbdev_op_turbo_enc { 419 struct rte_bbdev_op_data input; 420 struct rte_bbdev_op_data output; 421 422 uint32_t op_flags; 423 uint8_t rv_index; 424 uint8_t code_block_mode; 425 union { 426 struct rte_bbdev_op_enc_cb_params cb_params; 427 struct rte_bbdev_op_enc_tb_params tb_params; 428 }; 429 }; 430 431 432**Turbo Decode Op structure** 433 434.. code-block:: c 435 436 struct rte_bbdev_op_turbo_dec { 437 struct rte_bbdev_op_data input; 438 struct rte_bbdev_op_data hard_output; 439 struct rte_bbdev_op_data soft_output; 440 441 uint32_t op_flags; 442 uint8_t rv_index; 443 uint8_t iter_min:4; 444 uint8_t iter_max:4; 445 uint8_t iter_count; 446 uint8_t ext_scale; 447 uint8_t num_maps; 448 uint8_t code_block_mode; 449 union { 450 struct rte_bbdev_op_dec_cb_params cb_params; 451 struct rte_bbdev_op_dec_tb_params tb_params; 452 }; 453 }; 454 455Input and output data buffers are identified by ``rte_bbdev_op_data`` structure. 456This structure has three elements: 457 458- ``data`` - This is the mbuf reference 459 460- ``offset`` - The starting point for the Turbo input/output, in bytes, from the 461 start of the data in the data buffer. It must be smaller than data_len of the 462 mbuf's first segment 463 464- ``length`` - The length, in bytes, of the buffer on which the Turbo operation 465 will or has been computed. For the input, the length is set by the application. 466 For the output(s), the length is computed by the bbdev PMD driver. 467 468Sample code 469----------- 470 471The baseband device sample application gives an introduction on how to use the 472bbdev framework, by giving a sample code performing a loop-back operation with a 473baseband processor capable of transceiving data packets. 474 475The following sample C-like pseudo-code shows the basic steps to encode several 476buffers using (**sw_trubo**) bbdev PMD. 477 478.. code-block:: c 479 480 /* EAL Init */ 481 ret = rte_eal_init(argc, argv); 482 if (ret < 0) 483 rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); 484 485 /* Get number of available bbdev devices */ 486 nb_bbdevs = rte_bbdev_count(); 487 if (nb_bbdevs == 0) 488 rte_exit(EXIT_FAILURE, "No bbdevs detected!\n"); 489 490 /* Create bbdev op pools */ 491 bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC] = 492 rte_bbdev_op_pool_create("bbdev_op_pool_enc", 493 RTE_BBDEV_OP_TURBO_ENC, NB_MBUF, 128, rte_socket_id()); 494 495 /* Get information for this device */ 496 rte_bbdev_info_get(dev_id, &info); 497 498 /* Setup BBDEV device queues */ 499 ret = rte_bbdev_setup_queues(dev_id, qs_nb, info.socket_id); 500 if (ret < 0) 501 rte_exit(EXIT_FAILURE, 502 "ERROR(%d): BBDEV %u not configured properly\n", 503 ret, dev_id); 504 505 /* setup device queues */ 506 qconf.socket = info.socket_id; 507 qconf.queue_size = info.drv.queue_size_lim; 508 qconf.op_type = RTE_BBDEV_OP_TURBO_ENC; 509 510 for (q_id = 0; q_id < qs_nb; q_id++) { 511 /* Configure all queues belonging to this bbdev device */ 512 ret = rte_bbdev_queue_configure(dev_id, q_id, &qconf); 513 if (ret < 0) 514 rte_exit(EXIT_FAILURE, 515 "ERROR(%d): BBDEV %u queue %u not configured properly\n", 516 ret, dev_id, q_id); 517 } 518 519 /* Start bbdev device */ 520 ret = rte_bbdev_start(dev_id); 521 522 /* Create the mbuf mempool for pkts */ 523 mbuf_pool = rte_pktmbuf_pool_create("bbdev_mbuf_pool", 524 NB_MBUF, MEMPOOL_CACHE_SIZE, 0, 525 RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id()); 526 if (mbuf_pool == NULL) 527 rte_exit(EXIT_FAILURE, 528 "Unable to create '%s' pool\n", pool_name); 529 530 while (!global_exit_flag) { 531 532 /* Allocate burst of op structures in preparation for enqueue */ 533 if (rte_bbdev_enc_op_alloc_bulk(bbdev_op_pool[RTE_BBDEV_OP_TURBO_ENC], 534 ops_burst, op_num) != 0) 535 continue; 536 537 /* Allocate input mbuf pkts */ 538 ret = rte_pktmbuf_alloc_bulk(mbuf_pool, input_pkts_burst, MAX_PKT_BURST); 539 if (ret < 0) 540 continue; 541 542 /* Allocate output mbuf pkts */ 543 ret = rte_pktmbuf_alloc_bulk(mbuf_pool, output_pkts_burst, MAX_PKT_BURST); 544 if (ret < 0) 545 continue; 546 547 for (j = 0; j < op_num; j++) { 548 /* Append the size of the ethernet header */ 549 rte_pktmbuf_append(input_pkts_burst[j], 550 sizeof(struct ether_hdr)); 551 552 /* set op */ 553 554 ops_burst[j]->turbo_enc.input.offset = 555 sizeof(struct ether_hdr); 556 557 ops_burst[j]->turbo_enc->input.length = 558 rte_pktmbuf_pkt_len(bbdev_pkts[j]); 559 560 ops_burst[j]->turbo_enc->input.data = 561 input_pkts_burst[j]; 562 563 ops_burst[j]->turbo_enc->output.offset = 564 sizeof(struct ether_hdr); 565 566 ops_burst[j]->turbo_enc->output.data = 567 output_pkts_burst[j]; 568 } 569 570 /* Enqueue packets on BBDEV device */ 571 op_num = rte_bbdev_enqueue_enc_ops(qconf->bbdev_id, 572 qconf->bbdev_qs[q], ops_burst, 573 MAX_PKT_BURST); 574 575 /* Dequeue packets from BBDEV device*/ 576 op_num = rte_bbdev_dequeue_enc_ops(qconf->bbdev_id, 577 qconf->bbdev_qs[q], ops_burst, 578 MAX_PKT_BURST); 579 } 580 581 582BBDEV Device API 583~~~~~~~~~~~~~~~~ 584 585The bbdev Library API is described in the *DPDK API Reference* document. 586