153f6d732SRadha Mohan Chintakuntla.. SPDX-License-Identifier: BSD-3-Clause 253f6d732SRadha Mohan Chintakuntla Copyright(c) 2021 Marvell International Ltd. 353f6d732SRadha Mohan Chintakuntla 453f6d732SRadha Mohan Chintakuntla.. include:: <isonum.txt> 553f6d732SRadha Mohan Chintakuntla 653f6d732SRadha Mohan ChintakuntlaCNXK DMA Device Driver 753f6d732SRadha Mohan Chintakuntla====================== 853f6d732SRadha Mohan Chintakuntla 953f6d732SRadha Mohan ChintakuntlaThe ``cnxk`` dmadev driver provides a poll-mode driver (PMD) for Marvell DPI DMA 1033e71acfSJerin JacobHardware Accelerator block found in OCTEON 9 and OCTEON 10 family of SoCs. 1153f6d732SRadha Mohan ChintakuntlaEach DMA queue is exposed as a VF function when SRIOV is enabled. 1253f6d732SRadha Mohan Chintakuntla 1353f6d732SRadha Mohan ChintakuntlaThe block supports following modes of DMA transfers: 1453f6d732SRadha Mohan Chintakuntla 1553f6d732SRadha Mohan Chintakuntla#. Internal - DMA within SoC DRAM to DRAM 1653f6d732SRadha Mohan Chintakuntla#. Inbound - Host DRAM to SoC DRAM when SoC is in PCIe Endpoint 1753f6d732SRadha Mohan Chintakuntla#. Outbound - SoC DRAM to Host DRAM when SoC is in PCIe Endpoint 1853f6d732SRadha Mohan Chintakuntla 1953f6d732SRadha Mohan ChintakuntlaPrerequisites and Compilation procedure 2053f6d732SRadha Mohan Chintakuntla--------------------------------------- 2153f6d732SRadha Mohan Chintakuntla 2253f6d732SRadha Mohan ChintakuntlaSee :doc:`../platform/cnxk` for setup information. 2353f6d732SRadha Mohan Chintakuntla 2453f6d732SRadha Mohan ChintakuntlaDevice Setup 2553f6d732SRadha Mohan Chintakuntla------------- 2653f6d732SRadha Mohan Chintakuntla 2753f6d732SRadha Mohan ChintakuntlaThe ``dpdk-devbind.py`` script, included with DPDK, 2853f6d732SRadha Mohan Chintakuntlacan be used to show the presence of supported hardware. 2953f6d732SRadha Mohan ChintakuntlaRunning ``dpdk-devbind.py --status-dev dma`` will show all the CNXK DMA devices. 3053f6d732SRadha Mohan Chintakuntla 3153f6d732SRadha Mohan ChintakuntlaDevices using VFIO drivers 3253f6d732SRadha Mohan Chintakuntla~~~~~~~~~~~~~~~~~~~~~~~~~~ 3353f6d732SRadha Mohan Chintakuntla 3453f6d732SRadha Mohan ChintakuntlaThe HW devices to be used will need to be bound to a user-space IO driver for use. 3553f6d732SRadha Mohan ChintakuntlaThe ``dpdk-devbind.py`` script can be used to view the state of the devices 3653f6d732SRadha Mohan Chintakuntlaand to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``. 3753f6d732SRadha Mohan ChintakuntlaFor example:: 3853f6d732SRadha Mohan Chintakuntla 3953f6d732SRadha Mohan Chintakuntla $ dpdk-devbind.py -b vfio-pci 0000:05:00.1 4053f6d732SRadha Mohan Chintakuntla 4153f6d732SRadha Mohan ChintakuntlaDevice Probing and Initialization 4253f6d732SRadha Mohan Chintakuntla~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4353f6d732SRadha Mohan Chintakuntla 4453f6d732SRadha Mohan ChintakuntlaTo use the devices from an application, the dmadev API can be used. 45b56f1e2dSRadha Mohan ChintakuntlaCNXK DMA device configuration requirements: 46b56f1e2dSRadha Mohan Chintakuntla 47b56f1e2dSRadha Mohan Chintakuntla* Only one ``vchan`` is supported per device. 48b56f1e2dSRadha Mohan Chintakuntla* CNXK DMA devices do not support silent mode. 49b56f1e2dSRadha Mohan Chintakuntla 50b56f1e2dSRadha Mohan ChintakuntlaOnce configured, the device can then be made ready for use 51b56f1e2dSRadha Mohan Chintakuntlaby calling the ``rte_dma_start()`` API. 52b56f1e2dSRadha Mohan Chintakuntla 53b56f1e2dSRadha Mohan ChintakuntlaPerforming Data Copies 54b56f1e2dSRadha Mohan Chintakuntla~~~~~~~~~~~~~~~~~~~~~~ 55b56f1e2dSRadha Mohan Chintakuntla 56b56f1e2dSRadha Mohan ChintakuntlaRefer to the :ref:`Enqueue / Dequeue APIs <dmadev_enqueue_dequeue>` section 57b56f1e2dSRadha Mohan Chintakuntlaof the dmadev library documentation 58b56f1e2dSRadha Mohan Chintakuntlafor details on operation enqueue and submission API usage. 59*ccbc66aaSAmit Prakash Shukla 60*ccbc66aaSAmit Prakash ShuklaPerformance Tuning Parameters 61*ccbc66aaSAmit Prakash Shukla~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 62*ccbc66aaSAmit Prakash Shukla 63*ccbc66aaSAmit Prakash ShuklaTo achieve higher performance, DMA device needs to be tuned 64*ccbc66aaSAmit Prakash Shuklausing PF kernel driver module parameters. 65*ccbc66aaSAmit Prakash ShuklaThe PF kernel driver is part of the OCTEON SDK. 66*ccbc66aaSAmit Prakash ShuklaModule parameters shall be configured during module insert as in below example:: 67*ccbc66aaSAmit Prakash Shukla 68*ccbc66aaSAmit Prakash Shukla $ sudo insmod octeontx2_dpi.ko mps=128 mrrs=128 eng_fifo_buf=0x101008080808 69*ccbc66aaSAmit Prakash Shukla 70*ccbc66aaSAmit Prakash Shukla``mps`` 71*ccbc66aaSAmit Prakash Shukla 72*ccbc66aaSAmit Prakash Shukla Maximum payload size. 73*ccbc66aaSAmit Prakash Shukla MPS size shall not exceed the size selected by PCI config. 74*ccbc66aaSAmit Prakash Shukla Maximum size that shall be configured can be found 75*ccbc66aaSAmit Prakash Shukla on executing ``lspci`` command for the device. 76*ccbc66aaSAmit Prakash Shukla 77*ccbc66aaSAmit Prakash Shukla``mrrs`` 78*ccbc66aaSAmit Prakash Shukla 79*ccbc66aaSAmit Prakash Shukla Maximum read request size. 80*ccbc66aaSAmit Prakash Shukla MRRS size shall not exceed the size selected by PCI config. 81*ccbc66aaSAmit Prakash Shukla Maximum size that shall be configured can be found 82*ccbc66aaSAmit Prakash Shukla on executing ``lspci`` command for the device. 83*ccbc66aaSAmit Prakash Shukla 84*ccbc66aaSAmit Prakash Shukla``eng_fifo_buf`` 85*ccbc66aaSAmit Prakash Shukla 86*ccbc66aaSAmit Prakash Shukla CNXK supports 6 DMA engines and each engine has an associated FIFO. 87*ccbc66aaSAmit Prakash Shukla By default, all engine's FIFO is configured to 8 KB. 88*ccbc66aaSAmit Prakash Shukla Engine FIFO size can be tuned using this 64-bit variable, 89*ccbc66aaSAmit Prakash Shukla where each byte represents an engine. 90*ccbc66aaSAmit Prakash Shukla In the example above, engine 0-3 FIFO are configure as 8 KB 91*ccbc66aaSAmit Prakash Shukla and engine 4-5 are configured as 16 KB. 92*ccbc66aaSAmit Prakash Shukla 93*ccbc66aaSAmit Prakash Shukla.. note:: 94*ccbc66aaSAmit Prakash Shukla 95*ccbc66aaSAmit Prakash Shukla MPS and MRRS performance tuning parameters help achieve higher performance 96*ccbc66aaSAmit Prakash Shukla only for inbound and outbound DMA transfers. 97*ccbc66aaSAmit Prakash Shukla The parameter has no effect for internal only DMA transfer. 98