1*d76c27e6SVidya Sagar Velumuri.. SPDX-License-Identifier: BSD-3-Clause 2*d76c27e6SVidya Sagar Velumuri Copyright(c) 2024 Marvell. 3*d76c27e6SVidya Sagar Velumuri 4*d76c27e6SVidya Sagar VelumuriOdyssey ODM DMA Device Driver 5*d76c27e6SVidya Sagar Velumuri============================= 6*d76c27e6SVidya Sagar Velumuri 7*d76c27e6SVidya Sagar VelumuriThe ``odm`` DMA device driver provides a poll-mode driver (PMD) 8*d76c27e6SVidya Sagar Velumurifor Marvell Odyssey DMA Hardware Accelerator block found in Odyssey SoC. 9*d76c27e6SVidya Sagar VelumuriThe block supports only mem to mem DMA transfers. 10*d76c27e6SVidya Sagar Velumuri 11*d76c27e6SVidya Sagar VelumuriODM DMA device can support up to 32 queues and 16 VFs. 12*d76c27e6SVidya Sagar Velumuri 13*d76c27e6SVidya Sagar VelumuriDevice Setup 14*d76c27e6SVidya Sagar Velumuri------------ 15*d76c27e6SVidya Sagar Velumuri 16*d76c27e6SVidya Sagar VelumuriODM DMA device is initialized by kernel PF driver. 17*d76c27e6SVidya Sagar VelumuriThe PF kernel driver is part of Marvell software packages for Odyssey. 18*d76c27e6SVidya Sagar Velumuri 19*d76c27e6SVidya Sagar VelumuriKernel module can be inserted as in below example:: 20*d76c27e6SVidya Sagar Velumuri 21*d76c27e6SVidya Sagar Velumuri sudo insmod odyssey_odm.ko 22*d76c27e6SVidya Sagar Velumuri 23*d76c27e6SVidya Sagar VelumuriODM DMA device can support up to 16 VFs:: 24*d76c27e6SVidya Sagar Velumuri 25*d76c27e6SVidya Sagar Velumuri sudo echo 16 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs 26*d76c27e6SVidya Sagar Velumuri 27*d76c27e6SVidya Sagar VelumuriAbove command creates 16 VFs with 2 queues each. 28*d76c27e6SVidya Sagar Velumuri 29*d76c27e6SVidya Sagar VelumuriThe ``dpdk-devbind.py`` script, included with DPDK, 30*d76c27e6SVidya Sagar Velumurican be used to show the presence of supported hardware. 31*d76c27e6SVidya Sagar VelumuriRunning ``dpdk-devbind.py --status-dev dma`` 32*d76c27e6SVidya Sagar Velumuriwill show all the Odyssey ODM DMA devices. 33*d76c27e6SVidya Sagar Velumuri 34*d76c27e6SVidya Sagar VelumuriDevices using VFIO drivers 35*d76c27e6SVidya Sagar Velumuri~~~~~~~~~~~~~~~~~~~~~~~~~~ 36*d76c27e6SVidya Sagar Velumuri 37*d76c27e6SVidya Sagar VelumuriThe HW devices to be used will need to be bound to a user-space IO driver. 38*d76c27e6SVidya Sagar VelumuriThe ``dpdk-devbind.py`` script can be used to view the state of the devices 39*d76c27e6SVidya Sagar Velumuriand to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``. 40*d76c27e6SVidya Sagar VelumuriFor example:: 41*d76c27e6SVidya Sagar Velumuri 42*d76c27e6SVidya Sagar Velumuri dpdk-devbind.py -b vfio-pci 0000:08:00.1 43*d76c27e6SVidya Sagar Velumuri 44*d76c27e6SVidya Sagar VelumuriDevice Probing and Initialization 45*d76c27e6SVidya Sagar Velumuri~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 46*d76c27e6SVidya Sagar Velumuri 47*d76c27e6SVidya Sagar VelumuriTo use the devices from an application, the dmadev API can be used. 48*d76c27e6SVidya Sagar Velumuri 49*d76c27e6SVidya Sagar VelumuriOnce configured, the device can then be made ready for use 50*d76c27e6SVidya Sagar Velumuriby calling the ``rte_dma_start()`` API. 51*d76c27e6SVidya Sagar Velumuri 52*d76c27e6SVidya Sagar VelumuriPerforming Data Copies 53*d76c27e6SVidya Sagar Velumuri~~~~~~~~~~~~~~~~~~~~~~ 54*d76c27e6SVidya Sagar Velumuri 55*d76c27e6SVidya Sagar VelumuriRefer to the :ref:`Enqueue / Dequeue API <dmadev_enqueue_dequeue>` 56*d76c27e6SVidya Sagar Velumurisection of the dmadev library documentation 57*d76c27e6SVidya Sagar Velumurifor details on operation enqueue and submission API usage. 58*d76c27e6SVidya Sagar Velumuri 59*d76c27e6SVidya Sagar VelumuriPerformance Tuning Parameters 60*d76c27e6SVidya Sagar Velumuri~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 61*d76c27e6SVidya Sagar Velumuri 62*d76c27e6SVidya Sagar VelumuriTo achieve higher performance, DMA device needs to be tuned 63*d76c27e6SVidya Sagar Velumuriusing PF kernel driver module parameters. 64*d76c27e6SVidya Sagar Velumuri 65*d76c27e6SVidya Sagar VelumuriFollowing options are exposed by kernel PF driver via devlink interface 66*d76c27e6SVidya Sagar Velumurifor tuning performance. 67*d76c27e6SVidya Sagar Velumuri 68*d76c27e6SVidya Sagar Velumuri``eng_sel`` 69*d76c27e6SVidya Sagar Velumuri 70*d76c27e6SVidya Sagar Velumuri ODM DMA device has 2 engines internally. Engine to queue mapping is decided 71*d76c27e6SVidya Sagar Velumuri by a hardware register which can be configured as below:: 72*d76c27e6SVidya Sagar Velumuri 73*d76c27e6SVidya Sagar Velumuri /sbin/devlink dev param set pci/0000:08:00.0 name eng_sel value 3435973836 cmode runtime 74*d76c27e6SVidya Sagar Velumuri 75*d76c27e6SVidya Sagar Velumuri Each bit in the register corresponds to one queue. 76*d76c27e6SVidya Sagar Velumuri Each queue would be associated with one engine. 77*d76c27e6SVidya Sagar Velumuri If the value of the bit corresponding to the queue is 0, then engine 0 would be picked. 78*d76c27e6SVidya Sagar Velumuri If it is 1, then engine 1 would be picked. 79*d76c27e6SVidya Sagar Velumuri 80*d76c27e6SVidya Sagar Velumuri In the above command, the register value is set as 81*d76c27e6SVidya Sagar Velumuri ``1100 1100 1100 1100 1100 1100 1100 1100`` 82*d76c27e6SVidya Sagar Velumuri which allows for alternate engines to be used with alternate VFs 83*d76c27e6SVidya Sagar Velumuri (assuming the system has 16 VFs with 2 queues each). 84*d76c27e6SVidya Sagar Velumuri 85*d76c27e6SVidya Sagar Velumuri``max_load_request`` 86*d76c27e6SVidya Sagar Velumuri 87*d76c27e6SVidya Sagar Velumuri Specifies maximum outstanding load requests on internal bus. 88*d76c27e6SVidya Sagar Velumuri Values can range from 1 to 512. 89*d76c27e6SVidya Sagar Velumuri Set to 512 for maximum requests in flight.:: 90*d76c27e6SVidya Sagar Velumuri 91*d76c27e6SVidya Sagar Velumuri /sbin/devlink dev param set pci/0000:08:00.0 name max_load_request value 512 cmode runtime 92