1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2018-2019 Cisco Systems, Inc. 3 4====================== 5Memif Poll Mode Driver 6====================== 7 8Shared memory packet interface (memif) PMD allows for DPDK and any other client 9using memif (DPDK, VPP, libmemif) to communicate using shared memory. Memif is 10Linux only. 11 12The created device transmits packets in a raw format. It can be used with 13Ethernet mode, IP mode, or Punt/Inject. At this moment, only Ethernet mode is 14supported in DPDK memif implementation. 15 16Memif works in two roles: master and slave. Slave connects to master over an 17existing socket. It is also a producer of shared memory file and initializes 18the shared memory. Each interface can be connected to one peer interface 19at same time. The peer interface is identified by id parameter. Master 20creates the socket and listens for any slave connection requests. The socket 21may already exist on the system. Be sure to remove any such sockets, if you 22are creating a master interface, or you will see an "Address already in use" 23error. Function ``rte_pmd_memif_remove()``, which removes memif interface, 24will also remove a listener socket, if it is not being used by any other 25interface. 26 27The method to enable one or more interfaces is to use the 28``--vdev=net_memif0`` option on the DPDK application command line. Each 29``--vdev=net_memif1`` option given will create an interface named net_memif0, 30net_memif1, and so on. Memif uses unix domain socket to transmit control 31messages. Each memif has a unique id per socket. This id is used to identify 32peer interface. If you are connecting multiple 33interfaces using same socket, be sure to specify unique ids ``id=0``, ``id=1``, 34etc. Note that if you assign a socket to a master interface it becomes a 35listener socket. Listener socket can not be used by a slave interface on same 36client. 37 38.. csv-table:: **Memif configuration options** 39 :header: "Option", "Description", "Default", "Valid value" 40 41 "id=0", "Used to identify peer interface", "0", "uint32_t" 42 "role=master", "Set memif role", "slave", "master|slave" 43 "bsize=1024", "Size of single packet buffer", "2048", "uint16_t" 44 "rsize=11", "Log2 of ring size. If rsize is 10, actual ring size is 1024", "10", "1-14" 45 "socket=/tmp/memif.sock", "Socket filename", "/tmp/memif.sock", "string len 256" 46 "mac=01:23:45:ab:cd:ef", "Mac address", "01:ab:23:cd:45:ef", "" 47 "secret=abc123", "Secret is an optional security option, which if specified, must be matched by peer", "", "string len 24" 48 "zero-copy=yes", "Enable/disable zero-copy slave mode", "no", "yes|no" 49 50**Connection establishment** 51 52In order to create memif connection, two memif interfaces, each in separate 53process, are needed. One interface in ``master`` role and other in 54``slave`` role. It is not possible to connect two interfaces in a single 55process. Each interface can be connected to one interface at same time, 56identified by matching id parameter. 57 58Memif driver uses unix domain socket to exchange required information between 59memif interfaces. Socket file path is specified at interface creation see 60*Memif configuration options* table above. If socket is used by ``master`` 61interface, it's marked as listener socket (in scope of current process) and 62listens to connection requests from other processes. One socket can be used by 63multiple interfaces. One process can have ``slave`` and ``master`` interfaces 64at the same time, provided each role is assigned unique socket. 65 66For detailed information on memif control messages, see: net/memif/memif.h. 67 68Slave interface attempts to make a connection on assigned socket. Process 69listening on this socket will extract the connection request and create a new 70connected socket (control channel). Then it sends the 'hello' message 71(``MEMIF_MSG_TYPE_HELLO``), containing configuration boundaries. Slave interface 72adjusts its configuration accordingly, and sends 'init' message 73(``MEMIF_MSG_TYPE_INIT``). This message among others contains interface id. Driver 74uses this id to find master interface, and assigns the control channel to this 75interface. If such interface is found, 'ack' message (``MEMIF_MSG_TYPE_ACK``) is 76sent. Slave interface sends 'add region' message (``MEMIF_MSG_TYPE_ADD_REGION``) for 77every region allocated. Master responds to each of these messages with 'ack' 78message. Same behavior applies to rings. Slave sends 'add ring' message 79(``MEMIF_MSG_TYPE_ADD_RING``) for every initialized ring. Master again responds to 80each message with 'ack' message. To finalize the connection, slave interface 81sends 'connect' message (``MEMIF_MSG_TYPE_CONNECT``). Upon receiving this message 82master maps regions to its address space, initializes rings and responds with 83'connected' message (``MEMIF_MSG_TYPE_CONNECTED``). Disconnect 84(``MEMIF_MSG_TYPE_DISCONNECT``) can be sent by both master and slave interfaces at 85any time, due to driver error or if the interface is being deleted. 86 87Files 88 89- net/memif/memif.h *- control messages definitions* 90- net/memif/memif_socket.h 91- net/memif/memif_socket.c 92 93Shared memory 94~~~~~~~~~~~~~ 95 96**Shared memory format** 97 98Slave is producer and master is consumer. Memory regions, are mapped shared memory files, 99created by memif slave and provided to master at connection establishment. 100Regions contain rings and buffers. Rings and buffers can also be separated into multiple 101regions. For no-zero-copy, rings and buffers are stored inside single memory 102region to reduce the number of opened files. 103 104region n (no-zero-copy): 105 106+-----------------------+-------------------------------------------------------------------------+ 107| Rings | Buffers | 108+-----------+-----------+-----------------+---+---------------------------------------------------+ 109| S2M rings | M2S rings | packet buffer 0 | . | pb ((1 << pmd->run.log2_ring_size)*(s2m + m2s))-1 | 110+-----------+-----------+-----------------+---+---------------------------------------------------+ 111 112S2M OR M2S Rings: 113 114+--------+--------+-----------------------+ 115| ring 0 | ring 1 | ring num_s2m_rings - 1| 116+--------+--------+-----------------------+ 117 118ring 0: 119 120+-------------+---------------------------------------+ 121| ring header | (1 << pmd->run.log2_ring_size) * desc | 122+-------------+---------------------------------------+ 123 124Descriptors are assigned packet buffers in order of rings creation. If we have one ring 125in each direction and ring size is 1024, then first 1024 buffers will belong to S2M ring and 126last 1024 will belong to M2S ring. In case of zero-copy, buffers are dequeued and 127enqueued as needed. 128 129**Descriptor format** 130 131+----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 132|Quad|6| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |3|3| | | | | | | | | | | | | | |1|1| | | | | | | | | | | | | | | | 133| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 134|Word|3| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |2|1| | | | | | | | | | | | | | |6|5| | | | | | | | | | | | | | |0| 135+----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 136|0 |length |region |flags | 137+----+---------------------------------------------------------------+-------------------------------+-------------------------------+ 138|1 |metadata |offset | 139+----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 140| |6| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |3|3| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 141| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 142| |3| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |2|1| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |0| 143+----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 144 145**Flags field - flags (Quad Word 0, bits 0:15)** 146 147+-----+--------------------+------------------------------------------------------------------------------------------------+ 148|Bits |Name |Functionality | 149+=====+====================+================================================================================================+ 150|0 |MEMIF_DESC_FLAG_NEXT|Is chained buffer. When set, the packet is divided into multiple buffers. May not be contiguous.| 151+-----+--------------------+------------------------------------------------------------------------------------------------+ 152 153**Region index - region (Quad Word 0, 16:31)** 154 155Index of memory region, the buffer is located in. 156 157**Data length - length (Quad Word 0, 32:63)** 158 159Length of transmitted/received data. 160 161**Data Offset - offset (Quad Word 1, 0:31)** 162 163Data start offset from memory region address. *.regions[desc->region].addr + desc->offset* 164 165**Metadata - metadata (Quad Word 1, 32:63)** 166 167Buffer metadata. 168 169Files 170 171- net/memif/memif.h *- descriptor and ring definitions* 172- net/memif/rte_eth_memif.c *- eth_memif_rx() eth_memif_tx()* 173 174Example: testpmd 175---------------------------- 176In this example we run two instances of testpmd application and transmit packets over memif. 177 178First create ``master`` interface:: 179 180 #./build/app/testpmd -l 0-1 --proc-type=primary --file-prefix=pmd1 --vdev=net_memif,role=master -- -i 181 182Now create ``slave`` interface (master must be already running so the slave will connect):: 183 184 #./build/app/testpmd -l 2-3 --proc-type=primary --file-prefix=pmd2 --vdev=net_memif -- -i 185 186Start forwarding packets:: 187 188 Slave: 189 testpmd> start 190 191 Master: 192 testpmd> start tx_first 193 194Show status:: 195 196 testpmd> show port stats 0 197 198For more details on testpmd please refer to :doc:`../testpmd_app_ug/index`. 199 200Example: testpmd and VPP 201------------------------ 202For information on how to get and run VPP please see `<https://wiki.fd.io/view/VPP>`_. 203 204Start VPP in interactive mode (should be by default). Create memif master interface in VPP:: 205 206 vpp# create interface memif id 0 master no-zero-copy 207 vpp# set interface state memif0/0 up 208 vpp# set interface ip address memif0/0 192.168.1.1/24 209 210To see socket filename use show memif command:: 211 212 vpp# show memif 213 sockets 214 id listener filename 215 0 yes (1) /run/vpp/memif.sock 216 ... 217 218Now create memif interface by running testpmd with these command line options:: 219 220 #./testpmd --vdev=net_memif,socket=/run/vpp/memif.sock -- -i 221 222Testpmd should now create memif slave interface and try to connect to master. 223In testpmd set forward option to icmpecho and start forwarding:: 224 225 testpmd> set fwd icmpecho 226 testpmd> start 227 228Send ping from VPP:: 229 230 vpp# ping 192.168.1.2 231 64 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=36.2918 ms 232 64 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=23.3927 ms 233 64 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=24.2975 ms 234 64 bytes from 192.168.1.2: icmp_seq=5 ttl=254 time=17.7049 ms 235