1.. BSD LICENSE 2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved. 3 All rights reserved. 4 5 Redistribution and use in source and binary forms, with or without 6 modification, are permitted provided that the following conditions 7 are met: 8 9 * Redistributions of source code must retain the above copyright 10 notice, this list of conditions and the following disclaimer. 11 * Redistributions in binary form must reproduce the above copyright 12 notice, this list of conditions and the following disclaimer in 13 the documentation and/or other materials provided with the 14 distribution. 15 * Neither the name of Intel Corporation nor the names of its 16 contributors may be used to endorse or promote products derived 17 from this software without specific prior written permission. 18 19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 31VMDQ and DCB Forwarding Sample Application 32========================================== 33 34The VMDQ and DCB Forwarding sample application is a simple example of packet processing using the DPDK. 35The application performs L2 forwarding using VMDQ and DCB to divide the incoming traffic into queues. 36The traffic splitting is performed in hardware by the VMDQ and DCB features of the Intel® 82599 and X710/XL710 Ethernet Controllers. 37 38Overview 39-------- 40 41This sample application can be used as a starting point for developing a new application that is based on the DPDK and 42uses VMDQ and DCB for traffic partitioning. 43 44The VMDQ and DCB filters work on MAC and VLAN traffic to divide the traffic into input queues on the basis of the Destination MAC 45address, VLAN ID and VLAN user priority fields. 46VMDQ filters split the traffic into 16 or 32 groups based on the Destination MAC and VLAN ID. 47Then, DCB places each packet into one of queues within that group, based upon the VLAN user priority field. 48 49All traffic is read from a single incoming port (port 0) and output on port 1, without any processing being performed. 50With Intel® 82599 NIC, for example, the traffic is split into 128 queues on input, where each thread of the application reads from 51multiple queues. When run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. 52 53As supplied, the sample application configures the VMDQ feature to have 32 pools with 4 queues each as indicated in :numref:`figure_vmdq_dcb_example`. 54The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 16 pools of 8 queues. While the 55Intel® X710 or XL710 Ethernet Controller NICs support many configurations of VMDQ pools of 4 or 8 queues each. For simplicity, only 16 56or 32 pools is supported in this sample. And queues numbers for each VMDQ pool can be changed by setting CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM 57in config/common_* file. 58The nb-pools, nb-tcs and enable-rss parameters can be passed on the command line, after the EAL parameters: 59 60.. code-block:: console 61 62 ./build/vmdq_dcb [EAL options] -- -p PORTMASK --nb-pools NP --nb-tcs TC --enable-rss 63 64where, NP can be 16 or 32, TC can be 4 or 8, rss is disabled by default. 65 66.. _figure_vmdq_dcb_example: 67 68.. figure:: img/vmdq_dcb_example.* 69 70 Packet Flow Through the VMDQ and DCB Sample Application 71 72 73In Linux* user space, the application can display statistics with the number of packets received on each queue. 74To have the application display the statistics, send a SIGHUP signal to the running application process. 75 76The VMDQ and DCB Forwarding sample application is in many ways simpler than the L2 Forwarding application 77(see :doc:`l2_forward_real_virtual`) 78as it performs unidirectional L2 forwarding of packets from one port to a second port. 79No command-line options are taken by this application apart from the standard EAL command-line options. 80 81.. note:: 82 83 Since VMD queues are being used for VMM, this application works correctly 84 when VTd is disabled in the BIOS or Linux* kernel (intel_iommu=off). 85 86Compiling the Application 87------------------------- 88 89 90 91To compile the sample application see :doc:`compiling`. 92 93The application is located in the ``vmdq_dcb`` sub-directory. 94 95Running the Application 96----------------------- 97 98To run the example in a linuxapp environment: 99 100.. code-block:: console 101 102 user@target:~$ ./build/vmdq_dcb -l 0-3 -n 4 -- -p 0x3 --nb-pools 32 --nb-tcs 4 103 104Refer to the *DPDK Getting Started Guide* for general information on running applications and 105the Environment Abstraction Layer (EAL) options. 106 107Explanation 108----------- 109 110The following sections provide some explanation of the code. 111 112Initialization 113~~~~~~~~~~~~~~ 114 115The EAL, driver and PCI configuration is performed largely as in the L2 Forwarding sample application, 116as is the creation of the mbuf pool. 117See :doc:`l2_forward_real_virtual`. 118Where this example application differs is in the configuration of the NIC port for RX. 119 120The VMDQ and DCB hardware feature is configured at port initialization time by setting the appropriate values in the 121rte_eth_conf structure passed to the rte_eth_dev_configure() API. 122Initially in the application, 123a default structure is provided for VMDQ and DCB configuration to be filled in later by the application. 124 125.. code-block:: c 126 127 /* empty vmdq+dcb configuration structure. Filled in programmatically */ 128 static const struct rte_eth_conf vmdq_dcb_conf_default = { 129 .rxmode = { 130 .mq_mode = ETH_MQ_RX_VMDQ_DCB, 131 .split_hdr_size = 0, 132 .header_split = 0, /**< Header Split disabled */ 133 .hw_ip_checksum = 0, /**< IP checksum offload disabled */ 134 .hw_vlan_filter = 0, /**< VLAN filtering disabled */ 135 .jumbo_frame = 0, /**< Jumbo Frame Support disabled */ 136 }, 137 .txmode = { 138 .mq_mode = ETH_MQ_TX_VMDQ_DCB, 139 }, 140 /* 141 * should be overridden separately in code with 142 * appropriate values 143 */ 144 .rx_adv_conf = { 145 .vmdq_dcb_conf = { 146 .nb_queue_pools = ETH_32_POOLS, 147 .enable_default_pool = 0, 148 .default_pool = 0, 149 .nb_pool_maps = 0, 150 .pool_map = {{0, 0},}, 151 .dcb_tc = {0}, 152 }, 153 .dcb_rx_conf = { 154 .nb_tcs = ETH_4_TCS, 155 /** Traffic class each UP mapped to. */ 156 .dcb_tc = {0}, 157 }, 158 .vmdq_rx_conf = { 159 .nb_queue_pools = ETH_32_POOLS, 160 .enable_default_pool = 0, 161 .default_pool = 0, 162 .nb_pool_maps = 0, 163 .pool_map = {{0, 0},}, 164 }, 165 }, 166 .tx_adv_conf = { 167 .vmdq_dcb_tx_conf = { 168 .nb_queue_pools = ETH_32_POOLS, 169 .dcb_tc = {0}, 170 }, 171 }, 172 }; 173 174The get_eth_conf() function fills in an rte_eth_conf structure with the appropriate values, 175based on the global vlan_tags array, 176and dividing up the possible user priority values equally among the individual queues 177(also referred to as traffic classes) within each pool. With Intel® 82599 NIC, 178if the number of pools is 32, then the user priority fields are allocated 2 to a queue. 179If 16 pools are used, then each of the 8 user priority fields is allocated to its own queue within the pool. 180With Intel® X710/XL710 NICs, if number of tcs is 4, and number of queues in pool is 8, 181then the user priority fields are allocated 2 to one tc, and a tc has 2 queues mapping to it, then 182RSS will determine the destination queue in 2. 183For the VLAN IDs, each one can be allocated to possibly multiple pools of queues, 184so the pools parameter in the rte_eth_vmdq_dcb_conf structure is specified as a bitmask value. 185For destination MAC, each VMDQ pool will be assigned with a MAC address. In this sample, each VMDQ pool 186is assigned to the MAC like 52:54:00:12:<port_id>:<pool_id>, that is, 187the MAC of VMDQ pool 2 on port 1 is 52:54:00:12:01:02. 188 189.. code-block:: c 190 191 const uint16_t vlan_tags[] = { 192 0, 1, 2, 3, 4, 5, 6, 7, 193 8, 9, 10, 11, 12, 13, 14, 15, 194 16, 17, 18, 19, 20, 21, 22, 23, 195 24, 25, 26, 27, 28, 29, 30, 31 196 }; 197 198 /* pool mac addr template, pool mac addr is like: 52 54 00 12 port# pool# */ 199 static struct ether_addr pool_addr_template = { 200 .addr_bytes = {0x52, 0x54, 0x00, 0x12, 0x00, 0x00} 201 }; 202 203 /* Builds up the correct configuration for vmdq+dcb based on the vlan tags array 204 * given above, and the number of traffic classes available for use. */ 205 static inline int 206 get_eth_conf(struct rte_eth_conf *eth_conf) 207 { 208 struct rte_eth_vmdq_dcb_conf conf; 209 struct rte_eth_vmdq_rx_conf vmdq_conf; 210 struct rte_eth_dcb_rx_conf dcb_conf; 211 struct rte_eth_vmdq_dcb_tx_conf tx_conf; 212 uint8_t i; 213 214 conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 215 vmdq_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 216 tx_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 217 conf.nb_pool_maps = num_pools; 218 vmdq_conf.nb_pool_maps = num_pools; 219 conf.enable_default_pool = 0; 220 vmdq_conf.enable_default_pool = 0; 221 conf.default_pool = 0; /* set explicit value, even if not used */ 222 vmdq_conf.default_pool = 0; 223 224 for (i = 0; i < conf.nb_pool_maps; i++) { 225 conf.pool_map[i].vlan_id = vlan_tags[i]; 226 vmdq_conf.pool_map[i].vlan_id = vlan_tags[i]; 227 conf.pool_map[i].pools = 1UL << i ; 228 vmdq_conf.pool_map[i].pools = 1UL << i; 229 } 230 for (i = 0; i < ETH_DCB_NUM_USER_PRIORITIES; i++){ 231 conf.dcb_tc[i] = i % num_tcs; 232 dcb_conf.dcb_tc[i] = i % num_tcs; 233 tx_conf.dcb_tc[i] = i % num_tcs; 234 } 235 dcb_conf.nb_tcs = (enum rte_eth_nb_tcs)num_tcs; 236 (void)(rte_memcpy(eth_conf, &vmdq_dcb_conf_default, sizeof(*eth_conf))); 237 (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_dcb_conf, &conf, 238 sizeof(conf))); 239 (void)(rte_memcpy(ð_conf->rx_adv_conf.dcb_rx_conf, &dcb_conf, 240 sizeof(dcb_conf))); 241 (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_rx_conf, &vmdq_conf, 242 sizeof(vmdq_conf))); 243 (void)(rte_memcpy(ð_conf->tx_adv_conf.vmdq_dcb_tx_conf, &tx_conf, 244 sizeof(tx_conf))); 245 if (rss_enable) { 246 eth_conf->rxmode.mq_mode= ETH_MQ_RX_VMDQ_DCB_RSS; 247 eth_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP | 248 ETH_RSS_UDP | 249 ETH_RSS_TCP | 250 ETH_RSS_SCTP; 251 } 252 return 0; 253 } 254 255 ...... 256 257 /* Set mac for each pool.*/ 258 for (q = 0; q < num_pools; q++) { 259 struct ether_addr mac; 260 mac = pool_addr_template; 261 mac.addr_bytes[4] = port; 262 mac.addr_bytes[5] = q; 263 printf("Port %u vmdq pool %u set mac %02x:%02x:%02x:%02x:%02x:%02x\n", 264 port, q, 265 mac.addr_bytes[0], mac.addr_bytes[1], 266 mac.addr_bytes[2], mac.addr_bytes[3], 267 mac.addr_bytes[4], mac.addr_bytes[5]); 268 retval = rte_eth_dev_mac_addr_add(port, &mac, 269 q + vmdq_pool_base); 270 if (retval) { 271 printf("mac addr add failed at pool %d\n", q); 272 return retval; 273 } 274 } 275 276Once the network port has been initialized using the correct VMDQ and DCB values, 277the initialization of the port's RX and TX hardware rings is performed similarly to that 278in the L2 Forwarding sample application. 279See :doc:`l2_forward_real_virtual` for more information. 280 281Statistics Display 282~~~~~~~~~~~~~~~~~~ 283 284When run in a linuxapp environment, 285the VMDQ and DCB Forwarding sample application can display statistics showing the number of packets read from each RX queue. 286This is provided by way of a signal handler for the SIGHUP signal, 287which simply prints to standard output the packet counts in grid form. 288Each row of the output is a single pool with the columns being the queue number within that pool. 289 290To generate the statistics output, use the following command: 291 292.. code-block:: console 293 294 user@host$ sudo killall -HUP vmdq_dcb_app 295 296Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, 297rather than the terminal from which the HUP signal was sent. 298