1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright(c) 2010-2014 Intel Corporation. 3 4VMDQ and DCB Forwarding Sample Application 5========================================== 6 7The VMDQ and DCB Forwarding sample application is a simple example of packet processing using the DPDK. 8The application performs L2 forwarding using VMDQ and DCB to divide the incoming traffic into queues. 9The traffic splitting is performed in hardware by the VMDQ and DCB features of the Intel® 82599 and X710/XL710 Ethernet Controllers. 10 11Overview 12-------- 13 14This sample application can be used as a starting point for developing a new application that is based on the DPDK and 15uses VMDQ and DCB for traffic partitioning. 16 17The VMDQ and DCB filters work on MAC and VLAN traffic to divide the traffic into input queues on the basis of the Destination MAC 18address, VLAN ID and VLAN user priority fields. 19VMDQ filters split the traffic into 16 or 32 groups based on the Destination MAC and VLAN ID. 20Then, DCB places each packet into one of queues within that group, based upon the VLAN user priority field. 21 22All traffic is read from a single incoming port (port 0) and output on port 1, without any processing being performed. 23With Intel® 82599 NIC, for example, the traffic is split into 128 queues on input, where each thread of the application reads from 24multiple queues. When run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. 25 26As supplied, the sample application configures the VMDQ feature to have 32 pools with 4 queues each as indicated in :numref:`figure_vmdq_dcb_example`. 27The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 16 pools of 8 queues. While the 28Intel® X710 or XL710 Ethernet Controller NICs support many configurations of VMDQ pools of 4 or 8 queues each. For simplicity, only 16 29or 32 pools is supported in this sample. And queues numbers for each VMDQ pool can be changed by setting RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM 30in config/rte_config.h file. 31The nb-pools, nb-tcs and enable-rss parameters can be passed on the command line, after the EAL parameters: 32 33.. code-block:: console 34 35 ./<build_dir>/examples/dpdk-vmdq_dcb [EAL options] -- -p PORTMASK --nb-pools NP --nb-tcs TC --enable-rss 36 37where, NP can be 16 or 32, TC can be 4 or 8, rss is disabled by default. 38 39.. _figure_vmdq_dcb_example: 40 41.. figure:: img/vmdq_dcb_example.* 42 43 Packet Flow Through the VMDQ and DCB Sample Application 44 45 46In Linux* user space, the application can display statistics with the number of packets received on each queue. 47To have the application display the statistics, send a SIGHUP signal to the running application process. 48 49The VMDQ and DCB Forwarding sample application is in many ways simpler than the L2 Forwarding application 50(see :doc:`l2_forward_real_virtual`) 51as it performs unidirectional L2 forwarding of packets from one port to a second port. 52No command-line options are taken by this application apart from the standard EAL command-line options. 53 54.. note:: 55 56 Since VMD queues are being used for VMM, this application works correctly 57 when VTd is disabled in the BIOS or Linux* kernel (intel_iommu=off). 58 59Compiling the Application 60------------------------- 61 62 63 64To compile the sample application see :doc:`compiling`. 65 66The application is located in the ``vmdq_dcb`` sub-directory. 67 68Running the Application 69----------------------- 70 71To run the example in a linux environment: 72 73.. code-block:: console 74 75 user@target:~$ ./<build_dir>/examples/dpdk-vmdq_dcb -l 0-3 -n 4 -- -p 0x3 --nb-pools 32 --nb-tcs 4 76 77Refer to the *DPDK Getting Started Guide* for general information on running applications and 78the Environment Abstraction Layer (EAL) options. 79 80Explanation 81----------- 82 83The following sections provide some explanation of the code. 84 85Initialization 86~~~~~~~~~~~~~~ 87 88The EAL, driver and PCI configuration is performed largely as in the L2 Forwarding sample application, 89as is the creation of the mbuf pool. 90See :doc:`l2_forward_real_virtual`. 91Where this example application differs is in the configuration of the NIC port for RX. 92 93The VMDQ and DCB hardware feature is configured at port initialization time by setting the appropriate values in the 94rte_eth_conf structure passed to the rte_eth_dev_configure() API. 95Initially in the application, 96a default structure is provided for VMDQ and DCB configuration to be filled in later by the application. 97 98.. code-block:: c 99 100 /* empty vmdq+dcb configuration structure. Filled in programmatically */ 101 static const struct rte_eth_conf vmdq_dcb_conf_default = { 102 .rxmode = { 103 .mq_mode = ETH_MQ_RX_VMDQ_DCB, 104 .split_hdr_size = 0, 105 }, 106 .txmode = { 107 .mq_mode = ETH_MQ_TX_VMDQ_DCB, 108 }, 109 /* 110 * should be overridden separately in code with 111 * appropriate values 112 */ 113 .rx_adv_conf = { 114 .vmdq_dcb_conf = { 115 .nb_queue_pools = ETH_32_POOLS, 116 .enable_default_pool = 0, 117 .default_pool = 0, 118 .nb_pool_maps = 0, 119 .pool_map = {{0, 0},}, 120 .dcb_tc = {0}, 121 }, 122 .dcb_rx_conf = { 123 .nb_tcs = ETH_4_TCS, 124 /** Traffic class each UP mapped to. */ 125 .dcb_tc = {0}, 126 }, 127 .vmdq_rx_conf = { 128 .nb_queue_pools = ETH_32_POOLS, 129 .enable_default_pool = 0, 130 .default_pool = 0, 131 .nb_pool_maps = 0, 132 .pool_map = {{0, 0},}, 133 }, 134 }, 135 .tx_adv_conf = { 136 .vmdq_dcb_tx_conf = { 137 .nb_queue_pools = ETH_32_POOLS, 138 .dcb_tc = {0}, 139 }, 140 }, 141 }; 142 143The get_eth_conf() function fills in an rte_eth_conf structure with the appropriate values, 144based on the global vlan_tags array, 145and dividing up the possible user priority values equally among the individual queues 146(also referred to as traffic classes) within each pool. With Intel® 82599 NIC, 147if the number of pools is 32, then the user priority fields are allocated 2 to a queue. 148If 16 pools are used, then each of the 8 user priority fields is allocated to its own queue within the pool. 149With Intel® X710/XL710 NICs, if number of tcs is 4, and number of queues in pool is 8, 150then the user priority fields are allocated 2 to one tc, and a tc has 2 queues mapping to it, then 151RSS will determine the destination queue in 2. 152For the VLAN IDs, each one can be allocated to possibly multiple pools of queues, 153so the pools parameter in the rte_eth_vmdq_dcb_conf structure is specified as a bitmask value. 154For destination MAC, each VMDQ pool will be assigned with a MAC address. In this sample, each VMDQ pool 155is assigned to the MAC like 52:54:00:12:<port_id>:<pool_id>, that is, 156the MAC of VMDQ pool 2 on port 1 is 52:54:00:12:01:02. 157 158.. code-block:: c 159 160 const uint16_t vlan_tags[] = { 161 0, 1, 2, 3, 4, 5, 6, 7, 162 8, 9, 10, 11, 12, 13, 14, 15, 163 16, 17, 18, 19, 20, 21, 22, 23, 164 24, 25, 26, 27, 28, 29, 30, 31 165 }; 166 167 /* pool mac addr template, pool mac addr is like: 52 54 00 12 port# pool# */ 168 static struct rte_ether_addr pool_addr_template = { 169 .addr_bytes = {0x52, 0x54, 0x00, 0x12, 0x00, 0x00} 170 }; 171 172 /* Builds up the correct configuration for vmdq+dcb based on the vlan tags array 173 * given above, and the number of traffic classes available for use. */ 174 static inline int 175 get_eth_conf(struct rte_eth_conf *eth_conf) 176 { 177 struct rte_eth_vmdq_dcb_conf conf; 178 struct rte_eth_vmdq_rx_conf vmdq_conf; 179 struct rte_eth_dcb_rx_conf dcb_conf; 180 struct rte_eth_vmdq_dcb_tx_conf tx_conf; 181 uint8_t i; 182 183 conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 184 vmdq_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 185 tx_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; 186 conf.nb_pool_maps = num_pools; 187 vmdq_conf.nb_pool_maps = num_pools; 188 conf.enable_default_pool = 0; 189 vmdq_conf.enable_default_pool = 0; 190 conf.default_pool = 0; /* set explicit value, even if not used */ 191 vmdq_conf.default_pool = 0; 192 193 for (i = 0; i < conf.nb_pool_maps; i++) { 194 conf.pool_map[i].vlan_id = vlan_tags[i]; 195 vmdq_conf.pool_map[i].vlan_id = vlan_tags[i]; 196 conf.pool_map[i].pools = 1UL << i ; 197 vmdq_conf.pool_map[i].pools = 1UL << i; 198 } 199 for (i = 0; i < ETH_DCB_NUM_USER_PRIORITIES; i++){ 200 conf.dcb_tc[i] = i % num_tcs; 201 dcb_conf.dcb_tc[i] = i % num_tcs; 202 tx_conf.dcb_tc[i] = i % num_tcs; 203 } 204 dcb_conf.nb_tcs = (enum rte_eth_nb_tcs)num_tcs; 205 (void)(rte_memcpy(eth_conf, &vmdq_dcb_conf_default, sizeof(*eth_conf))); 206 (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_dcb_conf, &conf, 207 sizeof(conf))); 208 (void)(rte_memcpy(ð_conf->rx_adv_conf.dcb_rx_conf, &dcb_conf, 209 sizeof(dcb_conf))); 210 (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_rx_conf, &vmdq_conf, 211 sizeof(vmdq_conf))); 212 (void)(rte_memcpy(ð_conf->tx_adv_conf.vmdq_dcb_tx_conf, &tx_conf, 213 sizeof(tx_conf))); 214 if (rss_enable) { 215 eth_conf->rxmode.mq_mode= ETH_MQ_RX_VMDQ_DCB_RSS; 216 eth_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP | 217 ETH_RSS_UDP | 218 ETH_RSS_TCP | 219 ETH_RSS_SCTP; 220 } 221 return 0; 222 } 223 224 ...... 225 226 /* Set mac for each pool.*/ 227 for (q = 0; q < num_pools; q++) { 228 struct rte_ether_addr mac; 229 mac = pool_addr_template; 230 mac.addr_bytes[4] = port; 231 mac.addr_bytes[5] = q; 232 printf("Port %u vmdq pool %u set mac %02x:%02x:%02x:%02x:%02x:%02x\n", 233 port, q, 234 mac.addr_bytes[0], mac.addr_bytes[1], 235 mac.addr_bytes[2], mac.addr_bytes[3], 236 mac.addr_bytes[4], mac.addr_bytes[5]); 237 retval = rte_eth_dev_mac_addr_add(port, &mac, 238 q + vmdq_pool_base); 239 if (retval) { 240 printf("mac addr add failed at pool %d\n", q); 241 return retval; 242 } 243 } 244 245Once the network port has been initialized using the correct VMDQ and DCB values, 246the initialization of the port's RX and TX hardware rings is performed similarly to that 247in the L2 Forwarding sample application. 248See :doc:`l2_forward_real_virtual` for more information. 249 250Statistics Display 251~~~~~~~~~~~~~~~~~~ 252 253When run in a linux environment, 254the VMDQ and DCB Forwarding sample application can display statistics showing the number of packets read from each RX queue. 255This is provided by way of a signal handler for the SIGHUP signal, 256which simply prints to standard output the packet counts in grid form. 257Each row of the output is a single pool with the columns being the queue number within that pool. 258 259To generate the statistics output, use the following command: 260 261.. code-block:: console 262 263 user@host$ sudo killall -HUP vmdq_dcb_app 264 265Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, 266rather than the terminal from which the HUP signal was sent. 267