1.. BSD LICENSE 2 Copyright(c) 2010-2014 Intel Corporation. All rights reserved. 3 All rights reserved. 4 5 Redistribution and use in source and binary forms, with or without 6 modification, are permitted provided that the following conditions 7 are met: 8 9 * Redistributions of source code must retain the above copyright 10 notice, this list of conditions and the following disclaimer. 11 * Redistributions in binary form must reproduce the above copyright 12 notice, this list of conditions and the following disclaimer in 13 the documentation and/or other materials provided with the 14 distribution. 15 * Neither the name of Intel Corporation nor the names of its 16 contributors may be used to endorse or promote products derived 17 from this software without specific prior written permission. 18 19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 20 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 21 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 22 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 23 OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 24 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 25 LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 31QoS Scheduler Sample Application 32================================ 33 34The QoS sample application demonstrates the use of the DPDK to provide QoS scheduling. 35 36Overview 37-------- 38 39The architecture of the QoS scheduler application is shown in the following figure. 40 41.. _figure_qos_sched_app_arch: 42 43.. figure:: img/qos_sched_app_arch.* 44 45 QoS Scheduler Application Architecture 46 47 48There are two flavors of the runtime execution for this application, 49with two or three threads per each packet flow configuration being used. 50The RX thread reads packets from the RX port, 51classifies the packets based on the double VLAN (outer and inner) and 52the lower two bytes of the IP destination address and puts them into the ring queue. 53The worker thread dequeues the packets from the ring and calls the QoS scheduler enqueue/dequeue functions. 54If a separate TX core is used, these are sent to the TX ring. 55Otherwise, they are sent directly to the TX port. 56The TX thread, if present, reads from the TX ring and write the packets to the TX port. 57 58Compiling the Application 59------------------------- 60 61To compile the sample application see :doc:`compiling`. 62 63The application is located in the ``qos_sched`` sub-directory. 64 65 .. note:: 66 67 This application is intended as a linuxapp only. 68 69.. note:: 70 71 To get statistics on the sample app using the command line interface as described in the next section, 72 DPDK must be compiled defining *CONFIG_RTE_SCHED_COLLECT_STATS*, 73 which can be done by changing the configuration file for the specific target to be compiled. 74 75Running the Application 76----------------------- 77 78.. note:: 79 80 In order to run the application, a total of at least 4 81 G of huge pages must be set up for each of the used sockets (depending on the cores in use). 82 83The application has a number of command line options: 84 85.. code-block:: console 86 87 ./qos_sched [EAL options] -- <APP PARAMS> 88 89Mandatory application parameters include: 90 91* --pfc "RX PORT, TX PORT, RX LCORE, WT LCORE, TX CORE": Packet flow configuration. 92 Multiple pfc entities can be configured in the command line, 93 having 4 or 5 items (if TX core defined or not). 94 95Optional application parameters include: 96 97* -i: It makes the application to start in the interactive mode. 98 In this mode, the application shows a command line that can be used for obtaining statistics while 99 scheduling is taking place (see interactive mode below for more information). 100 101* --mst n: Master core index (the default value is 1). 102 103* --rsz "A, B, C": Ring sizes: 104 105* A = Size (in number of buffer descriptors) of each of the NIC RX rings read 106 by the I/O RX lcores (the default value is 128). 107 108* B = Size (in number of elements) of each of the software rings used 109 by the I/O RX lcores to send packets to worker lcores (the default value is 8192). 110 111* C = Size (in number of buffer descriptors) of each of the NIC TX rings written 112 by worker lcores (the default value is 256) 113 114* --bsz "A, B, C, D": Burst sizes 115 116* A = I/O RX lcore read burst size from the NIC RX (the default value is 64) 117 118* B = I/O RX lcore write burst size to the output software rings, 119 worker lcore read burst size from input software rings,QoS enqueue size (the default value is 64) 120 121* C = QoS dequeue size (the default value is 32) 122 123* D = Worker lcore write burst size to the NIC TX (the default value is 64) 124 125* --msz M: Mempool size (in number of mbufs) for each pfc (default 2097152) 126 127* --rth "A, B, C": The RX queue threshold parameters 128 129* A = RX prefetch threshold (the default value is 8) 130 131* B = RX host threshold (the default value is 8) 132 133* C = RX write-back threshold (the default value is 4) 134 135* --tth "A, B, C": TX queue threshold parameters 136 137* A = TX prefetch threshold (the default value is 36) 138 139* B = TX host threshold (the default value is 0) 140 141* C = TX write-back threshold (the default value is 0) 142 143* --cfg FILE: Profile configuration to load 144 145Refer to *DPDK Getting Started Guide* for general information on running applications and 146the Environment Abstraction Layer (EAL) options. 147 148The profile configuration file defines all the port/subport/pipe/traffic class/queue parameters 149needed for the QoS scheduler configuration. 150 151The profile file has the following format: 152 153:: 154 155 ; port configuration [port] 156 157 frame overhead = 24 158 number of subports per port = 1 159 number of pipes per subport = 4096 160 queue sizes = 64 64 64 64 161 162 ; Subport configuration 163 164 [subport 0] 165 tb rate = 1250000000; Bytes per second 166 tb size = 1000000; Bytes 167 tc 0 rate = 1250000000; Bytes per second 168 tc 1 rate = 1250000000; Bytes per second 169 tc 2 rate = 1250000000; Bytes per second 170 tc 3 rate = 1250000000; Bytes per second 171 tc period = 10; Milliseconds 172 tc oversubscription period = 10; Milliseconds 173 174 pipe 0-4095 = 0; These pipes are configured with pipe profile 0 175 176 ; Pipe configuration 177 178 [pipe profile 0] 179 tb rate = 305175; Bytes per second 180 tb size = 1000000; Bytes 181 182 tc 0 rate = 305175; Bytes per second 183 tc 1 rate = 305175; Bytes per second 184 tc 2 rate = 305175; Bytes per second 185 tc 3 rate = 305175; Bytes per second 186 tc period = 40; Milliseconds 187 188 tc 0 oversubscription weight = 1 189 tc 1 oversubscription weight = 1 190 tc 2 oversubscription weight = 1 191 tc 3 oversubscription weight = 1 192 193 tc 0 wrr weights = 1 1 1 1 194 tc 1 wrr weights = 1 1 1 1 195 tc 2 wrr weights = 1 1 1 1 196 tc 3 wrr weights = 1 1 1 1 197 198 ; RED params per traffic class and color (Green / Yellow / Red) 199 200 [red] 201 tc 0 wred min = 48 40 32 202 tc 0 wred max = 64 64 64 203 tc 0 wred inv prob = 10 10 10 204 tc 0 wred weight = 9 9 9 205 206 tc 1 wred min = 48 40 32 207 tc 1 wred max = 64 64 64 208 tc 1 wred inv prob = 10 10 10 209 tc 1 wred weight = 9 9 9 210 211 tc 2 wred min = 48 40 32 212 tc 2 wred max = 64 64 64 213 tc 2 wred inv prob = 10 10 10 214 tc 2 wred weight = 9 9 9 215 216 tc 3 wred min = 48 40 32 217 tc 3 wred max = 64 64 64 218 tc 3 wred inv prob = 10 10 10 219 tc 3 wred weight = 9 9 9 220 221Interactive mode 222~~~~~~~~~~~~~~~~ 223 224These are the commands that are currently working under the command line interface: 225 226* Control Commands 227 228* --quit: Quits the application. 229 230* General Statistics 231 232 * stats app: Shows a table with in-app calculated statistics. 233 234 * stats port X subport Y: For a specific subport, it shows the number of packets that 235 went through the scheduler properly and the number of packets that were dropped. 236 The same information is shown in bytes. 237 The information is displayed in a table separating it in different traffic classes. 238 239 * stats port X subport Y pipe Z: For a specific pipe, it shows the number of packets that 240 went through the scheduler properly and the number of packets that were dropped. 241 The same information is shown in bytes. 242 This information is displayed in a table separating it in individual queues. 243 244* Average queue size 245 246All of these commands work the same way, averaging the number of packets throughout a specific subset of queues. 247 248Two parameters can be configured for this prior to calling any of these commands: 249 250 * qavg n X: n is the number of times that the calculation will take place. 251 Bigger numbers provide higher accuracy. The default value is 10. 252 253 * qavg period X: period is the number of microseconds that will be allowed between each calculation. 254 The default value is 100. 255 256The commands that can be used for measuring average queue size are: 257 258* qavg port X subport Y: Show average queue size per subport. 259 260* qavg port X subport Y tc Z: Show average queue size per subport for a specific traffic class. 261 262* qavg port X subport Y pipe Z: Show average queue size per pipe. 263 264* qavg port X subport Y pipe Z tc A: Show average queue size per pipe for a specific traffic class. 265 266* qavg port X subport Y pipe Z tc A q B: Show average queue size of a specific queue. 267 268Example 269~~~~~~~ 270 271The following is an example command with a single packet flow configuration: 272 273.. code-block:: console 274 275 ./qos_sched -l 1,5,7 -n 4 -- --pfc "3,2,5,7" --cfg ./profile.cfg 276 277This example uses a single packet flow configuration which creates one RX thread on lcore 5 reading 278from port 3 and a worker thread on lcore 7 writing to port 2. 279 280Another example with 2 packet flow configurations using different ports but sharing the same core for QoS scheduler is given below: 281 282.. code-block:: console 283 284 ./qos_sched -l 1,2,6,7 -n 4 -- --pfc "3,2,2,6,7" --pfc "1,0,2,6,7" --cfg ./profile.cfg 285 286Note that independent cores for the packet flow configurations for each of the RX, WT and TX thread are also supported, 287providing flexibility to balance the work. 288 289The EAL coremask/corelist is constrained to contain the default mastercore 1 and the RX, WT and TX cores only. 290 291Explanation 292----------- 293 294The Port/Subport/Pipe/Traffic Class/Queue are the hierarchical entities in a typical QoS application: 295 296* A subport represents a predefined group of users. 297 298* A pipe represents an individual user/subscriber. 299 300* A traffic class is the representation of a different traffic type with a specific loss rate, 301 delay and jitter requirements; such as data voice, video or data transfers. 302 303* A queue hosts packets from one or multiple connections of the same type belonging to the same user. 304 305The traffic flows that need to be configured are application dependent. 306This application classifies based on the QinQ double VLAN tags and the IP destination address as indicated in the following table. 307 308.. _table_qos_scheduler_1: 309 310.. table:: Entity Types 311 312 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 313 | **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | 314 | | | | | 315 +================+=========================+==================================================+==================================+ 316 | Port | - | Ethernet port | Physical port | 317 | | | | | 318 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 319 | Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | 320 | | | | | 321 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 322 | Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | 323 | | | | | 324 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 325 | Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | 326 | | | | | 327 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 328 | Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | 329 | | | | | 330 +----------------+-------------------------+--------------------------------------------------+----------------------------------+ 331 332Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. 333