1.. SPDX-License-Identifier: BSD-3-Clause 2 Copyright 2018 The DPDK contributors 3 4Design 5====== 6 7 8Environment or Architecture-specific Sources 9-------------------------------------------- 10 11In DPDK and DPDK applications, some code is specific to an architecture (i686, x86_64) or to an executive environment (freebsd or linux) and so on. 12As far as is possible, all such instances of architecture or env-specific code should be provided via standard APIs in the EAL. 13 14By convention, a file is common if it is not located in a directory indicating that it is specific. 15For instance, a file located in a subdir of "x86_64" directory is specific to this architecture. 16A file located in a subdir of "linux" is specific to this execution environment. 17 18.. note:: 19 20 Code in DPDK libraries and applications should be generic. 21 The correct location for architecture or executive environment specific code is in the EAL. 22 23When absolutely necessary, there are several ways to handle specific code: 24 25* Use a ``#ifdef`` with a build definition macro in the C code. 26 This can be done when the differences are small and they can be embedded in the same C file: 27 28 .. code-block:: c 29 30 #ifdef RTE_ARCH_I686 31 toto(); 32 #else 33 titi(); 34 #endif 35 36* Use build definition macros and conditions in the Meson build file. This is done when the differences are more significant. 37 In this case, the code is split into two separate files that are architecture or environment specific. 38 This should only apply inside the EAL library. 39 40Per Architecture Sources 41~~~~~~~~~~~~~~~~~~~~~~~~ 42 43The following macro options can be used: 44 45* ``RTE_ARCH`` is a string that contains the name of the architecture. 46* ``RTE_ARCH_I686``, ``RTE_ARCH_X86_64``, ``RTE_ARCH_X86_X32``, ``RTE_ARCH_PPC_64``, ``RTE_ARCH_RISCV``, ``RTE_ARCH_LOONGARCH``, ``RTE_ARCH_ARM``, ``RTE_ARCH_ARMv7`` or ``RTE_ARCH_ARM64`` are defined only if we are building for those architectures. 47 48Per Execution Environment Sources 49~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 50 51The following macro options can be used: 52 53* ``RTE_EXEC_ENV`` is a string that contains the name of the executive environment. 54* ``RTE_EXEC_ENV_FREEBSD``, ``RTE_EXEC_ENV_LINUX`` or ``RTE_EXEC_ENV_WINDOWS`` are defined only if we are building for this execution environment. 55 56Mbuf features 57------------- 58 59The ``rte_mbuf`` structure must be kept small (128 bytes). 60 61In order to add new features without wasting buffer space for unused features, 62some fields and flags can be registered dynamically in a shared area. 63The "dynamic" mbuf area is the default choice for the new features. 64 65The "dynamic" area is eating the remaining space in mbuf, 66and some existing "static" fields may need to become "dynamic". 67 68Adding a new static field or flag must be an exception matching many criteria 69like (non exhaustive): wide usage, performance, size. 70 71 72Runtime Information - Logging, Tracing and Telemetry 73---------------------------------------------------- 74 75It is often desirable to provide information to the end-user 76as to what is happening to the application at runtime. 77DPDK provides a number of built-in mechanisms to provide this introspection: 78 79* :ref:`Logging <dynamic_logging>` 80* :doc:`Tracing <../prog_guide/trace_lib>` 81* :doc:`Telemetry <../prog_guide/telemetry_lib>` 82 83Each of these has its own strengths and suitabilities for use within DPDK components. 84 85Below are some guidelines for when each should be used: 86 87* For reporting error conditions, or other abnormal runtime issues, *logging* should be used. 88 Depending on the severity of the issue, the appropriate log level, for example, 89 ``ERROR``, ``WARNING`` or ``NOTICE``, should be used. 90 91.. note:: 92 93 Drivers of all classes, including both bus and device drivers, 94 should not output any log information if the hardware they support is not present. 95 This is to avoid any changes in output for existing users when a new driver is added to DPDK. 96 97* For component initialization, or other cases where a path through the code 98 is only likely to be taken once, 99 either *logging* at ``DEBUG`` level or *tracing* may be used, or potentially both. 100 In the latter case, tracing can provide basic information as to the code path taken, 101 with debug-level logging providing additional details on internal state, 102 not possible to emit via tracing. 103 104* For a component's data-path, where a path is to be taken multiple times within a short timeframe, 105 *tracing* should be used. 106 Since DPDK tracing uses `Common Trace Format <https://diamon.org/ctf/>`_ for its tracing logs, 107 post-analysis can be done using a range of external tools. 108 109* For numerical or statistical data generated by a component, for example, per-packet statistics, 110 *telemetry* should be used. 111 112* For any data where the data may need to be gathered at any point in the execution 113 to help assess the state of the application component, 114 for example, core configuration, device information, *telemetry* should be used. 115 Telemetry callbacks should not modify any program state, but be "read-only". 116 117Many libraries also include a ``rte_<libname>_dump()`` function as part of their API, 118writing verbose internal details to a given file-handle. 119New libraries are encouraged to provide such functions where it makes sense to do so, 120as they provide an additional application-controlled mechanism 121to get details of the internals of a DPDK component. 122 123 124Library Statistics 125------------------ 126 127Description 128~~~~~~~~~~~ 129 130This document describes the guidelines for DPDK library-level statistics counter 131support. This includes guidelines for turning library statistics on and off and 132requirements for preventing ABI changes when implementing statistics. 133 134 135Mechanism to allow the application to turn library statistics on and off 136~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 137 138Having runtime support for enabling/disabling library statistics is recommended, 139as build-time options should be avoided. However, if build-time options are used, 140for example as in the table library, the options can be set using c_args. 141When this flag is set, all the counters supported by current library are 142collected for all the instances of every object type provided by the library. 143When this flag is cleared, none of the counters supported by the current library 144are collected for any instance of any object type provided by the library: 145 146 147Prevention of ABI changes due to library statistics support 148~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 149 150The layout of data structures and prototype of functions that are part of the 151library API should not be affected by whether the collection of statistics 152counters is turned on or off for the current library. In practical terms, this 153means that space should always be allocated in the API data structures for 154statistics counters and the statistics related API functions are always built 155into the code, regardless of whether the statistics counter collection is turned 156on or off for the current library. 157 158When the collection of statistics counters for the current library is turned 159off, the counters retrieved through the statistics related API functions should 160have a default value of zero. 161 162 163Motivation to allow the application to turn library statistics on and off 164~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 165 166It is highly recommended that each library provides statistics counters to allow 167an application to monitor the library-level run-time events. Typical counters 168are: number of packets received/dropped/transmitted, number of buffers 169allocated/freed, number of occurrences for specific events, etc. 170 171However, the resources consumed for library-level statistics counter collection 172have to be spent out of the application budget and the counters collected by 173some libraries might not be relevant to the current application. In order to 174avoid any unwanted waste of resources and/or performance impacts, the 175application should decide at build time whether the collection of library-level 176statistics counters should be turned on or off for each library individually. 177 178Library-level statistics counters can be relevant or not for specific 179applications: 180 181* For Application A, counters maintained by Library X are always relevant and 182 the application needs to use them to implement certain features, such as traffic 183 accounting, logging, application-level statistics, etc. In this case, 184 the application requires that collection of statistics counters for Library X is 185 always turned on. 186 187* For Application B, counters maintained by Library X are only useful during the 188 application debug stage and are not relevant once debug phase is over. In this 189 case, the application may decide to turn on the collection of Library X 190 statistics counters during the debug phase and at a later stage turn them off. 191 192* For Application C, counters maintained by Library X are not relevant at all. 193 It might be that the application maintains its own set of statistics counters 194 that monitor a different set of run-time events (e.g. number of connection 195 requests, number of active users, etc). It might also be that the application 196 uses multiple libraries (Library X, Library Y, etc) and it is interested in the 197 statistics counters of Library Y, but not in those of Library X. In this case, 198 the application may decide to turn the collection of statistics counters off for 199 Library X and on for Library Y. 200 201The statistics collection consumes a certain amount of CPU resources (cycles, 202cache bandwidth, memory bandwidth, etc) that depends on: 203 204* Number of libraries used by the current application that have statistics 205 counters collection turned on. 206 207* Number of statistics counters maintained by each library per object type 208 instance (e.g. per port, table, pipeline, thread, etc). 209 210* Number of instances created for each object type supported by each library. 211 212* Complexity of the statistics logic collection for each counter: when only 213 some occurrences of a specific event are valid, additional logic is typically 214 needed to decide whether the current occurrence of the event should be counted 215 or not. For example, in the event of packet reception, when only TCP packets 216 with destination port within a certain range should be recorded, conditional 217 branches are usually required. When processing a burst of packets that have been 218 validated for header integrity, counting the number of bits set in a bitmask 219 might be needed. 220 221PF and VF Considerations 222------------------------ 223 224The primary goal of DPDK is to provide a userspace dataplane. Managing VFs from 225a PF driver is a control plane feature and developers should generally rely on 226the Linux Kernel for that. 227 228Developers should work with the Linux Kernel community to get the required 229functionality upstream. PF functionality should only be added to DPDK for 230testing and prototyping purposes while the kernel work is ongoing. It should 231also be marked with an "EXPERIMENTAL" tag. If the functionality isn't 232upstreamable then a case can be made to maintain the PF functionality in DPDK 233without the EXPERIMENTAL tag. 234