1 2# An Overview of SPDK Applications {#app_overview} 3 4SPDK is primarily a development kit that delivers libraries and header files for 5use in other applications. However, SPDK also contains a number of applications. 6These applications are primarily used to test the libraries, but many are full 7featured and high quality. The major applications in SPDK are: 8 9- @ref iscsi 10- @ref nvmf 11- @ref vhost 12- SPDK Target (a unified application combining the above three) 13 14There are also a number of tools and examples in the `examples` directory. 15 16The SPDK targets are all based on a common framework so they have much in 17common. The framework defines a concept called a `subsystem` and all 18functionality is implemented in various subsystems. Subsystems have a unified 19initialization and teardown path. 20 21# Configuring SPDK Applications {#app_config} 22 23## Command Line Parameters {#app_cmd_line_args} 24 25The SPDK application framework defines a set of base command line flags for all 26applications that use it. Specific applications may implement additional flags. 27 28Param | Long Param | Type | Default | Description 29-------- | ---------------------- | -------- | ---------------------- | ----------- 30-c | --config | string | | @ref cmd_arg_config_file 31-d | --limit-coredump | flag | false | @ref cmd_arg_limit_coredump 32-e | --tpoint-group | integer | | @ref cmd_arg_limit_tpoint_group_mask 33-g | --single-file-segments | flag | | @ref cmd_arg_single_file_segments 34-h | --help | flag | | show all available parameters and exit 35-i | --shm-id | integer | | @ref cmd_arg_multi_process 36-m | --cpumask | CPU mask | 0x1 | application @ref cpu_mask 37-n | --mem-channels | integer | all channels | number of memory channels used for DPDK 38-p | --main-core | integer | first core in CPU mask | main (primary) core for DPDK 39-r | --rpc-socket | string | /var/tmp/spdk.sock | RPC listen address 40-s | --mem-size | integer | all hugepage memory | @ref cmd_arg_memory_size 41| | --silence-noticelog | flag | | disable notice level logging to `stderr` 42-u | --no-pci | flag | | @ref cmd_arg_disable_pci_access. 43| | --wait-for-rpc | flag | | @ref cmd_arg_deferred_initialization 44-B | --pci-blocked | B:D:F | | @ref cmd_arg_pci_blocked_allowed. 45-A | --pci-allowed | B:D:F | | @ref cmd_arg_pci_blocked_allowed. 46-R | --huge-unlink | flag | | @ref cmd_arg_huge_unlink 47| | --huge-dir | string | the first discovered | allocate hugepages from a specific mount 48-L | --logflag | string | | @ref cmd_arg_log_flags 49 50### Configuration file {#cmd_arg_config_file} 51 52SPDK applications are configured using a JSON RPC configuration file. 53See @ref jsonrpc for details. 54 55### Limit coredump {#cmd_arg_limit_coredump} 56 57By default, an SPDK application will set resource limits for core file sizes 58to RLIM_INFINITY. Specifying `--limit-coredump` will not set the resource limits. 59 60### Tracepoint group mask {#cmd_arg_limit_tpoint_group_mask} 61 62SPDK has an experimental low overhead tracing framework. Tracepoints in this 63framework are organized into tracepoint groups. By default, all tracepoint 64groups are disabled. `--tpoint-group` can be used to enable a specific 65subset of tracepoint groups in the application. 66 67Note: Additional documentation on the tracepoint framework is in progress. 68 69### Deferred initialization {#cmd_arg_deferred_initialization} 70 71SPDK applications progress through a set of states beginning with `STARTUP` and 72ending with `RUNTIME`. 73 74If the `--wait-for-rpc` parameter is provided SPDK will pause just before starting 75framework initialization. This state is called `STARTUP`. The JSON RPC server is 76ready but only a small subset of commands are available to set up initialization 77parameters. Those parameters can't be changed after the SPDK application enters 78`RUNTIME` state. When the client finishes configuring the SPDK subsystems it 79needs to issue the @ref rpc_framework_start_init RPC command to begin the 80initialization process. After `rpc_framework_start_init` returns `true` SPDK 81will enter the `RUNTIME` state and the list of available commands becomes much 82larger. 83 84To see which RPC methods are available in the current state, issue the 85`rpc_get_methods` with the parameter `current` set to `true`. 86 87For more details see @ref jsonrpc documentation. 88 89### Create just one hugetlbfs file {#cmd_arg_single_file_segments} 90 91Instead of creating one hugetlbfs file per page, this option makes SPDK create 92one file per hugepages per socket. This is needed for @ref virtio to be used 93with more than 8 hugepages. See @ref virtio_2mb. 94 95### Multi process mode {#cmd_arg_multi_process} 96 97When `--shm-id` is specified, the application is started in multi-process mode. 98Applications using the same shm-id share their memory and 99[NVMe devices](@ref nvme_multi_process). The first app to start with a given id 100becomes a primary process, with the rest, called secondary processes, only 101attaching to it. When the primary process exits, the secondary ones continue to 102operate, but no new processes can be attached at this point. All processes within 103the same shm-id group must use the same 104[--single-file-segments setting](@ref cmd_arg_single_file_segments). 105 106### Memory size {#cmd_arg_memory_size} 107 108Total size of the hugepage memory to reserve. If DPDK env layer is used, it will 109reserve memory from all available hugetlbfs mounts, starting with the one with 110the highest page size. This option accepts a number of bytes with a possible 111binary prefix, e.g. 1024, 1024M, 1G. The default unit is megabyte. 112 113Starting with DPDK 18.05.1, it's possible to reserve hugepages at runtime, meaning 114that SPDK application can be started with 0 pre-reserved memory. Unlike hugepages 115pre-reserved at the application startup, the hugepages reserved at runtime will be 116released to the system as soon as they're no longer used. 117 118### Disable PCI access {#cmd_arg_disable_pci_access} 119 120If SPDK is run with PCI access disabled it won't detect any PCI devices. This 121includes primarily NVMe and IOAT devices. Also, the VFIO and UIO kernel modules 122are not required in this mode. 123 124### PCI address blocked and allowed lists {#cmd_arg_pci_blocked_allowed} 125 126If blocked list is used, then all devices with the provided PCI address will be 127ignored. If an allowed list is used, only allowed devices will be probed. 128`-B` or `-A` can be used more than once, but cannot be mixed together. That is, 129`-B` and `-A` cannot be used at the same time. 130 131### Unlink hugepage files after initialization {#cmd_arg_huge_unlink} 132 133By default, each DPDK-based application tries to remove any orphaned hugetlbfs 134files during its initialization. This option removes hugetlbfs files of the current 135process as soon as they're created, but is not compatible with `--shm-id`. 136 137### Log flag {#cmd_arg_log_flags} 138 139Enable a specific log type. This option can be used more than once. A list of 140all available types is provided in the `--help` output, with `--logflag all` 141enabling all of them. Additionally enables debug print level in debug builds of SPDK. 142 143## CPU mask {#cpu_mask} 144 145Whenever the `CPU mask` is mentioned it is a string in one of the following formats: 146 147- Case insensitive hexadecimal string with or without "0x" prefix. 148- Comma separated list of CPUs or list of CPU ranges. Use '-' to define range. 149 150### Example 151 152The following CPU masks are equal and correspond to CPUs 0, 1, 2, 8, 9, 10, 11 and 12: 153 154~~~bash 1550x1f07 1560x1F07 1571f07 158[0,1,2,8-12] 159[0, 1, 2, 8, 9, 10, 11, 12] 160~~~ 161