1# VMD driver {#vmd} 2 3## In this document {#vmd_toc} 4 5* @ref vmd_intro 6* @ref vmd_interface 7* @ref vmd_key_functions 8* @ref vmd_config 9* @ref vmd_app_frame 10* @ref vmd_app 11* @ref vmd_led 12 13## Introduction {#vmd_intro} 14 15Intel Volume Management Device is a hardware logic inside processor's Root Complex 16responsible for management of PCIe NVMe SSDs. It provides robust Hot Plug support 17and Status LED management. 18 19The driver is responsible for enumeration and hooking NVMe devices behind VMD 20into SPDK PCIe subsystem. It also provides API for LED management and hot plug. 21 22## Public Interface {#vmd_interface} 23 24- spdk/vmd.h 25 26## Key Functions {#vmd_key_functions} 27 28Function | Description 29--------------------------------------- | ----------- 30spdk_vmd_init() | @copybrief spdk_vmd_init() 31spdk_vmd_pci_device_list() | @copybrief spdk_vmd_pci_device_list() 32spdk_vmd_set_led_state() | @copybrief spdk_vmd_set_led_state() 33spdk_vmd_get_led_state() | @copybrief spdk_vmd_get_led_state() 34spdk_vmd_hotplug_monitor() | @copybrief spdk_vmd_hotplug_monitor() 35 36## Configuration {#vmd_config} 37 38To enable VMD driver enumeration, the following steps are required: 39 40Check for available VMD devices (VMD needs to be properly set up in BIOS first). 41 42Example: 43``` 44$ lspci | grep 201d 45 46$ 5d:05.5 RAID bus controller: Intel Corporation Device 201d (rev 04) 47$ d7:05.5 RAID bus controller: Intel Corporation Device 201d (rev 04) 48``` 49 50Run setup.sh script with VMD devices set in PCI_ALLOWED. 51 52Example: 53``` 54$ PCI_ALLOWED="0000:5d:05.5 0000:d7:05.5" scripts/setup.sh 55``` 56 57Check for available devices behind the VMD with spdk_lspci. 58 59Example: 60``` 61$ ./build/bin/spdk_lspci 62 63 5d0505:01:00.0 (8086 a54) (NVMe disk behind VMD) 64 5d0505:03:00.0 (8086 a54) (NVMe disk behind VMD) 65 d70505:01:00.0 (8086 a54) (NVMe disk behind VMD) 66 d70505:03:00.0 (8086 a54) (NVMe disk behind VMD) 67 0000:5d:05.5 (8086 201d) (VMD) 68 0000:d7:05.5 (8086 201d) (VMD) 69``` 70 71VMD NVMe BDF could be used as regular NVMe BDF. 72 73Example: 74``` 75$ ./scripts/rpc.py bdev_nvme_attach_controller -b NVMe1 -t PCIe -a 5d0505:01:00.0 76``` 77 78## Application framework {#vmd_app_frame} 79 80When application framework is used, VMD section needs to be added to the configuration file: 81 82JSON config: 83``` 84{ 85 "subsystem": "vmd", 86 "config": [ 87 { 88 "method": "vmd_enable", 89 "params": {} 90 } 91 ] 92} 93``` 94 95or use RPC call before framework starts e.g. 96``` 97$ ./build/bin/spdk_tgt --wait_for_rpc 98$ ./scripts/rpc.py vmd_enable 99$ ./scripts/rpc.py framework_start_init 100``` 101## Applications w/o application framework {#vmd_app} 102 103To enable VMD enumeration in SPDK application that are not using application framework 104e.g spdk_nvme_perf, spdk_nvme_identify -V flag is required - please refer to app help if it supports VMD. 105 106Applications need to call spdk_vmd_init() to enumerate NVMe devices behind the VMD prior to calling 107spdk_nvme_(probe|connect). 108To support hot plugs spdk_vmd_hotplug_monitor() needs to be called periodically. 109 110## LED management {#vmd_led} 111 112VMD LED utility in the [examples/vmd/led](https://github.com/spdk/spdk/tree/master/examples/vmd/led) 113could be used to set LED states. 114 115In order to verify that a platform is correctly configured to support LED management, ledctl(8) can 116be utilized. For instructions on how to use it, consult the manual page of this utility. 117