1# GDB Macros User Guide {#gdb_macros} 2 3## Introduction 4 5When debugging an spdk application using gdb we may need to view data structures 6in lists, e.g. information about bdevs or threads. 7 8If, for example I have several bdevs, and I wish to get information on bdev by 9the name 'test_vols3', I will need to manually iterate over the list as follows: 10 11~~~{.sh} 12(gdb) p g_bdev_mgr->bdevs->tqh_first->name 13$5 = 0x7f7dcc0b21b0 "test_vols1" 14(gdb) p g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->name 15$6 = 0x7f7dcc0b1a70 "test_vols2" 16(gdb) p 17g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->internal->link->tqe_next->name 18$7 = 0x7f7dcc215a00 "test_vols3" 19(gdb) p 20g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->internal->link->tqe_next 21$8 = (struct spdk_bdev *) 0x7f7dcc2c7c08 22~~~ 23 24At this stage, we can start looking at the relevant fields of our bdev which now 25we know is in address 0x7f7dcc2c7c08. 26 27This can be somewhat troublesome if there are 100 bdevs, and the one we need is 2856th in the list... 29 30Instead, we can use a gdb macro in order to get information about all the 31devices. 32 33Examples: 34 35Printing bdevs: 36 37~~~{.sh} 38(gdb) spdk_print_bdevs 39 40SPDK object of type struct spdk_bdev at 0x7f7dcc1642a8 41((struct spdk_bdev*) 0x7f7dcc1642a8) 42name 0x7f7dcc0b21b0 "test_vols1" 43 44--------------- 45 46SPDK object of type struct spdk_bdev at 0x7f7dcc216008 47((struct spdk_bdev*) 0x7f7dcc216008) 48name 0x7f7dcc0b1a70 "test_vols2" 49 50--------------- 51 52SPDK object of type struct spdk_bdev at 0x7f7dcc2c7c08 53((struct spdk_bdev*) 0x7f7dcc2c7c08) 54name 0x7f7dcc215a00 "test_vols3" 55 56--------------- 57~~~ 58 59Finding a bdev by name: 60 61~~~{.sh} 62(gdb) spdk_find_bdev test_vols1 63test_vols1 64 65SPDK object of type struct spdk_bdev at 0x7f7dcc1642a8 66((struct spdk_bdev*) 0x7f7dcc1642a8) 67name 0x7f7dcc0b21b0 "test_vols1" 68~~~ 69 70Printing spdk threads: 71 72~~~{.sh} 73(gdb) spdk_print_threads 74 75SPDK object of type struct spdk_thread at 0x7fffd0008b50 76((struct spdk_thread*) 0x7fffd0008b50) 77name 0x7fffd00008e0 "reactor_1" 78IO Channels: 79 SPDK object of type struct spdk_io_channel at 0x7fffd0052610 80 ((struct spdk_io_channel*) 0x7fffd0052610) 81 name 82 ref 1 83 device 0x7fffd0008c80 (0x7fffd0008ce0 "nvmf_tgt") 84 --------------- 85 86 SPDK object of type struct spdk_io_channel at 0x7fffd0056cd0 87 ((struct spdk_io_channel*) 0x7fffd0056cd0) 88 name 89 ref 2 90 device 0x7fffd0056bf0 (0x7fffd0008e70 "test_vol1") 91 --------------- 92 93 SPDK object of type struct spdk_io_channel at 0x7fffd00582e0 94 ((struct spdk_io_channel*) 0x7fffd00582e0) 95 name 96 ref 1 97 device 0x7fffd0056c50 (0x7fffd0056cb0 "bdev_test_vol1") 98 --------------- 99 100 SPDK object of type struct spdk_io_channel at 0x7fffd00583b0 101 ((struct spdk_io_channel*) 0x7fffd00583b0) 102 name 103 ref 1 104 device 0x7fffd0005630 (0x7fffd0005690 "bdev_mgr") 105 --------------- 106~~~ 107 108Printing nvmf subsystems: 109 110~~~{.sh} 111(gdb) spdk_print_nvmf_subsystems 112 113SPDK object of type struct spdk_nvmf_subsystem at 0x7fffd0008d00 114((struct spdk_nvmf_subsystem*) 0x7fffd0008d00) 115name "nqn.2014-08.org.nvmexpress.discovery", '\000' <repeats 187 times> 116nqn "nqn.2014-08.org.nvmexpress.discovery", '\000' <repeats 187 times> 117ID 0 118 119--------------- 120 121SPDK object of type struct spdk_nvmf_subsystem at 0x7fffd0055760 122((struct spdk_nvmf_subsystem*) 0x7fffd0055760) 123name "nqn.2016-06.io.spdk.umgmt:cnode1", '\000' <repeats 191 times> 124nqn "nqn.2016-06.io.spdk.umgmt:cnode1", '\000' <repeats 191 times> 125ID 1 126~~~ 127 128Printing SPDK spinlocks: 129 130In this example, the spinlock has been initialized and locked but has never been unlocked. 131After it is unlocked the first time the last unlocked stack will be present and the 132`Locked by spdk_thread` line will say `not locked`. 133 134~~~{.sh} 135Breakpoint 2, spdk_spin_unlock (sspin=0x655110 <g_bdev_mgr+80>) at thread.c:2915 1362915 struct spdk_thread *thread = spdk_get_thread(); 137(gdb) print *sspin 138$2 = struct spdk_spinlock: 139 Locked by spdk_thread: 0x658080 140 Initialized at: 141 0x43e677 <spdk_spin_init+213> thread.c:2878 142 0x404feb <_bdev_init+16> /build/spdk/spdk-review-public/lib/bdev/bdev.c:116 143 0x44483d <__libc_csu_init+77> 144 0x7ffff62c9d18 <__libc_start_main+120> 145 0x40268e <_start+46> 146 Last locked at: 147 0x43e936 <spdk_spin_lock+436> thread.c:2909 148 0x40ca9c <bdev_name_add+129> /build/spdk/spdk-review-public/lib/bdev/bdev.c:3855 149 0x411a3c <bdev_register+641> /build/spdk/spdk-review-public/lib/bdev/bdev.c:6660 150 0x412e1e <spdk_bdev_register+24> /build/spdk/spdk-review-public/lib/bdev/bdev.c:7171 151 0x417895 <num_blocks_test+119> bdev_ut.c:878 152 0x7ffff7bc38cb <run_single_test.constprop+379> 153 0x7ffff7bc3b61 <run_single_suite.constprop+433> 154 0x7ffff7bc3f76 <CU_run_all_tests+118> 155 0x43351f <main+1439> bdev_ut.c:6295 156 0x7ffff62c9d85 <__libc_start_main+229> 157 0x40268e <_start+46> 158 Last unlocked at: 159~~~ 160 161Print a single spinlock stack: 162 163~~~{.sh} 164(gdb) print sspin->internal.lock_stack 165$1 = struct sspin_stack: 166 0x40c6a1 <spdk_spin_lock+436> /build/spdk/spdk-review-public/lib/thread/thread.c:2909 167 0x413f48 <spdk_spin+552> thread_ut.c:1831 168 0x7ffff7bc38cb <run_single_test.constprop+379> 169 0x7ffff7bc3b61 <run_single_suite.constprop+433> 170 0x7ffff7bc3f76 <CU_run_all_tests+118> 171 0x4148fa <main+547> thread_ut.c:1948 172 0x7ffff62c9d85 <__libc_start_main+229> 173 0x40248e <_start+46> 174~~~ 175 176## Loading The gdb Macros 177 178Copy the gdb macros to the host where you are about to debug. 179It is best to copy the file either to somewhere within the PYTHONPATH, or to add 180the destination directory to the PYTHONPATH. This is not mandatory, and can be 181worked around, but can save a few steps when loading the module to gdb. 182 183From gdb, with the application core open, invoke python and load the modules. 184 185In the example below, I copied the macros to the /tmp directory which is not in 186the PYTHONPATH, so I had to manually add the directory to the path. 187 188~~~{.sh} 189(gdb) python 190>import sys 191>sys.path.append('/tmp') 192>import gdb_macros 193>end 194(gdb) spdk_load_macros 195~~~ 196 197## Using the gdb Data Directory 198 199On most systems, the data directory is /usr/share/gdb. The python script should 200be copied into the python/gdb/function (or python/gdb/command) directory under 201the data directory, e.g. /usr/share/gdb/python/gdb/function. 202 203If the python script is in there, then the only thing you need to do when 204starting gdb is type "spdk_load_macros". 205 206## Using .gdbinit To Load The Macros 207 208.gdbinit can also be used in order to run automatically run the manual steps 209above prior to starting gdb. 210 211Example .gdbinit: 212 213~~~{.sh} 214source /opt/km/install/tools/gdb_macros/gdb_macros.py 215~~~ 216 217When starting gdb you still have to call spdk_load_macros. 218 219## Why Do We Need to Explicitly Call spdk_load_macros 220 221The reason is that the macros need to use globals provided by spdk in order to 222iterate the spdk lists and build iterable representations of the list objects. 223This will result in errors if these are not available which is very possible if 224gdb is used for reasons other than debugging spdk core dumps. 225 226In the example below, I attempted to load the macros when the globals are not 227available causing gdb to fail loading the gdb_macros: 228 229~~~{.sh} 230(gdb) spdk_load_macros 231Traceback (most recent call last): 232 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 257, in invoke 233 spdk_print_threads() 234 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 241, in __init__ 235 threads = SpdkThreads() 236 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 234, in __init__ 237 super(SpdkThreads, self).__init__('g_threads', SpdkThread) 238 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 25, in __init__ 239 ['tailq']) 240 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 10, in __init__ 241 self.list = gdb.parse_and_eval(self.list_pointer) 242RuntimeError: No symbol table is loaded. Use the "file" command. 243Error occurred in Python command: No symbol table is loaded. Use the "file" 244command. 245~~~ 246 247## Macros available 248 249- spdk_load_macros: load the macros (use --reload in order to reload them) 250- spdk_print_bdevs: information about bdevs 251- spdk_find_bdev: find a bdev (substring search) 252- spdk_print_io_devices: information about io devices 253- spdk_print_nvmf_subsystems: information about nvmf subsystems 254- spdk_print_threads: information about threads 255 256## Adding New Macros 257 258The list iteration macros are usually built from 3 layers: 259 260- SpdkPrintCommand: inherits from gdb.Command and invokes the list iteration 261- SpdkTailqList: Performs the iteration of a tailq list according to the tailq 262 member implementation 263- SpdkObject: Provides the __str__ function so that the list iteration can print 264 the object 265 266Other useful objects: 267 268- SpdkNormalTailqList: represents a list which has 'tailq' as the tailq object 269- SpdkArr: Iteration over an array (instead of a linked list) 270