1 # GDB Macros User Guide {#gdb_macros} 2 3 ## Introduction 4 5 When debugging an spdk application using gdb we may need to view data structures 6 in lists, e.g. information about bdevs or threads. 7 8 If, for example I have several bdevs, and I wish to get information on bdev by 9 the name 'test_vols3', I will need to manually iterate over the list as follows: 10 11 ~~~{.sh} 12 (gdb) p g_bdev_mgr->bdevs->tqh_first->name 13 $5 = 0x7f7dcc0b21b0 "test_vols1" 14 (gdb) p g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->name 15 $6 = 0x7f7dcc0b1a70 "test_vols2" 16 (gdb) p 17 g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->internal->link->tqe_next->name 18 $7 = 0x7f7dcc215a00 "test_vols3" 19 (gdb) p 20 g_bdev_mgr->bdevs->tqh_first->internal->link->tqe_next->internal->link->tqe_next 21 $8 = (struct spdk_bdev *) 0x7f7dcc2c7c08 22 ~~~ 23 24 At this stage, we can start looking at the relevant fields of our bdev which now 25 we know is in address 0x7f7dcc2c7c08. 26 27 This can be somewhat troublesome if there are 100 bdevs, and the one we need is 28 56th in the list... 29 30 Instead, we can use a gdb macro in order to get information about all the 31 devices. 32 33 Examples: 34 35 Printing bdevs: 36 37 ~~~{.sh} 38 (gdb) spdk_print_bdevs 39 40 SPDK object of type struct spdk_bdev at 0x7f7dcc1642a8 41 ((struct spdk_bdev*) 0x7f7dcc1642a8) 42 name 0x7f7dcc0b21b0 "test_vols1" 43 44 --------------- 45 46 SPDK object of type struct spdk_bdev at 0x7f7dcc216008 47 ((struct spdk_bdev*) 0x7f7dcc216008) 48 name 0x7f7dcc0b1a70 "test_vols2" 49 50 --------------- 51 52 SPDK object of type struct spdk_bdev at 0x7f7dcc2c7c08 53 ((struct spdk_bdev*) 0x7f7dcc2c7c08) 54 name 0x7f7dcc215a00 "test_vols3" 55 56 --------------- 57 ~~~ 58 59 Finding a bdev by name: 60 61 ~~~{.sh} 62 (gdb) spdk_find_bdev test_vols1 63 test_vols1 64 65 SPDK object of type struct spdk_bdev at 0x7f7dcc1642a8 66 ((struct spdk_bdev*) 0x7f7dcc1642a8) 67 name 0x7f7dcc0b21b0 "test_vols1" 68 ~~~ 69 70 Printing spdk threads: 71 72 ~~~{.sh} 73 (gdb) spdk_print_threads 74 75 SPDK object of type struct spdk_thread at 0x7fffd0008b50 76 ((struct spdk_thread*) 0x7fffd0008b50) 77 name 0x7fffd00008e0 "reactor_1" 78 IO Channels: 79 SPDK object of type struct spdk_io_channel at 0x7fffd0052610 80 ((struct spdk_io_channel*) 0x7fffd0052610) 81 name 82 ref 1 83 device 0x7fffd0008c80 (0x7fffd0008ce0 "nvmf_tgt") 84 --------------- 85 86 SPDK object of type struct spdk_io_channel at 0x7fffd0056cd0 87 ((struct spdk_io_channel*) 0x7fffd0056cd0) 88 name 89 ref 2 90 device 0x7fffd0056bf0 (0x7fffd0008e70 "test_vol1") 91 --------------- 92 93 SPDK object of type struct spdk_io_channel at 0x7fffd00582e0 94 ((struct spdk_io_channel*) 0x7fffd00582e0) 95 name 96 ref 1 97 device 0x7fffd0056c50 (0x7fffd0056cb0 "bdev_test_vol1") 98 --------------- 99 100 SPDK object of type struct spdk_io_channel at 0x7fffd00583b0 101 ((struct spdk_io_channel*) 0x7fffd00583b0) 102 name 103 ref 1 104 device 0x7fffd0005630 (0x7fffd0005690 "bdev_mgr") 105 --------------- 106 ~~~ 107 108 Printing nvmf subsystems: 109 110 ~~~{.sh} 111 (gdb) spdk_print_nvmf_subsystems 112 113 SPDK object of type struct spdk_nvmf_subsystem at 0x7fffd0008d00 114 ((struct spdk_nvmf_subsystem*) 0x7fffd0008d00) 115 name "nqn.2014-08.org.nvmexpress.discovery", '\000' <repeats 187 times> 116 nqn "nqn.2014-08.org.nvmexpress.discovery", '\000' <repeats 187 times> 117 ID 0 118 119 --------------- 120 121 SPDK object of type struct spdk_nvmf_subsystem at 0x7fffd0055760 122 ((struct spdk_nvmf_subsystem*) 0x7fffd0055760) 123 name "nqn.2016-06.io.spdk.umgmt:cnode1", '\000' <repeats 191 times> 124 nqn "nqn.2016-06.io.spdk.umgmt:cnode1", '\000' <repeats 191 times> 125 ID 1 126 ~~~ 127 128 Printing SPDK spinlocks: 129 130 In this example, the spinlock has been initialized and locked but has never been unlocked. 131 After it is unlocked the first time the last unlocked stack will be present and the 132 `Locked by spdk_thread` line will say `not locked`. 133 134 ~~~{.sh} 135 Breakpoint 2, spdk_spin_unlock (sspin=0x655110 <g_bdev_mgr+80>) at thread.c:2915 136 2915 struct spdk_thread *thread = spdk_get_thread(); 137 (gdb) print *sspin 138 $2 = struct spdk_spinlock: 139 Locked by spdk_thread: 0x658080 140 Initialized at: 141 0x43e677 <spdk_spin_init+213> thread.c:2878 142 0x404feb <_bdev_init+16> /build/spdk/spdk-review-public/lib/bdev/bdev.c:116 143 0x44483d <__libc_csu_init+77> 144 0x7ffff62c9d18 <__libc_start_main+120> 145 0x40268e <_start+46> 146 Last locked at: 147 0x43e936 <spdk_spin_lock+436> thread.c:2909 148 0x40ca9c <bdev_name_add+129> /build/spdk/spdk-review-public/lib/bdev/bdev.c:3855 149 0x411a3c <bdev_register+641> /build/spdk/spdk-review-public/lib/bdev/bdev.c:6660 150 0x412e1e <spdk_bdev_register+24> /build/spdk/spdk-review-public/lib/bdev/bdev.c:7171 151 0x417895 <num_blocks_test+119> bdev_ut.c:878 152 0x7ffff7bc38cb <run_single_test.constprop+379> 153 0x7ffff7bc3b61 <run_single_suite.constprop+433> 154 0x7ffff7bc3f76 <CU_run_all_tests+118> 155 0x43351f <main+1439> bdev_ut.c:6295 156 0x7ffff62c9d85 <__libc_start_main+229> 157 0x40268e <_start+46> 158 Last unlocked at: 159 ~~~ 160 161 Print a single spinlock stack: 162 163 ~~~{.sh} 164 (gdb) print sspin->internal.lock_stack 165 $1 = struct sspin_stack: 166 0x40c6a1 <spdk_spin_lock+436> /build/spdk/spdk-review-public/lib/thread/thread.c:2909 167 0x413f48 <spdk_spin+552> thread_ut.c:1831 168 0x7ffff7bc38cb <run_single_test.constprop+379> 169 0x7ffff7bc3b61 <run_single_suite.constprop+433> 170 0x7ffff7bc3f76 <CU_run_all_tests+118> 171 0x4148fa <main+547> thread_ut.c:1948 172 0x7ffff62c9d85 <__libc_start_main+229> 173 0x40248e <_start+46> 174 ~~~ 175 176 ## Loading The gdb Macros 177 178 Copy the gdb macros to the host where you are about to debug. 179 It is best to copy the file either to somewhere within the PYTHONPATH, or to add 180 the destination directory to the PYTHONPATH. This is not mandatory, and can be 181 worked around, but can save a few steps when loading the module to gdb. 182 183 From gdb, with the application core open, invoke python and load the modules. 184 185 In the example below, I copied the macros to the /tmp directory which is not in 186 the PYTHONPATH, so I had to manually add the directory to the path. 187 188 ~~~{.sh} 189 (gdb) python 190 >import sys 191 >sys.path.append('/tmp') 192 >import gdb_macros 193 >end 194 (gdb) spdk_load_macros 195 ~~~ 196 197 ## Using the gdb Data Directory 198 199 On most systems, the data directory is /usr/share/gdb. The python script should 200 be copied into the python/gdb/function (or python/gdb/command) directory under 201 the data directory, e.g. /usr/share/gdb/python/gdb/function. 202 203 If the python script is in there, then the only thing you need to do when 204 starting gdb is type "spdk_load_macros". 205 206 ## Using .gdbinit To Load The Macros 207 208 .gdbinit can also be used in order to run automatically run the manual steps 209 above prior to starting gdb. 210 211 Example .gdbinit: 212 213 ~~~{.sh} 214 source /opt/km/install/tools/gdb_macros/gdb_macros.py 215 ~~~ 216 217 When starting gdb you still have to call spdk_load_macros. 218 219 ## Why Do We Need to Explicitly Call spdk_load_macros 220 221 The reason is that the macros need to use globals provided by spdk in order to 222 iterate the spdk lists and build iterable representations of the list objects. 223 This will result in errors if these are not available which is very possible if 224 gdb is used for reasons other than debugging spdk core dumps. 225 226 In the example below, I attempted to load the macros when the globals are not 227 available causing gdb to fail loading the gdb_macros: 228 229 ~~~{.sh} 230 (gdb) spdk_load_macros 231 Traceback (most recent call last): 232 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 257, in invoke 233 spdk_print_threads() 234 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 241, in __init__ 235 threads = SpdkThreads() 236 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 234, in __init__ 237 super(SpdkThreads, self).__init__('g_threads', SpdkThread) 238 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 25, in __init__ 239 ['tailq']) 240 File "/opt/km/install/tools/gdb_macros/gdb_macros.py", line 10, in __init__ 241 self.list = gdb.parse_and_eval(self.list_pointer) 242 RuntimeError: No symbol table is loaded. Use the "file" command. 243 Error occurred in Python command: No symbol table is loaded. Use the "file" 244 command. 245 ~~~ 246 247 ## Macros available 248 249 - spdk_load_macros: load the macros (use --reload in order to reload them) 250 - spdk_print_bdevs: information about bdevs 251 - spdk_find_bdev: find a bdev (substring search) 252 - spdk_print_io_devices: information about io devices 253 - spdk_print_nvmf_subsystems: information about nvmf subsystems 254 - spdk_print_threads: information about threads 255 256 ## Adding New Macros 257 258 The list iteration macros are usually built from 3 layers: 259 260 - SpdkPrintCommand: inherits from gdb.Command and invokes the list iteration 261 - SpdkTailqList: Performs the iteration of a tailq list according to the tailq 262 member implementation 263 - SpdkObject: Provides the __str__ function so that the list iteration can print 264 the object 265 266 Other useful objects: 267 268 - SpdkNormalTailqList: represents a list which has 'tailq' as the tailq object 269 - SpdkArr: Iteration over an array (instead of a linked list) 270