1Symbols on macOS 2================ 3 4.. contents:: 5 :local: 6 7On macOS, debug symbols are often in stand alone bundles called **dSYM** files. 8These are bundles that contain DWARF debug information and other resources 9related to builds and debug info. 10 11The DebugSymbols.framework framework helps locate dSYM files when given a UUID. 12It can locate the symbols using a variety of methods: 13 14- Spotlight 15- Explicit search paths 16- Implicit search paths 17- File mapped UUID paths 18- Running one or more shell scripts 19 20DebugSymbols.framework also has global defaults that can be modified to allow 21all of the debug tools (lldb, gdb, sample, CoreSymbolication.framework) to 22easily find important debug symbols. The domain for the DebugSymbols.framework 23defaults is **com.apple.DebugSymbols**, and the defaults can be read, written 24or modified using the **defaults** shell command: 25 26:: 27 28 % defaults read com.apple.DebugSymbols 29 % defaults write com.apple.DebugSymbols KEY ... 30 % defaults delete com.apple.DebugSymbols KEY 31 32The following is a list of the defaults key value setting pairs that can 33be used to enhance symbol location: 34 35**DBGFileMappedPaths** 36 37This default can be specified as a single string, or an array of 38strings. Each string represents a directory that contains file mapped 39UUID values that point to dSYM files. See the "File Mapped UUID 40Directories" section below for more details. Whenever 41DebugSymbols.framework is asked to lookup a dSYM file, it will first 42look in any file mapped UUID directories for a quick match. 43 44:: 45 46 % defaults write com.apple.DebugSymbols DBGFileMappedPaths -string /path/to/uuidmap1 47 % defaults write com.apple.DebugSymbols DBGFileMappedPaths -array /path/to/uuidmap1 48 /path/to/uuidmap2 49 50**DBGShellCommands** 51 52This default can be specified as a single string, or an array of 53strings. Specifies a shell script that will get run in order to find the 54dSYM. The shell script will be run given a single UUID value as the 55shell command arguments and the shell command is expected to return a 56property list. See the property list format defined below. 57 58:: 59 60 % defaults write com.apple.DebugSymbols DBGShellCommands -string /path/to/script1 61 % defaults write com.apple.DebugSymbols DBGShellCommands -array /path/to/script1 62 /path/to/script2 63 64**DBGSpotlightPaths** 65 66Specifies the directories to limit spotlight searches to as a string or 67array of strings. When any other defaults are supplied to 68**com.apple.DebugSymbols**, spotlight searches will be disabled unless 69this default is set to an empty array: 70 71:: 72 73 # Specify an empty array to keep Spotlight searches enabled in all locations 74 % defaults write com.apple.DebugSymbols DBGSpotlightPaths -array 75 76 # Specify an array of paths to limit spotlight searches to certain directories 77 % defaults write com.apple.DebugSymbols DBGSpotlightPaths -array /path/dir1 /path/dir2 78 79Shell Script Property List Format 80--------------------------------- 81 82Shell scripts that are specified with the **DBGShellCommands** defaults key 83will be run in the order in which they are specified until a match is found. 84The shell script will be invoked with a single UUID string value like 85"23516BE4-29BE-350C-91C9-F36E7999F0F1". The shell script must respond with a 86property list being written to STDOUT. The property list returned must contain 87UUID string values as the root key values, with a dictionary for each UUID. The 88dictionaries can contain one or more of the following keys: 89 90+-----------------------------------+-----------------------------------+ 91| Key | Description | 92+-----------------------------------+-----------------------------------+ 93| **DBGArchitecture** | A textual architecture or target | 94| | triple like "x86_64", "i386", or | 95| | "x86_64-apple-macosx". | 96+-----------------------------------+-----------------------------------+ 97| **DBGBuildSourcePath** | A path prefix that was used when | 98| | building the dSYM file. The debug | 99| | information will contain paths | 100| | with this prefix. | 101+-----------------------------------+-----------------------------------+ 102| **DBGSourcePath** | A path prefix for where the | 103| | sources exist after the build has | 104| | completed. Often when building | 105| | projects, build machines will | 106| | host the sources in a temporary | 107| | directory while building, then | 108| | move the sources to another | 109| | location for archiving. If the | 110| | paths in the debug info don't | 111| | match where the sources are | 112| | currently hosted, then specifying | 113| | this path along with the | 114| | **DBGBuildSourcePath** will help | 115| | the developer tools always show | 116| | you sources when debugging or | 117| | symbolicating. | 118+-----------------------------------+-----------------------------------+ 119| **DBGDSYMPath** | A path to the dSYM mach-o file | 120| | inside the dSYM bundle. | 121+-----------------------------------+-----------------------------------+ 122| **DBGSymbolRichExecutable** | A path to the symbol rich | 123| | executable. Binaries are often | 124| | stripped after being built and | 125| | packaged into a release. If your | 126| | build systems saves an unstripped | 127| | executable a path to this | 128| | executable can be provided. | 129+-----------------------------------+-----------------------------------+ 130| **DBGError** | If a binary can not be located | 131| | for the supplied UUID, a user | 132| | readable error can be returned. | 133+-----------------------------------+-----------------------------------+ 134 135Below is a sample shell script output for a binary that contains two 136architectures: 137 138:: 139 140 <?xml version="1.0" encoding="UTF-8"?> 141 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> 142 <plist version="1.0"> 143 <dict> 144 <key>23516BE4-29BE-350C-91C9-F36E7999F0F1</key> 145 <dict> 146 <key>DBGArchitecture</key> 147 <string>i386</string> 148 <key>DBGBuildSourcePath</key> 149 <string>/path/to/build/sources</string> 150 <key>DBGSourcePath</key> 151 <string>/path/to/actual/sources</string> 152 <key>DBGDSYMPath</key> 153 <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string> 154 <key>DBGSymbolRichExecutable</key> 155 <string>/path/to/unstripped/executable</string> 156 </dict> 157 <key>A40597AA-5529-3337-8C09-D8A014EB1578</key> 158 <dict> 159 <key>DBGArchitecture</key> 160 <string>x86_64</string> 161 <key>DBGBuildSourcePath</key> 162 <string>/path/to/build/sources</string> 163 <key>DBGSourcePath</key> 164 <string>/path/to/actual/sources</string> 165 <key>DBGDSYMPath</key> 166 <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string> 167 <key>DBGSymbolRichExecutable</key> 168 <string>/path/to/unstripped/executable</string> 169 </dict> 170 </dict> 171 </plist> 172 173There is no timeout imposed on a shell script when is it asked to locate a dSYM 174file, so be careful to not make a shell script that has high latency or takes a 175long time to download unless this is really what you want. This can slow down 176debug sessions in LLDB and GDB, symbolication with CoreSymbolication or Report 177Crash, with no visible feedback to the user. You can quickly return a plist 178with a single **DBGError** key that indicates a timeout has been reached. You 179might also want to exec new processes to do the downloads so that if you return 180an error that indicates a timeout, your download can still proceed after your 181shell script has exited so subsequent debug sessions can use the cached files. 182It is also important to track when a current download is in progress in case 183you get multiple requests for the same UUID so that you don't end up 184downloading the same file simultaneously. Also you will want to verify the 185download was successful and then and only then place the file into the cache 186for tools that will cache files locally. 187 188Embedding UUID property lists inside the dSYM bundles 189----------------------------------------------------- 190 191Since dSYM files are bundles, you can also place UUID info plists files inside 192your dSYM bundles in the **Contents/Resources** directory. One of the main 193reasons to create the UUID plists inside the dSYM bundles is that it will help 194LLDB and other developer tools show you source. LLDB currently knows how to 195check for these plist files so it can automatically remap the source location 196information in the debug info. 197 198If we take the two UUID values from the returns plist above, we can split them 199out and save then in the dSYM bundle: 200 201:: 202 203 % ls /path/to/foo.dSYM/Contents/Resources 204 23516BE4-29BE-350C-91C9-F36E7999F0F1.plist 205 A40597AA-5529-3337-8C09-D8A014EB1578.plist 206 207 % cat /path/to/foo.dSYM/Contents/Resources/23516BE4-29BE-350C-91C9-F36E7999F0F1.plist 208 <?xml version="1.0" encoding="UTF-8"?> 209 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> 210 <plist version="1.0"> 211 <dict> 212 <key>DBGArchitecture</key> 213 <string>i386</string> 214 <key>DBGBuildSourcePath</key> 215 <string>/path/to/build/sources</string> 216 <key>DBGSourcePath</key> 217 <string>/path/to/actual/sources</string> 218 <key>DBGDSYMPath</key> 219 <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string> 220 <key>DBGSymbolRichExecutable</key> 221 <string>/path/to/unstripped/executable</string> 222 <key>DBGVersion</key> 223 <string>3</string> 224 <key>DBGSourcePathRemapping</key> 225 <dict> 226 <key>/path/to/build/time/src/location1</key> 227 <string>/path/to/debug/time/src/location</string> 228 <key>/path/to/build/time/src/location2</key> 229 <string>/path/to/debug/time/src/location</string> 230 </dict> 231 <key>DBGSymbolRichExecutable</key> 232 <string>/path/to/unstripped/executable</string> 233 </dict> 234 </plist> 235 236Note that the output is very close to what is needed by shell script output, so 237making the results of your shell script will be very easy to create by 238combining two plists into a single one where you take the UUID and use it a 239string key, and the value is the contents of the plist. 240 241LLDB will read the following entries from the per-UUID plist file in the dSYM 242bundle: **DBGSymbolRichExecutable**, **DBGBuildSourcePath** and 243**DBGSourcePath**, and **DBGSourcePathRemapping** if **DBGVersion** is 3 or 244higher. **DBGBuildSourcePath** and **DBGSourcePath** are for remapping a single 245file path. For instance, the files may be in /BuildDir/SheetApp/SheetApp-37 246when built, but they are in /SourceDir/SheetApp/SheetApp-37 at debug time, 247those two paths could be listed in those keys. If there are multiple source 248path remappings, the **DBGSourcePathRemapping** dictionary can be used, where 249an arbitrary number of entries may be present. **DBGVersion** should be 3 or 250**DBGSourcePathRemapping** will not be read. If both **DBGSourcePathRemapping** 251AND **DBGBuildSourcePath**/**DBGSourcePath** are present in the plist, the 252**DBGSourcePathRemapping** entries will be used for path remapping first. This 253may allow for more specific remappings in the **DBGSourcePathRemapping** 254dictionary and a less specific remapping in the 255**DBGBuildSourcePath**/**DBGSourcePath** pair as a last resort. 256 257File Mapped UUID Directories 258---------------------------- 259 260File Mapped directories can be used for efficient dSYM file lookups for local 261or remote dSYM files. The UUID is broken up by splitting the first 20 hex 262digits into 4 character chunks, and a directory is created for each chunk, and 263each subsequent directory is created inside the previous one. A symlink is then 264created whose name is the last 12 hex digits in the deepest directory. The 265symlinks value is a full path to the mach-o files inside the dSYM bundle which 266contains the DWARF. Whenever DebugSymbols.framework is asked to lookup a dSYM 267file, it will first look in any file mapped UUID directories for a quick match 268if the defaults are appropriately set. 269 270For example, if we take the sample UUID plist information from above, we can 271create a File Mapped UUID directory cache in 272**~/Library/SymbolCache/dsyms/uuids**. We can easily see how things are laid 273out: 274 275:: 276 277 % find ~/Library/SymbolCache/dsyms/uuids -type l 278 ~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1 279 ~/Library/SymbolCache/dsyms/uuids/A405/97AA/5529/3337/8C09/D8A014EB1578 280 281The last entries in these file mapped directories are symlinks to the actual 282dsym mach file in the dsym bundle: 283 284:: 285 286 % ls -lAF ~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1 287 ~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1@ -> ../../../../../../dsyms/foo.dSYM/Contents/Resources/DWARF/foo 288 289Then you can also tell DebugSymbols to check this UUID file map cache using: 290 291:: 292 293 % defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids 294 295dSYM Locating Shell Script Tips 296------------------------------- 297 298One possible implementation of a dSYM finding shell script is to have the 299script download and cache files locally in a known location. Then create a UUID 300map for each UUID value that was found in a local UUID File Map cache so the 301next query for the dSYM file will be able to use the cached version. So the 302shell script is used to initially download and cache the file, and subsequent 303accesses will use the cache and avoid calling the shell script. 304 305Then the defaults for DebugSymbols.framework will entail enabling your shell 306script, enabling the file mapped path setting so that already downloaded dSYMS 307fill quickly be found without needing to run the shell script every time, and 308also leaving spotlight enabled so that other normal dSYM files are still found: 309 310:: 311 312 % defaults write com.apple.DebugSymbols DBGShellCommands /path/to/shellscript 313 % defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids 314 % defaults write com.apple.DebugSymbols DBGSpotlightPaths -array 315 316Hopefully this helps explain how DebugSymbols.framework can help any company 317implement a smart symbol finding and caching with minimal overhead. 318