1Remote Debugging 2================ 3 4Remote debugging refers to the act of debugging a process which is running on a 5different system, than the debugger itself. We shall refer to the system 6running the debugger as the local system, while the system running the debugged 7process will be the remote system. 8 9To enable remote debugging, LLDB employs a client-server architecture. The 10client part runs on the local system and the remote system runs the server. The 11client and server communicate using the gdb-remote protocol, usually 12transported over TCP/IP. More information on the protocol can be found here and 13the LLDB-specific extensions are documented in docs/lldb-gdb-remote.txt file 14inside LLDB source repository. Besides the gdb-remote stub, the server part of 15LLDB also consists of a platform binary, which is responsible for performing 16advanced debugging operations, like copying files from/to the remote system and 17can be used to execute arbitrary shell commands on the remote system. 18 19In order to reduce code complexity and improve remote debugging experience LLDB 20on Linux and macOS uses the remote debugging stub even when debugging a process 21locally. This is achieved by spawning a remote stub process locally and 22communicating with it over the loopback interface. In the case of local 23debugging this whole process is transparent to the user. The platform binary is 24not used in this case, since no file transfers are needed. 25 26Preparation for Remote Debugging 27--------------------------------- 28 29While the process of actual debugging (stepping, backtraces, evaluating 30expressions) is same as in the local case, in the case of remote debugging, 31more preparation is needed as the required binaries cannot started on the 32remote system automatically. Also, if the remote system runs a different OS or 33architecture, the server component needs to be compiled separately. 34 35Remote system 36************* 37 38On Linux and Android, all required remote functionality is contained in the 39lldb-server binary. This binary combines the functionality of the platform and 40gdb-remote stub. A single binary facilitates deployment and reduces code size, 41since the two functions share a lot of code. The lldb-server binary is also 42statically linked with the rest of LLDB (unlike lldb, which dynamically links 43to liblldb.so by default), so it does not have any dependencies on the rest of 44lldb. On macOS and iOS, the remote-gdb functionality is implemented by the 45debugserver binary, which you will need to deploy alongside lldb-server. 46 47The binaries mentioned above need to be present on the remote system to enable 48remote debugging. You can either compile on the remote system directly or copy 49them from the local machine. If compiling locally and the remote architecture 50differs from the local one, you will need to cross-compile the correct version 51of the binaries. More information on cross-compiling LLDB can be found on the 52build page. 53 54Once the binaries are in place, you just need to run the lldb-server in 55platform mode and specify the port it should listen on. For example, the 56command 57 58:: 59 60 remote% lldb-server platform --listen "*:1234" --server 61 62will start the LLDB platform and wait for incoming connections from any address 63to port 1234. Specifying an address instead of * will only allow connections 64originating from that address. Adding a --server parameter to the command line 65will fork off a new process for every incoming connection, allowing multiple 66parallel debug sessions. 67 68Local system 69************ 70 71On the local system, you need to let LLDB know that you intend to do remote 72debugging. This is achieved through the platform command and its sub-commands. 73As a first step you need to choose the correct platform plug-in for your remote 74system. A list of available plug-ins can be obtained through platform list. 75 76:: 77 78 local% lldb 79 (lldb) platform list 80 Available platforms: 81 host: Local macOS user platform plug-in. 82 remote-freebsd: Remote FreeBSD user platform plug-in. 83 remote-linux: Remote Linux user platform plug-in. 84 remote-netbsd: Remote NetBSD user platform plug-in. 85 remote-windows: Remote Windows user platform plug-in. 86 remote-android: Remote Android user platform plug-in. 87 remote-ios: Remote iOS platform plug-in. 88 remote-macosx: Remote macOS user platform plug-in. 89 ios-simulator: iOS simulator platform plug-in. 90 darwin-kernel: Darwin Kernel platform plug-in. 91 tvos-simulator: Apple TV simulator platform plug-in. 92 watchos-simulator: Apple Watch simulator platform plug-in. 93 remote-tvos: Remote Apple TV platform plug-in. 94 remote-watchos: Remote Apple Watch platform plug-in. 95 remote-gdb-server: A platform that uses the GDB remote protocol as the communication transport. 96 97The default platform is the platform host which is used for local debugging. 98Apart from this, the list should contain a number of plug-ins, for debugging 99different kinds of systems. The remote plug-ins are prefixed with "remote-". 100For example, to debug a remote Linux application: 101 102:: 103 104 (lldb) platform select remote-linux 105 106After selecting the platform plug-in, you should receive a prompt which 107confirms the selected platform, and states that you are not connected. This is 108because remote plug-ins need to be connected to their remote platform 109counterpart to operate. This is achieved using the platform connect command. 110This command takes a number of arguments (as always, use the help command to 111find out more), but normally you only need to specify the address to connect 112to, e.g.: 113 114:: 115 116 (lldb) platform connect connect://remote:1234 117 Platform: remote-linux 118 Triple: x86_64-gnu-linux 119 Hostname: remote 120 Connected: yes 121 WorkingDir: /tmp 122 123Note that the platform has a working directory of /tmp. This directory will be 124used as the directory that executables will be uploaded to by default when 125launching a process from local. 126 127After this, you should be able to debug normally. You can use the process 128attach to attach to an existing remote process or target create, process launch 129to start a new one. The platform plugin will transparently take care of 130uploading or downloading the executable in order to be able to debug. If your 131application needs additional files, you can transfer them using the platform 132commands: get-file, put-file, mkdir, etc. The environment can be prepared 133further using the platform shell command. 134 135When using the "remote-android" platform, the client LLDB forwards two ports, one 136for connecting to the platform, and another for connecting to the gdbserver. 137The client ports are configurable through the environment variables 138ANDROID_PLATFORM_LOCAL_PORT and ANDROID_PLATFORM_LOCAL_GDB_PORT, respectively. 139 140Launching a locally built process on the remote machine 141------------------------------------------------------- 142 143Install and run in the platform working directory 144************************************************* 145 146To launch a locally built process on the remote system in the platform working 147directory: 148 149:: 150 151 (lldb) file a.out 152 (lldb) run 153 154This will cause LLDB to create a target with the "a.out" executable that you 155cross built. The "run" command will cause LLDB to upload "a.out" to the 156platform's current working directory only if the file has changed. The platform 157connection allows us to transfer files, but also allows us to get the MD5 158checksum of the file on the other end and only upload the file if it has 159changed. LLDB will automatically launch a lldb-server in gdbremote mode to 160allow you to debug this executable, connect to it and start your debug session 161for you. 162 163Changing the platform working directory 164*************************************** 165 166You can change the platform working directory while connected to the platform 167with: 168 169:: 170 171 (lldb) platform settings -w /usr/local/bin 172 173And you can verify it worked using "platform status": 174 175:: 176 177 (lldb) platform status 178 Platform: remote-linux 179 Triple: x86_64-gnu-linux 180 Hostname: remote 181 Connected: yes 182 WorkingDir: /usr/local/bin 183 184If we run again, the program will be installed into ``/usr/local/bin``. 185 186Install and run by specifying a remote install path 187*************************************************** 188 189If you want the "a.out" executable to be installed into "/bin/a.out" instead of 190the platform's current working directory, we can set the platform file 191specification using python: 192 193:: 194 195 (lldb) file a.out 196 (lldb) script lldb.target.module['a.out'].SetPlatformFileSpec("/bin/a.out") 197 (lldb) run 198 199Now when you run your program, the program will be uploaded to "/bin/a.out" 200instead of the platform current working directory. Only the main executable is 201uploaded to the remote system by default when launching the application. If you 202have shared libraries that should also be uploaded, then you can add the 203locally build shared library to the current target and set its platform file 204specification: 205 206:: 207 208 (lldb) file a.out 209 (lldb) target module add /local/build/libfoo.so 210 (lldb) target module add /local/build/libbar.so 211 (lldb) script lldb.target.module['libfoo.so'].SetPlatformFileSpec("/usr/lib/libfoo.so") 212 (lldb) script lldb.target.module['libbar.so'].SetPlatformFileSpec("/usr/local/lib/libbar.so") 213 (lldb) run 214 215Attaching to a remote process 216***************************** 217 218If you want to attach to a remote process, you can first list the processes on 219the remote system: 220 221:: 222 223 (lldb) platform process list 224 223 matching processes were found on "remote-linux" 225 PID PARENT USER TRIPLE NAME 226 ====== ====== ========== ======================== ============================ 227 68639 90652 x86_64-apple-macosx lldb 228 ... 229 230Then attaching is as simple as specifying the remote process ID: 231 232:: 233 234 (lldb) attach 68639 235