[lldb] Have crashlog warn when remapped paths are inaccessible.

It can be tricky to troubleshoot why the crashlog script can't show
inline sources. The two most common causes are that we couldn't fi

[lldb] Have crashlog warn when remapped paths are inaccessible.

It can be tricky to troubleshoot why the crashlog script can't show
inline sources. The two most common causes are that we couldn't find the
dSYM or, if we find the dSYM, that the path remapping included in the
dSYMForUUID output isn't accessible. The former is already easy to
diagnose, but the latter is harder because you'd have to manually invoke
dsymForUUID on the UUID and check the remapped path. This patch
automates that process and prints a warning if the remapped path doesn't
exist or is not accessible.

Differential revision: https://reviews.llvm.org/D152886

show more ...

[lldb] Reinvoke crashlog under lldb when run with -i from the command line

Run crashlog inside lldb when invoked in interactive mode from the
command line. Currently, when passing -i to crashlog fro

[lldb] Reinvoke crashlog under lldb when run with -i from the command line

Run crashlog inside lldb when invoked in interactive mode from the
command line. Currently, when passing -i to crashlog from the command
line, we symbolicate in LLDB and immediately exit right after. This
pretty much defeats the purpose of interactive mode. That said, we
wouldn't want to re-implement the driver from the crashlog script.
Re-invoking the crashlog command from inside LLDB solves the issue.

rdar://97801509

Differential revision: https://reviews.llvm.org/D152319

show more ...

[lldb] Fix "NameError: name 'self' is not defined" when using crashlog -c

This fixes a regression introduced by 27f27d15f6c9 that results in a
NameError: (name 'self' is not defined) when using cras

[lldb] Fix "NameError: name 'self' is not defined" when using crashlog -c

This fixes a regression introduced by 27f27d15f6c9 that results in a
NameError: (name 'self' is not defined) when using crashlog with the -c
option.

rdar://110007391

show more ...

[lldb/crashlog] Expand crash report file path before parsing

This patch should fix a crash in the opening a crash report that was
passed with a relative path.

This patch expands the crash report pa

[lldb/crashlog] Expand crash report file path before parsing

This patch should fix a crash in the opening a crash report that was
passed with a relative path.

This patch expands the crash report path before parsing it and raises a
`FileNotFoundError` exception if the file doesn't exist.

Differential Revision: https://reviews.llvm.org/D152012

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb/Commands] Add support to auto-completion for user commands

This patch should allow the user to set specific auto-completion type
for their custom commands.

To do so, we had to hoist the `Comp

[lldb/Commands] Add support to auto-completion for user commands

This patch should allow the user to set specific auto-completion type
for their custom commands.

To do so, we had to hoist the `CompletionType` enum so the user can
access it and add a new completion type flag to the CommandScriptAdd
Command Object.

So now, the user can specify which completion type will be used with
their custom command, when they register it.

This also makes the `crashlog` custom commands use disk-file completion
type, to browse through the user file system and load the report.

Differential Revision: https://reviews.llvm.org/D152011

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb/crashlog] Run python formatter (nfc)

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

[lldb/crashlog] Create interactive crashlog with no binary

This patch changes the way we load a crash report into a scripted
process by creating a empty target.

To do so, it parses the architecture

[lldb/crashlog] Create interactive crashlog with no binary

This patch changes the way we load a crash report into a scripted
process by creating a empty target.

To do so, it parses the architecture information from the report (for
both the legacy and json format) and uses that to create a target that
doesn't have any executable, like what we do when attaching to a process.

For the legacy format, we mostly rely on the `Code Type` line, since the
architure is an optional field on the `Binary Images` sections.

However for the json format, we first try to get the architecture while
parsing the image dictionary if we couldn't find it, we try to infer it
using the "flavor" key when parsing the frame's registers.

If the architecture is still not set after parsing the report, we raise
an exception.

rdar://107850263

Differential Revision: https://reviews.llvm.org/D151849

Differential

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb/crashlog] Fix crash when loading non-symbolicated report

This patch should address the crashes when parsing a the crash report
frame dictionary.

If the crash report is not symbolicated, the `

[lldb/crashlog] Fix crash when loading non-symbolicated report

This patch should address the crashes when parsing a the crash report
frame dictionary.

If the crash report is not symbolicated, the `symbolLocation` key will
be missing. In that case, we should just use the `imageOffset`.

rdar://109836386

Differential Revision: https://reviews.llvm.org/D151844

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problem

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.

RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Differential revision: https://reviews.llvm.org/D151460

show more ...

[lldb] Fix racing issue when loading inlined symbols from crash report

Following abba5de72466, some tests started failing on green-dragon:

https://green.lab.llvm.org/green/job/lldb-cmake/55460/cons

[lldb] Fix racing issue when loading inlined symbols from crash report

Following abba5de72466, some tests started failing on green-dragon:

https://green.lab.llvm.org/green/job/lldb-cmake/55460/console

Looking at the backtrace, there seems to be a racing issue when deleting
the temporary directory containing all the JSON object files:

```
Traceback (most recent call last):
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1115, in __call__
SymbolicateCrashLogs(debugger, shlex.split(command), result)
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1457, in SymbolicateCrashLogs
SymbolicateCrashLog(crash_log, options)
File "/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/python3.10/site-packages/lldb/macosx/crashlog.py", line 1158, in SymbolicateCrashLog
with tempfile.TemporaryDirectory() as obj_dir:
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 869, in __exit__
self.cleanup()
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 873, in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 731, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 729, in rmtree
os.rmdir(path)
OSError: [Errno 66] Directory not empty: '/var/folders/09/r4vw4v8n5kb67jl66zvlbljw0000gn/T/tmp6qfifxk7'
```

This patch should fix that issue since it won't delete the object file
directory until we're sure that the modules adding tasks completed.

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb/crashlog] Remove tempfile prefix from inlined symbol object file

This patch changes the way we generate the ObjectFileJSON files
containing the inlined symbols from the crash report to remove

[lldb/crashlog] Remove tempfile prefix from inlined symbol object file

This patch changes the way we generate the ObjectFileJSON files
containing the inlined symbols from the crash report to remove the
tempfile prefix from the object file name.

To do so, instead of creating a new tempfile for each module, we create a
temporary directory that contains each module object file with the same
name as the module.

This makes the backtraces only contain the module name without the
temfile prefix which makes it look like a regular stackframe.

Differential Revision: https://reviews.llvm.org/D151045

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb/crashlog] Load inlined symbol into interactive crashlog

Sometimes, crash reports come with inlined symbols. These provide the
exact stacktrace from the user binary.

However, when investigatin

[lldb/crashlog] Load inlined symbol into interactive crashlog

Sometimes, crash reports come with inlined symbols. These provide the
exact stacktrace from the user binary.

However, when investigating a crash, it's very likely that the images related
to the crashed thread are not available on the debugging user system or
that the versions don't match. This causes interactive crashlog to show
a degraded backtrace in lldb.

This patch aims to address that issue, by parsing the inlined symbols
from the crash report and load them into lldb's target.

This patch is a follow-up to 27f27d1, focusing on inlined symbols
loading from legacy (non-json) crash reports.

To do so, it updates the stack frame regular expression to make the
capture groups more granular, to be able to extract the symbol name, the
offset and the source location if available, while making it more
maintainable.

So now, when parsing the crash report, we build a data structure
containing all the symbol information for each stackframe. Then, after
launching the scripted process for interactive mode, we write a JSON
symbol file for each module, only containing the symbols that it contains.

Finally, we load the json symbol file into lldb, before showing the user
the process status and backtrace.

rdar://97345586

Differential Revision: https://reviews.llvm.org/D146765

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb] Use ObjectFileJSON to create modules for interactive crashlogs

Create an artificial module using a JSON object file when we can't
locate the module and dSYM through dsymForUUID (or however
lo

[lldb] Use ObjectFileJSON to create modules for interactive crashlogs

Create an artificial module using a JSON object file when we can't
locate the module and dSYM through dsymForUUID (or however
locate_module_and_debug_symbols is implemented). By parsing the symbols
from the crashlog and making them part of the JSON object file, LLDB can
symbolicate frames it otherwise wouldn't be able to, as there is no
module for it.

For non-interactive crashlogs, that never was a problem because we could
simply show the "pre-symbolicated" frame from the input. For interactive
crashlogs, we need a way to pass the symbol information to LLDB so that
it can symbolicate the frames, which is what motivated the JSON object
file format.

Differential revision: https://reviews.llvm.org/D148172

show more ...

[lldb] Parse the crashlog only once

Now that we can pass Python objects to the scripted process instance, we
don't need to parse the crashlog twice anymore.

Differential revision: https://reviews.l

[lldb] Parse the crashlog only once

Now that we can pass Python objects to the scripted process instance, we
don't need to parse the crashlog twice anymore.

Differential revision: https://reviews.llvm.org/D148063

show more ...

[lldb] Parse image high address from JSON crashlogs

Use the base + size to correctly populate the image high address when
parsing JSON crashlogs.

Differential revision: https://reviews.llvm.org/D14

[lldb] Parse image high address from JSON crashlogs

Use the base + size to correctly populate the image high address when
parsing JSON crashlogs.

Differential revision: https://reviews.llvm.org/D148053

show more ...

[lldb] Force override when adding crashlog command

When using interactive crashlog from an IDE, it can happen that the user
already have the `command script import lldb.macosx.crashlog` command on
t

[lldb] Force override when adding crashlog command

When using interactive crashlog from an IDE, it can happen that the user
already have the `command script import lldb.macosx.crashlog` command on
their `lldbinit` file.

That leads to showing some message:

```
error: cannot add command: user command exists and force replace not set
error: cannot add command: user command exists and force replace not set
```

This leads to confusion because the crashlog symbolication continues and
succeeds even after these errors.

To address that, the crashlog commands get overridden everytime the
script get re-imported.

rdar://103403943

Differential Revision: https://reviews.llvm.org/D140113

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Refactor CrashLogParser into a Factory pattern

This patch should fix an undefined behaviour that's happening when
parsing a crash report from an IDE. In the previous implementation,

[lldb/crashlog] Refactor CrashLogParser into a Factory pattern

This patch should fix an undefined behaviour that's happening when
parsing a crash report from an IDE. In the previous implementation, the
CrashLogParser base class would use the `__new__` static class method to
create the right parser instance depending on the crash report type.

For some reasons, the derived parser initializer wouldn't be called when
running the command from an IDE, so this patch refactors the
CrashLogParser code to replace the use of the `__new__` method with a
factory `create` static method.

rdar://100527640

Differential Revision: https://reviews.llvm.org/D139951

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Standardize file path key in the ScriptedProcess Dictionary

This patch replaces the backing file path key to "file_path" to keep it
consistent.

rdar://101652618

Signed-off-by: Med

[lldb/crashlog] Standardize file path key in the ScriptedProcess Dictionary

This patch replaces the backing file path key to "file_path" to keep it
consistent.

rdar://101652618

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Fix frame parser regex for when there is no source info

It can happen that the originator of a crash report doesn't have access
to certain images. When that's the case, ReportCrash w

[lldb/crashlog] Fix frame parser regex for when there is no source info

It can happen that the originator of a crash report doesn't have access
to certain images. When that's the case, ReportCrash won't show the
source info in the crash report stack frames, but only the stack address
and image name.

This patch fixes a bug in the crashlog stackframe parser regular
expression to optionally match the source info group.

rdar://101934135

Differential Revision: https://reviews.llvm.org/D137466

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Add support for Application Specific Backtraces & Information

For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Appli

[lldb/crashlog] Add support for Application Specific Backtraces & Information

For an exception crashlog, the thread backtraces aren't usually very helpful
and instead, developpers look at the "Application Specific Backtrace" that
was generated by `objc_exception_throw`.

LLDB could already parse and symbolicate these Application Specific Backtraces
for regular textual-based crashlog, so this patch adds support to parse them
in JSON crashlogs, and materialize them a HistoryThread extending the
crashed ScriptedThread.

This patch also includes the Application Specific Information messages
as part of the process extended crash information log. To do so, the
ScriptedProcess Python interface has a new GetMetadata method that
returns an arbitrary dictionary with data related to the process.

rdar://93207586

Differential Revision: https://reviews.llvm.org/D126260

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Fix the image_regex_uuid to skip null UUID images

This patch updates the image_regex_uuid matcher to match null-UUID
images in the plain text crashlog parser.

It updates the regex t

[lldb/crashlog] Fix the image_regex_uuid to skip null UUID images

This patch updates the image_regex_uuid matcher to match null-UUID
images in the plain text crashlog parser.

It updates the regex to match one or more '?' characters or the image
full path.

rdar://100904019

Differential Revision: https://reviews.llvm.org/D135482

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/crashlog] Add support for 32bit frame addresses

This patch adds support for 32bit stack frame addresses in the `crashlog`
command.

For crash reports that are generated from a arm64_32 process

[lldb/crashlog] Add support for 32bit frame addresses

This patch adds support for 32bit stack frame addresses in the `crashlog`
command.

For crash reports that are generated from a arm64_32 process, `PAGEZERO`
is loaded at 0x00004000 so no code address will be less than 0x4000.

This patch changes the crashlog frame address regex group to match
addresses as small as 4 hex characters.

rdar://100805026

Differential Revision: https://reviews.llvm.org/D135310

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb] Explicitly open file to write with utf-8 encoding in crashlog.py

The python "open" function will use the default encoding for the
locale (the result of "locale.getpreferredencoding()"). Expli

[lldb] Explicitly open file to write with utf-8 encoding in crashlog.py

The python "open" function will use the default encoding for the
locale (the result of "locale.getpreferredencoding()"). Explicitly set
the locale to utf-8 when opening the crashlog for writing, as there may
be non-ascii symbols in there (for example, Swift uses "τ" to indicate
generic parameters).

rdar://101402755

Differential Revision: https://reviews.llvm.org/D136798

show more ...

[lldb/crashlog] Adapt raw text crashlog exception to json format

This patch parses CrashLog exception data from the raw
text format and adapts it to the new JSON format.

This is necessary for featu

[lldb/crashlog] Adapt raw text crashlog exception to json format

This patch parses CrashLog exception data from the raw
text format and adapts it to the new JSON format.

This is necessary for feature parity between the 2 formats.

Differential Revision: https://reviews.llvm.org/D131719

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb] Tidy some regex in crashlog.py (NFC)

A spiritual follow up to D131032. I noticed some regex could be simplified.

This does some of the following:
1. Removes unused capture groups
2. Uses non

[lldb] Tidy some regex in crashlog.py (NFC)

A spiritual follow up to D131032. I noticed some regex could be simplified.

This does some of the following:
1. Removes unused capture groups
2. Uses non-capturing `(?:...)` groups where grouping is needed but capturing isn't
3. Removes trailing `.*`
4. Uses `\d` over `[0-9]`
5. Uses raw strings
6. Uses `{N,}` to indicate N-or-more

Also improves the call site of a `re.findall`.

Differential Revision: https://reviews.llvm.org/D131305

show more ...