[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optiona

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the am

[lldb/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb/Plugins] Improve error reporting with reading memory in Scripted Process

This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python im

[lldb/Plugins] Improve error reporting with reading memory in Scripted Process

This patch improves the ScriptedPythonInterface::Dispatch method to
support passing lldb_private types to the python implementation.

This will allow, for instance, the Scripted Process python implementation
to report errors when reading memory back to lldb.

To do so, the Dispatch method will transform the private types in the
parameter pack into `PythonObject`s to be able to pass them down to the
python methods.

Then, if the call succeeded, the transformed arguments will be converted
back to their original type and re-assigned in the parameter pack, to
ensure pointers and references behaviours are preserved.

This patch also updates various scripted process python class and tests
to reflect this change.

rdar://100030995

Differential Revision: https://reviews.llvm.org/D134033

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb] Add matching based on Python callbacks for data formatters.

This patch adds a new matching method for data formatters, in addition
to the existing exact typename and regex-based matching. The

[lldb] Add matching based on Python callbacks for data formatters.

This patch adds a new matching method for data formatters, in addition
to the existing exact typename and regex-based matching. The new method
allows users to specify the name of a Python callback function that
takes a `SBType` object and decides whether the type is a match or not.

Here is an overview of the changes performed:

- Add a new `eFormatterMatchCallback` matching type, and logic to handle
it in `TypeMatcher` and `SBTypeNameSpecifier`.

- Extend `FormattersMatchCandidate` instances with a pointer to the
current `ScriptInterpreter` and the `TypeImpl` corresponding to the
candidate type, so we can run registered callbacks and pass the type
to them. All matcher search functions now receive a
`FormattersMatchCandidate` instead of a type name.

- Add some glue code to ScriptInterpreterPython and the SWIG bindings to
allow calling a formatter matching callback. Most of this code is
modeled after the equivalent code for watchpoint callback functions.

- Add an API test for the new callback-based matching feature.

For more context, please check the RFC thread where this feature was
originally discussed:
https://discourse.llvm.org/t/rfc-python-callback-for-data-formatters-type-matching/64204/11

Differential Revision: https://reviews.llvm.org/D135648

show more ...

Remove Python 2 support from the ScriptInterpreter plugin

We dropped downstream support for Python 2 in the previous release. Now
that we have branched for the next release the window where this kin

Remove Python 2 support from the ScriptInterpreter plugin

We dropped downstream support for Python 2 in the previous release. Now
that we have branched for the next release the window where this kind of
change could introduce conflicts is closing too. Start by getting rid of
Python 2 support in the Script Interpreter plugin.

Differential revision: https://reviews.llvm.org/D124429

show more ...

[lldb] Initialize Python exactly once

We got a few crash reports that showed LLDB initializing Python on two
separate threads. Make sure Python is initialized exactly once.

rdar://87287005

Differe

[lldb] Initialize Python exactly once

We got a few crash reports that showed LLDB initializing Python on two
separate threads. Make sure Python is initialized exactly once.

rdar://87287005

Differential revision: https://reviews.llvm.org/D117601

show more ...

[lldb/python] Use PythonObject in LLDBSwigPython functions

Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python

[lldb/python] Use PythonObject in LLDBSwigPython functions

Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python
objects is automatically tracked.

Differential Revision: https://reviews.llvm.org/D117462

show more ...

[lldb/python] Fix (some) dangling pointers in our glue code

This starts to fix the other half of the lifetime problems in this code
-- dangling references. SB objects created on the stack will go aw

[lldb/python] Fix (some) dangling pointers in our glue code

This starts to fix the other half of the lifetime problems in this code
-- dangling references. SB objects created on the stack will go away
when the function returns, which is a problem if the python code they
were meant for stashes a reference to them somewhere. Most of the time
this goes by unnoticed, as the code rarely has a reason to store these,
but in case it does, we shouldn't respond by crashing.

This patch fixes the management for a couple of SB objects (Debugger,
Frame, Thread). The SB objects are now created on the heap, and
their ownership is immediately passed on to SWIG, which will ensure they
are destroyed when the last python reference goes away. I will handle
the other objects in separate patches.

I include one test which demonstrates the lifetime issue for SBDebugger.
Strictly speaking, one should create a test case for each of these
objects and each of the contexts they are being used. That would require
figuring out how to persist (and later access) each of these objects.
Some of those may involve a lot of hoop-jumping (we can run python code
from within a frame-format string). I don't think that is
necessary/worth it since the new wrapper functions make it very hard to
get this wrong.

Differential Revision: https://reviews.llvm.org/D115925

show more ...

[lldb] Clarify StructuredDataImpl ownership

StructuredDataImpl ownership semantics is unclear at best. Various
structures were holding a non-owning pointer to it, with a comment that
the object is o

[lldb] Clarify StructuredDataImpl ownership

StructuredDataImpl ownership semantics is unclear at best. Various
structures were holding a non-owning pointer to it, with a comment that
the object is owned somewhere else. From what I was able to gather that
"somewhere else" was the SBStructuredData object, but I am not sure that
all created object eventually made its way there. (It wouldn't matter
even if they did, as we are leaking most of our SBStructuredData
objects.)

Since StructuredDataImpl is just a collection of two (shared) pointers,
there's really no point in elaborate lifetime management, so this patch
replaces all StructuredDataImpl pointers with actual objects or
unique_ptrs to it. This makes it much easier to resolve SBStructuredData
leaks in a follow-up patch.

Differential Revision: https://reviews.llvm.org/D114791

show more ...

[lldb] Remove 'extern "C"' from the lldb-swig-python interface

The LLDBSWIGPython functions had (at least) two problems:
- There wasn't a single source of truth (a header file) for the
prototypes

[lldb] Remove 'extern "C"' from the lldb-swig-python interface

The LLDBSWIGPython functions had (at least) two problems:
- There wasn't a single source of truth (a header file) for the
prototypes of these functions. This meant that subtle differences
in copies of function declarations could go by undetected. And
not-so-subtle differences would result in strange runtime failures.
- All of the declarations had to have an extern "C" interface, because
the function definitions were being placed inside and extert "C" block
generated by swig.

This patch fixes both problems by moving the function definitions to the
%header block of the swig files. This block is not surrounded by extern
"C", and seems more appropriate anyway, as swig docs say it is meant for
"user-defined support code" (whereas the previous %wrapper code was for
automatically-generated wrappers).

It also puts the declarations into the SWIGPythonBridge header file
(which seems to have been created for this purpose), and ensures it is
included by all code wishing to define or use these functions. This
means that any differences in the declaration become a compiler error
instead of a runtime failure.

Differential Revision: https://reviews.llvm.org/D114369

show more ...

[lldb] Fix [some] leaks in python bindings

Using an lldb_private object in the bindings involves three steps
- wrapping the object in it's lldb::SB variant
- using swig to convert/wrap that to a PyO

[lldb] Fix [some] leaks in python bindings

Using an lldb_private object in the bindings involves three steps
- wrapping the object in it's lldb::SB variant
- using swig to convert/wrap that to a PyObject
- wrapping *that* in a lldb_private::python::PythonObject

Our SBTypeToSWIGWrapper was only handling the middle part. This doesn't
just result in increased boilerplate in the callers, but is also a
functionality problem, as it's very hard to get the lifetime of of all
of these objects right. Most of the callers are creating the SB object
(step 1) on the stack, which means that we end up with dangling python
objects after the function terminates. Most of the time this isn't a
problem, because the python code does not need to persist the objects.
However, there are legitimate cases where they can do it (and even if
the use case is not completely legitimate, crashing is not the best
response to that).

For this reason, some of our code creates the SB object on the heap, but
it has another problem -- it never gets cleaned up.

This patch begins to add a new function (ToSWIGWrapper), which does all
of the three steps, while properly taking care of ownership. In the
first step, I have converted most of the leaky code (except for
SBStructuredData, which needs a bit more work).

Differential Revision: https://reviews.llvm.org/D114259

show more ...

[lldb/bindings] Change ScriptedThread initializer parameters

This patch changes the `ScriptedThread` initializer in couple of ways:
- It replaces the `SBTarget` parameter by a `SBProcess` (pointing

[lldb/bindings] Change ScriptedThread initializer parameters

This patch changes the `ScriptedThread` initializer in couple of ways:
- It replaces the `SBTarget` parameter by a `SBProcess` (pointing to the
`ScriptedProcess` that "owns" the `ScriptedThread`).
- It adds a reference to the `ScriptedProcessInfo` Dictionary, to pass
arbitrary user-input to the `ScriptedThread`.

This patch also fixes the SWIG bindings methods that call the
`ScriptedProcess` and `ScriptedThread` initializers by passing all the
arguments to the appropriate `PythonCallable` object.

Differential Revision: https://reviews.llvm.org/D112046

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/Plugins] Add memory region support in ScriptedProcess

This patch adds support for memory regions in Scripted Processes.
This is necessary to read the stack memory region in order to
reconstruc

[lldb/Plugins] Add memory region support in ScriptedProcess

This patch adds support for memory regions in Scripted Processes.
This is necessary to read the stack memory region in order to
reconstruct each stackframe of the program.

In order to do so, this patch makes some changes to the SBAPI, namely:
- Add a new constructor for `SBMemoryRegionInfo` that takes arguments
such as the memory region name, address range, permissions ...
This is used when reading memory at some address to compute the offset
in the binary blob provided by the user.
- Add a `GetMemoryRegionContainingAddress` method to `SBMemoryRegionInfoList`
to simplify the access to a specific memory region.

With these changes, lldb is now able to unwind the stack and reconstruct
each frame. On top of that, reloading the target module at offset 0 allows
lldb to symbolicate the `ScriptedProcess` using debug info, similarly to an
ordinary Process.

To test this, I wrote a simple program with multiple function calls, ran it in
lldb, stopped at a leaf function and read the registers values and copied
the stack memory into a binary file. These are then used in the python script.

Differential Revision: https://reviews.llvm.org/D108953

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/Plugins] Add support for ScriptedThread in ScriptedProcess

This patch introduces the `ScriptedThread` class with its python
interface.

When used with `ScriptedProcess`, `ScriptedThreaad` can

[lldb/Plugins] Add support for ScriptedThread in ScriptedProcess

This patch introduces the `ScriptedThread` class with its python
interface.

When used with `ScriptedProcess`, `ScriptedThreaad` can provide various
information such as the thread state, stop reason or even its register
context.

This can be used to reconstruct the program stack frames using lldb's unwinder.

rdar://74503836

Differential Revision: https://reviews.llvm.org/D107585

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/Plugins] Introduce Scripted Interface Factory

This patch splits the previous `ScriptedProcessPythonInterface` into
multiple specific classes:

1. The `ScriptedInterface` abstract class that ca

[lldb/Plugins] Introduce Scripted Interface Factory

This patch splits the previous `ScriptedProcessPythonInterface` into
multiple specific classes:

1. The `ScriptedInterface` abstract class that carries the interface
instance object and its virtual pure abstract creation method.

2. The `ScriptedPythonInterface` that holds a generic `Dispatch` method that
can be used by various interfaces to call python methods and also keeps a
reference to the Python Script Interpreter instance.

3. The `ScriptedProcessInterface` that describes the base Scripted
Process model with all the methods used in the underlying script.

All these components are used to refactor the `ScriptedProcessPythonInterface`
class, making it more modular.

This patch is also a requirement for the upcoming work on `ScriptedThread`.

Differential Revision: https://reviews.llvm.org/D107521

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/Interpreter] Add ScriptInterpreter Wrapper for ScriptedProcess

This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.

This

[lldb/Interpreter] Add ScriptInterpreter Wrapper for ScriptedProcess

This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.

This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.

At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.

This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).

rdar://65508855

Differential Revision: https://reviews.llvm.org/D95711

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb] Revert ScriptedProcess patches

This patch reverts the following commits:
- 5a9c34918bb1526b7e8c29aa5e4fb8d8e27e27b4
- 46796762afe76496ec4dd900f64d0cf4cdc30e99
- 2cff3dec1171188ce04ab1a4373cc1

[lldb] Revert ScriptedProcess patches

This patch reverts the following commits:
- 5a9c34918bb1526b7e8c29aa5e4fb8d8e27e27b4
- 46796762afe76496ec4dd900f64d0cf4cdc30e99
- 2cff3dec1171188ce04ab1a4373cc1885ab97be1
- 182f0d1a34419445bb19d67581d6ac1afc98b7fa
- d62a53aaf1d38a55d1affbd3a30d564a4e9d3171

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

[lldb/Interpreter] Add ScriptInterpreter Wrapper for ScriptedProcess

This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.

This

[lldb/Interpreter] Add ScriptInterpreter Wrapper for ScriptedProcess

This patch adds a ScriptedProcess interface to the ScriptInterpreter and
more specifically, to the ScriptInterpreterPython.

This interface will be used in the C++ `ScriptProcess` Process Plugin to
call the script methods.

At the moment, not all methods are implemented, they will upstreamed in
upcoming patches.

This patch also adds helper methods to the ScriptInterpreter to
convert `SBAPI` Types (SBData & SBError) to `lldb_private` types
(DataExtractor & Status).

rdar://65508855

Differential Revision: https://reviews.llvm.org/D95711

Signed-off-by: Med Ismail Bennani <medismail.bennani@gmail.com>

show more ...

Revert "Revert "Add the ability to write target stop-hooks using the ScriptInterpreter.""

This reverts commit f775fe59640a2e837ad059a8f40e26989d4f9831.

I fixed a return type error in the original p

Revert "Revert "Add the ability to write target stop-hooks using the ScriptInterpreter.""

This reverts commit f775fe59640a2e837ad059a8f40e26989d4f9831.

I fixed a return type error in the original patch that was causing a test failure.
Also added a REQUIRES: python to the shell test so we'll skip this for
people who build lldb w/o Python.
Also added another test for the error printing.

show more ...

Revert "Add the ability to write target stop-hooks using the ScriptInterpreter."

This temporarily reverts commit b65966cff65bfb66de59621347ffd97238d3f645
while Jim figures out why the test is failin

Revert "Add the ability to write target stop-hooks using the ScriptInterpreter."

This temporarily reverts commit b65966cff65bfb66de59621347ffd97238d3f645
while Jim figures out why the test is failing on the bots.

show more ...

Add the ability to write target stop-hooks using the ScriptInterpreter.

Differential Revision: https://reviews.llvm.org/D88123

[lldb/ScriptInterpreter] Fix Windows error C2371: 'pid_t': redefinition

pyconfig.h(194): error C2371: 'pid_t': redefinition; different basic types
PosixApi.h(82): note: see declaration of 'pid_t'

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp ----------------------------------------

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).

This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).

Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere

Reviewed By: JDevlieghere

Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73258

show more ...

[LLDB] Disable MSVC warning C4190: 'LLDBSwigPythonBreakpointCallbackFunction' has C-linkage specified, but returns UDT 'llvm::Expected<bool>' which is incompatible with C

Differential Revision: http

[LLDB] Disable MSVC warning C4190: 'LLDBSwigPythonBreakpointCallbackFunction' has C-linkage specified, but returns UDT 'llvm::Expected<bool>' which is incompatible with C

Differential Revision: https://reviews.llvm.org/D70830

show more ...

[LLDB][Python] warning fix for LLDBSwigPythonBreakpointCallbackFunction

This is a quick followup to this commit:

https://reviews.llvm.org/rGa69bbe02a2352271e8b14542073f177e24c499c1

In that, I #pra

[LLDB][Python] warning fix for LLDBSwigPythonBreakpointCallbackFunction

This is a quick followup to this commit:

https://reviews.llvm.org/rGa69bbe02a2352271e8b14542073f177e24c499c1

In that, I #pragma-squelch this warning in `ScriptInterpreterPython.cpp`
but we get the same warning in `PythonTestSuite.cpp`.

This patch squelches the same warning in the same way as the
reviweed commit. I'm submitting it without review under the
"obviously correct" rule.

At least if this is incorrect the main commit was also incorrect.

By the way, as far as I can tell, these functions are extern "C" because
SWIG does that to everything, not because they particularly need to be.

show more ...