[lldb] Make PythonDataObjects work with Python 2

I considered keeping this change strictly downstream. Since we still
have a bunch of places that check for Python 2, I figured it doesn't
harm to lan

[lldb] Make PythonDataObjects work with Python 2

I considered keeping this change strictly downstream. Since we still
have a bunch of places that check for Python 2, I figured it doesn't
harm to land it upstream and avoid the conflict when I eventually do
remove them (hopefully soon!).

show more ...

[lldb] Fix D114722 for python<=3.6

_Py_IsFinalizing was called _Py_Finalizing back then (and it was a
variable instead of a function).

[lldb/python] Use PythonObject in LLDBSwigPython functions

Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python

[lldb/python] Use PythonObject in LLDBSwigPython functions

Return our PythonObject wrappers instead of raw PyObjects (obfuscated as
void *). This ensures that ownership (reference counts) of python
objects is automatically tracked.

Differential Revision: https://reviews.llvm.org/D117462

show more ...

[LLDB] Fix Python GIL-not-held issues

The GIL must be held when calling any Python C API functions. In multithreaded applications that use callbacks this requirement can easily be violated by accide

[LLDB] Fix Python GIL-not-held issues

The GIL must be held when calling any Python C API functions. In multithreaded applications that use callbacks this requirement can easily be violated by accident. A general tool to ensure GIL health is not available, but patching Python Py_INCREF to add an assert provides a basic health check:

```
+int PyGILState_Check(void); /* Include/internal/pystate.h */
+
#define Py_INCREF(op) ( \
+ assert(PyGILState_Check()), \
_Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
((PyObject *)(op))->ob_refcnt++)

#define Py_DECREF(op) \
do { \
+ assert(PyGILState_Check()); \
PyObject *_py_decref_tmp = (PyObject *)(op); \
if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \
--(_py_decref_tmp)->ob_refcnt != 0) \
```

Adding this assertion causes around 50 test failures in LLDB. Adjusting the scope of things guarded by `py_lock` fixes them.

More background: https://docs.python.org/3/glossary.html#term-global-interpreter-lock

Patch by Ralf Grosse-Kunstleve

Differential Revision: https://reviews.llvm.org/D114722

show more ...

[lldb] make it easier to find LLDB's python

It is surprisingly difficult to write a simple python script that
can reliably `import lldb` without failing, or crashing. I'm
currently resorting to co

[lldb] make it easier to find LLDB's python

It is surprisingly difficult to write a simple python script that
can reliably `import lldb` without failing, or crashing. I'm
currently resorting to convolutions like this:

def find_lldb(may_reexec=False):
if prefix := os.environ.get('LLDB_PYTHON_PREFIX'):
if os.path.realpath(prefix) != os.path.realpath(sys.prefix):
raise Exception("cannot import lldb.\n"
f" sys.prefix should be: {prefix}\n"
f" but it is: {sys.prefix}")
else:
line1, line2 = subprocess.run(
['lldb', '-x', '-b', '-o', 'script print(sys.prefix)'],
encoding='utf8', stdout=subprocess.PIPE,
check=True).stdout.strip().splitlines()
assert line1.strip() == '(lldb) script print(sys.prefix)'
prefix = line2.strip()
os.environ['LLDB_PYTHON_PREFIX'] = prefix

if sys.prefix != prefix:
if not may_reexec:
raise Exception(
"cannot import lldb.\n" +
f" This python, at {sys.prefix}\n"
f" does not math LLDB's python at {prefix}")
os.environ['LLDB_PYTHON_PREFIX'] = prefix
python_exe = os.path.join(prefix, 'bin', 'python3')
os.execl(python_exe, python_exe, *sys.argv)

lldb_path = subprocess.run(['lldb', '-P'],
check=True, stdout=subprocess.PIPE,
encoding='utf8').stdout.strip()

sys.path = [lldb_path] + sys.path

This patch aims to replace all that with:

#!/usr/bin/env lldb-python
import lldb
...

... by adding the following features:

* new command line option: --print-script-interpreter-info. This
prints language-specific information about the script interpreter
in JSON format.

* new tool (unix only): lldb-python which finds python and exec's it.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D112973

show more ...

[lldb] [gdb-remote] Add eOpenOptionReadWrite for future gdb compat

Modify OpenOptions enum to open the future path into synchronizing
vFile:open bits with GDB. Currently, LLDB and GDB use different

[lldb] [gdb-remote] Add eOpenOptionReadWrite for future gdb compat

Modify OpenOptions enum to open the future path into synchronizing
vFile:open bits with GDB. Currently, LLDB and GDB use different flag
models effectively making it impossible to match bits. Notably, LLDB
uses two bits to indicate read and write status, and uses union of both
for read/write. GDB uses a value of 0 for read-only, 1 for write-only
and 2 for read/write.

In order to future-proof the code for the GDB variant:

1. Add a distinct eOpenOptionReadWrite constant to be used instead
of (eOpenOptionRead | eOpenOptionWrite) when R/W access is required.

2. Rename eOpenOptionRead and eOpenOptionWrite to eOpenOptionReadOnly
and eOpenOptionWriteOnly respectively, to make it clear that they
do not mean to be combined and require update to all call sites.

3. Use the intersection of all three flags when matching against
the three possible values.

This commit does not change the actual bits used by LLDB.

Differential Revision: https://reviews.llvm.org/D106984

show more ...

[lldb][NFC] Use C++ versions of the deprecated C standard library headers

The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivale

[lldb][NFC] Use C++ versions of the deprecated C standard library headers

The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivalent.

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D103084

show more ...

[lldb] Use PyUnicode_GetLength instead of PyUnicode_GetSize

PyUnicode_GetSize is deprecated since Python version 3.3.

Re-land "get rid of PythonInteger::GetInteger()"

This was reverted due to a python2-specific bug. Re-landing with a fix
for python2.

Summary:
One small step in my long running quest to improve pyt

Re-land "get rid of PythonInteger::GetInteger()"

This was reverted due to a python2-specific bug. Re-landing with a fix
for python2.

Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions

Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn, omjavaid

Reviewed By: labath, omjavaid

Subscribers: omjavaid, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D78462

show more ...

Revert "get rid of PythonInteger::GetInteger()"

This reverts commit 7375212172951d2fc283c81d03c1a8588c3280c6.

This causes multiple test failures on LLDB AArch64 Linux buildbot.
http://lab.llvm.org:

Revert "get rid of PythonInteger::GetInteger()"

This reverts commit 7375212172951d2fc283c81d03c1a8588c3280c6.

This causes multiple test failures on LLDB AArch64 Linux buildbot.
http://lab.llvm.org:8011/builders/lldb-aarch64-ubuntu/builds/3695

Differential Revision: https://reviews.llvm.org/D78462

show more ...

get rid of PythonInteger::GetInteger()

Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long lon

get rid of PythonInteger::GetInteger()

Summary:
One small step in my long running quest to improve python exception handling in
LLDB. Replace GetInteger() which just returns an int with As<long long> and
friends, which return Expected types that can track python exceptions

Reviewers: labath, jasonmolenda, JDevlieghere, vadimcn

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D78462

show more ...

[lldb] NFC: Fix trivial typo in comments, documents, and messages

Differential Revision: https://reviews.llvm.org/D77460

Fix implicit conversion in the lldb Python plugin

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp ----------------------------------------

[lldb][NFC] Fix all formatting errors in .cpp file headers

Summary:
A *.cpp file header in LLDB (and in LLDB) should like this:
```
//===-- TestUtilities.cpp -------------------------------------------------===//
```
However in LLDB most of our source files have arbitrary changes to this format and
these changes are spreading through LLDB as folks usually just use the existing
source files as templates for their new files (most notably the unnecessary
editor language indicator `-*- C++ -*-` is spreading and in every review
someone is pointing out that this is wrong, resulting in people pointing out that this
is done in the same way in other files).

This patch removes most of these inconsistencies including the editor language indicators,
all the different missing/additional '-' characters, files that center the file name, missing
trailing `===//` (mostly caused by clang-format breaking the line).

Reviewers: aprantl, espindola, jfb, shafik, JDevlieghere

Reviewed By: JDevlieghere

Subscribers: dexonsmith, wuzish, emaste, sdardis, nemanjai, kbarton, MaskRay, atanasyan, arphaman, jfb, abidh, jsji, JDevlieghere, usaxena95, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D73258

show more ...

[lldb/CMake] Rename LLDB_DISABLE_PYTHON to LLDB_ENABLE_PYTHON

This matches the naming scheme used by LLVM and all the other optional
dependencies in LLDB.

Differential revision: https://reviews.llv

[lldb/CMake] Rename LLDB_DISABLE_PYTHON to LLDB_ENABLE_PYTHON

This matches the naming scheme used by LLVM and all the other optional
dependencies in LLDB.

Differential revision: https://reviews.llvm.org/D71482

show more ...

[lldb/Host] Use Host/Config.h entries instead of a global define.

As suggested by Pavel in a code review:

> Can we replace this (and maybe python too, while at it) with a
> Host/Config.h entry? A g

[lldb/Host] Use Host/Config.h entries instead of a global define.

As suggested by Pavel in a code review:

> Can we replace this (and maybe python too, while at it) with a
> Host/Config.h entry? A global definition means that one has to
> recompile everything when these change in any way, whereas in
> practice only a handful of files need this..

Differential revision: https://reviews.llvm.org/D71280

show more ...

[LLDB][Python] remove ArgInfo::count

Summary:
This patch updates the last user of ArgInfo::count and deletes
it. I also delete `GetNumInitArguments()` and `GetInitArgInfo()`.
Classess are callable

[LLDB][Python] remove ArgInfo::count

Summary:
This patch updates the last user of ArgInfo::count and deletes
it. I also delete `GetNumInitArguments()` and `GetInitArgInfo()`.
Classess are callables and `GetArgInfo()` should work on them.

On python 3 it already works, of course. `inspect` is good.

On python 2 we have to add yet another special case. But hey if
python 2 wasn't crufty we wouln't need python 3.

I also delete `is_bound_method` becuase it is unused.

This path is tested in `TestStepScripted.py`

Reviewers: labath, mgorny, JDevlieghere

Reviewed By: labath, JDevlieghere

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69742

show more ...

[LLDB][PythonFile] fix dangerous borrow semantics on python2

Summary:
It is inherently unsafe to allow a python program to manipulate borrowed
memory from a python object's destructor. It would

[LLDB][PythonFile] fix dangerous borrow semantics on python2

Summary:
It is inherently unsafe to allow a python program to manipulate borrowed
memory from a python object's destructor. It would be nice to
flush a borrowed file when python is finished with it, but it's not safe
to do on python 2.

Python 3 does not suffer from this issue.

Reviewers: labath, mgorny

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69532

show more ...

[LLDB][Python] fix another fflush issue on NetBSD

Summary:
Here's another instance where we were calling fflush on an input
stream, which is illegal on NetBSD.

Reviewers: labath, mgorny

Reviewed B

[LLDB][Python] fix another fflush issue on NetBSD

Summary:
Here's another instance where we were calling fflush on an input
stream, which is illegal on NetBSD.

Reviewers: labath, mgorny

Reviewed By: mgorny

Subscribers: krytarowski, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69488

show more ...

[lldb] [Python] Do not attempt to flush() a read-only fd

Summary:
When creating a FileSP object, do not flush() the underlying file unless
it is open for writing. Attempting to flush() a read-only

[lldb] [Python] Do not attempt to flush() a read-only fd

Summary:
When creating a FileSP object, do not flush() the underlying file unless
it is open for writing. Attempting to flush() a read-only fd results
in EBADF on NetBSD.

Reviewers: lawrence_danna, labath, krytarowski

Reviewed By: lawrence_danna, labath

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D69320

show more ...

remove multi-argument form of PythonObject::Reset()

Summary:
With this patch, only the no-argument form of `Reset()` remains in
PythonDataObjects. It also deletes PythonExceptionState in favor of

remove multi-argument form of PythonObject::Reset()

Summary:
With this patch, only the no-argument form of `Reset()` remains in
PythonDataObjects. It also deletes PythonExceptionState in favor of
PythonException, because the only call-site of PythonExceptionState was
also using Reset, so I cleaned up both while I was there.

Reviewers: JDevlieghere, clayborg, labath, jingham

Reviewed By: labath

Subscribers: mgorny, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69214

llvm-svn: 375475

show more ...

eliminate nontrivial Reset(...) from TypedPythonObject

Summary:
This deletes `Reset(...)`, except for the no-argument form `Reset()`
from `TypedPythonObject`, and therefore from `PythonString`, `Pyt

eliminate nontrivial Reset(...) from TypedPythonObject

Summary:
This deletes `Reset(...)`, except for the no-argument form `Reset()`
from `TypedPythonObject`, and therefore from `PythonString`, `PythonList`,
etc.

It updates the various callers to use assignment, `As<>`, `Take<>`,
and `Retain<>`, as appropriate.

followon to https://reviews.llvm.org/D69080

Reviewers: JDevlieghere, clayborg, labath, jingham

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69133

llvm-svn: 375350

show more ...

[LLDB] bugfix: command script add -f doesn't work for some callables

Summary:
When users define a debugger command from python, they provide a callable
object. Because the signature of the functio

[LLDB] bugfix: command script add -f doesn't work for some callables

Summary:
When users define a debugger command from python, they provide a callable
object. Because the signature of the function has been extended, LLDB
needs to inspect the number of parameters the callable can take.

The rule it was using to decide was weird, apparently not tested, and
giving wrong results for some kinds of python callables.

This patch replaces the weird rule with a simple one: if the callable can
take 5 arguments, it gets the 5 argument version of the signature.
Otherwise it gets the old 4 argument version.

It also adds tests with a bunch of different kinds of python callables
with both 4 and 5 arguments.

Reviewers: JDevlieghere, clayborg, labath, jingham

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69014

llvm-svn: 375333

show more ...

eliminate one form of PythonObject::Reset()

Summary:
I'd like to eliminate all forms of Reset() and all public constructors
on these objects, so the only way to make them is with Take<> and Retain<>

eliminate one form of PythonObject::Reset()

Summary:
I'd like to eliminate all forms of Reset() and all public constructors
on these objects, so the only way to make them is with Take<> and Retain<>
and the only way to copy or move them is with actual c++ copy, move, or
assignment.

This is a simple place to start.

Reviewers: JDevlieghere, clayborg, labath, jingham

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D69080

llvm-svn: 375182

show more ...

clean up the implementation of PythonCallable::GetNumArguments

Summary:
The current implementation of PythonCallable::GetNumArguments
is not exception safe, has weird semantics, and is just plain
in

clean up the implementation of PythonCallable::GetNumArguments

Summary:
The current implementation of PythonCallable::GetNumArguments
is not exception safe, has weird semantics, and is just plain
incorrect for some kinds of functions.

Python 3.3 introduces inspect.signature, which lets us easily
query for function signatures in a sane and documented way.

This patch leaves the old implementation in place for < 3.3,
but uses inspect.signature for modern pythons. It also leaves
the old weird semantics in place, but with FIXMEs grousing about
it. We should update the callers and fix the semantics in a
subsequent patch. It also adds some tests.

Reviewers: JDevlieghere, clayborg, labath, jingham

Reviewed By: labath

Subscribers: lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D68995

llvm-svn: 375181

show more ...