Revert "[lldb] Implement basic support for reverse-continue" (#123906)

Reverts llvm/llvm-project#112079 due to failures on the arm bot.

[lldb] Implement basic support for reverse-continue (#112079)

This commit adds support for a
`SBProcess::ContinueInDirection()` API. A user-accessible command for
this will follow in a later commit.

[lldb] Implement basic support for reverse-continue (#112079)

This commit adds support for a
`SBProcess::ContinueInDirection()` API. A user-accessible command for
this will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. For testing
purposes, this commit adds a Python implementation of *very limited*
record-and-reverse-execute functionality, implemented as a proxy between
lldb and lldb-server in `lldbreverse.py`. This should not (and in
practice cannot) be used for anything except testing.

The tests here are quite minimal but we test that simple breakpoints and
watchpoints work as expected during reverse execution, and that
conditional breakpoints and watchpoints work when the condition calls a
function that must be executed in the forward direction.

show more ...

[lldb][Linux] Mark memory regions used for shadow stacks (#117861)

This is intended for use with Arm's Guarded Control Stack extension
(GCS). Which reuses some existing shadow stack support in Linu

[lldb][Linux] Mark memory regions used for shadow stacks (#117861)

This is intended for use with Arm's Guarded Control Stack extension
(GCS). Which reuses some existing shadow stack support in Linux. It
should also work with the x86 equivalent.

A "ss" flag is added to the "VmFlags" line of shadow stack memory
regions in `/proc/<pid>/smaps`. To keep the naming generic I've called
it shadow stack instead of guarded control stack.

Also the wording is "shadow stack: yes" because the shadow stack region
is just where it's stored. It's enabled for the whole process or it
isn't. As opposed to memory tagging which can be enabled per region, so
"memory tagging: enabled" fits better for that.

I've added a test case that is also intended to be the start of a set of
tests for GCS. This should help me avoid duplicating the inline assembly
needed.

Note that no special compiler support is needed for the test. However,
for the intial enabling of GCS (assuming the libc isn't doing it) we do
need to use an inline assembly version of prctl.

This is because as soon as you enable GCS, all returns are checked
against the GCS. If the GCS is empty, the program will fault. In other
words, you can never return from the function that enabled GCS, unless
you push values onto it (which is possible but not needed here).

So you cannot use the libc's prctl wrapper for this reason. You can use
that wrapper for anything else, as we do to check if GCS is enabled.

show more ...

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

Reverting this again; I added a commit which added @skipIfDarwin
markers to the TestReverseContinueBreakpoints.py and
TestRevers

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

Reverting this again; I added a commit which added @skipIfDarwin
markers to the TestReverseContinueBreakpoints.py and
TestReverseContinueNotSupported.py API tests, which use lldb-server
in gdbserver mode which does not work on Darwin. But the aarch64 ubuntu
bot reported a failure on TestReverseContinueBreakpoints.py,
https://lab.llvm.org/buildbot/#/builders/59/builds/6397

File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/reverse-execution/TestReverseContinueBreakpoints.py", line 63, in test_reverse_continue_skip_breakpoint
self.reverse_continue_skip_breakpoint_internal(async_mode=False)
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/reverse-execution/TestReverseContinueBreakpoints.py", line 81, in reverse_continue_skip_breakpoint_internal
self.expect(
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2372, in expect
self.runCmd(
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1002, in runCmd
self.assertTrue(self.res.Succeeded(), msg + output)
AssertionError: False is not true : Process should be stopped due to history boundary
Error output:
error: Process must be launched.

This reverts commit 4f297566b3150097de26c6a23a987d2bd5fc19c5.

show more ...

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later com

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. So, for testing
purposes, `lldbreverse.py` wraps `lldb-server` with a Python
implementation of *very limited* record-and-replay functionality for use
by *tests only*.

The majority of this PR is test infrastructure (about 700 of the 950
lines added).

show more ...

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

This reverts commit d5e1de6da96c1ab3b8cae68447e8ed3696a7006e.

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later com

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. So, for testing
purposes, `lldbreverse.py` wraps `lldb-server` with a Python
implementation of *very limited* record-and-replay functionality for use
by *tests only*.

The majority of this PR is test infrastructure (about 700 of the 950
lines added).

show more ...

[lldb] Make conversions from llvm::Error explicit with Status::FromEr… (#107163)

…ror() [NFC]

[lldb/linux] Make truncated reads work (#106532)

Previously, we were returning an error if we couldn't read the whole
region. This doesn't matter most of the time, because lldb caches memory
reads

[lldb/linux] Make truncated reads work (#106532)

Previously, we were returning an error if we couldn't read the whole
region. This doesn't matter most of the time, because lldb caches memory
reads, and in that process it aligns them to cache line boundaries. As
(LLDB) cache lines are smaller than pages, the reads are unlikely to
cross page boundaries.

Nonetheless, this can cause a problem for large reads (which bypass the
cache), where we're unable to read anything even if just a single byte
of the memory is unreadable. This patch fixes the lldb-server to do
that, and also changes the linux implementation, to reuse any partial
results it got from the process_vm_readv call (to avoid having to
re-read everything again using ptrace, only to find that it stopped at
the same place).

This matches debugserver behavior. It is also consistent with the gdb
remote protocol documentation, but -- notably -- not with actual
gdbserver behavior (which returns errors instead of partial results). We
filed a
[clarification
bug](https://sourceware.org/bugzilla/show_bug.cgi?id=24751) several
years ago. Though we did not really reach a conclusion there, I think
this is the most logical behavior.

The associated test does not currently pass on windows, because the
windows memory read APIs don't support partial reads (I have a WIP patch
to work around that).

show more ...

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
ant

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.

This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()

Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form

` ResultTy DoFoo(Status& error)
`
to

` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?

The interesting changes are in Status.h and Status.cpp, all other
changes are mostly

` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.

show more ...

[lldb] Claim to support swbreak and hwbreak packets when debugging a gdbremote (#102873)

This fixes https://github.com/llvm/llvm-project/issues/56125 and
https://github.com/vadimcn/codelldb/issues/

[lldb] Claim to support swbreak and hwbreak packets when debugging a gdbremote (#102873)

This fixes https://github.com/llvm/llvm-project/issues/56125 and
https://github.com/vadimcn/codelldb/issues/666, as well as the
downstream issue in our binary ninja debugger:
https://github.com/Vector35/debugger/issues/535

Basically, lldb does not claim to support the `swbreak` packet so the
gdbserver would not use it. As a result, the gdbserver always sends the
unmodified program counter value which, on systems like x86, causes the
program counter to be off-by-one (or otherwise wrong). For reference,
the lldb-server always sends the modified program counter value so it
works perfectly with lldb.

https://sourceware.org/gdb/current/onlinedocs/gdb.html/Stop-Reply-Packets.html#swbreak-stop-reason

No new code is added to add support `swbreak`, since the way lldb works
already expects the remote to have adjusted the program counter. The
change just lets the gdbserver know that lldb supports it, so that it
will send the adjusted program counter.

To test this PR, you can use lldb to connect to a gdbserver running on
e.g., Ubuntu 22.04, and see the program counter is off-by-one without
the patch. With the patch, things work as expected

show more ...

New ThreadPlanSingleThreadTimeout to resolve potential deadlock in single thread stepping (#90930)

This PR introduces a new `ThreadPlanSingleThreadTimeout` that will be
used to address potential de

New ThreadPlanSingleThreadTimeout to resolve potential deadlock in single thread stepping (#90930)

This PR introduces a new `ThreadPlanSingleThreadTimeout` that will be
used to address potential deadlock during single-thread stepping.

While debugging a target with a non-trivial number of threads (around
5000 threads in one example target), we noticed that a simple step over
can take as long as 10 seconds. Enabling single-thread stepping mode
significantly reduces the stepping time to around 3 seconds. However,
this can introduce deadlock if we try to step over a method that depends
on other threads to release a lock.

To address this issue, we introduce a new
`ThreadPlanSingleThreadTimeout` that can be controlled by the
`target.process.thread.single-thread-plan-timeout` setting during
single-thread stepping mode. The concept involves counting the elapsed
time since the last internal stop to detect overall stepping progress.
Once a timeout occurs, we assume the target is not making progress due
to a potential deadlock, as mentioned above. We then send a new async
interrupt, resume all threads, and `ThreadPlanSingleThreadTimeout`
completes its task.

To support this design, the major changes made in this PR are:
1. `ThreadPlanSingleThreadTimeout` is popped during every internal stop
and reset (re-pushed) to the top of the stack (as a leaf node) during
resume. This is achieved by always returning `true` from
`ThreadPlanSingleThreadTimeout::DoPlanExplainsStop()` and
`ThreadPlanSingleThreadTimeout::MischiefManaged()`.
2. A new thread-specific async interrupt stop is introduced, which can
be detected/consumed by `ThreadPlanSingleThreadTimeout`.
3. The clearing of branch breakpoints in the range thread plan has been
moved from `DoPlanExplainsStop()` to `ShouldStop()`, as it is not
guaranteed that it will be called.

The detailed design is discussed in the RFC below:

[https://discourse.llvm.org/t/improve-single-thread-stepping/74599](https://discourse.llvm.org/t/improve-single-thread-stepping/74599)

---------

Co-authored-by: jeffreytan81 <jeffreytan@fb.com>

show more ...

[lldb][AArch64] Add register field enum information (#96887)

This enables XML output for enums and adds enums for 2 fields on AArch64
Linux:
* mte_ctrl.tcf, which controls how tag faults are deliv

[lldb][AArch64] Add register field enum information (#96887)

This enables XML output for enums and adds enums for 2 fields on AArch64
Linux:
* mte_ctrl.tcf, which controls how tag faults are delivered.
* fpcr.rmode, which sets the rounding mode for floating point
operations.

The other one we could do is cpsr.btype, but it is not clear what would
be useful here so I'm not including it in this change.

show more ...

[lldb] [llgs] Fix assertion in Handle_qfThreadInfo (#88301)

Currently, GDBRemoteCommunicationServerLLGS::Handle_qfThreadInfo asserts
if the number of processes under debug isn’t 1 and the multiproc

[lldb] [llgs] Fix assertion in Handle_qfThreadInfo (#88301)

Currently, GDBRemoteCommunicationServerLLGS::Handle_qfThreadInfo asserts
if the number of processes under debug isn’t 1 and the multiprocess
feature isn’t supported. This is so that we don't string IDs of threads
belonging to different processes together without including the IDs of
the processes themselves in the response when there are multiple
processes under debug. However, it’s conceivable that we have no process
under debug and the multiprocess feature isn’t supported. So, have
GDBRemoteCommunicationServerLLGS::Handle_qfThreadInfo assert if the
number of processes under debug is greater than 1 and the multiprocess
feature isn’t supported.

show more ...

[lldb] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,end

[lldb] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[lldb][lldb-server] Enable sending RegisterFlags as XML (#69951)

This adds ToXML methods to encode RegisterFlags and its fields into XML
according to GDB's target XML format:

https://sourceware.

[lldb][lldb-server] Enable sending RegisterFlags as XML (#69951)

This adds ToXML methods to encode RegisterFlags and its fields into XML
according to GDB's target XML format:

https://sourceware.org/gdb/onlinedocs/gdb/Target-Description-Format.html#Target-Description-Format

lldb-server does not use libXML to build XML, so this follows the
existing code that uses strings. Indentation is used so the result is
still human readable.

```
<flags id=\"Foo\" size=\"4\">
<field name=\"abc\" start=\"0\" end=\"0\"/>
</flags>
```

This is used by lldb-server when building target XML, though no one sets
any fields yet. That'll come in a later commit.

show more ...

[lldb] Implement thread local storage for linux (#67470)

This patch implements the thread local storage support for linux
(https://github.com/llvm/llvm-project/issues/28766).

TLS feature is orig

[lldb] Implement thread local storage for linux (#67470)

This patch implements the thread local storage support for linux
(https://github.com/llvm/llvm-project/issues/28766).

TLS feature is originally only implemented for Mac. With my previous
patch to enable `fs_base` register for Linux
(https://reviews.llvm.org/D155256), now it is feasible to implement this
feature for Linux.

The major changes are:
* Track the main module's link address during launch
* Fetch thread pointer from `fs_base` register
* Create register alias for thread pointer
* Read pthread metadata from target memory instead of process so that it
works for coredump

With the patch the failing test is passing now. Note: I am only enabling
this test for Mac and Linux because I do not have machine to test for
FreeBSD/NetBSD.

---------

Co-authored-by: jeffreytan81 <jeffreytan@fb.com>

show more ...

Get LLDB building with clang-6 on Ubuntu 18.04

This patch gets clang-6 building with LLDB. The move from makeArrayRef
to deduction guides in 984b800a036fc61ccb129a8da7592af9cadc94dd is
tripping up c

Get LLDB building with clang-6 on Ubuntu 18.04

This patch gets clang-6 building with LLDB. The move from makeArrayRef
to deduction guides in 984b800a036fc61ccb129a8da7592af9cadc94dd is
tripping up clang-6 on Ubuntu 18.04.

Related to issue #64782.

show more ...

[lldb] Fix building with latest libc++

Since https://reviews.llvm.org/D157058 in libc++,
the base template for char_traits has been removed - it is only
provided for char, wchar_t, char8_t, char16_t

[lldb] Fix building with latest libc++

Since https://reviews.llvm.org/D157058 in libc++,
the base template for char_traits has been removed - it is only
provided for char, wchar_t, char8_t, char16_t and char32_t.
(Thus, to use basic_string with a type other than those, we'd need
to supply suitable traits ourselves.)

For this particular use, a vector works just as well as basic_string.

Differential Revision: https://reviews.llvm.org/D157589

show more ...

[lldb] Use SmallVector for handling register data

Previously lldb was using arrays of size kMaxRegisterByteSize to handle
registers. This was set to 256 because the largest possible register
we supp

[lldb] Use SmallVector for handling register data

Previously lldb was using arrays of size kMaxRegisterByteSize to handle
registers. This was set to 256 because the largest possible register
we support is Arm's scalable vectors (SVE) which can be up to 256 bytes long.

This means for most operations aside from SVE, we're wasting 192 bytes
of it. Which is ok given that we don't have to pay the cost of a heap
alocation and 256 bytes isn't all that much overall.

With the introduction of the Arm Scalable Matrix extension there is a new
array storage register, ZA. This register is essentially a square made up of
SVE vectors. Therefore ZA could be up to 64kb in size.

https://developer.arm.com/documentation/ddi0616/latest/

"The Effective Streaming SVE vector length, SVL, is a power of two in the range 128 to 2048 bits inclusive."

"The ZA storage is architectural register state consisting of a two-dimensional ZA array of [SVLB × SVLB] bytes."

99% of operations will never touch ZA and making every stack frame 64kb+ just
for that slim chance is a bad idea.

Instead I'm switching register handling to use SmallVector with a stack allocation
size of kTypicalRegisterByteSize. kMaxRegisterByteSize will be used in places
where we can't predict the size of register we're reading (in the GDB remote client).

The result is that the 99% of small register operations can use the stack
as before and the actual ZA operations will move to the heap as needed.

I tested this by first working out -wframe-larger-than values for all the
libraries using the arrays previously. With this change I was able to increase
kMaxRegisterByteSize to 256*256 without hitting those limits. With the
exception of the GDB server which needs to use a max size buffer.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D153626

show more ...

[lldb-server/linux] Use waitpid(-1) to collect inferior events

This is a follow-up to D116372, which had a rather unfortunate side
effect of making the processing of a single SIGCHLD quadratic in th

[lldb-server/linux] Use waitpid(-1) to collect inferior events

This is a follow-up to D116372, which had a rather unfortunate side
effect of making the processing of a single SIGCHLD quadratic in the
number of threads -- which does not matter for simple applications, but
can get really bad for applications with thousands of threads.

This patch fixes the problem by implementing the other possibility
mentioned in the first patch -- doing waitpid(-1) centrally and then
routing the events to the correct process instance. The "uncollected"
threads are held in the process factory class -- which I've renamed to
Manager for this purpose, as it now does more than creating processes.

Differential Revision: https://reviews.llvm.org/D146977

show more ...

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

Move from llvm::makeArrayRef to ArrayRef deduction guides - last part

This is a follow-up to https://reviews.llvm.org/D140896, split into
several parts as it touches a lot of files.

Differential Re

Move from llvm::makeArrayRef to ArrayRef deduction guides - last part

This is a follow-up to https://reviews.llvm.org/D140896, split into
several parts as it touches a lot of files.

Differential Revision: https://reviews.llvm.org/D141298

show more ...

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/ADT/Optional.h", etc.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optiona

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...