Revert "[lldb] Implement basic support for reverse-continue" (#123906)

Reverts llvm/llvm-project#112079 due to failures on the arm bot.

[lldb] Implement basic support for reverse-continue (#112079)

This commit adds support for a
`SBProcess::ContinueInDirection()` API. A user-accessible command for
this will follow in a later commit.

[lldb] Implement basic support for reverse-continue (#112079)

This commit adds support for a
`SBProcess::ContinueInDirection()` API. A user-accessible command for
this will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. For testing
purposes, this commit adds a Python implementation of *very limited*
record-and-reverse-execute functionality, implemented as a proxy between
lldb and lldb-server in `lldbreverse.py`. This should not (and in
practice cannot) be used for anything except testing.

The tests here are quite minimal but we test that simple breakpoints and
watchpoints work as expected during reverse execution, and that
conditional breakpoints and watchpoints work when the condition calls a
function that must be executed in the forward direction.

show more ...

[lldb] Avoid repeated map lookups (NFC) (#112655)

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

Reverting this again; I added a commit which added @skipIfDarwin
markers to the TestReverseContinueBreakpoints.py and
TestRevers

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

Reverting this again; I added a commit which added @skipIfDarwin
markers to the TestReverseContinueBreakpoints.py and
TestReverseContinueNotSupported.py API tests, which use lldb-server
in gdbserver mode which does not work on Darwin. But the aarch64 ubuntu
bot reported a failure on TestReverseContinueBreakpoints.py,
https://lab.llvm.org/buildbot/#/builders/59/builds/6397

File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/reverse-execution/TestReverseContinueBreakpoints.py", line 63, in test_reverse_continue_skip_breakpoint
self.reverse_continue_skip_breakpoint_internal(async_mode=False)
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/reverse-execution/TestReverseContinueBreakpoints.py", line 81, in reverse_continue_skip_breakpoint_internal
self.expect(
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2372, in expect
self.runCmd(
File "/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1002, in runCmd
self.assertTrue(self.res.Succeeded(), msg + output)
AssertionError: False is not true : Process should be stopped due to history boundary
Error output:
error: Process must be launched.

This reverts commit 4f297566b3150097de26c6a23a987d2bd5fc19c5.

show more ...

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later com

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. So, for testing
purposes, `lldbreverse.py` wraps `lldb-server` with a Python
implementation of *very limited* record-and-replay functionality for use
by *tests only*.

The majority of this PR is test infrastructure (about 700 of the 950
lines added).

show more ...

Revert "[lldb] Implement basic support for reverse-continue (#99736)"

This reverts commit d5e1de6da96c1ab3b8cae68447e8ed3696a7006e.

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later com

[lldb] Implement basic support for reverse-continue (#99736)

This commit only adds support for the
`SBProcess::ReverseContinue()` API. A user-accessible command for this
will follow in a later commit.

This feature depends on a gdbserver implementation (e.g. `rr`) providing
support for the `bc` and `bs` packets. `lldb-server` does not support
those packets, and there is no plan to change that. So, for testing
purposes, `lldbreverse.py` wraps `lldb-server` with a Python
implementation of *very limited* record-and-replay functionality for use
by *tests only*.

The majority of this PR is test infrastructure (about 700 of the 950
lines added).

show more ...

[lldb][AArch64] Create Neon subregs when XML only includes SVE (#108365)

Fixes #107864

QEMU decided that when SVE is enabled it will only tell us about SVE
registers in the XML, and not include

[lldb][AArch64] Create Neon subregs when XML only includes SVE (#108365)

Fixes #107864

QEMU decided that when SVE is enabled it will only tell us about SVE
registers in the XML, and not include Neon registers. On the grounds
that the Neon V registers can be read from the bottom 128 bits of a SVE
Z register (SVE's vector length is always >= 128 bits).

To support this we create sub-registers just as we do for S and D
registers of the V registers. Except this time we use part of the Z
registers.

This change also updates our fallback for registers with unknown types
that are > 128 bit. This is detailed in
https://github.com/llvm/llvm-project/issues/87471, though that covers
more than this change fixes.

We'll now treat any register of unknown type that is >= 128 bit as a
vector of bytes. So that the user gets to see something
even if the order might be wrong.

And until lldb supports vector and union types for registers, this is
also the only way we can get a value to apply the sub-reg to, to make
the V registers.

show more ...

[lldb] Don't invalid register context after setting thread pc's (#109499)

Some gdb remote serial protocol stubs will send the thread IDs and PCs
for all threads in a process in the stop-reply packe

[lldb] Don't invalid register context after setting thread pc's (#109499)

Some gdb remote serial protocol stubs will send the thread IDs and PCs
for all threads in a process in the stop-reply packet. lldb often needs
to know the pc values for all threads while at a private stop, and that
results in <n-1> read-register packets for <n> threads, and can be a big
performance problem when this is a hot code path.

GDBRemoteRegisterContext tracks the StopID of when its values were set,
and when the thread's StopID has incremented, it marks all values it has
as Invalid, and knows to refetch them.

We have a code path that resulted in setting the PCs for all the
threads, and then `ProcessGDBRemote::CalculateThreadStopInfo` *forcing*
an invalidation of all the register contexts, forcing us to re-read the
pc values for all threads except the one that stopped.

There are times when it is valid to force an invalidation of the
regsiter cache - for instance, if the layout of the registers has
changed because the processor state is different, or we've sent a
write-all-registers packet to the inferior and we want to make sure we
stay in sync with the inferior. But there was no reason for this method
to be forcing the register context to be invalid.

I added a test when running on Darwin systems, where debugserver always
sends the thread IDs and PCs, which turns on packet logging. The test
runs against an inferior which has 4 threads; it steps over a dlopen()
call, steps in to a user function with debug info, steps-over and
steps-in across source lines with multiple function calls, and then
examines the packet log and flags it as an error if lldb asked for the
pc value of any thread at any point in the debug session.

For this program and the operations we're doing, with debugserver that
provides thread IDs and PCs, we should never ask for the value of a pc
register.

rdar://136247381

show more ...

[lldb] Nits on uses of llvm::raw_string_ostream (NFC) (#108745)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610

[lldb] Nits on uses of llvm::raw_string_ostream (NFC) (#108745)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610a5226b84889b923bae884ba395ad084d for further reference )

* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Avoid unneeded calls to raw_string_ostream::str(), to avoid excess
indirection.

show more ...

[lldb] Set the stop reason when receiving swbreak/hwbreak (#108518)

xusheng added support for swbreak/hwbreak a month ago, and no special
support was needed in ProcessGDBRemote when they're receive

[lldb] Set the stop reason when receiving swbreak/hwbreak (#108518)

xusheng added support for swbreak/hwbreak a month ago, and no special
support was needed in ProcessGDBRemote when they're received because
lldb already marks a thread as having hit a breakpoint when it stops at
a breakpoint site. However, with changes I am working on, we need to
know the real stop reason a thread stopped or the breakpoint hit will
not be recognized.

This is similar to how lldb processes the "watch/rwatch/awatch" keys in
a thread stop packet -- we set the `reason` to `watchpoint`, and these
set it to `breakpoint` so we set the stop reason correctly later in
these methods.

show more ...

[lldb][NFC] Used shared_fd_t (#107553)

Replaced `int connection_fd = -1` with `shared_fd_t connection_fd =
SharedSocket::kInvalidFD`.

This is prerequisite for #104238.

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
ant

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.

This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()

Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form

` ResultTy DoFoo(Status& error)
`
to

` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?

The interesting changes are in Status.h and Status.cpp, all other
changes are mostly

` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.

show more ...

[lldb][AIX] 1. Avoid namespace collision on other platforms (#104679)

This PR is in reference to porting LLDB on AIX.

Link to discussions on llvm discourse and github:
1. https://discourse.llvm

[lldb][AIX] 1. Avoid namespace collision on other platforms (#104679)

This PR is in reference to porting LLDB on AIX.

Link to discussions on llvm discourse and github:
1. https://discourse.llvm.org/t/port-lldb-to-ibm-aix/80640
2. #101657

The complete changes for porting are present in this draft PR:
https://github.com/llvm/llvm-project/pull/102601

The changes on this PR are intended to avoid namespace collision for
certain typedefs between lldb and other platforms:
1. tid_t --> lldb::tid_t
2. offset_t --> lldb::offset_t

show more ...

[lldb] Claim to support swbreak and hwbreak packets when debugging a gdbremote (#102873)

This fixes https://github.com/llvm/llvm-project/issues/56125 and
https://github.com/vadimcn/codelldb/issues/

[lldb] Claim to support swbreak and hwbreak packets when debugging a gdbremote (#102873)

This fixes https://github.com/llvm/llvm-project/issues/56125 and
https://github.com/vadimcn/codelldb/issues/666, as well as the
downstream issue in our binary ninja debugger:
https://github.com/Vector35/debugger/issues/535

Basically, lldb does not claim to support the `swbreak` packet so the
gdbserver would not use it. As a result, the gdbserver always sends the
unmodified program counter value which, on systems like x86, causes the
program counter to be off-by-one (or otherwise wrong). For reference,
the lldb-server always sends the modified program counter value so it
works perfectly with lldb.

https://sourceware.org/gdb/current/onlinedocs/gdb.html/Stop-Reply-Packets.html#swbreak-stop-reason

No new code is added to add support `swbreak`, since the way lldb works
already expects the remote to have adjusted the program counter. The
change just lets the gdbserver know that lldb supports it, so that it
will send the adjusted program counter.

To test this PR, you can use lldb to connect to a gdbserver running on
e.g., Ubuntu 22.04, and see the program counter is off-by-one without
the patch. With the patch, things work as expected

show more ...

New ThreadPlanSingleThreadTimeout to resolve potential deadlock in single thread stepping (#90930)

This PR introduces a new `ThreadPlanSingleThreadTimeout` that will be
used to address potential de

New ThreadPlanSingleThreadTimeout to resolve potential deadlock in single thread stepping (#90930)

This PR introduces a new `ThreadPlanSingleThreadTimeout` that will be
used to address potential deadlock during single-thread stepping.

While debugging a target with a non-trivial number of threads (around
5000 threads in one example target), we noticed that a simple step over
can take as long as 10 seconds. Enabling single-thread stepping mode
significantly reduces the stepping time to around 3 seconds. However,
this can introduce deadlock if we try to step over a method that depends
on other threads to release a lock.

To address this issue, we introduce a new
`ThreadPlanSingleThreadTimeout` that can be controlled by the
`target.process.thread.single-thread-plan-timeout` setting during
single-thread stepping mode. The concept involves counting the elapsed
time since the last internal stop to detect overall stepping progress.
Once a timeout occurs, we assume the target is not making progress due
to a potential deadlock, as mentioned above. We then send a new async
interrupt, resume all threads, and `ThreadPlanSingleThreadTimeout`
completes its task.

To support this design, the major changes made in this PR are:
1. `ThreadPlanSingleThreadTimeout` is popped during every internal stop
and reset (re-pushed) to the top of the stack (as a leaf node) during
resume. This is achieved by always returning `true` from
`ThreadPlanSingleThreadTimeout::DoPlanExplainsStop()` and
`ThreadPlanSingleThreadTimeout::MischiefManaged()`.
2. A new thread-specific async interrupt stop is introduced, which can
be detected/consumed by `ThreadPlanSingleThreadTimeout`.
3. The clearing of branch breakpoints in the range thread plan has been
moved from `DoPlanExplainsStop()` to `ShouldStop()`, as it is not
guaranteed that it will be called.

The detailed design is discussed in the RFC below:

[https://discourse.llvm.org/t/improve-single-thread-stepping/74599](https://discourse.llvm.org/t/improve-single-thread-stepping/74599)

---------

Co-authored-by: jeffreytan81 <jeffreytan@fb.com>

show more ...

Revert "[lldb] Change lldb's breakpoint handling behavior (#96260)"

This reverts commit 05f0e86cc895181b3d2210458c78938f83353002.

The debuginfo dexter tests are failing, probably because the way
st

Revert "[lldb] Change lldb's breakpoint handling behavior (#96260)"

This reverts commit 05f0e86cc895181b3d2210458c78938f83353002.

The debuginfo dexter tests are failing, probably because the way
stepping over breakpoints has changed with my patches. And there
are two API tests fails on the ubuntu-arm (32-bit) bot. I'll need
to investigate both of these, neither has an obvious failure reason.

show more ...

[lldb] Change lldb's breakpoint handling behavior (#96260)

lldb today has two rules: When a thread stops at a BreakpointSite, we
set the thread's StopReason to be "breakpoint hit" (regardless if we

[lldb] Change lldb's breakpoint handling behavior (#96260)

lldb today has two rules: When a thread stops at a BreakpointSite, we
set the thread's StopReason to be "breakpoint hit" (regardless if we've
actually hit the breakpoint, or if we've merely stopped *at* the
breakpoint instruction/point and haven't tripped it yet). And second,
when resuming a process, any thread sitting at a BreakpointSite is
silently stepped over the BreakpointSite -- because we've already
flagged the breakpoint hit when we stopped there originally.

In this patch, I change lldb to only set a thread's stop reason to
breakpoint-hit when we've actually executed the instruction/triggered
the breakpoint. When we resume, we only silently step past a
BreakpointSite that we've registered as hit. We preserve this state
across inferior function calls that the user may do while stopped, etc.

Also, when a user adds a new breakpoint at $pc while stopped, or changes
$pc to be the address of a BreakpointSite, we will silently step past
that breakpoint when the process resumes. This is purely a UX call, I
don't think there's any person who wants to set a breakpoint at $pc and
then hit it immediately on resuming.

One non-intuitive UX from this change, but I'm convinced it is
necessary: If you're stopped at a BreakpointSite that has not yet
executed, you `stepi`, you will hit the breakpoint and the pc will not
yet advance. This thread has not completed its stepi, and the thread
plan is still on the stack. If you then `continue` the thread, lldb will
now stop and say, "instruction step completed", one instruction past the
BreakpointSite. You can continue a second time to resume execution. I
discussed this with Jim, and trying to paper over this behavior will
lead to more complicated scenarios behaving non-intuitively. And mostly
it's the testsuite that was trying to instruction step past a breakpoint
and getting thrown off -- and I changed those tests to expect the new
behavior.

The bugs driving this change are all from lldb dropping the real stop
reason for a thread and setting it to breakpoint-hit when that was not
the case. Jim hit one where we have an aarch64 watchpoint that triggers
one instruction before a BreakpointSite. On this arch we are notified of
the watchpoint hit after the instruction has been unrolled -- we disable
the watchpoint, instruction step, re-enable the watchpoint and collect
the new value. But now we're on a BreakpointSite so the watchpoint-hit
stop reason is lost.

Another was reported by ZequanWu in
https://discourse.llvm.org/t/lldb-unable-to-break-at-start/78282 we
attach to/launch a process with the pc at a BreakpointSite and
misbehave. Caroline Tice mentioned it is also a problem they've had with
putting a breakpoint on _dl_debug_state.

The change to each Process plugin that does execution control is that

1. If we've stopped at a BreakpointSite that has not been executed yet,
we will call Thread::SetThreadStoppedAtUnexecutedBP(pc) to record
that. When the thread resumes, if the pc is still at the same site, we
will continue, hit the breakpoint, and stop again.

2. When we've actually hit a breakpoint (enabled for this thread or not),
the Process plugin should call Thread::SetThreadHitBreakpointSite().
When we go to resume the thread, we will push a step-over-breakpoint
ThreadPlan before resuming.

The biggest set of changes is to StopInfoMachException where we
translate a Mach Exception into a stop reason. The Mach exception codes
differ in a few places depending on the target (unambiguously), and I
didn't want to duplicate the new code for each target so I've tested
what mach exceptions we get for each action on each target, and
reorganized StopInfoMachException::CreateStopReasonWithMachException to
document these possible values, and handle them without specializing
based on the target arch.

rdar://123942164

show more ...

Reland "[lldb] Parse and display register field enums" (#97258)" (#97270)

This reverts commit d9e659c538516036e40330b6a98160cbda4ff100.

I could not reproduce the Mac OS ASAN failure locally but I

Reland "[lldb] Parse and display register field enums" (#97258)" (#97270)

This reverts commit d9e659c538516036e40330b6a98160cbda4ff100.

I could not reproduce the Mac OS ASAN failure locally but I narrowed it
down to the test `test_many_fields_same_enum`. This test shares an enum
between x0, which is 64 bit, and cpsr, which is 32 bit.

My theory is that when it does `register read x0`, an enum type is
created where the undlerying enumerators are 64 bit, matching the
register size.

Then it does `register read cpsr` which used the cached enum type, but
this register is 32 bit. This caused lldb to try to read an 8 byte value
out of a 4 byte allocation:
READ of size 8 at 0x60200014b874 thread T0
<...>
=>0x60200014b800: fa fa fd fa fa fa fd fa fa fa fd fa fa fa[04]fa

To fix this I've added the register's size in bytes to the constructed
enum type's name. This means that x0 uses:
__lldb_register_fields_enum_some_enum_8
And cpsr uses:
__lldb_register_fields_enum_some_enum_4

If any other registers use this enum and are read, they will use the
cached type as long as their size matches, otherwise we make a new type.

show more ...

Revert "[lldb] Parse and display register field enums" (#97258)

Reverts llvm/llvm-project#95768 due to a test failure on macOS with
ASAN:

https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/ll

Revert "[lldb] Parse and display register field enums" (#97258)

Reverts llvm/llvm-project#95768 due to a test failure on macOS with
ASAN:

https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/lldb-cmake-sanitized/425/console

show more ...

[lldb] Don't call AddRemoteRegisters if the target XML did not include any registers (#96907)

Fixes #92541

When e69a3d18f48bc0d81b5dd12e735a2ec898ce64d added fallback register
layouts, it assume

[lldb] Don't call AddRemoteRegisters if the target XML did not include any registers (#96907)

Fixes #92541

When e69a3d18f48bc0d81b5dd12e735a2ec898ce64d added fallback register
layouts, it assumed that the choices were target XML with registers, or
no target XML at all.

In the linked issue, a user has a debug stub that does have target XML,
but it's missing register information.

This caused us to finalize the register information using an empty set
of registers got from target XML, then fail an assert when we attempted
to add the fallback set. Since we think we've already completed the
register information.

This change adds a check to prevent that first call and expands the
existing tests to check each architecture without target XML and with
target XML missing register information.

show more ...

[lldb] Parse and display register field enums (#95768)

This teaches lldb to parse the enum XML elements sent by lldb-server,
and make use of the information in `register read` and `register info`.

[lldb] Parse and display register field enums (#95768)

This teaches lldb to parse the enum XML elements sent by lldb-server,
and make use of the information in `register read` and `register info`.

The format is described in

https://sourceware.org/gdb/current/onlinedocs/gdb.html/Enum-Target-Types.html.

The target XML parser will drop any invalid enum or evalue. If we find
multiple evalue for the same value, we will use the last one we find.

The order of evalues from the XML is preserved as there may be good
reason they are not in numerical order.

show more ...

[lldb] More descriptive name for register flags logging functions

This was requested on a review for enum code that added new log
functions.

[lldb] Use operator==(StringRef, StringRef) instead of StringRef::equals (NFC) (#92476)

Note that StringRef::equals has been deprecated in favor of
operator==(StringRef, StringRef).

[lldb] Invert relationship between Process and AddressableBits (#85858)

AddressableBits is in the Utility module of LLDB. It currently directly
refers to Process, which is from the Target LLDB modu

[lldb] Invert relationship between Process and AddressableBits (#85858)

AddressableBits is in the Utility module of LLDB. It currently directly
refers to Process, which is from the Target LLDB module. This is a
layering violation which concretely means that it is impossible to link
anything that uses Utility without it also using Target as well. This is
generally not an issue for LLDB (since everything is built together) but
it may make it difficult to write unit tests for AddressableBits later
on.

show more ...