Fix statistics dump to report per-target (#113723)

"statistics dump" currently report the statistics of all targets in
debugger instead of current target. This is wrong because there is a
"statist

Fix statistics dump to report per-target (#113723)

"statistics dump" currently report the statistics of all targets in
debugger instead of current target. This is wrong because there is a
"statistics dump --all-targets" option that supposed to include
everything.

This PR fixes the issue by only report statistics for current target
instead of all. It also includes the change to reset statistics debug
info/symbol table parsing/indexing time during debugger destroy. This is
required so that we report current statistics if we plan to reuse
lldb/lldb-dap across debug sessions

---------

Co-authored-by: jeffreytan81 <jeffreytan@fb.com>

show more ...

More aggressively deduplicate global warnings based on contents. (#112801)

I've been getting complaints from users being spammed by -gmodules
missing file warnings going out of control because eac

More aggressively deduplicate global warnings based on contents. (#112801)

I've been getting complaints from users being spammed by -gmodules
missing file warnings going out of control because each object file
depends on an entire DAG of PCM files that usually are all missing at
once. To reduce this problem, this patch does two things:

1. Module now maintains a DenseMap<hash, once> that is used to display
each warning only once, based on its actual text.

2. The PCM warning itself is reworded to include less details, such as
the DIE offset, which is only useful to LLDB developers, who can get
this from the dwarf log if they need it. Because the detail is omitted
the hashing from (1) deduplicates the warnings.

rdar://138144624

show more ...

[lldb] Nits on uses of llvm::raw_string_ostream (NFC) (#108745)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610

[lldb] Nits on uses of llvm::raw_string_ostream (NFC) (#108745)

As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly

( 65b13610a5226b84889b923bae884ba395ad084d for further reference )

* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Avoid unneeded calls to raw_string_ostream::str(), to avoid excess
indirection.

show more ...

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
ant

[lldb] Turn lldb_private::Status into a value type. (#106163)

This patch removes all of the Set.* methods from Status.

This cleanup is part of a series of patches that make it harder use the
anti-pattern of keeping a long-lives Status object around and updating
it while dropping any errors it contains on the floor.

This patch is largely NFC, the more interesting next steps this enables
is to:
1. remove Status.Clear()
2. assert that Status::operator=() never overwrites an error
3. remove Status::operator=()

Note that step (2) will bring 90% of the benefits for users, and step
(3) will dramatically clean up the error handling code in various
places. In the end my goal is to convert all APIs that are of the form

` ResultTy DoFoo(Status& error)
`
to

` llvm::Expected<ResultTy> DoFoo()
`
How to read this patch?

The interesting changes are in Status.h and Status.cpp, all other
changes are mostly

` perl -pi -e 's/\.SetErrorString/ = Status::FromErrorString/g' $(git
grep -l SetErrorString lldb/source)
`
plus the occasional manual cleanup.

show more ...

[lldb] Change Module to have a concrete UnwindTable, update (#101130)

Currently a Module has a std::optional<UnwindTable> which is created
when the UnwindTable is requested from outside the Module.

[lldb] Change Module to have a concrete UnwindTable, update (#101130)

Currently a Module has a std::optional<UnwindTable> which is created
when the UnwindTable is requested from outside the Module. The idea is
to delay its creation until the Module has an ObjectFile initialized,
which will have been done by the time we're doing an unwind.

However, Module::GetUnwindTable wasn't doing any locking, so it was
possible for two threads to ask for the UnwindTable for the first time,
one would be created and returned while another thread would create one,
destroy the first in the process of emplacing it. It was an uncommon
crash, but it was possible.

Grabbing the Module's mutex would be one way to address it, but when
loading ELF binaries, we start creating the SymbolTable on one thread
(ObjectFileELF) grabbing the Module's mutex, and then spin up worker
threads to parse the individual DWARF compilation units, which then try
to also get the UnwindTable and deadlock if they try to get the Module's
mutex.

This changes Module to have a concrete UnwindTable as an ivar, and when
it adds an ObjectFile or SymbolFileVendor, it will call the Update
method on it, which will re-evaluate which sections exist in the
ObjectFile/SymbolFile. UnwindTable used to have an Initialize method
which set all the sections, and an Update method which would set some of
them if they weren't set. I unified these with the Initialize method
taking a `force` option to re-initialize the section pointers even if
they had been done already before.

This is addressing a rare crash report we've received, and also a
failure Adrian spotted on the -fsanitize=address CI bot last week, it's
still uncommon with ASAN but it can happen with the standard testsuite.

rdar://128876433

show more ...

[lldb] Revive shell test after updating UnwindTable (#86770)

In
commit 2f63718f8567413a1c596bda803663eb58d6da5a
Author: Jason Molenda <jmolenda@apple.com>
Date: Tue Mar 26 09:07:

[lldb] Revive shell test after updating UnwindTable (#86770)

In
commit 2f63718f8567413a1c596bda803663eb58d6da5a
Author: Jason Molenda <jmolenda@apple.com>
Date: Tue Mar 26 09:07:15 2024 -0700

[lldb] Don't clear a Module's UnwindTable when adding a SymbolFile
(#86603)

I stopped clearing a Module's UnwindTable when we add a SymbolFile to
avoid the memory management problems with adding a symbol file
asynchronously while the UnwindTable is being accessed on another
thread. This broke the target-symbols-add-unwind.test shell test on
Linux which removes the DWARF debub_frame section from a binary, loads
it, then loads the unstripped binary with the DWARF debug_frame section
and checks that the UnwindPlans for a function include debug_frame.

I originally decided that I was willing to sacrifice the possiblity of
additional unwind sources from a symbol file because we rely on assembly
emulation so heavily, they're rarely critical. But there are targets
where we we don't have emluation and rely on things like DWARF
debug_frame a lot more, so this probably wasn't a good choice.

This patch adds a new UnwindTable::Update method which looks for any new
sources of unwind information and adds it to the UnwindTable, and calls
that after a new SymbolFile has been added to a Module.

show more ...

[lldb] Don't clear a Module's UnwindTable when adding a SymbolFile (#86603)

Fixing a crash in lldb when `symbols.auto-download` setting is enabled.
When doing a backtrace, this feature has lldb sea

[lldb] Don't clear a Module's UnwindTable when adding a SymbolFile (#86603)

Fixing a crash in lldb when `symbols.auto-download` setting is enabled.
When doing a backtrace, this feature has lldb search for a SymbolFile
for stack frames when we are backtracing, and add them either
synchoronously or asynchronously, depending on the specific setting
used.

Module::SetSymbolFileFileSpec clears the Module's UnwindTable, once we
find a new SymbolFile. We may be adding a source of unwind information
that we did not have when lldb was working only with the executable
binary.

What happens in practice is that we're using a reference to the Module's
UnwindTable, and then the other thread getting the SymbolFile clears it
and now the first thread is referring to freed memory and we can crash.
When built with address sanitizer, it crashes much more reliably.

Given that unwind information used for exception handling -- eh_frame,
compact unwind -- is present in executable binaries, the only thing
we're likely to *add* would be DWARF's `debug_frame` if that was also
available. The actual value of re-creating the UnwindTable when we have
added a SymbolFile is not large.

I also tried fixing this by changing the Module to have a shared_ptr to
the UnwindTable, so we could have two different UnwindTable's in use
simultaneously for a brief period. This would be fine TODAY, but it
introduces a very subtle bug that someone will have a heck of a time
figuring out in the future.

In the end, I believe the safest approach is to sacrifice the possible
marginal gain of reconstructing the UnwindTable once a SymbolFile has
been added, to sidestep this whole problem area.

Also, in `Module::GetUnwindTable()`, call `DownloadSymbolFileAsync`
before we create the UnwindTable for the first time, in case the symbol
file is fetched synchronously, we will have it for that possible
marginal gain.

show more ...

[lldb] Fix trailing whitespace & formatting in Core/Module.cpp (NFC)

I have my editor configured to remove trailing whitespace and every time
I touch this file I end up with a bunch of clang-format

[lldb] Fix trailing whitespace & formatting in Core/Module.cpp (NFC)

I have my editor configured to remove trailing whitespace and every time
I touch this file I end up with a bunch of clang-format changes to lines
that were modified because of it. Nobody likes trailing whitespace so
this cleans up the file.

show more ...

[lldb] DWARFDIE: Follow DW_AT_specification when computing CompilerCo… (#77157)

…ntext

Following the specification chain seems to be clearly the expected
behavior of GetDeclContext(). Otherwise

[lldb] DWARFDIE: Follow DW_AT_specification when computing CompilerCo… (#77157)

…ntext

Following the specification chain seems to be clearly the expected
behavior of GetDeclContext(). Otherwise C++ methods have an empty
CompilerContext instead of being nested in their struct/class.

Theprimary motivation for this functionality is the Swift plugin. In
order to test the change I added a proof-of-concept implementation of a
Module::FindFunction() variant that takes a CompilerContext, expesed via
lldb-test.

rdar://120553412

show more ...

[lldb] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,end

[lldb] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[lldb] Make only one function that needs to be implemented when searching for types (#74786)

This patch revives the effort to get this Phabricator patch into
upstream:

https://reviews.llvm.org/D

[lldb] Make only one function that needs to be implemented when searching for types (#74786)

This patch revives the effort to get this Phabricator patch into
upstream:

https://reviews.llvm.org/D137900

This patch was accepted before in Phabricator but I found some
-gsimple-template-names issues that are fixed in this patch.

A fixed up version of the description from the original patch starts
now.

This patch started off trying to fix Module::FindFirstType() as it
sometimes didn't work. The issue was the SymbolFile plug-ins didn't do
any filtering of the matching types they produced, and they only looked
up types using the type basename. This means if you have two types with
the same basename, your type lookup can fail when only looking up a
single type. We would ask the Module::FindFirstType to lookup "Foo::Bar"
and it would ask the symbol file to find only 1 type matching the
basename "Bar", and then we would filter out any matches that didn't
match "Foo::Bar". So if the SymbolFile found "Foo::Bar" first, then it
would work, but if it found "Baz::Bar" first, it would return only that
type and it would be filtered out.

Discovering this issue lead me to think of the patch Alex Langford did a
few months ago that was done for finding functions, where he allowed
SymbolFile objects to make sure something fully matched before parsing
the debug information into an AST type and other LLDB types. So this
patch aimed to allow type lookups to also be much more efficient.

As LLDB has been developed over the years, we added more ways to to type
lookups. These functions have lots of arguments. This patch aims to make
one API that needs to be implemented that serves all previous lookups:

- Find a single type
- Find all types
- Find types in a namespace

This patch introduces a `TypeQuery` class that contains all of the state
needed to perform the lookup which is powerful enough to perform all of
the type searches that used to be in our API. It contain a vector of
CompilerContext objects that can fully or partially specify the lookup
that needs to take place.

If you just want to lookup all types with a matching basename,
regardless of the containing context, you can specify just a single
CompilerContext entry that has a name and a CompilerContextKind mask of
CompilerContextKind::AnyType.

Or you can fully specify the exact context to use when doing lookups
like: CompilerContextKind::Namespace "std"
CompilerContextKind::Class "foo"
CompilerContextKind::Typedef "size_type"

This change expands on the clang modules code that already used a
vector<CompilerContext> items, but it modifies it to work with
expression type lookups which have contexts, or user lookups where users
query for types. The clang modules type lookup is still an option that
can be enabled on the `TypeQuery` objects.

This mirrors the most recent addition of type lookups that took a
vector<CompilerContext> that allowed lookups to happen for the
expression parser in certain places.

Prior to this we had the following APIs in Module:

```
void
Module::FindTypes(ConstString type_name, bool exact_match, size_t max_matches,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeList &types);

void
Module::FindTypes(llvm::ArrayRef<CompilerContext> pattern, LanguageSet languages,
llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
TypeMap &types);

void Module::FindTypesInNamespace(ConstString type_name,
const CompilerDeclContext &parent_decl_ctx,
size_t max_matches, TypeList &type_list);
```

The new Module API is much simpler. It gets rid of all three above
functions and replaces them with:

```
void FindTypes(const TypeQuery &query, TypeResults &results);
```
The `TypeQuery` class contains all of the needed settings:

- The vector<CompilerContext> that allow efficient lookups in the symbol
file classes since they can look at basename matches only realize fully
matching types. Before this any basename that matched was fully realized
only to be removed later by code outside of the SymbolFile layer which
could cause many types to be realized when they didn't need to.
- If the lookup is exact or not. If not exact, then the compiler context
must match the bottom most items that match the compiler context,
otherwise it must match exactly
- If the compiler context match is for clang modules or not. Clang
modules matches include a Module compiler context kind that allows types
to be matched only from certain modules and these matches are not needed
when d oing user type lookups.
- An optional list of languages to use to limit the search to only
certain languages

The `TypeResults` object contains all state required to do the lookup
and store the results:
- The max number of matches
- The set of SymbolFile objects that have already been searched
- The matching type list for any matches that are found

The benefits of this approach are:
- Simpler API, and only one API to implement in SymbolFile classes
- Replaces the FindTypesInNamespace that used a CompilerDeclContext as a
way to limit the search, but this only worked if the TypeSystem matched
the current symbol file's type system, so you couldn't use it to lookup
a type in another module
- Fixes a serious bug in our FindFirstType functions where if we were
searching for "foo::bar", and we found a "baz::bar" first, the basename
would match and we would only fetch 1 type using the basename, only to
drop it from the matching list and returning no results

show more ...

[lldb] Remove LocateSymbolFile (#71301)

This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really f

[lldb] Remove LocateSymbolFile (#71301)

This completes the conversion of LocateSymbolFile into a SymbolLocator
plugin. The only remaining function is DownloadSymbolFileAsync which
doesn't really fit into the plugin model, and therefore moves into the
SymbolLocator class, while still relying on the plugins to do the
underlying work.

show more ...

[lldb][NFCI] Module constructor should take ConstString by value

ConstStrings are super cheap to copy around. It is often more expensive
to pass a pointer and potentially dereference it than just to

[lldb][NFCI] Module constructor should take ConstString by value

ConstStrings are super cheap to copy around. It is often more expensive
to pass a pointer and potentially dereference it than just to always copy it.

Differential Revision: https://reviews.llvm.org/D158043

show more ...

[BSDArchive] NULL check the child object file ptr before accessing its member

Recently we've observed lldb crashes caused by missing object file linked to a thin archive (.a) files. The crash is due

[BSDArchive] NULL check the child object file ptr before accessing its member

Recently we've observed lldb crashes caused by missing object file linked to a thin archive (.a) files. The crash is due to a missing NULL check in the code when looking for child object file referred by the thin archive. Malformed archive file should not crash LLDB. Instead, it should report the error and continue.

New error message will look like the following

```
error: libfoo.a(__objects__/foo/barAppDelegate.mm.o) failed to load objfile for path/to/libfoo.a.
Debugging will be degraded for this module.
```

Test Plan:

llvm-lit test
```
./bin/llvm-lit -sv ../llvm-project/lldb/test/API/functionalities/archives/TestBSDArchives.py
```

Test without code change will error out with LLDB crash
```
--
Command Output (stderr):
--
PASS: LLDB (~/llvm-upstream/Debug/bin/clang-arm64) :: test (TestBSDArchives.BSDArchivesTestCase)
PASS: LLDB (~/llvm-upstream/Debug/bin/clang-arm64) :: test_frame_var_errors_when_archive_missing (TestBSDArchives.BSDArchivesTestCase)
FAIL: LLDB (~/llvm-upstream/Debug/bin/clang-arm64) :: test_frame_var_errors_when_mtime_mistmatch_for_object_in_archive (TestBSDArchives.BSDArchivesTestCase)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. HandleCommand(command = "b a")
1. HandleCommand(command = "breakpoint set --name 'a'")
Fatal Python error: Segmentation fault

Current thread 0x00000001f7b99e00 (most recent call first):
File "~/llvm-upstream/Debug/bin/LLDB.framework/Resources/Python/lldb/__init__.py", line 3270 in HandleCommand
File "~/llvm-upstream/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2070 in runCmd
File "~/llvm-upstream/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2421 in expect
File "~/llvm-upstream/llvm-project/lldb/test/API/functionalities/archives/TestBSDArchives.py", line 156 in test_frame_var_errors_when_thin_archive_malformed
...
```

Differential Revision: https://reviews.llvm.org/D156367

show more ...

[lldb][NFCI] Methods to load scripting resources should take a Stream by reference

These methods all take a `Stream *` to get feedback about what's going
on. By default, it's a nullptr, but we alway

[lldb][NFCI] Methods to load scripting resources should take a Stream by reference

These methods all take a `Stream *` to get feedback about what's going
on. By default, it's a nullptr, but we always feed it with a valid
pointer. It would therefore make more sense to have this take a
reference.

Differential Revision: https://reviews.llvm.org/D154883

show more ...

Refine the reporting mechanism for interruption.

Also, make it possible for new Targets which haven't been added to
the TargetList yet to check for interruption, and add a few more
places in buildin

Refine the reporting mechanism for interruption.

Also, make it possible for new Targets which haven't been added to
the TargetList yet to check for interruption, and add a few more
places in building modules where we can check for interruption.

Differential Revision: https://reviews.llvm.org/D154542

show more ...

Revert "Revert "Host: generalise `GetXcodeSDKPath`""

This reverts commit c46d9af26cefb0b24646d3235b75ae7a1b8548d4.

Rename the variable to avoid `-Wchanges-meaning` warning. Although, it
might be b

Revert "Revert "Host: generalise `GetXcodeSDKPath`""

This reverts commit c46d9af26cefb0b24646d3235b75ae7a1b8548d4.

Rename the variable to avoid `-Wchanges-meaning` warning. Although, it
might be better to squelch the warning as it is of low value IMO.

show more ...

[lldb] Move Core/FileSpecList -> Utility/FileSpecList (NFC)

There's no reason for FileSpecList to live in lldb/Core while FileSpec
lives in lldb/Utility. Move FileSpecList next to FileSpec.

Revert "Host: generalise `GetXcodeSDKPath`"

This reverts commit ade3c6a6a88ed3a9b06c076406f196da9d3cc1b9.

This breaks the build with GCC and affects at least 2 build bots:
https://lab.llvm.org/buil

Revert "Host: generalise `GetXcodeSDKPath`"

This reverts commit ade3c6a6a88ed3a9b06c076406f196da9d3cc1b9.

This breaks the build with GCC and affects at least 2 build bots:
https://lab.llvm.org/buildbot/#/builders/217/builds/20568
https://lab.llvm.org/buildbot/#/builders/243/builds/5576

show more ...

Host: generalise `GetXcodeSDKPath`

This generalises the GetXcodeSDKPath hook to a GetSDKRoot path which
will be re-used for the Windows support to compute a language specific
SDK path on the platfor

Host: generalise `GetXcodeSDKPath`

This generalises the GetXcodeSDKPath hook to a GetSDKRoot path which
will be re-used for the Windows support to compute a language specific
SDK path on the platform. Because there may be other options that we
wish to use to compute the SDK path, sink the XcodeSDK parameter into
a structure which can pass a disaggregated set of options. Furthermore,
optionalise the parameter as Xcode is not available for all platforms.

Differential Revision: https://reviews.llvm.org/D149397
Reviewed By: JDevlieghere

show more ...

[lldb] Add `operator StringRef` to ConstString

Add a `StringRef` conversion function to `ConstString`.

This will make using llvm, and other non-ConstString, APIs more convenient.

For demonstration

[lldb] Add `operator StringRef` to ConstString

Add a `StringRef` conversion function to `ConstString`.

This will make using llvm, and other non-ConstString, APIs more convenient.

For demonstration, this updates Module.cpp.

Differential Revision: https://reviews.llvm.org/D148175

show more ...

[LLDB] Change formatting to use llvm::formatv

In preparation for eanbling 64bit support in LLDB switching to use llvm::formatv
instead of format MACROs.

Reviewed By: labath, JDevlieghere

Different

[LLDB] Change formatting to use llvm::formatv

In preparation for eanbling 64bit support in LLDB switching to use llvm::formatv
instead of format MACROs.

Reviewed By: labath, JDevlieghere

Differential Revision: https://reviews.llvm.org/D139955

show more ...

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/ADT/Optional.h", etc.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optiona

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of ma

[lldb] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...