[lldb] Search main function with lldb::eFunctionNameTypeFull when getting default file and line. (#113980)

This is to work around the fact that
`SymbolFileNativePDB::FindFunctions` only support
`l

[lldb] Search main function with lldb::eFunctionNameTypeFull when getting default file and line. (#113980)

This is to work around the fact that
`SymbolFileNativePDB::FindFunctions` only support
`lldb::eFunctionNameTypeFull` and `lldb::eFunctionNameTypeMethod` now.
Since `main`'s full name is the same as base name (`main`), it's okay to
search with `lldb::eFunctionNameTypeFull` when trying to get the default
file and line. With this, `lldb/test/Shell/Driver/TestSingleQuote.test`
passes on Windows with NativePDB plugin.

show more ...

[lldb] Print a warning on checksum mismatch (#107968)

Print a warning when the debugger detects a mismatch between the MD5
checksum in the DWARF 5 line table and the file on disk. The warning is
p

[lldb] Print a warning on checksum mismatch (#107968)

Print a warning when the debugger detects a mismatch between the MD5
checksum in the DWARF 5 line table and the file on disk. The warning is
printed only once per file.

show more ...

[lldb] Avoid FileSpec indirection where we can use SupportFiles directly

Now that more parts of LLDB know about SupportFiles, avoid going through
FileSpec (and losing the Checksum in the process). I

[lldb] Avoid FileSpec indirection where we can use SupportFiles directly

Now that more parts of LLDB know about SupportFiles, avoid going through
FileSpec (and losing the Checksum in the process). Instead, use the
SupportFile directly.

show more ...

[lldb] Include checksum in source cache dump (#106773)

This patch updates the source cache dump command to print both the
actual (on-disk) checksum and the expected (line table) checksum. To
achie

[lldb] Include checksum in source cache dump (#106773)

This patch updates the source cache dump command to print both the
actual (on-disk) checksum and the expected (line table) checksum. To
achieve that we now read and store the on-disk checksum in the cached
object. The same information will be used in a future path to print a
warning when the checksums differ.

show more ...

[lldb] Deal with SupportFiles in SourceManager (NFC) (#106740)

To support detecting MD5 checksum mismatches, deal with SupportFiles
rather than a plain FileSpecs in the SourceManager.

[lldb] Store SupportFiles in SourceManager::File (NFC) (#106639)

To support detecting MD5 checksum mismatches, store a SupportFile rather
than a plain FileSpec in SourceManager::File.

[lldb] Reland: Store SupportFile in FileEntry (NFC) (#85892)

This is another step towards supporting DWARF5 checksums and inline
source code in LLDB. This is a reland of #85468 but without the
fun

[lldb] Reland: Store SupportFile in FileEntry (NFC) (#85892)

This is another step towards supporting DWARF5 checksums and inline
source code in LLDB. This is a reland of #85468 but without the
functional change of storing the support file from the line table (yet).

show more ...

Revert "[lldb] Store SupportFile in FileEntry (NFC)" (#85885)

Reverts llvm/llvm-project#85468 because @slackito reports this broke
stepping in one of their tests [1] and this patch was meant to be

Revert "[lldb] Store SupportFile in FileEntry (NFC)" (#85885)

Reverts llvm/llvm-project#85468 because @slackito reports this broke
stepping in one of their tests [1] and this patch was meant to be NFC.

[1]
https://github.com/llvm/llvm-project/commit/d5a277d309e92b1d3e493da6036cffdf815105b1#commitcomment-139991120

show more ...

[lldb] Store SupportFile in FileEntry (NFC) (#85468)

This is another step towards supporting DWARF5 checksums and inline
source code in LLDB.

[lldb] Add two-level caching in the source manager

We recently saw an uptick in internal reports complaining that LLDB is
slow when sources on network file systems are inaccessible. I looked at
the

[lldb] Add two-level caching in the source manager

We recently saw an uptick in internal reports complaining that LLDB is
slow when sources on network file systems are inaccessible. I looked at
the SourceManger and its cache and I think there’s some room for
improvement in terms of reducing file system accesses:

1. We always resolve the path.
2. We always check the timestamp.
3. We always recheck the file system for negative cache hits.

D153726 fixes (1) but (2) and (3) are necessary because of the cache’s
current design. Source files are cached at the debugger level which
means that the source file cache can span multiple targets and
processes. It wouldn't be correct to not reload a modified or new file
from disk.

We can however significantly reduce the number of file system accesses
by using a two level cache design: one cache at the debugger level and
one at the process level:

- The cache at the debugger level works the way it does today. There is
no negative cache: if we can't find the file on disk, we'll try again
next time the cache is queried. If a cached file's timestamp changes
or if its path remapping changes, the cached file is evicted and we
reload it from disk.
- The cache at the process level is design to avoid accessing the file
system. It doesn't check the file's modification time. It caches
negative results, so if a file didn't exist, it doesn't try to reread
it from disk. Checking if the path remapping changed is cheap
(doesn't involve checking the file system) and is the only way for a
file to get evicted from the process cache.

The result of this patch is that LLDB will not show you new content if a
file is modified or created while a process is running. I would argue
that this is what most people would expect, but it is a change from how
LLDB behaves today.

For an average stop, we query the source cache 4 times. With the current
implementation, that's 4 stats to get the modification time, If the file
doesn't exist on disk, that's an additional 4 stats. Before D153726, if
the path starts with a ~ there are another additional 4 calls to
realpath. When debugging sources on a slow (network) file system, this
quickly adds up.

In addition to the two level caching, this patch also adds a source
logging channel and synchronization to the source file cache. The
logging was helpful during development and hopefully will help us triage
issues in the future. The synchronization isn't a new requirement: as
the cache is shared across targets, there is no guarantees that it can't
be accessed concurrently. The patch also fixes a bug where we would only
set the source remapping ID if the un-remapped file didn't exist, which
led to the file getting evicted from the cache on every access.

rdar://110787562

Differential revision: https://reviews.llvm.org/D153834

show more ...

[lldb] Avoid FileSystem::Resolve for cached files in the SourceManager

Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have t

[lldb] Avoid FileSystem::Resolve for cached files in the SourceManager

Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have to resolve the
path, which can be slow. This patch avoids the call to FileSystem::Resolve
by caching both the resolved and unresolved path. We now only resolve
the path once when we create and cache a new file.

rdar://110787562

Differential revision: https://reviews.llvm.org/D153726

show more ...

[lldb] Add `source cache dump` and `source cache clear` subcommand

Add two new source subcommands: source cache dump and source cache
clear. As the name implies the first one dumps the source cache

[lldb] Add `source cache dump` and `source cache clear` subcommand

Add two new source subcommands: source cache dump and source cache
clear. As the name implies the first one dumps the source cache while
the later clears the cache.

This patch was motivated by a handful of (internal) bug reports related
to sources not being available. Right now those issues can be hard to
diagnose. The new commands give users, as well as us as developers, more
insight into and control over the source cache.

Differential revision: https://reviews.llvm.org/D153685

show more ...

Re-land "[lldb] Expose a const iterator for SymbolContextList"

Re-lands 04aa943be8ed5c03092e2a90112ac638360ec253 with modifications
to fix tests.
I originally reverted this because it caused a test

Re-land "[lldb] Expose a const iterator for SymbolContextList"

Re-lands 04aa943be8ed5c03092e2a90112ac638360ec253 with modifications
to fix tests.
I originally reverted this because it caused a test to fail on Linux.
The problem was that I inverted a condition on accident.

show more ...

Revert "[lldb] Expose a const iterator for SymbolContextList"

This reverts commit 04aa943be8ed5c03092e2a90112ac638360ec253.

This broke the debian buildbot and I'm not sure why. Reverting so I can
i

Revert "[lldb] Expose a const iterator for SymbolContextList"

This reverts commit 04aa943be8ed5c03092e2a90112ac638360ec253.

This broke the debian buildbot and I'm not sure why. Reverting so I can
investigate.

show more ...

[lldb] Expose a const iterator for SymbolContextList

There are many situations where we'll iterate over a SymbolContextList
with the pattern:
```
SymbolContextList sc_list;
// Fill in sc_list here
f

[lldb] Expose a const iterator for SymbolContextList

There are many situations where we'll iterate over a SymbolContextList
with the pattern:
```
SymbolContextList sc_list;
// Fill in sc_list here
for (auto i = 0; i < sc_list.GetSize(); i++) {
SymbolContext sc;
sc_list.GetSymbolAtContext(i, sc);

// Do work with sc
}
```
Adding an iterator to iterate over the instances directly means we don't
have to do bounds checking or create a copy of every element of the
SymbolContextList.

Differential Revision: https://reviews.llvm.org/D149900

show more ...

[lldb] Replace sprintf with snprintf (NFC)

On macOS, `sprintf` is deprecated, using `snprintf` is recommended instead.

Differential Revision: https://reviews.llvm.org/D147246

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/

[lldb] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to clean up the "using" declarations, #include
"llvm/ADT/Optional.h", etc.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optiona

[lldb] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Check that a FileSpec has a Directory component before using

A follow on to my patch for https://reviews.llvm.org/D126435
hit by an x86_64 linux bot; I assumed that a FileSpec had a
directory compon

Check that a FileSpec has a Directory component before using

A follow on to my patch for https://reviews.llvm.org/D126435
hit by an x86_64 linux bot; I assumed that a FileSpec had a
directory component and checked if the first character was a
'~'. This was not a valid assumption.

show more ...

Defer source path remap tilde expansion until source file use

When reading source path remappings out of a dSYM, lldb currently
does tilde expansion -- expanding the tilde-username and checking
that

Defer source path remap tilde expansion until source file use

When reading source path remappings out of a dSYM, lldb currently
does tilde expansion -- expanding the tilde-username and checking
that the destination pathname exists, for each dSYM with the path
remappings. This cost happens during lldb's initial process launch
/ load, an especially perf-sensitive time. Inside Apple, we have
dSYMs with source path remappings pointing to NFS directories where
these extra stats for every dSYM can be very expensive if the network
is slow.

This patch instead keeps the source path mapping in the original
tilde-username terms and does the tilde expansion when we need
to read a specific source file from one of the modules. We'll
be stat'ing all of those inodes to load the source file anyway,
so the fact that we do the tilde expansion on every source file
we load, it doesn't cost us significantly.

Differential Revision: https://reviews.llvm.org/D126435
rdar://77091379

(cherry picked from commit c274b6e5830ea88d3f55d6dc1d2b99e38cf6595e)

show more ...

Revert "Defer source path remap tilde expansion until source file use"

This reverts commit c274b6e5830ea88d3f55d6dc1d2b99e38cf6595e.

The x86_64 debian bot got a failure with this patch,
https://lab

Revert "Defer source path remap tilde expansion until source file use"

This reverts commit c274b6e5830ea88d3f55d6dc1d2b99e38cf6595e.

The x86_64 debian bot got a failure with this patch,
https://lab.llvm.org/buildbot#builders/68/builds/33078
where
SymbolFile/DWARF/x86/DW_TAG_variable-DW_AT_decl_file-DW_AT_abstract_origin-crosscu1.s
is crashing here -

#2 0x0000000000425a9f SignalHandler(int) Signals.cpp:0:0
#3 0x00007f57160e9140 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14140)
#4 0x00007f570d911e43 lldb_private::SourceManager::GetFile(lldb_private::FileSpec const&) crtstuff.c:0:0
#5 0x00007f570d914270 lldb_private::SourceManager::DisplaySourceLinesWithLineNumbers(lldb_private::FileSpec const&, unsigned int, unsigned int, unsigned int, unsigned int, char const*, lldb_private::Stream*, lldb_private::SymbolContextList const*) crtstuff.c:0:0
#6 0x00007f570da662c8 lldb_private::StackFrame::GetStatus(lldb_private::Stream&, bool, bool, bool, char const*) crtstuff.c:0:0

I don't get a failure here my mac, I'll review this method more
closely tomorrow.

show more ...

Defer source path remap tilde expansion until source file use

When reading source path remappings out of a dSYM, lldb currently
does tilde expansion -- expanding the tilde-username and checking
that

Defer source path remap tilde expansion until source file use

When reading source path remappings out of a dSYM, lldb currently
does tilde expansion -- expanding the tilde-username and checking
that the destination pathname exists, for each dSYM with the path
remappings. This cost happens during lldb's initial process launch
/ load, an especially perf-sensitive time. Inside Apple, we have
dSYMs with source path remappings pointing to NFS directories where
these extra stats for every dSYM can be very expensive if the network
is slow.

This patch instead keeps the source path mapping in the original
tilde-username terms and does the tilde expansion when we need
to read a specific source file from one of the modules. We'll
be stat'ing all of those inodes to load the source file anyway,
so the fact that we do the tilde expansion on every source file
we load, it doesn't cost us significantly.

Differential Revision: https://reviews.llvm.org/D126435
rdar://77091379

show more ...

[lldb] Refactor DataBuffer so we can map files as read-only

Currently, all data buffers are assumed to be writable. This is a
problem on macOS where it's not allowed to load unsigned binaries in
mem

[lldb] Refactor DataBuffer so we can map files as read-only

Currently, all data buffers are assumed to be writable. This is a
problem on macOS where it's not allowed to load unsigned binaries in
memory as writable. To be more precise, MAP_RESILIENT_CODESIGN and
MAP_RESILIENT_MEDIA need to be set for mapped (unsigned) binaries on our
platform.

Binaries are mapped through FileSystem::CreateDataBuffer which returns a
DataBufferLLVM. The latter is backed by a llvm::WritableMemoryBuffer
because every DataBuffer in LLDB is considered to be writable. In order
to use a read-only llvm::MemoryBuffer I had to split our abstraction
around it.

This patch distinguishes between a DataBuffer (read-only) and
WritableDataBuffer (read-write) and updates LLDB to use the appropriate
one.

rdar://74890607

Differential revision: https://reviews.llvm.org/D122856

show more ...

[lldb] Return a DataBuffer from FileSystem::CreateDataBuffer (NFC)

The concrete class (DataBufferLLVM) is an implementation detail.

[lldb] Use a struct to pass function search options to Module::FindFunction

Rather than passing two booleans around, which is especially error prone
with them being next to each other, use a struct

[lldb] Use a struct to pass function search options to Module::FindFunction

Rather than passing two booleans around, which is especially error prone
with them being next to each other, use a struct with named fields
instead.

Differential revision: https://reviews.llvm.org/D107295

show more ...