Revert "[SystemZ][z/OS] Propagate IsText parameter to open text files as text (#107906)"

This reverts commit edf3b277a5f2ebe144827ed47463c22743cac5f9.

[SystemZ][z/OS] Propagate IsText parameter to open text files as text (#107906)

This patch adds an IsText parameter to the following functions
openFileForRead, getBufferForFile, getBufferForFileImp

[SystemZ][z/OS] Propagate IsText parameter to open text files as text (#107906)

This patch adds an IsText parameter to the following functions
openFileForRead, getBufferForFile, getBufferForFileImpl and determines
whether a file is text by querying the file tag on z/OS. The default is
set to OF_Text instead of OF_None, this change in value does not affect
any other platforms other than z/OS.

show more ...

[llvm][unittests] Don't call raw_string_ostream::flush() (NFC)

raw_string_ostream::flush() is essentially a no-op (also specified in docs).
Don't call it in tests that aren't meant to test 'raw_stri

[llvm][unittests] Don't call raw_string_ostream::flush() (NFC)

raw_string_ostream::flush() is essentially a no-op (also specified in docs).
Don't call it in tests that aren't meant to test 'raw_string_ostream' itself.

p.s. remove a few redundant calls to raw_string_ostream::str()

show more ...

[llvm][support] Implement tracing virtual file system (#88326)

LLVM-based tools often use the `llvm::vfs::FileSystem` instrastructure
to access the file system. This patch adds new kind of a VFS th

[llvm][support] Implement tracing virtual file system (#88326)

LLVM-based tools often use the `llvm::vfs::FileSystem` instrastructure
to access the file system. This patch adds new kind of a VFS that
performs lightweight tracing of file system operations on an underlying
VFS. This is supposed to aid in investigating file system traffic
without resorting to instrumentation on the operating system level.
There will be follow-up patches that integrate this into Clang and its
dependency scanner.

show more ...

[llvm] Prefer StringRef::substr to StringRef::slice (NFC) (#106330)

S.substr(N) is simpler than S.slice(N, StringRef::npos). Also, substr
is probably better recognizable than slice thanks to
std::

[llvm] Prefer StringRef::substr to StringRef::slice (NFC) (#106330)

S.substr(N) is simpler than S.slice(N, StringRef::npos). Also, substr
is probably better recognizable than slice thanks to
std::string_view::substr.

show more ...

[llvm] Construct SmallVector with ArrayRef (NFC) (#101872)

[clang] Fix handling of adding a file with the same name as an existing dir to VFS (#94461)

When trying to add a file to clang's VFS via `addFile` and a directory
of the same name already exists, w

[clang] Fix handling of adding a file with the same name as an existing dir to VFS (#94461)

When trying to add a file to clang's VFS via `addFile` and a directory
of the same name already exists, we run into a [out-of-bound
access](https://github.com/llvm/llvm-project/blob/145815c180fc82c5a55bf568d01d98d250490a55/llvm/lib/Support/Path.cpp#L244).

The problem is that the file name is [recognised as existing path](
https://github.com/llvm/llvm-project/blob/145815c180fc82c5a55bf568d01d98d250490a55/llvm/lib/Support/VirtualFileSystem.cpp#L896)
and thus continues to process the next part of the path which doesn't
exist.

This patch adds a check if we have reached the last part of the filename
and return false in that case.
This we reject to add a file if a directory of the same name already
exists.

This is in sync with [this
check](https://github.com/llvm/llvm-project/blob/145815c180fc82c5a55bf568d01d98d250490a55/llvm/lib/Support/VirtualFileSystem.cpp#L903)
that rejects adding a path if a file of the same name already exists.

show more ...

Apply modernize-use-starts-ends-with on llvm-project (#89140)

Run `modernize-use-starts-ends-with` on llvm-project. Two instances are
flagged, minor readability improvements, extremely minor perfor

Apply modernize-use-starts-ends-with on llvm-project (#89140)

Run `modernize-use-starts-ends-with` on llvm-project. Two instances are
flagged, minor readability improvements, extremely minor performance
improvements.

```
python3 clang-tools-extra/clang-tidy/tool/run-clang-tidy.py \
-clang-tidy-binary="build/bin/clang-tidy" \
-clang-apply-replacements-binary="build/bin/clang-apply-replacements" \
-checks="-*,modernize-use-starts-ends-with" \
-header-filter=".*" \
-fix -format
```

I am working on some additions to this check, but they don't seem to
flag any additional cases anyway.

show more ...

[llvm][vfs] Make vfs::FileSystem::exists() virtual NFC (#88575)

Allow a `vfs::FileSystem` to provide a more efficient implementation of
`exists()` if they are able to. The existing `FileSystem`
im

[llvm][vfs] Make vfs::FileSystem::exists() virtual NFC (#88575)

Allow a `vfs::FileSystem` to provide a more efficient implementation of
`exists()` if they are able to. The existing `FileSystem`
implementations continue to default to using `status()` except that
overlay, proxy, and redirecting filesystems are taught to forward calls
to `exists()` correctly to their wrapped/external filesystem.

Co-authored-by: Ben Langmuir <blangmuir@apple.com>

show more ...

[llvm][vfs] NFCI: Remove `const` from `VFS::getRealPath()`

This is an NFC change split from https://github.com/llvm/llvm-project/pull/68645.

[llvm][vfs] Preserve paths for fallback/fallthrough in RedirectingFileSystem (#85307)

When we lookup in the external filesystem, do not remove . and ..
components from the original path. For .. thi

[llvm][vfs] Preserve paths for fallback/fallthrough in RedirectingFileSystem (#85307)

When we lookup in the external filesystem, do not remove . and ..
components from the original path. For .. this is a correctness issue in
the presence of symlinks, while for . it is simply better practice to
preserve the original path to better match the behaviour of other
filesystems. The only modification we need is to apply the working
directory, since it could differ from the external filesystem.

rdar://123655660

show more ...

[clang][DependencyScanner] Remove unused -ivfsoverlay files (#73734)

`-ivfsoverlay` files are unused when building most modules. Enable
removing them by,
* adding a way to visit the filesystem tre

[clang][DependencyScanner] Remove unused -ivfsoverlay files (#73734)

`-ivfsoverlay` files are unused when building most modules. Enable
removing them by,
* adding a way to visit the filesystem tree with extensible RTTI to
access each `RedirectingFileSystem`.
* Adding tracking to `RedirectingFileSystem` to record when it
actually redirects a file access.
* Storing this information in each PCM.

Usage tracking is only enabled when iterating over the source manager
and affecting modulemaps. Here each path is stated to cause an access.
During scanning these stats all hit the cache.

show more ...

[clang] NFC: Remove `{File,Directory}Entry::getName()` (#74910)

The files and directories that Clang accesses are uniqued by their
inode. For each inode `FileManager` will create exactly one `FileE

[clang] NFC: Remove `{File,Directory}Entry::getName()` (#74910)

The files and directories that Clang accesses are uniqued by their
inode. For each inode `FileManager` will create exactly one `FileEntry`
or `DirectoryEntry` object, which makes answering the question _"Are
these two files/directories the same?"_ a simple pointer equality check.

However, since the same inode can be accessed through multiple different
paths, asking the `FileEntry` or `DirectoryEntry` object _"What is your
name?"_ doesn't have clear semantics. In c0ff9908 we started reporting
the most recent name used to access the entry, which turned out to be
necessary for Clang modules. However, the long-term solution has always
been to explicitly track the as-requested name. This has been
implemented in 4dc5573a as `FileEntryRef` and `DirectoryEntryRef`.

The `DirectoryEntry::getName()` interface has been deprecated since the
Clang 17 release and `FileEntry::getName()` since Clang 18. We have
replaced uses of these deprecated APIs in `main` with
`DirectoryEntryRef::getName()` and `FileEntryRef::getName()`
respectively.

This makes it possible to remove `{File,Directory}Entry::getName()` for
good along with the `FileManager` code that implements them.

show more ...

[llvm] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,end

[llvm] Use StringRef::{starts,ends}_with (NFC)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[llvm][vfs] For virtual directories, use the virtual path as the real path

A follow-up to D135841. This patch returns the virtual path for directories from `RedirectingFileSystem`. This ensures the

[llvm][vfs] For virtual directories, use the virtual path as the real path

A follow-up to D135841. This patch returns the virtual path for directories from `RedirectingFileSystem`. This ensures the contents of `Path` are the same as the contents of `FS->getRealPath(Path)`. This also means we can drop the workaround in Clang's module map canonicalization, where we couldn't use the real path for a directory if it resolved to a different `DirectoryEntry`. In addition to that, we can also avoid introducing new workaround for a bug triggered by the newly introduced test case.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D135849

show more ...

[llvm][test] Skip physical filesystem test if unsupported

We are relying on deleting the working directory to trigger a later
error in getcwd, but some platforms (e.g. Solaris) disallow deleting the

[llvm][test] Skip physical filesystem test if unsupported

We are relying on deleting the working directory to trigger a later
error in getcwd, but some platforms (e.g. Solaris) disallow deleting the
working directory of the process. Skip the test in that case.

Differential Revision: https://reviews.llvm.org/D149760

show more ...

[llvm][vfs] Avoid silent fallback to process-wide working directory

In createPhysicalFileSystem, preserve the per-instance working
directory, even after the first call to getcwd fails.

rdar://10821

[llvm][vfs] Avoid silent fallback to process-wide working directory

In createPhysicalFileSystem, preserve the per-instance working
directory, even after the first call to getcwd fails.

rdar://108213753

Differential Revision: https://reviews.llvm.org/D149173

show more ...

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC w

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.

show more ...

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

Revert "[Clang] Give Clang the ability to use a shared stat cache"

This reverts commit c5abe893120b115907376359a5809229a9f9608a.
This reverts commit a033dbbe5c43247b60869b008e67ed86ed230eaa.

This b

Revert "[Clang] Give Clang the ability to use a shared stat cache"

This reverts commit c5abe893120b115907376359a5809229a9f9608a.
This reverts commit a033dbbe5c43247b60869b008e67ed86ed230eaa.

This broke the build with -DLLVM_LINK_LLVM_DYLIB=ON. Reverting while I
investigate.

show more ...

[Clang] Give Clang the ability to use a shared stat cache

Every Clang instance uses an internal FileSystemStatCache to avoid
stating the same content multiple times. However, different instances
of

[Clang] Give Clang the ability to use a shared stat cache

Every Clang instance uses an internal FileSystemStatCache to avoid
stating the same content multiple times. However, different instances
of Clang will contend for filesystem access for their initial stats
during HeaderSearch or module validation.

On some workloads, the time spent in the kernel in these concurrent
stat calls has been measured to be over 20% of the overall compilation
time. This is extremly wassteful when most of the stat calls target
mostly immutable content like a SDK.

This commit introduces a new tool `clang-stat-cache` able to generate
an OnDiskHashmap containing the stat data for a given filesystem
hierarchy.

The driver part of this has been modeled after -ivfsoverlay given
the similarities with what it influences. It introduces a new
-ivfsstatcache driver option to instruct Clang to use a stat cache
generated by `clang-stat-cache`. These stat caches are inserted at
the bottom of the VFS stack (right above the real filesystem).

Differential Revision: https://reviews.llvm.org/D136651

show more ...

[vfs] Allow root paths relative to the vfsoverlay YAML file

This change adds 'root-relative' option in vfsoverlay YAML file format
so the root patchs can be relative to the YAML file directory inste

[vfs] Allow root paths relative to the vfsoverlay YAML file

This change adds 'root-relative' option in vfsoverlay YAML file format
so the root patchs can be relative to the YAML file directory instead of
the current working directory.

Differential Revision: https://reviews.llvm.org/D137473

show more ...

[llvm/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the am

[llvm/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[llvm][vfs] Implement in-memory symlinks

This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Reviewed By: sammccall

Differential

[llvm][vfs] Implement in-memory symlinks

This patch implements symlinks for the in-memory VFS. Original author: @erik.pilkington.

Depends on D117648 & D117649.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D117650

show more ...

Fix a buglet in remove_dots().

The function promises to canonicalize the path, but neglected to do so
for the root component.

For example, calling remove_dots("/tmp/foo.c", Style::windows_backslash

Fix a buglet in remove_dots().

The function promises to canonicalize the path, but neglected to do so
for the root component.

For example, calling remove_dots("/tmp/foo.c", Style::windows_backslash)
resulted in "/tmp\foo.c". Now it produces "\tmp\foo.c".

Also fix FIXME in the corresponding test.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D126412

show more ...