[llvm][Support][Windows] Fix slash in path for remove_directories (#121448)

Before 925471ed903dad871042d7ed0bab89ab6566a564 remove_directories
supports path with slash (instead of backslash).
The

[llvm][Support][Windows] Fix slash in path for remove_directories (#121448)

Before 925471ed903dad871042d7ed0bab89ab6566a564 remove_directories
supports path with slash (instead of backslash).
The ILCreateFromPathW in new implementation requires backslash path,
so the call to remove_directories will fail if the path contains slash.

This is to normalize the path to make sure remove_directories still
support path with slash as well.

show more ...

[NFC] Add explicit #include llvm-config.h where its macros are used. (#106621)

Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial si

[NFC] Add explicit #include llvm-config.h where its macros are used. (#106621)

Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial side effects.

show more ...

[llvm-project] Fix typo "seperate" (#95373)

[Support] Report EISDIR when opening a directory (#79880)

The test `llvm/unittests/Support/CommandLineTest.cpp` that handles
errors in expansion of response files was previously disabled for AIX.

[Support] Report EISDIR when opening a directory (#79880)

The test `llvm/unittests/Support/CommandLineTest.cpp` that handles
errors in expansion of response files was previously disabled for AIX.
Originally the code was dependent on `read` returning `EISDIR` which
occurs on platforms such as Linux. However, other platforms such as AIX
allow use of `read` on file descriptors for directories. This change
updates `readNativeFile` to produce `EISDIR` on AIX and z/OS when used
on a directory (instead of relying on the call to `read` to do so).

---------

Co-authored-by: Alison Zhang <alisonzhang@ibm.com>
Co-authored-by: James Henderson <46713263+jh7370@users.noreply.github.com>

show more ...

[Path] Fix off-by-one in finding filename for win style paths (#78055)

This fixes a crash where `path::parent_path` causes an invalid access on
a string upon receiving a path that consists of a sin

[Path] Fix off-by-one in finding filename for win style paths (#78055)

This fixes a crash where `path::parent_path` causes an invalid access on
a string upon receiving a path that consists of a single colon.

On Windows machine, with runtime checks enabled build, upon `clang -I:
test.cc` produces:
```
Assertion failed: Index < Length && "Invalid index!", file llvm\include\llvm/ADT/StringRef.h, line 232
...
#6 0x00007ff7816201eb `anonymous namespace'::parent_path_end llvm\lib\Support\Path.cpp:144:0
#7 0x00007ff781620135 llvm::sys::path::parent_path(class llvm::StringRef, enum llvm::sys::path::Style) llvm\lib\Support\Path.cpp:470:0
```

Ideally, we can look for the last colon starting from the last
character, but we can instead start from second to last, and handle
empty paths by abusing `0 - 1 == npos`.

show more ...

[ADT] Rename SmallString::{starts,ends}with to {starts,ends}_with (#74916)

This patch renames {starts,ends}with to {starts,ends}_with for
consistency with std::{string,string_view}::{starts,ends}_w

[ADT] Rename SmallString::{starts,ends}with to {starts,ends}_with (#74916)

This patch renames {starts,ends}with to {starts,ends}_with for
consistency with std::{string,string_view}::{starts,ends}_with in
C++20. Since there are only a handful of occurrences, this patch
skips the deprecation phase and simply renames them.

show more ...

[Windows] Avoid using FileIndex for unique IDs

The FileIndex values returned from GetFileInformationByHandle are
considered stable and uniquely identifying a file, as long as the
handle is open. Whe

[Windows] Avoid using FileIndex for unique IDs

The FileIndex values returned from GetFileInformationByHandle are
considered stable and uniquely identifying a file, as long as the
handle is open. When handles are closed, there are no guarantees
for their stability or uniqueness. On some file systems (such as
NTFS), the indices are documented to be stable even across handles.
But with some file systems, in particular network mounts, file
indices can be reused very soon after handles are closed.

When such file indices are used for LLVM's UniqueID, files are
considered duplicates as soon as the filesystem driver happens to
have used the same file index for the handle used to inspect the
file. This caused widespread, non-obvious (seemingly random)
breakage. This can happen e.g. if running on a directory that is
shared via Remote Desktop or VirtualBox.

To avoid the issue, use a hash of the canonicalized path for the
file as unique identifier, instead of using FileIndex.

This fixes https://github.com/llvm/llvm-project/issues/61401 and
https://github.com/llvm/llvm-project/issues/22079.

Performance wise, this adds (usually) one extra call to
GetFinalPathNameByHandleW for each call to getStatus(). A test
cases such as running clang-scan-deps becomes around 1% slower
by this, which is considered tolerable.

Change the equivalent() function to use getUniqueID instead of
checking individual file_status fields. The
equivalent(Twine,Twine,bool& result) function calls status() on
each path successively, without keeping the file handles open,
which also is prone to such false positives. This also gets rid
of checks of other superfluous fields in the
equivalent(file_status, file_status) function - the unique ID of
a file should be enough (that is what is done for Unix anyway).

This comes with one known caveat: For hardlinks, each name for
the file now gets a different UniqueID, and equivalent() considers
them different. While that's not ideal, occasional false negatives
for equivalent() is usually that fatal (the cases where we strictly
do need to deduplicate files with different path names are quite
rare) compared to the issues caused by false positives for
equivalent() (where we'd deduplicate and omit totally distinct files).

The FileIndex is documented to be stable on NTFS though, so ideally
we could maybe have used it in the majority of cases. That would
require a heuristic for whether we can rely on FileIndex or not.
We considered using the existing function is_local_internal for that;
however that caused an unacceptable performance regression
(clang-scan-deps became 38% slower in one test, even more than that
in another test).

Differential Revision: https://reviews.llvm.org/D155579

show more ...

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC w

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.

show more ...

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

[unittests] Use GTEST_SKIP() instead of return when appropriate

Basically NFC: A TEST/TEST_F/etc that bails out early (usually because
setup failed or some other runtime condition wasn't met) genera

[unittests] Use GTEST_SKIP() instead of return when appropriate

Basically NFC: A TEST/TEST_F/etc that bails out early (usually because
setup failed or some other runtime condition wasn't met) generally
should use GTEST_SKIP() to report its status correctly, unless it
takes steps to report another status (e.g., FAIL()).

I did see a handful of tests show up as SKIPPED after this change,
which is not unexpected. The status seemed appropriate in all the new
cases.

show more ...

[llvm][ADT] Replace uses of `makeMutableArrayRef` with deduction guides

Similar to how `makeArrayRef` is deprecated in favor of deduction guides, do the
same for `makeMutableArrayRef`.

Once all of

[llvm][ADT] Replace uses of `makeMutableArrayRef` with deduction guides

Similar to how `makeArrayRef` is deprecated in favor of deduction guides, do the
same for `makeMutableArrayRef`.

Once all of the places in-tree are using the deduction guides for
`MutableArrayRef`, we can mark `makeMutableArrayRef` as deprecated.

Differential Revision: https://reviews.llvm.org/D141814

show more ...

[Support] llvm::Optional => std::optional

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

[llvm/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the am

[llvm/unittests] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Revert "Support: Add mapped_file_region::sync(), equivalent to msync"

This reverts commit 142f51fc2f448845f6a32e767ffaa2b665eea11f.

This shouldn't be committed, it got committed accidentally.

Support: Add mapped_file_region::sync(), equivalent to msync

Add mapped_file_region::sync(), equivalent to POSIX msync,
synchronizing written content to disk without unmapping the region.
Asserts if

Support: Add mapped_file_region::sync(), equivalent to msync

Add mapped_file_region::sync(), equivalent to POSIX msync,
synchronizing written content to disk without unmapping the region.
Asserts if the mode is not mapped_file_region::readwrite.

Note that I don't have access to a Windows machine, so I can't
easily run those unit tests.

Change by dexonsmith

Differential Revision: https://reviews.llvm.org/D95494

show more ...

[llvm] Update FileSystem test that failed spuriously

This test failed spuriously in an environment that appears to ignore the
'x' bit permission on directories. Allow for that possibility.

[llvm] Fix assertion when stat fails in remove_directories

We were dereferencing an empty Optional if IgnoreErrors was true and the
stat failed.

rdar://60887887

Differential Revision: https://revi

[llvm] Fix assertion when stat fails in remove_directories

We were dereferencing an empty Optional if IgnoreErrors was true and the
stat failed.

rdar://60887887

Differential Revision: https://reviews.llvm.org/D131791

show more ...

Fix a buglet in remove_dots().

The function promises to canonicalize the path, but neglected to do so
for the root component.

For example, calling remove_dots("/tmp/foo.c", Style::windows_backslash

Fix a buglet in remove_dots().

The function promises to canonicalize the path, but neglected to do so
for the root component.

For example, calling remove_dots("/tmp/foo.c", Style::windows_backslash)
resulted in "/tmp\foo.c". Now it produces "\tmp\foo.c".

Also fix FIXME in the corresponding test.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D126412

show more ...

[RGT] Use GTEST_SKIP() in more places where we skip a test

Simply returning will report the test as PASSED when it didn't
really do anything. SKIPPED is the correct result for these.

Found by the R

[RGT] Use GTEST_SKIP() in more places where we skip a test

Simply returning will report the test as PASSED when it didn't
really do anything. SKIPPED is the correct result for these.

Found by the Rotten Green Tests project.

show more ...

[Support][unittests] Silence warning when building with Clang 13 on
Windows.

Remove dependency from raw_ostream on <chrono>

The tryLockFor method from raw_fd_sotreamis the sole user of that
header, and it's not referenced in the mono repo. I still chose to keep
it (may be us

Remove dependency from raw_ostream on <chrono>

The tryLockFor method from raw_fd_sotreamis the sole user of that
header, and it's not referenced in the mono repo. I still chose to keep
it (may be useful for downstream user) but added a transient type that's
forward declared to hold the duration parameter.

Notable changes:

- "llvm/Support/Duration.h" must be included in order to use tryLockFor.
- "llvm/Support/raw_ostream.h" no longer includes <chrono>

This sole change has an interesting impact on the number of processed
line, as measured by:

clang++ -E -Iinclude -I../llvm/include ../llvm/lib/Support/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l

before: 7917500
after: 7835142

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup/5831

show more ...

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPE

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPECT_TRUE(A <operator> B).

This commit updates all of llvm/unittests/Support to use these macros
where possible.

This change has not been possible in:
- llvm/unittests/Support/FSUniqueIDTest.cpp - due to not overloading
operators beyond ==, != and <.
- llvm/unittests/Support/BranchProbabilityTest.cpp - where the unchanged
tests are of the operator overloads themselves.

There are other possibilities of this conversion not being valid, which
have not applied in these tests, as they do not use NULL (they use
nullptr), and they do not use const char* (they use std::string or
StringRef).

Reviewed By: mubashar_

Differential Revision: https://reviews.llvm.org/D117319

show more ...

Support: Extract sys::fs::readNativeFileToEOF() from MemoryBuffer

Extract the `readNativeFile()` loop from
`MemoryBuffer::getMemoryBufferForStream()` into `readNativeFileToEOF()`
to allow reuse. The

Support: Extract sys::fs::readNativeFileToEOF() from MemoryBuffer

Extract the `readNativeFile()` loop from
`MemoryBuffer::getMemoryBufferForStream()` into `readNativeFileToEOF()`
to allow reuse. The chunk size is configurable; the default of `4*4096`
is exposed as `sys::fs::DefaultReadChunkSize` to allow sizing of
SmallVectors.

There's somewhere I'd like to read a usually-small file without overhead
of a MemoryBuffer; extracting existing logic rather than duplicating it.

Differential Revision: https://reviews.llvm.org/D115397

show more ...

[Support] [Windows] Convert paths to the preferred form

This normalizes most paths (except ones input from the user as command
line arguments) into the preferred form, if `real_style()` evaluates to

[Support] [Windows] Convert paths to the preferred form

This normalizes most paths (except ones input from the user as command
line arguments) into the preferred form, if `real_style()` evaluates to
`windows_forward`.

Differential Revision: https://reviews.llvm.org/D111880

show more ...

[Support] Add a new path style for Windows with forward slashes

This behaves just like the regular Windows style, with both separator
forms accepted, but with get_separator() returning forward slash

[Support] Add a new path style for Windows with forward slashes

This behaves just like the regular Windows style, with both separator
forms accepted, but with get_separator() returning forward slashes.

Add a more descriptive name for the existing style, keeping the old
name around as an alias initially.

Add a new function `make_preferred()` (like the C++17
`std::filesystem::path` function with the same name), which converts
windows paths to the preferred separator form (while this one works on
any platform and takes a `path::Style` argument).

Contrary to `native()` (just like `make_preferred()` in `std::filesystem`),
this doesn't do anything at all on Posix, it doesn't try to reinterpret
backslashes into forward slashes there.

Differential Revision: https://reviews.llvm.org/D111879

show more ...