Revert "[clang] Canonicalize absolute paths in dependency file" (#121638)

Reverts llvm/llvm-project#117458

https://github.com/llvm/llvm-project/pull/117458#issuecomment-2568804774

https://gith

Revert "[clang] Canonicalize absolute paths in dependency file" (#121638)

Reverts llvm/llvm-project#117458

https://github.com/llvm/llvm-project/pull/117458#issuecomment-2568804774

https://github.com/ninja-build/ninja/issues/2528

show more ...

[clang] Canonicalize absolute paths in dependency file (#117458)

This fixes #117438.

If paths in dependency file are not absoulte, make (or ninja) will
canonicalize them.
While their canonicali

[clang] Canonicalize absolute paths in dependency file (#117458)

This fixes #117438.

If paths in dependency file are not absoulte, make (or ninja) will
canonicalize them.
While their canonicalization does not involves symbolic links expansion
(for IO performance concerns), leaving a non-absolute path in dependency
file may lead to unexpected canonicalization.
For example, '/a/../b', where '/a' is a symlink to '/c/d', it should be
'/c/b' but make (and ninja) canonicalizes it as '/b', and fails for file
not found.

show more ...

Reland [clang][Sema, Lex, Parse] Preprocessor embed in C and C++ (#95802)

This commit implements the entirety of the now-accepted [N3017
-Preprocessor
Embed](https://www.open-std.org/jtc1/sc22/wg1

Reland [clang][Sema, Lex, Parse] Preprocessor embed in C and C++ (#95802)

This commit implements the entirety of the now-accepted [N3017
-Preprocessor
Embed](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3017.htm) and
its sister C++ paper [p1967](https://wg21.link/p1967). It implements
everything in the specification, and includes an implementation that
drastically improves the time it takes to embed data in specific
scenarios (the initialization of character type arrays). The mechanisms
used to do this are used under the "as-if" rule, and in general when the
system cannot detect it is initializing an array object in a variable
declaration, will generate EmbedExpr AST node which will be expanded by
AST consumers (CodeGen or constant expression evaluators) or expand
embed directive as a comma expression.

This reverts commit
https://github.com/llvm/llvm-project/commit/682d461d5a231cee54d65910e6341769419a67d7.

---------

Co-authored-by: The Phantom Derpstorm <phdofthehouse@gmail.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
Co-authored-by: H. Vetinari <h.vetinari@gmx.com>

show more ...

Revert "✨ [Sema, Lex, Parse] Preprocessor embed in C and C++ (and Obj-C and Obj-C++ by-proxy)" (#95299)

Reverts llvm/llvm-project#68620

Introduce or expose a memory leak and UB, see llvm/llvm-pro

Revert "✨ [Sema, Lex, Parse] Preprocessor embed in C and C++ (and Obj-C and Obj-C++ by-proxy)" (#95299)

Reverts llvm/llvm-project#68620

Introduce or expose a memory leak and UB, see llvm/llvm-project#68620

show more ...

[clang][Sema, Lex, Parse] Preprocessor embed in C and C++ (and Obj-C and Obj-C++ by-proxy) (#68620)

This commit implements the entirety of the now-accepted [N3017 -
Preprocessor
Embed](https://www

[clang][Sema, Lex, Parse] Preprocessor embed in C and C++ (and Obj-C and Obj-C++ by-proxy) (#68620)

This commit implements the entirety of the now-accepted [N3017 -
Preprocessor
Embed](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3017.htm) and
its sister C++ paper [p1967](https://wg21.link/p1967). It implements
everything in the specification, and includes an implementation that
drastically improves the time it takes to embed data in specific
scenarios (the initialization of character type arrays). The mechanisms
used to do this are used under the "as-if" rule, and in general when the
system cannot detect it is initializing an array object in a variable
declaration, will generate EmbedExpr AST node which will be expanded
by AST consumers (CodeGen or constant expression evaluators) or
expand embed directive as a comma expression.

---------

Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
Co-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Podchishchaeva, Mariya <mariya.podchishchaeva@intel.com>

show more ...

[clang][lex] Always pass suggested module to `InclusionDirective()` callback (#81061)

This patch provides more information to the
`PPCallbacks::InclusionDirective()` hook. We now always pass the
s

[clang][lex] Always pass suggested module to `InclusionDirective()` callback (#81061)

This patch provides more information to the
`PPCallbacks::InclusionDirective()` hook. We now always pass the
suggested module, regardless of whether it was actually imported or not.
The extra `bool ModuleImported` parameter then denotes whether the
header `#include` will be automatically translated into import the the
module.

The main change is in `clang/lib/Lex/PPDirectives.cpp`, where we take
care to not modify `SuggestedModule` after it's been populated by
`LookupHeaderIncludeOrImport()`. We now exclusively use the `SM`
(`ModuleToImport`) variable instead, which has been equivalent to
`SuggestedModule` until now. This allows us to use the original
non-modified `SuggestedModule` for the callback itself.

(This patch turns out to be necessary for
https://github.com/apple/llvm-project/pull/8011).

show more ...

Revert "Reland [clang] Canonicalize system headers in dependency file when -canonical-prefixes" (#71697)

This reverts commit 578a4716f549167165a2ec3bac89c86706136d4e.

This causes multiple issues.

Revert "Reland [clang] Canonicalize system headers in dependency file when -canonical-prefixes" (#71697)

This reverts commit 578a4716f549167165a2ec3bac89c86706136d4e.

This causes multiple issues. Compile time slowdown due to more path
canonicalization, and weird behavior on Windows.

Will reland under a separate flag `-f[no-]canonical-system-headers` to
match gcc in the future and further limit when it's passed by default.

Fixes #70011.

show more ...

[clang] NFCI: Use `FileEntryRef` in `FileManager::getCanonicalName()`

[clang] NFCI: Use `FileEntryRef` in `ModuleMapCallbacks`

Reland [clang] Canonicalize system headers in dependency file when -canonical-prefixes

Clang was writing paths to the dependency file that don't exist when using a sysroot with symlinks, causing eve

Reland [clang] Canonicalize system headers in dependency file when -canonical-prefixes

Clang was writing paths to the dependency file that don't exist when using a sysroot with symlinks, causing everything to get rebuilt every time. This is reproducible on Linux by creating a symlink to '/', using that as the sysroot, and trying to build something with ninja that includes the C++ stdlib (e.g. a typical build of LLVM).

This fixes https://github.com/ninja-build/ninja/issues/1330 and somewhat matches gcc.

gcc canonicalizes system headers in dependency files under a -f[no-]canonical-system-headers, but it makes more sense to look at -canonical-prefixes.

D37954 was a previous attempt at this.

Fixed use of %T instead of %t in test, causing bots to fail the test on the initial commit.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D149187

show more ...

Revert "[clang] Canonicalize system headers in dependency file when -canonical-prefixes"

This reverts commit 8f0dd4ef3ed2137d1e2554204127434fc46ee190.

Causes bot failures: https://lab.llvm.org/buil

Revert "[clang] Canonicalize system headers in dependency file when -canonical-prefixes"

This reverts commit 8f0dd4ef3ed2137d1e2554204127434fc46ee190.

Causes bot failures: https://lab.llvm.org/buildbot/#builders/139/builds/39964

show more ...

[clang] Canonicalize system headers in dependency file when -canonical-prefixes

Clang was writing paths to the dependency file that don't exist when using a sysroot with symlinks, causing everything

[clang] Canonicalize system headers in dependency file when -canonical-prefixes

Clang was writing paths to the dependency file that don't exist when using a sysroot with symlinks, causing everything to get rebuilt every time. This is reproducible on Linux by creating a symlink to '/', using that as the sysroot, and trying to build something with ninja that includes the C++ stdlib (e.g. a typical build of LLVM).

This fixes https://github.com/ninja-build/ninja/issues/1330 and somewhat matches gcc.

gcc canonicalizes system headers in dependency files under a -f[no-]canonical-system-headers, but it makes more sense to look at -canonical-prefixes.

D37954 was a previous attempt at this.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D149187

show more ...

[clang] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".

This is p

[clang] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[clang] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Option

[clang] Add #include <optional> (NFC)

This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[clang] Report the on-disk paths for inputs to module compiles

Since D135636, PCM files contain the "as requested" path of input files. The machinery for generating dependency files reports those pa

[clang] Report the on-disk paths for inputs to module compiles

Since D135636, PCM files contain the "as requested" path of input files. The machinery for generating dependency files reports those paths as they appeared in the PCM file, which may confuse consumers that are not aware of VFS overlays that might've been in place at compile-time.

This patch makes sure the "use-external-name" setting is being respected when generating dependency files in modular builds by piping the paths serialized in PCMs through `FileEntryRef::getName()` before putting them into dependency files.

rdar://103459532

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D141644

show more ...

[Clang] Prepare for llvm::Optional becoming std::optional.

The needed tweaks are mostly trivial, the one nasty bit is Clang's usage
of OptionalStorage. To keep this working old Optional stays around

[Clang] Prepare for llvm::Optional becoming std::optional.

The needed tweaks are mostly trivial, the one nasty bit is Clang's usage
of OptionalStorage. To keep this working old Optional stays around as
clang::CustomizableOptional, with the default Storage removed.
Optional<File/DirectoryEntryRef> is replaced with a typedef.

I tested this with GCC 7.5, the oldest supported GCC I had around.

Differential Revision: https://reviews.llvm.org/D140332

show more ...

Revert "[clang] Convert OptionalFileEntryRefDegradesToFileEntryPtr to std::optional"

This reverts commit 8f0df9f3bbc6d7f3d5cbfd955c5ee4404c53a75d.

The Optional*RefDegradesTo*EntryPtr types want to

Revert "[clang] Convert OptionalFileEntryRefDegradesToFileEntryPtr to std::optional"

This reverts commit 8f0df9f3bbc6d7f3d5cbfd955c5ee4404c53a75d.

The Optional*RefDegradesTo*EntryPtr types want to keep the same size as
the underlying type, which std::optional doesn't guarantee. For use with
llvm::Optional, they define their own storage class, and there is no way
to do that in std::optional.

On top of that, that commit broke builds with older GCCs, where
std::optional was not trivially copyable (static_assert in the clang
sources was failing).

show more ...

[clang] Convert OptionalFileEntryRefDegradesToFileEntryPtr to std::optional

Don't include StringSwitch (NFC)

These files do not use llvm::StringSwitch.

[Frontend] -MP: remove blank lines

GCC 10 removed blank lines for phony targets during a refactoring.
The blank lines seems unuseful, so let's follow suit.

[Frontend] Fix -MP when main file is <stdin>

rC220726 had a bug: `echo "<cstdlib>" | clang -M -MP -x c++ - 2>/dev/null`
(used by glibc/configure.ac find_cxx_header) omitted a `cstdlib:` line. Instea

[Frontend] Fix -MP when main file is <stdin>

rC220726 had a bug: `echo "<cstdlib>" | clang -M -MP -x c++ - 2>/dev/null`
(used by glibc/configure.ac find_cxx_header) omitted a `cstdlib:` line. Instead
of filtering out `<stdin>` in `Dependencies`, retain it (so that the number of
entries does not change whether or not main file is `<stdin>`) and filter the
`PhonyTarget` output.

show more ...

[Lex] Introduce `PPCallbacks::LexedFileChanged()` preprocessor callback

This is a preprocessor callback focused on the lexed file changing, without conflating effects of line number directives and o

[Lex] Introduce `PPCallbacks::LexedFileChanged()` preprocessor callback

This is a preprocessor callback focused on the lexed file changing, without conflating effects of line number directives and other pragmas.
A client that only cares about what files the lexer processes, like dependency generation, can use this more straightforward
callback instead of `PPCallbacks::FileChanged()`. Clients that want the pragma directive effects as well can keep using `FileChanged()`.

A use case where `PPCallbacks::LexedFileChanged()` is particularly simpler to use than `FileChanged()` is in a situation
where a client wants to keep track of lexed file changes that include changes from/to the predefines buffer, where it becomes
unnecessary complicated trying to use `FileChanged()` while filtering out the pragma directives effects callbacks.

Also take the opportunity to provide information about the prior `FileID` the `Lexer` moved from, even when entering a new file.

Differential Revision: https://reviews.llvm.org/D128947

show more ...

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<

[clang][lex] NFCI: Use FileEntryRef in PPCallbacks::InclusionDirective()

This patch changes type of the `File` parameter in `PPCallbacks::InclusionDirective()` from `const FileEntry *` to `Optional<FileEntryRef>`.

With the API change in place, this patch then removes some uses of the deprecated `FileEntry::getName()` (e.g. in `DependencyGraph.cpp` and `ModuleDependencyCollector.cpp`).

Reviewed By: dexonsmith, bnbarham

Differential Revision: https://reviews.llvm.org/D123574

show more ...

[clang] In DependencyCollector on Windows, ignore case and separators when discarding duplicate dependency file paths.

This patch removes duplicates also encountered in the output of clang-scan-deps

[clang] In DependencyCollector on Windows, ignore case and separators when discarding duplicate dependency file paths.

This patch removes duplicates also encountered in the output of clang-scan-deps when one same header file is encountered with different casing and/or different separators ('/' vs '\').

The case of separators can appear when the same file is included externally by
`#include <folder/file.h>`

whereas a file from the same folder does
`#include "file.h"`

Under Windows, clang computes the paths using '/' from the include directive, the `\` from the -I options, and the concatenations use the native `\`, leading to internal paths containing a mix of both separators.

Differential Revision: https://reviews.llvm.org/D102339

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...