[clang][Serialization] Stop including Frontend headers from Serialization (NFC) (#123140)

The Frontend library depends on Serialization. This is an explicit
dependency encoded in the CMake target. H

[clang][Serialization] Stop including Frontend headers from Serialization (NFC) (#123140)

The Frontend library depends on Serialization. This is an explicit
dependency encoded in the CMake target. However, Serialization currently
has an implicit dependency on Frontend, as it includes one of its
headers. This is not reflected in the CMake build rules, but Bazel is
stricter so, in order to avoid a dependency cycle, it hackily declares
the Frontend headers as source files for Serialization.

Fortunately, the only Frontend header used by Serialization is
clang/Frontend/FrontendDiagnostic.h, which is a legacy header that just
includes clang/Basic/DiagnosticFrontend since
d076608d58d1ec55016eb747a995511e3a3f72aa, back in 2018.

This commit changes Serialization to use the underlying header from
Basic instead. Both Serialization and Frontend depend on Basic, so this
breaks the dependency cycle.

show more ...

[clang] Allow generating module interfaces with parsing errors (#121485)

Fixes a regression introduced in commit
da00c60dae0040185dc45039c4397f6e746548e9

This functionality was originally added

[clang] Allow generating module interfaces with parsing errors (#121485)

Fixes a regression introduced in commit
da00c60dae0040185dc45039c4397f6e746548e9

This functionality was originally added in commit
5834996fefc937d6211dc8c8a5b200068753391a

Co-authored-by: Tomasz Kaminski <tomasz.kaminski@sonarsource.com>

show more ...

[clang][deps] Only write preprocessor info into PCMs (#115239)

This patch builds on top of
https://github.com/llvm/llvm-project/pull/115237 and
https://github.com/llvm/llvm-project/pull/115235, on

[clang][deps] Only write preprocessor info into PCMs (#115239)

This patch builds on top of
https://github.com/llvm/llvm-project/pull/115237 and
https://github.com/llvm/llvm-project/pull/115235, only passing the
`Preprocessor` object to `ASTWriter`. This reduces the size of scanning
PCM files by 1/3 and speeds up scans by 16%.

show more ...

[clang][serialization] Enable `ASTWriter` to work with `Preprocessor` only (#115237)

This PR builds on top of
https://github.com/llvm/llvm-project/pull/115235 and makes it possible
to call `ASTWri

[clang][serialization] Enable `ASTWriter` to work with `Preprocessor` only (#115237)

This PR builds on top of
https://github.com/llvm/llvm-project/pull/115235 and makes it possible
to call `ASTWriter::WriteAST()` with `Preprocessor` only instead of full
`Sema` object. So far, there are no clients that leverage the new
capability - that will come in a follow-up commit.

show more ...

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to genera… (#90570)

…te module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the mo

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to genera… (#90570)

…te module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C++20 modules. But this is slighty more or less odd. This patch tries to
use a new class 'CXX20ModulesGenerator' to generate the module file for
C++20 modules.

show more ...

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrec

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrectly. And this problem get revealed by another change
recently. So this patch tried to rever the previous "optimization"
partially.

show more ...

Revert "[C++20] [Modules] Don't skip pragma diagnostic mappings"
and "[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to
generate module file for C++20 modules instead of PCHGenerator"

T

Revert "[C++20] [Modules] Don't skip pragma diagnostic mappings"
and "[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to
generate module file for C++20 modules instead of PCHGenerator"

This reverts commit fb21343473e33e9a886b42d2fe95d1cec1cd0030.
and commit 18268ac0f48d93c2bcddb69732761971669c09ab.

It looks like there are some problems about linking the compiler

show more ...

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrec

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrectly. And this problem get revealed by another change
recently. So this patch tried to rever the previous "optimization"
partially.

show more ...

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to generate module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to generate module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C++20 modules. But this is slighty more or less odd. This patch tries
to use a new class 'CXX20ModulesGenerator' to generate the module file
for C++20 modules.

show more ...

[unused-includes][Serialization] Remove unused includes. NFC. (#88790)

[NFC] Check the nullness of pointer before dereference it in the assertion

This was part of https://github.com/llvm/llvm-project/pull/85050.

It is suggested to split the unrelated change as much as

[NFC] Check the nullness of pointer before dereference it in the assertion

This was part of https://github.com/llvm/llvm-project/pull/85050.

It is suggested to split the unrelated change as much as possible. So
here is the patch.

show more ...

[NFC] [C++20] [Modules] [Reduced BMI] Make sure the size of reduced BMI is not large than full BMI

Before this patch, the size of the reduced BMI may be large than the
full BMI when the source codes

[NFC] [C++20] [Modules] [Reduced BMI] Make sure the size of reduced BMI is not large than full BMI

Before this patch, the size of the reduced BMI may be large than the
full BMI when the source codes is pretty small. This violates the design
principles. The root cause is an oversight that we skipped something
in full BMI but forgot to make it in reduced BMI.

show more ...

[NFC] [C++20] [Modules] Refactor ReducedBMIGenerator

Changes:
- Don't lookup the emitting module from HeaderSearch. We will use the
module from the ASTContext directly.
- Remove some useless argum

[NFC] [C++20] [Modules] Refactor ReducedBMIGenerator

Changes:
- Don't lookup the emitting module from HeaderSearch. We will use the
module from the ASTContext directly.
- Remove some useless arguments. Let's addback in the future if
required.

show more ...

[C++20] [Modules] Introduce reduced BMI (#75894)

Close https://github.com/llvm/llvm-project/issues/71034

See

https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/747

[C++20] [Modules] Introduce reduced BMI (#75894)

Close https://github.com/llvm/llvm-project/issues/71034

See

https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755

This patch introduces reduced BMI, which doesn't contain the definitions
of functions and variables if its definitions won't contribute to the
ABI.

Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

The user interfaces part it left to following patches to ease the
reviewing.

show more ...

Revert "Revert "Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)""

This reverts commit a6acf3fd49a20c570a390af2a3c84e10b9545b

Revert "Revert "Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)""

This reverts commit a6acf3fd49a20c570a390af2a3c84e10b9545b68 and
relands a50e63b38b931d945f97eac882278068221eca17. The original revert
was done by mistake.

show more ...

Revert "Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)"

This reverts commit a50e63b38b931d945f97eac882278068221eca17.

With

Revert "Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)"

This reverts commit a50e63b38b931d945f97eac882278068221eca17.

With clang-14.0.6 as the host compiler, I'm getting:

ld.lld: error: undefined symbol: clang::ASTWriter::WriteAST(clang::Sema&, llvm::StringRef, clang::Module*, llvm::StringRef, bool, bool)
>>> referenced by ASTUnit.cpp
>>> ASTUnit.cpp.o:(clang::ASTUnit::serialize(llvm::raw_ostream&)) in archive lib/libclangFrontend.a

show more ...

Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)

The issue #53952 is reported indicating clang is giving a crashing pch
file

Fixes and closes #53952. Setting the ASTHasCompilerErrors member variable correctly based on the PP diagnostics. (#68127)

The issue #53952 is reported indicating clang is giving a crashing pch
file, when hasErrors is been passed incorrectly to WriteAST method.

To fix the issue, the parameter has been removed and instead we're
relying on the results of `hasUncompilableErrorOccured()` instead of
letting the caller override it.

Fixes https://github.com/llvm/llvm-project/issues/53952

show more ...

[clang][modules] Avoid storing command-line macro definitions into implicitly built PCM files

With implicit modules, it's impossible to load a PCM file that was built using different command-line ma

[clang][modules] Avoid storing command-line macro definitions into implicitly built PCM files

With implicit modules, it's impossible to load a PCM file that was built using different command-line macro definitions. This is guaranteed by the fact that they contribute to the context hash. This means that we don't need to store those macros into PCM files for validation purposes. This patch avoids serializing them in those circumstances, since there's no other use for command-line macro definitions (besides "-module-file-info").

For a typical Apple project, this speeds up the dependency scan by 5.6% and shrinks the cache with scanning PCMs by 26%.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D158136

show more ...

[Serialization] Remove `ORIGINAL_PCH_DIR` record

Use of `ORIGINAL_PCH_DIR` record has been superseeded by making PCH/PCM files with relocatable paths at write time.
Removing this record is useful fo

[Serialization] Remove `ORIGINAL_PCH_DIR` record

Use of `ORIGINAL_PCH_DIR` record has been superseeded by making PCH/PCM files with relocatable paths at write time.
Removing this record is useful for producing an output-path-independent PCH file and enable sharing of the same PCH file even
when it was intended for a different output path.

Differential Revision: https://reviews.llvm.org/D131124

show more ...

[ASTWriter] Provide capability to output a PCM/PCH file that does not write out information about its output path

This is useful to enable sharing of the same PCH file even when it's intended for a

[ASTWriter] Provide capability to output a PCM/PCH file that does not write out information about its output path

This is useful to enable sharing of the same PCH file even when it's intended for a different output path.

The only information this option disables writing is for `ORIGINAL_PCH_DIR` record which is treated as optional and (when present) used as fallback for resolving input file paths relative to it.

Differential Revision: https://reviews.llvm.org/D130710

show more ...

[clang][modules] NFC: Propagate import `SourceLocation` into `HeaderSearch::lookupModule`

This patch propagates the import `SourceLocation` into `HeaderSearch::lookupModule`. This enables remarks on

[clang][modules] NFC: Propagate import `SourceLocation` into `HeaderSearch::lookupModule`

This patch propagates the import `SourceLocation` into `HeaderSearch::lookupModule`. This enables remarks on search path usage (implemented in D102923) to point to the source code that initiated header search.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D111557

show more ...

[pch] Honour -fallow-pch-with-compiler-errors for overall compilation status

Previously we would emit a PCH with errors, but fail the overall
compilation. If run using the driver, that would result

[pch] Honour -fallow-pch-with-compiler-errors for overall compilation status

Previously we would emit a PCH with errors, but fail the overall
compilation. If run using the driver, that would result in removing the
just-produced PCH. Instead, we should have the compilation result match
whether we were able to emit the PCH.

Differential Revision: https://reviews.llvm.org/D77159

rdar://61110294

show more ...

[Bitcode] Move Bitstream to a separate library

This moves Bitcode/Bitstream*, Bitcode/BitCodes.h to Bitstream/.

This is needed to avoid a circular dependency when using the bitstream
code for parsi

[Bitcode] Move Bitstream to a separate library

This moves Bitcode/Bitstream*, Bitcode/BitCodes.h to Bitstream/.

This is needed to avoid a circular dependency when using the bitstream
code for parsing optimization remarks.

Since Bitcode uses Core for the IR part:

libLLVMRemarks -> Bitcode -> Core

and Core uses libLLVMRemarks to generate remarks (see
IR/RemarkStreamer.cpp):

Core -> libLLVMRemarks

we need to separate the Bitstream and Bitcode part.

For clang-doc, it seems that it doesn't need the whole bitcode layer, so
I updated the CMake to only use the bitstream part.

Differential Revision: https://reviews.llvm.org/D63899

llvm-svn: 365091

show more ...

Modules: Add LangOptions::CacheGeneratedPCH

Add an option to cache the generated PCH in the ModuleCache when
emitting it. This protects clients that build PCHs and read them in the
same process, al

Modules: Add LangOptions::CacheGeneratedPCH

Add an option to cache the generated PCH in the ModuleCache when
emitting it. This protects clients that build PCHs and read them in the
same process, allowing them to avoid race conditions between parallel
jobs the same way that Clang's implicit module build system does.

rdar://problem/48740787

llvm-svn: 355950

show more ...

Modules: Rename MemoryBufferCache to InMemoryModuleCache

Change MemoryBufferCache to InMemoryModuleCache, moving it from Basic to
Serialization. Another patch will start using it to manage module b

Modules: Rename MemoryBufferCache to InMemoryModuleCache

Change MemoryBufferCache to InMemoryModuleCache, moving it from Basic to
Serialization. Another patch will start using it to manage module build
more explicitly, but this is split out because it's mostly mechanical.

Because of the move to Serialization we can no longer abuse the
Preprocessor to forward it to the ASTReader. Besides the rename and
file move, that means Preprocessor::Preprocessor has one fewer parameter
and ASTReader::ASTReader has one more.

llvm-svn: 355777

show more ...