[Frontend] Add flag to allow PCM generation despite compiler errors

As with precompiled headers, it's useful for indexers to be able to
continue through compiler errors in dependent modules.

Resolv

[Frontend] Add flag to allow PCM generation despite compiler errors

As with precompiled headers, it's useful for indexers to be able to
continue through compiler errors in dependent modules.

Resolves rdar://69816264

Reviewed By: akyrtzi

Differential Revision: https://reviews.llvm.org/D91580

show more ...

Allow searching for prebuilt implicit modules.

This reverts commit c67656b994c87224e0b33e2c4b09093986a5cfa6, and addresses the
build issue.

Revert "Allow searching for prebuilt implicit modules."

This reverts commit 71e108cd86e70b06c5fa3a63689dcb3555c3d13f.

This change caused a build failure on Windows:
http://lab.llvm.org:8011/#/build

Revert "Allow searching for prebuilt implicit modules."

This reverts commit 71e108cd86e70b06c5fa3a63689dcb3555c3d13f.

This change caused a build failure on Windows:
http://lab.llvm.org:8011/#/builders/83/builds/570

show more ...

Allow searching for prebuilt implicit modules.

The behavior is controlled by the `-fprebuilt-implicit-modules` option, and
allows searching for implicit modules in the prebuilt module cache paths.

Allow searching for prebuilt implicit modules.

The behavior is controlled by the `-fprebuilt-implicit-modules` option, and
allows searching for implicit modules in the prebuilt module cache paths.

The current command-line options for prebuilt modules do not allow to easily
maintain and use multiple versions of modules. Both the producer and users of
prebuilt modules are required to know the relationships between compilation
options and module file paths. Using a particular version of a prebuilt module
requires passing a particular option on the command line (e.g.
`-fmodule-file=[<name>=]<file>` or `-fprebuilt-module-path=<directory>`).

However the compiler already knows how to distinguish and automatically locate
implicit modules. Hence this proposal to introduce the
`-fprebuilt-implicit-modules` option. When set, it enables searching for
implicit modules in the prebuilt module paths (specified via
`-fprebuilt-module-path`). To not modify existing behavior, this search takes
place after the standard search for prebuilt modules. If not

Here is a workflow illustrating how both the producer and consumer of prebuilt
modules would need to know what versions of prebuilt modules are available and
where they are located.

clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v1 <config 1 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v2 <config 2 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules_v3 <config 3 options>

clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules_v1 <config 1 options>
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap <non-prebuilt config options>

With prebuilt implicit modules, the producer can generate prebuilt modules as
usual, all in the same output directory. The same mechanisms as for implicit
modules take care of incorporating hashes in the path to distinguish between
module versions.

Note that we do not specify the output module filename, so `-o` implicit modules are generated in the cache path `prebuilt_modules`.

clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 1 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 2 options>
clang -cc1 -x c modulemap -fmodules -emit-module -fmodule-name=foo -fmodules-cache-path=prebuilt_modules <config 3 options>

The user can now simply enable prebuilt implicit modules and point to the
prebuilt modules cache. No need to "parse" command-line options to decide
what prebuilt modules (paths) to use.

clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules -fprebuilt-implicit-modules <config 1 options>
clang -cc1 -x c use.c -fmodules fmodule-map-file=modulemap -fprebuilt-module-path=prebuilt_modules -fprebuilt-implicit-modules <non-prebuilt config options>

This is for example particularly useful in a use-case where compilation is
expensive, and the configurations expected to be used are predictable, but not
controlled by the producer of prebuilt modules. Modules for the set of
predictable configurations can be prebuilt, and using them does not require
"parsing" the configuration (command-line options).

Reviewed By: Bigcheese

Differential Revision: https://reviews.llvm.org/D68997

show more ...

ContentCache: Simplify by always owning the MemoryBuffer

This changes `ContentCache::Buffer` to use
`std::unique_ptr<MemoryBuffer>` instead of the `PointerIntPair`. It
drops the (mostly unused) `DoN

ContentCache: Simplify by always owning the MemoryBuffer

This changes `ContentCache::Buffer` to use
`std::unique_ptr<MemoryBuffer>` instead of the `PointerIntPair`. It
drops the (mostly unused) `DoNotFree` bit, instead creating a (new)
non-owning `MemoryBuffer` instance when passed a `MemoryBufferRef`.

Differential Revision: https://reviews.llvm.org/D67030

show more ...

clang/Frontend: Use MemoryBufferRef in FrontendInputFile (and remove SourceManager::getBuffer)

In order to drop the final callers to `SourceManager::getBuffer`, change
`FrontendInputFile` to use `Op

clang/Frontend: Use MemoryBufferRef in FrontendInputFile (and remove SourceManager::getBuffer)

In order to drop the final callers to `SourceManager::getBuffer`, change
`FrontendInputFile` to use `Optional<MemoryBufferRef>`. Also updated
the "unowned" version of `SourceManager::createFileID` to take a
`MemoryBufferRef` (it now calls `MemoryBuffer::getMemBuffer`, which
creates a `MemoryBuffer` that does not own the buffer data).

Differential Revision: https://reviews.llvm.org/D89427

show more ...

Revert "Reland "[Modules] Add stats to measure performance of building and loading modules.""

This reverts commit 4000c9ee18ecebe3ff0f197af8c1fb434ad986e5.

Test "LLVM :: Other/statistic.ll" is fail

Revert "Reland "[Modules] Add stats to measure performance of building and loading modules.""

This reverts commit 4000c9ee18ecebe3ff0f197af8c1fb434ad986e5.

Test "LLVM :: Other/statistic.ll" is failing on Windows.

show more ...

Reland "[Modules] Add stats to measure performance of building and loading modules."

Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header

Reland "[Modules] Add stats to measure performance of building and loading modules."

Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header search. High-level operations
like building a module or processing a .pcm file are motivated by
previous issues where clang was re-building modules or re-reading .pcm
files unnecessarily. Fixed-cost operations like `stat` calls are tracked
because clang cannot change how long each operation takes but it can
perform fewer of such operations to improve the compile time.

Also tracking such stats over time can help us detect compile-time
regressions. Added stats are more stable than the actual measured
compilation time, so expect the detected regressions to be less noisy.

On relanding drop stats in MemoryBuffer.cpp as their value is pretty low
but affects a lot of clients and many of those aren't interested in
modules and header search.

rdar://problem/55715134

Reviewed By: aprantl, bruno

Differential Revision: https://reviews.llvm.org/D86895

show more ...

Revert "[Modules] Add stats to measure performance of building and loading modules."

This reverts commit c4bacc3c9b333bb7032fb96f41d6f5b851623132.

Test "LLVM :: ThinLTO/X86/funcimport-stats.ll" is

Revert "[Modules] Add stats to measure performance of building and loading modules."

This reverts commit c4bacc3c9b333bb7032fb96f41d6f5b851623132.

Test "LLVM :: ThinLTO/X86/funcimport-stats.ll" is failing. Reverting now
and will recommit after making the test not fail with the added stats.

show more ...

[Modules] Add stats to measure performance of building and loading modules.

Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header search.

[Modules] Add stats to measure performance of building and loading modules.

Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header search. High-level operations
like building a module or processing a .pcm file are motivated by
previous issues where clang was re-building modules or re-reading .pcm
files unnecessarily. Fixed-cost operations like `stat` calls are tracked
because clang cannot change how long each operation takes but it can
perform fewer of such operations to improve the compile time.

Also tracking such stats over time can help us detect compile-time
regressions. Added stats are more stable than the actual measured
compilation time, so expect the detected regressions to be less noisy.

rdar://problem/55715134

Reviewed By: aprantl, bruno

Differential Revision: https://reviews.llvm.org/D86895

show more ...

Reland "[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support."

We currently have strict floating point/constrained floating point enabled
for all targets. Constrained

Reland "[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support."

We currently have strict floating point/constrained floating point enabled
for all targets. Constrained SDAG nodes get converted to the regular ones
before reaching the target layer. In theory this should be fine.

However, the changes are exposed to users through multiple clang options
already in use in the field, and the changes are _completely_ _untested_
on almost all of our targets. Bugs have already been found, like
"https://bugs.llvm.org/show_bug.cgi?id=45274".

This patch disables constrained floating point options in clang everywhere
except X86 and SystemZ. A warning will be printed when this happens.

Use the new -fexperimental-strict-floating-point flag to force allowing
strict floating point on hosts that aren't already marked as supporting
it (X86 and SystemZ).

Differential Revision: https://reviews.llvm.org/D80952

show more ...

Revert "[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support."

My mistake, I had a blocking reviewer.

This reverts commit 39d2ae0afb2312a15e4d15a0855b35b4e1c49fc4.
T

Revert "[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support."

My mistake, I had a blocking reviewer.

This reverts commit 39d2ae0afb2312a15e4d15a0855b35b4e1c49fc4.
This reverts commit bfdafa32a0fa4b2745627fe57dd253db10ac3fcf.
This reverts commit 2b35511350454dd22997f129ee529e3fdb129ac2.

Differential Revision: https://reviews.llvm.org/D80952

show more ...

[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support.

We currently have strict floating point/constrained floating point enabled
for all targets. Constrained SDAG nod

[FPEnv][Clang][Driver] Disable constrained floating point on targets lacking support.

We currently have strict floating point/constrained floating point enabled
for all targets. Constrained SDAG nodes get converted to the regular ones
before reaching the target layer. In theory this should be fine.

However, the changes are exposed to users through multiple clang options
already in use in the field, and the changes are _completely_ _untested_
on almost all of our targets. Bugs have already been found, like
"https://bugs.llvm.org/show_bug.cgi?id=45274".

This patch disables constrained floating point options in clang everywhere
except X86 and SystemZ. A warning will be printed when this happens.

Differential Revision: https://reviews.llvm.org/D80952

show more ...

[clang][NFC] Removed unused parameters in InitializeSourceManager

Rename IsMissingRequirement to IsUnimportable and set it for shadowed
modules too.

This more accurately reflects the semantics of this flag, as distinct
from "IsAvailable", which (in an explicit mod

Rename IsMissingRequirement to IsUnimportable and set it for shadowed
modules too.

This more accurately reflects the semantics of this flag, as distinct
from "IsAvailable", which (in an explicit modules world) only describes
whether a module is buildable, not whether it's importable.

show more ...

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in FileManager.h and LLVM's FS headers.

The main code change here is to sink SM::createFileID into the cpp file.
I reason that this is not performance critical because it doesn't happen
on the diagnostic path, it happens along the paths of macro expansion
(could be hot) and new includes (less hot).

Saves some includes:
309 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileManager.h
272 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileSystemOptions.h
271 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/VirtualFileSystem.h
267 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/FileSystem.h
266 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Chrono.h

Differential Revision: https://reviews.llvm.org/D75406

show more ...

clang/Modules: Finish renaming CompilerInstance::ModuleManager, NFC.

Follow-up to 20d51b2f14ac4488f684f8fc57cb0ba718a6b91d, rename the setter to
make it consistent with the getter. Also fixed a few

clang/Modules: Finish renaming CompilerInstance::ModuleManager, NFC.

Follow-up to 20d51b2f14ac4488f684f8fc57cb0ba718a6b91d, rename the setter to
make it consistent with the getter. Also fixed a few comments along the
way, didn't try to find all references to a module manager.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D74939

show more ...

Use heterogenous lookup for std;:map<std::string with a StringRef. NFCI.

[hip] Properly populate macros based on host processor.

Summary:
- The device compilation needs to have a consistent source code compared
to the corresponding host compilation. If macros based on

[hip] Properly populate macros based on host processor.

Summary:
- The device compilation needs to have a consistent source code compared
to the corresponding host compilation. If macros based on the
host-specific target processor is not properly populated, the device
compilation may fail due to the inconsistent source after the
preprocessor. So far, only the host triple is used to build the
macros. If a detailed host CPU target or certain features are
specified, macros derived from them won't be populated properly, e.g.
`__SSE3__` won't be added unless `+sse3` feature is present. On
Windows compilation compatible with MSVC, that missing macros result
in that intrinsics are not included and cause device compilation
failure on the host-side source.

- This patch addresses this issue by introducing two `cc1` options,
i.e., `-aux-target-cpu` and `-aux-target-feature`. If a specific host
CPU target or certain features are specified, the compiler driver will
append them during the construction of the offline compilation
actions. Then, the toolchain in `cc1` phase will populate macros
accordingly.

- An internal option `--gpu-use-aux-triple-only` is added to fall back
the original behavior to help diagnosing potential issues from the new
behavior.

Reviewers: tra, yaxunl

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73942

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

Re-land "Add an -fno-temp-file flag for compilation"

This time making sure to initialize FrontendOptions::UseTemporary.

Patch by Zachary Henkel!

Differential revision: https://reviews.llvm.org/D70

Re-land "Add an -fno-temp-file flag for compilation"

This time making sure to initialize FrontendOptions::UseTemporary.

Patch by Zachary Henkel!

Differential revision: https://reviews.llvm.org/D70615

show more ...

Revert "Add an -fno-temp-file flag for compilation"

This reverts commit d129aa1d5369781deff6c6b854cb612e160d3fb2.

This broke the MSan buildbots. More information available in the
original PR: https

Revert "Add an -fno-temp-file flag for compilation"

This reverts commit d129aa1d5369781deff6c6b854cb612e160d3fb2.

This broke the MSan buildbots. More information available in the
original PR: https://reviews.llvm.org/D70615

show more ...

Add an -fno-temp-file flag for compilation

Our build system does not handle randomly named files created during
the build well. We'd prefer to write compilation output directly
without creating a te

Add an -fno-temp-file flag for compilation

Our build system does not handle randomly named files created during
the build well. We'd prefer to write compilation output directly
without creating a temporary file. Function parameters already
existed to control this behavior but were not exposed all the way out
to the command line.

Patch by Zachary Henkel!

Differential revision: https://reviews.llvm.org/D70615

show more ...

[SYCL] Add support for auxiliary triple specification to Frontend

Summary:
Add host predefined macros to compilation for SYCL device, which is
required for pre-processing host specific includes (e.g

[SYCL] Add support for auxiliary triple specification to Frontend

Summary:
Add host predefined macros to compilation for SYCL device, which is
required for pre-processing host specific includes (e.g. system
headers).

Reviewers: ABataev, jdoerfert

Subscribers: ebevhan, Anastasia, cfe-commits, keryell, Naghasan, Fznamznon

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71286

Signed-off-by: Alexey Bader <alexey.bader@intel.com>

show more ...

clang/Modules: Rename CompilerInstance::ModuleManager, NFC

Fix the confusing naming of `CompilerInstance::ModuleManager`. This is
actually an instance of `ASTReader`, which contains an instance of

clang/Modules: Rename CompilerInstance::ModuleManager, NFC

Fix the confusing naming of `CompilerInstance::ModuleManager`. This is
actually an instance of `ASTReader`, which contains an instance of
`ModuleManager`. I have to assume there was a point in the past where
they were just one class, but it's been pretty confusing for a while. I
think it's time to fix it.

The new name is `TheASTReader`; the annoying `The` prefix is so that we
don't shadow the `ASTReader` class. I tried out `ASTRdr` but that
seemed less clear, and this choice matches `ThePCHContainerOperations`
just a couple of declarations below.

Also rename `CompilerInstance::getModuleManager` and
`CompilerInstance::createModuleManager` to `*ASTReader`, making some
cases of `getModuleManager().getModuleManager()` a little more clear.

https://reviews.llvm.org/D70583

show more ...