[clang][modules] Serialize VFS overlay paths into PCMs

With implicitly built modules, the importing `CompilerInstance` assumes PCMs were built in a "compatible way" (i.e. with similarly set up insta

[clang][modules] Serialize VFS overlay paths into PCMs

With implicitly built modules, the importing `CompilerInstance` assumes PCMs were built in a "compatible way" (i.e. with similarly set up instance). Either because their context hash matches, or because this instance has just built them.

There are some use-cases, however, where this assumption doesn't hold, libclang/c-index-test being one of them. There, the importing instance (or `ASTUnit`) is being set up while the PCM file is being deserialized. Until now, we've assumed the serialized paths to input files are the actual on-disk files, meaning the default physical VFS was always able to resolve them. This won't be the case after D135636. Therefore, this patch makes sure `ASTUnit` is initialized with the same VFS as the PCM it's deserializing - by storing paths to the VFS overlay files into the PCM itself.

For the VFS overlay files to be adopted at the very start of PCM deserialization, they are stored in a new section in the unhashed control block, together with header search paths and system header prefixes. The move to the unhashed control block should be safe: if two modules were built with different header search paths and they produced different results, the hashed part of the PCM file will reflect that.

Reviewed By: akyrtzi, benlangmuir

Differential Revision: https://reviews.llvm.org/D135634

show more ...

[analyzer] Deprecate FAM analyzer-config, recommend -fstrict-flex-arrays instead

By default, clang assumes that all trailing array objects could be a
FAM. So, an array of undefined size, size 0, siz

[analyzer] Deprecate FAM analyzer-config, recommend -fstrict-flex-arrays instead

By default, clang assumes that all trailing array objects could be a
FAM. So, an array of undefined size, size 0, size 1, or even size 42 is
considered as FAMs for optimizations at least.

One needs to override the default behavior by supplying the
`-fstrict-flex-arrays=<N>` flag, with `N > 0` value to reduce the set of
FAM candidates. Value `3` is the most restrictive and `0` is the most
permissive on this scale.

0: all trailing arrays are FAMs
1: only incomplete, zero and one-element arrays are FAMs
2: only incomplete, zero-element arrays are FAMs
3: only incomplete arrays are FAMs

If the user is happy with consdering single-element arrays as FAMs, they
just need to remove the
`consider-single-element-arrays-as-flexible-array-members` from the
command line.
Otherwise, if they don't want to recognize such cases as FAMs, they
should specify `-fstrict-flex-arrays` anyway, which will be picked up by
CSA.

Any use of the deprecated analyzer-config value will trigger a warning
explaining what to use instead.
The `-analyzer-config-help` is updated accordingly.

Depends on D138657

Reviewed By: xazax.hun

Differential Revision: https://reviews.llvm.org/D138659

show more ...

Remove -cc1 -fconcepts-ts flag

The -fconcepts-ts flag has been deprecated for 5 releases now, so just
remove it, concepts is supported in C++20 mode.

[C++20] [Modules] Remove unmaintained Header Module

Currently there is a -emit-header-module mode, which can combine several
headers together as a module interface. However, this breaks our
assumpti

[C++20] [Modules] Remove unmaintained Header Module

Currently there is a -emit-header-module mode, which can combine several
headers together as a module interface. However, this breaks our
assumption (for standard c++ modules) about module interface. The module
interface should come from a module interface unit. And if it is a
header, it should be a header unit. And currently we have no ideas to
combine several headers together.

So I think this mode is an experimental one and it is not maintained and
it is not used. So it will be better to remove them.

Reviewed By: Bigcheese, dblaikie, bruno

Differential Revision: https://reviews.llvm.org/D137609

show more ...

The handling of 'funsafe-math-optimizations' doesn't update the 'MathErrno'
flag. But the driver checks for 'fno-math-errno' before passing
'funsafe-math-optimizations' to the FE. In GCC, the option

The handling of 'funsafe-math-optimizations' doesn't update the 'MathErrno'
flag. But the driver checks for 'fno-math-errno' before passing
'funsafe-math-optimizations' to the FE. In GCC, the option
'funsafe-math-optimizations' doesn't affect the 'fmath-errno' flag.
This patch aligns clang with GCC.

'-ffast-math' sets the FPContract to 'fast'. But 'funsafe-math-optimizations'
the driver doesn't consider the FPContract when handling the option.
Unfortunately there are places in the BE that interpret unsafe math
mode as allowing FMA. This patch makes -ffast-math' and
'funsafe-math-optimizations' behave similarly in regard to the setting of the
FPContract.

Differential Revision: https://reviews.llvm.org/D137578

show more ...

[Driver] Refactor err_drv_unsupported_option_argument call sites to use llvm::opt::Arg::getSpelling

For `-foo=bar`, getSpelling return `-foo=` which is exactly what we need from
the diagnostic. Drop

[Driver] Refactor err_drv_unsupported_option_argument call sites to use llvm::opt::Arg::getSpelling

For `-foo=bar`, getSpelling return `-foo=` which is exactly what we need from
the diagnostic. Drop `-` from the err_drv_unsupported_option_argument template.
This change makes `--` long option diagnostics more convenient.

Reviewed By: lenary

Differential Revision: https://reviews.llvm.org/D137659

show more ...

[clang][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D

[clang][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D137491

show more ...

[CodeView][clang] Add flag to disable emitting command line into CodeView

In https://reviews.llvm.org/D80833, there were concerns about
determinism emitting the commandline into CodeView. We're actu

[CodeView][clang] Add flag to disable emitting command line into CodeView

In https://reviews.llvm.org/D80833, there were concerns about
determinism emitting the commandline into CodeView. We're actually
hitting these when running clang-cl on Linux (cross compiling) versus on
Windows (e.g. -fmessage-length being inferred on terminals).

Add -g[no-]codeview-command-line to enable/disable this feature.
It's still on by default to preserve the current state of clang.

Reviewed By: thakis, rnk

Differential Revision: https://reviews.llvm.org/D136474

show more ...

[clang] Move getenv call for SOURCE_DATE_EPOCH out of frontend NFC

Move the check for SOURCE_DATE_EPOCH to the driver and use a cc1 option
to pass it to the frontend. This avoids hidden state in the

[clang] Move getenv call for SOURCE_DATE_EPOCH out of frontend NFC

Move the check for SOURCE_DATE_EPOCH to the driver and use a cc1 option
to pass it to the frontend. This avoids hidden state in the cc1
invocation and makes this env variable behave more like other env
variables that clang handles in the driver.

Differential Revision: https://reviews.llvm.org/D136717

show more ...

[Frontend] Recognize environment variable SOURCE_DATE_EPOCH

See https://reproducible-builds.org/docs/source-date-epoch/ . The environment
variable ``SOURCE_DATE_EPOCH`` been recognized by many compi

[Frontend] Recognize environment variable SOURCE_DATE_EPOCH

See https://reproducible-builds.org/docs/source-date-epoch/ . The environment
variable ``SOURCE_DATE_EPOCH`` been recognized by many compilers.

In GCC, if `SOURCE_DATE_EPOCH` is set, it specifies a UNIX timestamp to be used
in replacement of the current date and time in the `__DATE__` and `__TIME__`
macros. Note: GCC as of today does not update `__TIMESTAMP__` (the modification
time of the current source file) but
https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros expresses the
intention to update it.

This patches parses SOURCE_DATE_EPOCH and changes all the three macros.

In addition, in case gmtime/localtime returns null (e.g. on 64-bit Windows
gmtime returns null when the timestamp is larger than 32536850399
(3001-01-19T21:59:59Z)), use `??? ?? ????` as used by GCC.

Reviewed By: ychen

Differential Revision: https://reviews.llvm.org/D135045

show more ...

Restore "[MemProf] Memprof profile matching and annotation"

This reverts commit 794b7ea960ccc3222f2af582efadbc5e5c464292, and
thus restores commit a212d8da94d08e229aa8d65283e4b116310bba10, and
follo

Restore "[MemProf] Memprof profile matching and annotation"

This reverts commit 794b7ea960ccc3222f2af582efadbc5e5c464292, and
thus restores commit a212d8da94d08e229aa8d65283e4b116310bba10, and
follow on fixes 0cd6763fa93159b84d70a5bb602c24996acaafaa,
e9ff53d42feac7fc157718523275619a8106f2f3, and
37c6a25e9ab230e5e21fa34e246d9fec55275df0.

Use a hash function (BLAKE3) instead of hash_combine/hash_code which are
not guaranteed to be stable across executions.

Additionally, it adds a "REQUIRES: x86_64-linux" to the tests that have
raw profile inputs to avoid failures on big endian bots.

Reviewers: snehasish, davidxl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D128142

show more ...

Revert "[MemProf] Memprof profile matching and annotation"

This reverts commit a212d8da94d08e229aa8d65283e4b116310bba10, and follow
on fixes 0cd6763fa93159b84d70a5bb602c24996acaafaa,
e9ff53d42feac7f

Revert "[MemProf] Memprof profile matching and annotation"

This reverts commit a212d8da94d08e229aa8d65283e4b116310bba10, and follow
on fixes 0cd6763fa93159b84d70a5bb602c24996acaafaa,
e9ff53d42feac7fc157718523275619a8106f2f3, and
37c6a25e9ab230e5e21fa34e246d9fec55275df0.

After re-reading the documentation for hash_combine, I don't think this
is the appropriate hash function to use for computing the hash to use as
a stack id in the metadata, since it is not guaranteed to produce stable
values across executions. I have not hit this problem, but plan to
switch to using an MD5 hash. I am hitting an issue with one of the bots
(https://lab.llvm.org/buildbot/#/builders/171/builds/20732)
where the values produced are only the lower 32 bits of the expected
hash values, however, which I assume is related to the implementation of
hash_combine and hash_code.

I believe I fixed all of the other bot failures with the follow on fixes,
which I'll merge into the new version before reapplying.

show more ...

[MemProf] Memprof profile matching and annotation

Profile matching and IR annotation for memprof profiles.

See also related RFCs:
RFC: Sanitizer-based Heap Profiler [1]
RFC: A binary serialization

[MemProf] Memprof profile matching and annotation

Profile matching and IR annotation for memprof profiles.

See also related RFCs:
RFC: Sanitizer-based Heap Profiler [1]
RFC: A binary serialization format for MemProf [2]
RFC: IR metadata format for MemProf [3]*

* Note that the IR metadata format has changed from the RFC during
implementation, as described in the preceeding patch adding the basic
metadata and verification support.

The matching is performed during the normal PGO annotation phase, to
ensure that the inlines applied in the IR at that point are a subset
of the inlines in the profiled binary and thus reflected in the
profile's call stacks. This is important because the call frames are
associated with functions in the profile based on the inlining in the
symbolized call stacks, and this simplifies locating the subset of
profile data relevant for matching onto each function's IR.

The PGOInstrumentationUse pass is enhanced to perform matching for
whatever combination of memprof and regular PGO profile data exists in
the profile.

Using the utilities introduced in D128854:
The memprof profile data for each context is converted to "cold" or
"notcold" based on parameterized thresholds for size, access count, and
lifetime. The memprof allocation contexts are trimmed to the minimal
amount of context required to uniquely identify whether the context is
cold or not cold. For allocations where all profiled contexts have the
same allocation type, no memprof metadata is attached and instead the
allocation call is directly annotated with an attribute specifying the
alloction type. This is the same attributed that will be applied to
allocation calls once cloned for different contexts, and later used
during LibCall simplification to emit allocation hints [4].

Depends on D128141 and D128854.

[1] https://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html
[2] https://lists.llvm.org/pipermail/llvm-dev/2021-September/153007.html
[3] https://discourse.llvm.org/t/rfc-ir-metadata-format-for-memprof/59165
[4] https://github.com/google/tcmalloc/commit/ab87cf382dc56784f783f3aaa43d6d0465d5f385

Differential Revision: https://reviews.llvm.org/D128142

show more ...

[Clang] Give error message for invalid profile path when compiling IR

Before this patch, when compiling an IR file (eg the .llvmbc section
from an object file compiled with -Xclang -fembed-bitcode=a

[Clang] Give error message for invalid profile path when compiling IR

Before this patch, when compiling an IR file (eg the .llvmbc section
from an object file compiled with -Xclang -fembed-bitcode=all) and
profile data was passed in using the -fprofile-instrument-use-path
flag, there would be no error printed (as the previous implementation
relied on the error getting caught again in the constructor of
CodeGenModule which isn't called when -x ir is set). This patch
moves the error checking directly to where the error is caught
originally rather than failing silently in setPGOUseInstrumentor and
waiting to catch it in CodeGenModule to print diagnostic information to
the user.

Regression test added.

Reviewed By: xur, mtrofin

Differential Revision: https://reviews.llvm.org/D132991

show more ...

[OpenMP] Remove simplified device runtime handling

The old device runtime had a "simplified" version that prevented many of
the runtime features from being initialized. The old device runtime was
de

[OpenMP] Remove simplified device runtime handling

The old device runtime had a "simplified" version that prevented many of
the runtime features from being initialized. The old device runtime was
deleted in LLVM 14 and is no longer in use. Selectively deactivating
features is now done using specific flags rather than the old technique.
This patch simply removes the extra logic required for handling the old
simple runtime scheme.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D133802

show more ...

[clang] Explicitly set the EmulatedTLS codegen option. NFC.

Set the EmulatedTLS option based on `Triple::hasDefaultEmulatedTLS()`
if the user didn't specify it; set `ExplicitEmulatedTLS` to true
in

[clang] Explicitly set the EmulatedTLS codegen option. NFC.

Set the EmulatedTLS option based on `Triple::hasDefaultEmulatedTLS()`
if the user didn't specify it; set `ExplicitEmulatedTLS` to true
in `llvm::TargetOptions` and set `EmulatedTLS` to Clang's
opinion of what the default or preference is.

This avoids any risk of deviance between the two.

This affects one check of `getCodeGenOpts().EmulatedTLS` in
`shouldAssumeDSOLocal` in CodeGenModule, but as that check only
is done for `TT.isWindowsGNUEnvironment()`, and
`hasDefaultEmulatedTLS()` returns false for such environments
it doesn't make any current testable difference - thus NFC.

Some mingw distributions carry a downstream patch, that enables
emulated TLS by default for mingw targets in `hasDefaultEmulatedTLS()`
- and for such cases, this patch does make a difference and fixes the
detection of emulated TLS, if it is implicitly enabled.

Differential Revision: https://reviews.llvm.org/D132916

show more ...

[HLSL] Restrict to supported targets

Someday we would like to support HLSL on a wider range of targets, but
today targeting anything other than `dxil` is likly to cause lots of
headaches. This adds

[HLSL] Restrict to supported targets

Someday we would like to support HLSL on a wider range of targets, but
today targeting anything other than `dxil` is likly to cause lots of
headaches. This adds an error and tests to validate that the expected
target is `dxil-?-shadermodel`.

We will continue to do a best effort to ensure the code we write makes
it easy to support other targets (like SPIR-V), but this error will
prevent users from hitting frustrating errors for unsupported cases.

Reviewed By: jcranmer-intel, Anastasia

Differential Revision: https://reviews.llvm.org/D132056

show more ...

[Clang] follow-up D128745, use ClangABICompat15 instead of ClangABICompat14

Since the patch missed release 15.x and will be included in release 16.x. Also, simplify related tests.

Reviewed By: aaro

[Clang] follow-up D128745, use ClangABICompat15 instead of ClangABICompat14

Since the patch missed release 15.x and will be included in release 16.x. Also, simplify related tests.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D132414

show more ...

[analyzer] Drop deprecated flags

As proposed in D126215 (ffe7950ebc62380c3afc7c71f454a1db3f6f5c76),
I'm dropping the `-analyzer-store` and
`-analyzer-opt-analyze-nested-blocks` clang frontend flags.

[analyzer] Drop deprecated flags

As proposed in D126215 (ffe7950ebc62380c3afc7c71f454a1db3f6f5c76),
I'm dropping the `-analyzer-store` and
`-analyzer-opt-analyze-nested-blocks` clang frontend flags.
I'm also dropping the corresponding commandline handlers of `scanbuild`.

This behavior is planned to be part of `clang-16`.

Reviewed By: xazax.hun

Differential Revision: https://reviews.llvm.org/D132289

show more ...

[clang] Pull some utility functions into CompilerInvocation NFC

Move copying compiler arguments to a vector<string> and modifying
common module-related options into CompilerInvocation in preparation

[clang] Pull some utility functions into CompilerInvocation NFC

Move copying compiler arguments to a vector<string> and modifying
common module-related options into CompilerInvocation in preparation for
using some of them in more places and to avoid duplicating this code
accidentally in the future.

Differential Revision: https://reviews.llvm.org/D132419

show more ...

Revert "Use std::is_same_v instead of std::is_same (NFC)"

This reverts commit c5da37e42d388947a40654b7011f2a820ec51601.

This patch seems to break builds with some versions of MSVC.

Use std::is_same_v instead of std::is_same (NFC)

[clang][llvm][NFC] Change misexpect's tolerance option to be 32-bit

In D131869 we noticed that we jump through some hoops because we parse the
tolerance option used in MisExpect.cpp into a 64-bit in

[clang][llvm][NFC] Change misexpect's tolerance option to be 32-bit

In D131869 we noticed that we jump through some hoops because we parse the
tolerance option used in MisExpect.cpp into a 64-bit integer. This is
unnecessary, since the value can only be in the range [0, 100).

This patch changes the underlying type to be 32-bit from where it is
parsed in Clang through to it's use in LLVM.

Reviewed By: jloser

Differential Revision: https://reviews.llvm.org/D131935

show more ...

[HLSL] Support -E option for HLSL.

-E option will set entry function for hlsl.
The format is -E entry_name.

To avoid conflict with existing option with name 'E', add an extra prefix '--'.

A new fi

[HLSL] Support -E option for HLSL.

-E option will set entry function for hlsl.
The format is -E entry_name.

To avoid conflict with existing option with name 'E', add an extra prefix '--'.

A new field HLSLEntry is added to TargetOption.
To share code with HLSLShaderAttr, entry function will be add HLSLShaderAttr attribute too.

Reviewed By: beanz

Differential Revision: https://reviews.llvm.org/D124751

show more ...

Fix lack of cc1 flag in llvmcmd sections when assertions are enabled

Currently when assertions are enabled, the cc1 flag is not
inserted into the llvmcmd section of object files with embedded
bitcod

Fix lack of cc1 flag in llvmcmd sections when assertions are enabled

Currently when assertions are enabled, the cc1 flag is not
inserted into the llvmcmd section of object files with embedded
bitcode. This deviates from the normal behavior where this is
the first flag that is inserted. This error stems from incorrect
use of the function generateCC1CommandLine() which requires
manually adding in the -cc1 flag which is currently not done.

Reviewed By: jansvoboda11

Differential Revision: https://reviews.llvm.org/D130620

show more ...