Reland: [clang] unified CWG2398 and P0522 changes; finishes implementation of P3310 (#124137)

This patch relands the following PRs:
* #111711
* #107350
* #111457

All of these patches were reve

Reland: [clang] unified CWG2398 and P0522 changes; finishes implementation of P3310 (#124137)

This patch relands the following PRs:
* #111711
* #107350
* #111457

All of these patches were reverted due to an issue reported in
https://github.com/llvm/llvm-project/pull/111711#issuecomment-2406491485,
due to interdependencies.

---
[clang] Finish implementation of P0522

This finishes the clang implementation of P0522, getting rid
of the fallback to the old, pre-P0522 rules.

Before this patch, when partial ordering template template parameters,
we would perform, in order:
* If the old rules would match, we would accept it. Otherwise, don't
generate diagnostics yet.
* If the new rules would match, just accept it. Otherwise, don't
generate any diagnostics yet again.
* Apply the old rules again, this time with diagnostics.

This situation was far from ideal, as we would sometimes:
* Accept some things we shouldn't.
* Reject some things we shouldn't.
* Only diagnose rejection in terms of the old rules.

With this patch, we apply the P0522 rules throughout.

This needed to extend template argument deduction in order
to accept the historial rule for TTP matching pack parameter to non-pack
arguments.
This change also makes us accept some combinations of historical and P0522
allowances we wouldn't before.

It also fixes a bunch of bugs that were documented in the test suite,
which I am not sure there are issues already created for them.

This causes a lot of changes to the way these failures are diagnosed,
with related test suite churn.

The problem here is that the old rules were very simple and
non-recursive, making it easy to provide customized diagnostics,
and to keep them consistent with each other.

The new rules are a lot more complex and rely on template argument
deduction, substitutions, and they are recursive.

The approach taken here is to mostly rely on existing diagnostics,
and create a new instantiation context that keeps track of this context.

So for example when a substitution failure occurs, we use the error
produced there unmodified, and just attach notes to it explaining
that it occurred in the context of partial ordering this template
argument against that template parameter.

This diverges from the old diagnostics, which would lead with an
error pointing to the template argument, explain the problem
in subsequent notes, and produce a final note pointing to the parameter.

---
[clang] CWG2398: improve overload resolution backwards compat

With this change, we discriminate if the primary template and which partial
specializations would have participated in overload resolution prior to
P0522 changes.

We collect those in an initial set. If this set is not empty, or the
primary template would have matched, we proceed with this set as the
candidates for overload resolution.

Otherwise, we build a new overload set with everything else, and proceed
as usual.

---
[clang] Implement TTP 'reversed' pack matching for deduced function template calls.

Clang previously missed implementing P0522 pack matching
for deduced function template calls.

show more ...

[clang] Allow generating module interfaces with parsing errors (#121485)

Fixes a regression introduced in commit
da00c60dae0040185dc45039c4397f6e746548e9

This functionality was originally added

[clang] Allow generating module interfaces with parsing errors (#121485)

Fixes a regression introduced in commit
da00c60dae0040185dc45039c4397f6e746548e9

This functionality was originally added in commit
5834996fefc937d6211dc8c8a5b200068753391a

Co-authored-by: Tomasz Kaminski <tomasz.kaminski@sonarsource.com>

show more ...

Remove support for RenderScript (#112916)

See
https://discourse.llvm.org/t/rfc-deprecate-and-eventually-remove-renderscript-support/81284
for the RFC

Revert "Reland: [clang] Finish implementation of P0522 (#111711)"

See discussion in https://github.com/llvm/llvm-project/pull/111711

This reverts commit 6213aa5e58a7d32bdc82dd40322fb1bab83c4783.

Reland: [clang] Finish implementation of P0522 (#111711)

This finishes the clang implementation of P0522, getting rid of the
fallback to the old, pre-P0522 rules.

Before this patch, when partial

Reland: [clang] Finish implementation of P0522 (#111711)

This finishes the clang implementation of P0522, getting rid of the
fallback to the old, pre-P0522 rules.

Before this patch, when partial ordering template template parameters,
we would perform, in order:
* If the old rules would match, we would accept it. Otherwise, don't
generate diagnostics yet.
* If the new rules would match, just accept it. Otherwise, don't
generate any diagnostics yet again.
* Apply the old rules again, this time with diagnostics.

This situation was far from ideal, as we would sometimes:
* Accept some things we shouldn't.
* Reject some things we shouldn't.
* Only diagnose rejection in terms of the old rules.

With this patch, we apply the P0522 rules throughout.

This needed to extend template argument deduction in order to accept the
historial rule for TTP matching pack parameter to non-pack arguments.
This change also makes us accept some combinations of historical and
P0522 allowances we wouldn't before.

It also fixes a bunch of bugs that were documented in the test suite,
which I am not sure there are issues already created for them.

This causes a lot of changes to the way these failures are diagnosed,
with related test suite churn.

The problem here is that the old rules were very simple and
non-recursive, making it easy to provide customized diagnostics, and to
keep them consistent with each other.

The new rules are a lot more complex and rely on template argument
deduction, substitutions, and they are recursive.

The approach taken here is to mostly rely on existing diagnostics, and
create a new instantiation context that keeps track of this context.

So for example when a substitution failure occurs, we use the error
produced there unmodified, and just attach notes to it explaining that
it occurred in the context of partial ordering this template argument
against that template parameter.

This diverges from the old diagnostics, which would lead with an error
pointing to the template argument, explain the problem in subsequent
notes, and produce a final note pointing to the parameter.

show more ...

Revert "[clang] Finish implementation of P0522 (#96023)"

This caused Clang to reject valid code, see discussion on the PR
https://github.com/llvm/llvm-project/pull/96023#issuecomment-2393228464
and

Revert "[clang] Finish implementation of P0522 (#96023)"

This caused Clang to reject valid code, see discussion on the PR
https://github.com/llvm/llvm-project/pull/96023#issuecomment-2393228464
and https://github.com/llvm/llvm-project/issues/111363

This reverts commit 6afe56732a172d3f2cbd0330b1fcb34bbfd002a9 and
follow-up commit 9abb97f9663a27fe5b8e346ed557b3435aa9ec2f.

show more ...

[clang] Finish implementation of P0522 (#96023)

This finishes the clang implementation of P0522, getting rid of the
fallback to the old, pre-P0522 rules.

Before this patch, when partial ordering

[clang] Finish implementation of P0522 (#96023)

This finishes the clang implementation of P0522, getting rid of the
fallback to the old, pre-P0522 rules.

Before this patch, when partial ordering template template parameters,
we would perform, in order:
* If the old rules would match, we would accept it. Otherwise, don't
generate diagnostics yet.
* If the new rules would match, just accept it. Otherwise, don't
generate any diagnostics yet again.
* Apply the old rules again, this time with diagnostics.

This situation was far from ideal, as we would sometimes:
* Accept some things we shouldn't.
* Reject some things we shouldn't.
* Only diagnose rejection in terms of the old rules.

With this patch, we apply the P0522 rules throughout.

This needed to extend template argument deduction in order to accept the
historial rule for TTP matching pack parameter to non-pack arguments.
This change also makes us accept some combinations of historical and
P0522 allowances we wouldn't before.

It also fixes a bunch of bugs that were documented in the test suite,
which I am not sure there are issues already created for them.

This causes a lot of changes to the way these failures are diagnosed,
with related test suite churn.

The problem here is that the old rules were very simple and
non-recursive, making it easy to provide customized diagnostics, and to
keep them consistent with each other.

The new rules are a lot more complex and rely on template argument
deduction, substitutions, and they are recursive.

The approach taken here is to mostly rely on existing diagnostics, and
create a new instantiation context that keeps track of things.

So for example when a substitution failure occurs, we use the error
produced there unmodified, and just attach notes to it explaining that
it occurred in the context of partial ordering this template argument
against that template parameter.

This diverges from the old diagnostics, which would lead with an error
pointing to the template argument, explain the problem in subsequent
notes, and produce a final note pointing to the parameter.

show more ...

[NFC] Add explicit #include llvm-config.h where its macros are used, clang part. (#107301)

(this is clang related part)

Without these explicit includes, removing other headers, who implicitly
in

[NFC] Add explicit #include llvm-config.h where its macros are used, clang part. (#107301)

(this is clang related part)

Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial side effects. For example,
`clagd` may report even `llvm-config.h` as "no used" in case it defines
a macro, that is explicitly used with #ifdef. It is actually amplified
with different build configs which use different set of macros.

show more ...

Revert "[C++20] [Modules] Embed all source files for C++20 Modules (#102444)"

This reverts commit 2eeeff842f993a694159183a2834b4d305549cad.

See the post commit discussion in
https://github.com/llvm

Revert "[C++20] [Modules] Embed all source files for C++20 Modules (#102444)"

This reverts commit 2eeeff842f993a694159183a2834b4d305549cad.

See the post commit discussion in
https://github.com/llvm/llvm-project/commit/2eeeff842f993a694159183a2834b4d305549cad

show more ...

[C++20] [Modules] Embed all source files for C++20 Modules (#102444)

Close https://github.com/llvm/llvm-project/issues/72383

The implementation rationale is, I don't want to pass
`-fmodules-embe

[C++20] [Modules] Embed all source files for C++20 Modules (#102444)

Close https://github.com/llvm/llvm-project/issues/72383

The implementation rationale is, I don't want to pass
`-fmodules-embed-all-files` all the time since we can't test it in lit
tests (we're using `clang_cc1`). So I tried to set it in FrontendActions
for modules.

show more ...

[re-format][Modules] Follow-up formatting to "Mention which AST file's options differ from the current TU options." (#102484)

Fix formatting for fdf8e3e31103bc81917cdb27150877f524bb2669.

[Modules][Diagnostic] Mention which AST file's options differ from the current TU options. (#101413)

Claiming a mismatch is always in a precompiled header is wrong and
misleading as a mismatch can

[Modules][Diagnostic] Mention which AST file's options differ from the current TU options. (#101413)

Claiming a mismatch is always in a precompiled header is wrong and
misleading as a mismatch can happen in any provided AST file. Emitting a
path for a file with a problem allows to disambiguate between multiple
input files.

Use generic term "AST file" because we don't always know a kind of the
provided file (for example, see `ASTReader::readASTFileControlBlock`).

rdar://65005546

show more ...

[Clang] Don't crash if input file is not a module. (#98439)

Currently clang crashes with `-module-file-info` and input file which is
not a module
Emit error instead of segfaulting.
Fix #98365

Revert "Pass LangOpts from CompilerInstance to DependencyScanningWorker (#93753)" (#94488)

This reverts commit 9862080b1cbf685c0d462b29596e3f7206d24aa2.

Pass LangOpts from CompilerInstance to DependencyScanningWorker (#93753)

This commit fixes https://github.com/llvm/llvm-project/issues/88896 by
passing LangOpts from the CompilerInstance to
Depend

Pass LangOpts from CompilerInstance to DependencyScanningWorker (#93753)

This commit fixes https://github.com/llvm/llvm-project/issues/88896 by
passing LangOpts from the CompilerInstance to
DependencyScanningWorker so that the original LangOpts are
preserved/respected.
This makes for more accurate parsing/lexing when certain language
versions or features specific to versions are to be used.

show more ...

[NFC] [C++20] [Modules] [Reduced BMI] Reorder Emitting reduced BMI and normal BMI for named modules

When we generate the reduced BMI on the fly, the order of the emitting
phase is different within `

[NFC] [C++20] [Modules] [Reduced BMI] Reorder Emitting reduced BMI and normal BMI for named modules

When we generate the reduced BMI on the fly, the order of the emitting
phase is different within `-emit-obj` and `-emit-module-interface`.
Although this is meant to be fine, we observed it in
https://github.com/llvm/llvm-project/issues/93859 (that the different phase order may cause problems).
Also it turns out to be a different fundamental reason to the orders.

But it might be fine to make the order of emitting reducing BMI at first
to avoid such confusions in the future.

show more ...

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to genera… (#90570)

…te module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the mo

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to genera… (#90570)

…te module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C++20 modules. But this is slighty more or less odd. This patch tries to
use a new class 'CXX20ModulesGenerator' to generate the module file for
C++20 modules.

show more ...

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrec

[C++20] [Modules] Don't skip pragma diagnostic mappings

Close https://github.com/llvm/llvm-project/issues/75057

Previously, I thought the diagnostic mappings is not meaningful with
modules incorrectly. And this problem get revealed by another change
recently. So this patch tried to rever the previous "optimization"
partially.

show more ...

Revert "[C++20] [Modules] Don't skip pragma diagnostic mappings"
and "[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to
generate module file for C++20 modules instead of PCHGenerator"

T

Revert "[C++20] [Modules] Don't skip pragma diagnostic mappings"
and "[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to
generate module file for C++20 modules instead of PCHGenerator"

This reverts commit fb21343473e33e9a886b42d2fe95d1cec1cd0030.
and commit 18268ac0f48d93c2bcddb69732761971669c09ab.

It looks like there are some problems about linking the compiler

show more ...

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to generate module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C

[NFC] [C++20] [Modules] Use new class CXX20ModulesGenerator to generate module file for C++20 modules instead of PCHGenerator

Previously we're re-using PCHGenerator to generate the module file for
C++20 modules. But this is slighty more or less odd. This patch tries
to use a new class 'CXX20ModulesGenerator' to generate the module file
for C++20 modules.

show more ...

[C++20] [Modules] Introduce -fexperimental-modules-reduced-bmi (#85050)

This is the driver part of
https://github.com/llvm/llvm-project/pull/75894.

This patch introduces '-fexperimental-modules-

[C++20] [Modules] Introduce -fexperimental-modules-reduced-bmi (#85050)

This is the driver part of
https://github.com/llvm/llvm-project/pull/75894.

This patch introduces '-fexperimental-modules-reduced-bmi' to enable
generating the reduced BMI.

This patch did:
- When `-fexperimental-modules-reduced-bmi` is specified but
`--precompile` is not specified for a module unit, we'll skip the
precompile phase to avoid unnecessary two-phase compilation phases. Then
if `-c` is specified, we will generate the reduced BMI in CodeGenAction
as a by-product.
- When `-fexperimental-modules-reduced-bmi` is specified and
`--precompile` is specified, we will generate the reduced BMI in
GenerateModuleInterfaceAction as a by-product.
- When `-fexperimental-modules-reduced-bmi` is specified for a
non-module unit. We don't do anything nor try to give a warn. This is
more user friendly so that the end users can try to test and experiment
with the feature without asking help from the build systems.

The core design idea is that users should be able to enable this easily
with the existing cmake mechanisms.

The future plan for the flag is:
- Add this to clang19 and make it opt-in for 1~2 releases. It depends on
the testing feedback to decide how long we like to make it opt-in.
- Then we can announce the existing BMI generating may be deprecated and
suggesting people (end users or build systems) to enable this for 1~2
releases.
- Finally we will enable this by default. When that time comes, the term
`BMI` will refer to the reduced BMI today and the existing BMI will only
be meaningful to build systems which loves to support two phase
compilations.

I'll send release notes and document in seperate commits after this get
landed.

show more ...

Revert "[clang] Move state out of `PreprocessorOptions` (1/n) (#86358)"

This reverts commit 407a2f23 which stopped propagating the callback to module compiles, effectively disabling dependency direc

Revert "[clang] Move state out of `PreprocessorOptions` (1/n) (#86358)"

This reverts commit 407a2f23 which stopped propagating the callback to module compiles, effectively disabling dependency directive scanning for all modular dependencies. Also added a regression test.

show more ...

[Sema][NFC] Cleanups after 843cc474f (#87996)

I forgot to tidy up these lines that should've been done in the previous
commit, specifically:

1. Merge two `CodeSynthesisContext`s into one in `Che

[Sema][NFC] Cleanups after 843cc474f (#87996)

I forgot to tidy up these lines that should've been done in the previous
commit, specifically:

1. Merge two `CodeSynthesisContext`s into one in `CheckTemplateIdType`.
2. Remove some gratuitous `Sema::` specifiers.
3. Rename the parameter `Template` to `Entity` to avoid confusion.

show more ...

[Clang][Sema] Fix the lambda call expression inside of a type alias declaration (#82310)

This patch attempts to fix the lambda call expression inside of a type
alias declaration from two aspects:

[Clang][Sema] Fix the lambda call expression inside of a type alias declaration (#82310)

This patch attempts to fix the lambda call expression inside of a type
alias declaration from two aspects:
1. Defer the lambda call expression building until after we have
sufficient template arguments. This avoids the overeager (and often
wrong) semantic checking before the type alias instantiation.
2. Properly obtain template arguments involving a template type alias
for constraint checking.

It is unfortunate that a `TypeAliasTemplateDecl` (or a `TypeAliasDecl`)
is never a `DeclContext`, nor does it have an associated specialization
Decl from which we could collect these template arguments. Thus, I added
a new CodeSynthesisContext to record template arguments for alias
declarations.

Fixes https://github.com/llvm/llvm-project/issues/70601
Fixes https://github.com/llvm/llvm-project/issues/76674
Fixes https://github.com/llvm/llvm-project/issues/79555
Fixes https://github.com/llvm/llvm-project/issues/81145
Fixes https://github.com/llvm/llvm-project/issues/82104

Note that this doesn't involve the fix for
https://github.com/llvm/llvm-project/issues/28461. That seems different,
and I'd like to leave it as a follow-up.

show more ...

[clang] Move state out of `PreprocessorOptions` (1/n) (#86358)

An instance of `PreprocessorOptions` is part of `CompilerInvocation`
which is supposed to be a value type. The `DependencyDirectivesFo

[clang] Move state out of `PreprocessorOptions` (1/n) (#86358)

An instance of `PreprocessorOptions` is part of `CompilerInvocation`
which is supposed to be a value type. The `DependencyDirectivesForFile`
member is problematic, since it holds an owning reference of the
scanning VFS. This makes it not a true value type, and it can keep
potentially large chunk of memory (the local cache in the scanning VFS)
alive for longer than clients might expect. Let's move it into the
`Preprocessor` instead.

show more ...