[Serialization] Support loading template specializations lazily (#119333)

Reland https://github.com/llvm/llvm-project/pull/83237

---

(Original comments)

Currently all the specializations of

[Serialization] Support loading template specializations lazily (#119333)

Reland https://github.com/llvm/llvm-project/pull/83237

---

(Original comments)

Currently all the specializations of a template (including
instantiation, specialization and partial specializations) will be
loaded at once if we want to instantiate another instance for the
template, or find instantiation for the template, or just want to
complete the redecl chain.

This means basically we need to load every specializations for the
template once the template declaration got loaded. This is bad since
when we load a specialization, we need to load all of its template
arguments. Then we have to deserialize a lot of unnecessary
declarations.

For example,

```
// M.cppm
export module M;
export template <class T>
class A {};

export class ShouldNotBeLoaded {};

export class Temp {
A<ShouldNotBeLoaded> AS;
};

// use.cpp
import M;
A<int> a;
```

We have a specialization ` A<ShouldNotBeLoaded>` in `M.cppm` and we
instantiate the template `A` in `use.cpp`. Then we will deserialize
`ShouldNotBeLoaded` surprisingly when compiling `use.cpp`. And this
patch tries to avoid that.

Given that the templates are heavily used in C++, this is a pain point
for the performance.

This patch adds MultiOnDiskHashTable for specializations in the
ASTReader. Then we will only deserialize the specializations with the
same template arguments. We made that by using ODRHash for the template
arguments as the key of the hash table.

To review this patch, I think `ASTReaderDecl::AddLazySpecializations`
may be a good entry point.

show more ...

Revert "[Serialization] Support load lazy specialization lazily"

This reverts commit b5bd19211118c6d43bc525a4e3fb65d2c750d61e.
It brokes multiple llvm bots including clang-x64-windows-msvc

[Serialization] Support load lazy specialization lazily

Currently all the specializations of a template (including
instantiation, specialization and partial specializations) will be
loaded at once

[Serialization] Support load lazy specialization lazily

Currently all the specializations of a template (including
instantiation, specialization and partial specializations) will be
loaded at once if we want to instantiate another instance for the
template, or find instantiation for the template, or just want to
complete the redecl chain.

This means basically we need to load every specializations for the
template once the template declaration got loaded. This is bad since
when we load a specialization, we need to load all of its template
arguments. Then we have to deserialize a lot of unnecessary
declarations.

For example,

```
// M.cppm
export module M;
export template <class T>
class A {};

export class ShouldNotBeLoaded {};

export class Temp {
A<ShouldNotBeLoaded> AS;
};

// use.cpp
import M;
A<int> a;
```

We should a specialization ` A<ShouldNotBeLoaded>` in `M.cppm` and we
instantiate the template `A` in `use.cpp`. Then we will deserialize
`ShouldNotBeLoaded` surprisingly when compiling `use.cpp`. And this
patch tries to avoid that.

Given that the templates are heavily used in C++, this is a pain point
for the performance.

This patch adds MultiOnDiskHashTable for specializations in the
ASTReader. Then we will only deserialize the specializations with the
same template arguments. We made that by using ODRHash for the template
arguments as the key of the hash table.

To review this patch, I think `ASTReaderDecl::AddLazySpecializations`
may be a good entry point.

The patch was reviewed in
https://github.com/llvm/llvm-project/pull/83237 but that PR is a stacked
PR. But I feel the intention of the stacked PRs get lost during the
review process. So I feel it is better to merge the commits into a
single commit instead of merging them in the PR page. It is better for
us to cherry-pick and revert.

show more ...

[AST] Remove unused includes (NFC) (#116549)

Identified with misc-include-cleaner.

[clang] Implement CWG2398 provisional TTP matching to class templates (#94981)

This extends default argument deduction to cover class templates as
well, applying only to partial ordering, adding to

[clang] Implement CWG2398 provisional TTP matching to class templates (#94981)

This extends default argument deduction to cover class templates as
well, applying only to partial ordering, adding to the provisional
wording introduced in https://github.com/llvm/llvm-project/pull/89807.

This solves some ambuguity introduced in P0522 regarding how template
template parameters are partially ordered, and should reduce the
negative impact of enabling `-frelaxed-template-template-args` by
default.

Given the following example:
```C++
template <class T1, class T2 = float> struct A;
template <class T3> struct B;

template <template <class T4> class TT1, class T5> struct B<TT1<T5>>; // #1
template <class T6, class T7> struct B<A<T6, T7>>; // #2

template struct B<A<int>>;
```
Prior to P0522, `#2` was picked. Afterwards, this became ambiguous. This
patch restores the pre-P0522 behavior, `#2` is picked again.

show more ...

[Clang] Implement C++26’s P2893R3 ‘Variadic friends’ (#101448)

Implement P2893R3 ‘Variadic friends’ for C++26.

This closes #98587.

Co-authored-by: Younan Zhang <zyn7109@gmail.com>

[Serialization] Use stable hash functions

clangSerialization currently uses hash_combine/hash_value from
Hashing.h, which are not guaranteed to be deterministic.
Replace these uses with xxh3_64bits.

[Serialization] Use stable hash functions

clangSerialization currently uses hash_combine/hash_value from
Hashing.h, which are not guaranteed to be deterministic.
Replace these uses with xxh3_64bits.

Pull Request: https://github.com/llvm/llvm-project/pull/96136

show more ...

[clang] Preserve Qualifiers and type sugar in TemplateNames (#93433)

This patch improves the preservation of qualifiers and loss of type
sugar in TemplateNames.

This problem is analogous to https:/

[clang] Preserve Qualifiers and type sugar in TemplateNames (#93433)

This patch improves the preservation of qualifiers and loss of type
sugar in TemplateNames.

This problem is analogous to https://reviews.llvm.org/D112374 and this
patch takes a very similar approach to that patch, except the impact
here is much lesser.

When a TemplateName was written bare, without qualifications, we
wouldn't produce a QualifiedTemplate which could be used to disambiguate
it from a Canonical TemplateName. This had effects in the TemplateName
printer, which had workarounds to deal with this, and wouldn't print the
TemplateName as-written in most situations.

There are also some related fixes to help preserve this type sugar along
the way into diagnostics, so that this patch can be properly tested.

- Fix dropping the template keyword.
- Fix type deduction to preserve sugar in TST TemplateNames.

show more ...

[clang] NFCI: use TemplateArgumentLoc for NTTP DefaultArgument (#92852)

This is an enabler for https://github.com/llvm/llvm-project/pull/92855

This allows an NTTP default argument to be set as an a

[clang] NFCI: use TemplateArgumentLoc for NTTP DefaultArgument (#92852)

This is an enabler for https://github.com/llvm/llvm-project/pull/92855

This allows an NTTP default argument to be set as an arbitrary
TemplateArgument, not just an expression.
This allows template parameter packs to have default arguments in the
AST, even though the language proper doesn't support the syntax for it.

This allows NTTP default arguments to be other kinds of arguments, like
packs, integral constants, and such.

show more ...

[clang] NFCI: use TemplateArgumentLoc for type-param DefaultArgument (#92854)

This is an enabler for a future patch.

This allows an type-parameter default argument to be set as an arbitrary
Temp

[clang] NFCI: use TemplateArgumentLoc for type-param DefaultArgument (#92854)

This is an enabler for a future patch.

This allows an type-parameter default argument to be set as an arbitrary
TemplateArgument, not just a type.
This allows template parameter packs to have default arguments in the
AST, even though the language proper doesn't support the syntax for it.

This will be used in a later patch which synthesizes template parameter
lists with arbitrary default arguments taken from template
specializations.

There are a few places we used SubsType, because we only had a type, now
we use SubstTemplateArgument.
SubstTemplateArgument was missing arguments for setting Instantiation
location and entity names.
Adding those is needed so we don't regress in diagnostics.

show more ...

[Clang] [C++26] Implement P2573R2: `= delete("should have a reason");` (#86526)

This implements support for the `= delete("message")` syntax that was
only just added to C++26
([P2573R2](https://is

[Clang] [C++26] Implement P2573R2: `= delete("should have a reason");` (#86526)

This implements support for the `= delete("message")` syntax that was
only just added to C++26
([P2573R2](https://isocpp.org/files/papers/P2573R2.html#proposal-scope)).

show more ...

[HLSL] Implement array temporary support (#79382)

HLSL constant sized array function parameters do not decay to pointers.
Instead constant sized array types are preserved as unique types for
overl

[HLSL] Implement array temporary support (#79382)

HLSL constant sized array function parameters do not decay to pointers.
Instead constant sized array types are preserved as unique types for
overload resolution, template instantiation and name mangling.

This implements the change by adding a new `ArrayParameterType` which
represents a non-decaying `ConstantArrayType`. The new type behaves the
same as `ConstantArrayType` except that it does not decay to a pointer.

Values of `ConstantArrayType` in HLSL decay during overload resolution
via a new `HLSLArrayRValue` cast to `ArrayParameterType`.

`ArrayParamterType` values are passed indirectly by-value to functions
in IR generation resulting in callee generated memcpy instructions.

The behavior of HLSL function calls is documented in the [draft language
specification](https://microsoft.github.io/hlsl-specs/specs/hlsl.pdf)
under the Expr.Post.Call heading.

Additionally the design of this implementation approach is documented in
[Clang's
documentation](https://clang.llvm.org/docs/HLSL/FunctionCalls.html)

Resolves #70123

show more ...

[C++20] [Modules] Remove previous workaround for odr hashing enums

Previosly we land
https://github.com/llvm/llvm-project/commit/085eae6b863881fb9fda323e5b672b04a00ed19e
to workaround the false posi

[C++20] [Modules] Remove previous workaround for odr hashing enums

Previosly we land
https://github.com/llvm/llvm-project/commit/085eae6b863881fb9fda323e5b672b04a00ed19e
to workaround the false positive ODR violations in
https://github.com/llvm/llvm-project/issues/76638.

However, we decided to not perform ODR checks for decls from GMF in
https://github.com/llvm/llvm-project/issues/79240 and we land the
corresponding change. So we should be able to remove the workaround now.

The original tests get remained.

show more ...

[c++20] P1907R1: Support for generalized non-type template arguments of scalar type. (#78041)

Previously committed as 9e08e51a20d0d2b1c5724bb17e969d036fced4cd, and
reverted because a dependency comm

[c++20] P1907R1: Support for generalized non-type template arguments of scalar type. (#78041)

Previously committed as 9e08e51a20d0d2b1c5724bb17e969d036fced4cd, and
reverted because a dependency commit was reverted, then committed again
as 4b574008aef5a7235c1f894ab065fe300d26e786 and reverted again because
"dependency commit" 5a391d38ac6c561ba908334d427f26124ed9132e was
reverted. But it doesn't seem that 5a391d38ac6c was a real dependency
for this.

This commit incorporates 4b574008aef5a7235c1f894ab065fe300d26e786 and
18e093faf726d15f210ab4917142beec51848258 by Richard Smith (@zygoloid),
with some minor fixes, most notably:

- `UncommonValue` renamed to `StructuralValue`

- `VK_PRValue` instead of `VK_RValue` as default kind in lvalue and
member pointer handling branch in
`BuildExpressionFromNonTypeTemplateArgumentValue`;

- handling of `StructuralValue` in `IsTypeDeclaredInsideVisitor`;

- filling in `SugaredConverted` along with `CanonicalConverted`
parameter in `Sema::CheckTemplateArgument`;

- minor cleanup in
`TemplateInstantiator::transformNonTypeTemplateParmRef`;

- `TemplateArgument` constructors refactored;

- `ODRHash` calculation for `UncommonValue`;

- USR generation for `UncommonValue`;

- more correct MS compatibility mangling algorithm (tested on MSVC ver.
19.35; toolset ver. 143);

- IR emitting fixed on using a subobject as a template argument when the
corresponding template parameter is used in an lvalue context;

- `noundef` attribute and opaque pointers in `template-arguments` test;

- analysis for C++17 mode is turned off for templates in
`warn-bool-conversion` test; in C++17 and C++20 mode, array reference
used as a template argument of pointer type produces template argument
of UncommonValue type, and
`BuildExpressionFromNonTypeTemplateArgumentValue` makes
`OpaqueValueExpr` for it, and `DiagnoseAlwaysNonNullPointer` cannot see
through it; despite of "These cases should not warn" comment, I'm not
sure about correct behavior; I'd expect a suggestion to replace `if` by
`if constexpr`;

- `temp.arg.nontype/p1.cpp` and `dr18xx.cpp` tests fixed.

show more ...

[Clang][NFC] Rename CXXMethodDecl::isPure -> is VirtualPure (#78463)

To avoid any possible confusion with the notion of pure function and the
gnu::pure attribute.

[C++20] [Modules] Allow to merge enums with the same underlying interger types

Close https://github.com/llvm/llvm-project/issues/76638. See the issue
for the context of the change.

[clang][NFC] Refactor `ImplicitParamDecl::ImplicitParamKind`

This patch converts `ImplicitParamDecl::ImplicitParamKind` into a scoped enum at namespace scope, making it eligible for forward declarin

[clang][NFC] Refactor `ImplicitParamDecl::ImplicitParamKind`

This patch converts `ImplicitParamDecl::ImplicitParamKind` into a scoped enum at namespace scope, making it eligible for forward declaring. This is useful for `preferred_type` annotations on bit-fields.

show more ...

[clang][NFC] Refactor `ObjCMethodDecl::ImplementationControl`

This patch moves `ObjCMethodDecl::ImplementationControl` to a DeclBase.h so that it's complete at the point where corresponsing bit-fiel

[clang][NFC] Refactor `ObjCMethodDecl::ImplementationControl`

This patch moves `ObjCMethodDecl::ImplementationControl` to a DeclBase.h so that it's complete at the point where corresponsing bit-field is declared. This patch also converts it to a scoped enum `clang::ObjCImplementationControl`.

show more ...

[clang][NFC] Refactor `VectorType::VectorKind`

This patch moves `VectorKind` to namespace scope, and make it complete at the point its bit-field is declared. It also converts it to a scoped enum.

[clang][NFC] Refactor ElaboratedTypeKeyword

This patch moves ElaboratedTypeKeyword before `Type` definition so that the enum is complete where bit-field for it is declared. It also converts it to sc

[clang][NFC] Refactor ElaboratedTypeKeyword

This patch moves ElaboratedTypeKeyword before `Type` definition so that the enum is complete where bit-field for it is declared. It also converts it to scoped enum and removes `ETK_` prefix.

show more ...

[clang][NFC] Refactor `ArrayType::ArraySizeModifier`

This patch moves `ArraySizeModifier` before `Type` declaration so that it's complete at `ArrayTypeBitfields` declaration. It's also converted to

[clang][NFC] Refactor `ArrayType::ArraySizeModifier`

This patch moves `ArraySizeModifier` before `Type` declaration so that it's complete at `ArrayTypeBitfields` declaration. It's also converted to scoped enum along the way.

show more ...

[clang] remove ClassScopeFunctionSpecializationDecl (#66636)

This removes the `ClassScopeFunctionSpecializationDecl` `Decl` node, and
instead uses `DependentFunctionTemplateSpecializationInfo` to h

[clang] remove ClassScopeFunctionSpecializationDecl (#66636)

This removes the `ClassScopeFunctionSpecializationDecl` `Decl` node, and
instead uses `DependentFunctionTemplateSpecializationInfo` to handle
such declarations. `DependentFunctionTemplateSpecializationInfo` is also
changed to store a `const ASTTemplateArgumentListInfo*` to be more in
line with `FunctionTemplateSpecializationInfo`.

This also changes `FunctionDecl::isFunctionTemplateSpecialization` to
return `true` for dependent specializations, and
`FunctionDecl::getTemplateSpecializationKind`/`FunctionDecl::getTemplateSpecializationKindForInstantiation`
to return `TSK_ExplicitSpecialization` for non-friend dependent
specializations (the same behavior as dependent class scope
`ClassTemplateSepcializationDecl` & `VarTemplateSepcializationDecl`).

show more ...

[ODRHash] Hash type-as-written

Close https://github.com/llvm/llvm-project/issues/63947
Close https://github.com/llvm/llvm-project/issues/63595

This is suggested by @rsmith in
https://reviews.llvm.o

[ODRHash] Hash type-as-written

Close https://github.com/llvm/llvm-project/issues/63947
Close https://github.com/llvm/llvm-project/issues/63595

This is suggested by @rsmith in
https://reviews.llvm.org/D154324#inline-1508868

Reviewed By: rsmith

Differential Revision: https://reviews.llvm.org/D156210

show more ...

Revert "[C++20] [Modules] Use CanonicalType for base classes"

Close https://github.com/llvm/llvm-project/issues/64091

This reverts commit f82df0b285acd8a7115f0bfc55ce44474251c2d1 and add a
test fro

Revert "[C++20] [Modules] Use CanonicalType for base classes"

Close https://github.com/llvm/llvm-project/issues/64091

This reverts commit f82df0b285acd8a7115f0bfc55ce44474251c2d1 and add a
test from https://github.com/llvm/llvm-project/issues/64091

show more ...

[C++20] [Modules] Use CanonicalType for base classes

This comes from https://reviews.llvm.org/D153003

By @rsmith, the test case is valid since:

> Per [temp.type]/1.4 (http://eel.is/c++draft/temp.t

[C++20] [Modules] Use CanonicalType for base classes

This comes from https://reviews.llvm.org/D153003

By @rsmith, the test case is valid since:

> Per [temp.type]/1.4 (http://eel.is/c++draft/temp.type#1.4),
>
>> Two template-ids are the same if [...] their corresponding template
>> template-arguments refer to the same template.
> so B<A> and B<NS::A> are the same type. The stricter "same sequence of
> tokens" rule doesn't apply here, because using-declarations are not
> definitions.

> we should either (preferably) be including only the syntactic form of
> the base specifier (because local syntax is what the ODR covers), or
> the canonical type (which should be the same for both
> using-declarations).

Here we adopt the second suggested solutions.

Reviewed By: cor3ntin, v.g.vassilev

Differential Revision: https://reviews.llvm.org/D154324

show more ...