[clang][test] add TestLanguage.def to specify all tested language versions (#94243)

Adds a def file to have a single location where tested language versions
are specified. Removes the need to updat

[clang][test] add TestLanguage.def to specify all tested language versions (#94243)

Adds a def file to have a single location where tested language versions
are specified. Removes the need to update multiple locations in the
testing infrastructure to add a new language version to be tested. Test
instatiation can now include all languages without needing to specify
them.
This patch also adds pretty printing for instantiated test names. That
means, that a test instantiated with C++23 will have the name
`...TestSuite/TestName/CXX23` instead ending with some number (index of
the argument for instantiation of the test), which provides a better
experience when encountering a test failure with a specific language
version. The suffix will also contain an `_win` if the target contains
`win`.

---------

Co-authored-by: Sirraide <aeternalmail@gmail.com>

show more ...

[clang][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D

[clang][NFC] Use c++17 style variable type traits

This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: shafik

Differential Revision: https://reviews.llvm.org/D137491

show more ...

[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

[syntax] Introduce a TokenManager interface.

TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

It decouples the syntax-tree from a particular token implementation (TokenBuffer
previously). This enables us to use a different underlying token implementation
for the syntax Leaf node -- in clang pseudoparser, we want to produce a
syntax-tree with its own pseudo::Token rather than syntax::Token.

Differential Revision: https://reviews.llvm.org/D128411

show more ...

[NFC][testing] Return underlying strings directly instead of OS.str()

This avoids an unnecessary copy required by 'return OS.str()', allowing
instead for NRVO or implicit move. The .str() call (whic

[NFC][testing] Return underlying strings directly instead of OS.str()

This avoids an unnecessary copy required by 'return OS.str()', allowing
instead for NRVO or implicit move. The .str() call (which flushes the
stream) is no longer required since 65b13610a5226b84889b923bae884ba395ad084d,
which made raw_string_ostream unbuffered by default.

Differential Revision: https://reviews.llvm.org/D115374

show more ...

Bump googletest to 1.10.0

[Syntax] Add iterators over children of syntax trees.

This gives us slightly nicer syntax (foreach) for idioms currently expressed
as a loop, and the option to use range algorithms where it makes se

[Syntax] Add iterators over children of syntax trees.

This gives us slightly nicer syntax (foreach) for idioms currently expressed
as a loop, and the option to use range algorithms where it makes sense
(e.g. llvm::all_of et al encapsulate the needed flow control in a useful way).

It's also a building block for iteration over filtered views (e.g. iterate over
all Stmt children, with the right type):
for (const Statement &S : filter<Statement>(N.children()))
...

I realize the recent direction has been mostly towards strongly-typed
node-specific facilities, but I think it's important we have convenient
generic facilities too.

Differential Revision: https://reviews.llvm.org/D90023

show more ...

[SyntaxTree] Test the List API

Differential Revision: https://reviews.llvm.org/D87839

[SyntaxTree][Synthesis] Fix: `deepCopy` -> `deepCopyExpandingMacros`.

There can be Macros that are tagged with `modifiable`. Thus verifying
`canModifyAllDescendants` is not sufficient to avoid macro

[SyntaxTree][Synthesis] Fix: `deepCopy` -> `deepCopyExpandingMacros`.

There can be Macros that are tagged with `modifiable`. Thus verifying
`canModifyAllDescendants` is not sufficient to avoid macros when deep
copying.

We think the `TokenBuffer` could inform us whether a `Token` comes from
a macro. We'll look into that when we can surface this information
easily, for instance in unit tests for `ComputeReplacements`.

Differential Revision: https://reviews.llvm.org/D88034

show more ...

[SyntaxTree] Test `findFirstLeaf` and `findLastLeaf`

* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `

[SyntaxTree] Test `findFirstLeaf` and `findLastLeaf`

* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `findLastLeaf`
* Fix implementations of `findFirstLeaf` and `findLastLeaf` that had
been broken when empty `Tree` were present.

Differential Revision: https://reviews.llvm.org/D87779

show more ...

[SyntaxTree] Rename tests following `TestSuite_TestCase` + nits

[SyntaxTree] Split tests for expressions

We do that because:
* Big tests generated big tree dumps that could hardly serve as documentation.
* In most cases the tests didn't share setup, thus there w

[SyntaxTree] Split tests for expressions

We do that because:
* Big tests generated big tree dumps that could hardly serve as documentation.
* In most cases the tests didn't share setup, thus there was not much addition in lines of code.

We split tests for:
* `UserDefinedLiteral`
* `NestedBinaryOperator`
* `UserDefinedBinaryOperator`
* `UserDefinedPrefixOperator`
* `QualifiedId`

Differential Revision: https://reviews.llvm.org/D85819

show more ...

[SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-U

[SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-IdExpression
| `-UnqualifiedId
| `-<
`-IdExpression
`-UnqualifiedId
`-y
```
But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-<
`-IdExpression
`-UnqualifiedId
`-y
```

Differential Revision: https://reviews.llvm.org/D85750

show more ...

[SyntaxTree] Expand support for `NestedNameSpecifier`

Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.

To ac

[SyntaxTree] Expand support for `NestedNameSpecifier`

Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.

To achieve this we:
* Use the `RecursiveASTVisitor`'s API to traverse
`NestedNameSpecifierLoc`s and automatically create its syntax nodes
* Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.

In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.

show more ...

[SyntaxTree] Use simplified grammar rule for `NestedNameSpecifier` grammar nodes

This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
/*empty*/
simple-template-specifier:
te

[SyntaxTree] Use simplified grammar rule for `NestedNameSpecifier` grammar nodes

This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
/*empty*/
simple-template-specifier:
template_opt simple-template-id
name-specifier:
global-specifier
decltype-specifier
identifier
simple-template-specifier
nested-name-specifier:
list(name-specifier, ::, non-empty, terminated)

It is a relaxed version of C++ [expr.prim.id] and quite simpler to map to our API.

TODO: refine name specifiers, `simple-template-name-specifier` and
decltype-name-specifier` are token soup for now.

show more ...

[SyntaxTree][NFC] remove redundant namespace-specifiers

Differential Revision: https://reviews.llvm.org/D85427

[SyntaxTree] Add test coverage for `->*` operator

This was the last binary operator that we supported but didn't have any
test coverage. The recent fix in a crash in member pointers allowed us
to ad

[SyntaxTree] Add test coverage for `->*` operator

This was the last binary operator that we supported but didn't have any
test coverage. The recent fix in a crash in member pointers allowed us
to add this test.

Differential Revision: https://reviews.llvm.org/D85185

show more ...

[SyntaxTree] Fix crash on pointer to member function

Differential Revision: https://reviews.llvm.org/D85146

[clang][Syntax] syntax::Arena doesnt own TokenBuffer

Currently an Arena can only be built while consuming a TokenBuffer,
some users (like clangd) might want to share a TokenBuffer with multiple
comp

[clang][Syntax] syntax::Arena doesnt own TokenBuffer

Currently an Arena can only be built while consuming a TokenBuffer,
some users (like clangd) might want to share a TokenBuffer with multiple
compenents. This patch changes Arena's TokenBuffer member to be a reference so
that it can be created with read-only token buffers.

Differential Revision: https://reviews.llvm.org/D84973

show more ...

Add kinded UDL for raw literal operator and numeric literal operator template

Fix crash on `user defined literals`

Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the op

Fix crash on `user defined literals`

Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the operator and interpret it as the underlying literal.
e.g.: `1.2_w` token generates syntax node IntegerLiteral(1.2_w)

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82157

show more ...

Fix crash on overloaded postfix unary operators due to invalid sloc

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82954

Add parenthesized expression to SyntaxTree

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82960

Move TestClangConfig into libClangTesting and use it in AST Matchers tests

Summary:
Previously, AST Matchers tests were using a custom way to run a test
with a specific C++ standard version. I'm mig

Move TestClangConfig into libClangTesting and use it in AST Matchers tests

Summary:
Previously, AST Matchers tests were using a custom way to run a test
with a specific C++ standard version. I'm migrating them to a shared
infrastructure to specify a Clang target from libClangTesting. I'm also
changing tests for AST Matchers to run in multiple language standards
versions, and under multiple triples that have different behavior with
regards to templates.

To keep the size of the patch manageable, in this patch I'm only
migrating one file to get the process started and get feedback on this
approach.

One caveat is that increasing the number of test configuration does
significantly increase the runtime of AST Matchers tests. On my machine,
the test runtime increases from 2.0 to 6.0s. I think it is worth the
improved test coverage.

Reviewers: jdoerfert, ymandel

Reviewed By: ymandel

Subscribers: gribozavr2, jfb, sstefan1, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82179

show more ...

Work around a bug in MSVC in the syntax tree test

Summary:
MSVC does not handle raw string literals with embedded double quotes
correctly. I switched the affected test case to use regular string
lit

Work around a bug in MSVC in the syntax tree test

Summary:
MSVC does not handle raw string literals with embedded double quotes
correctly. I switched the affected test case to use regular string
literals insetad.

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82636

show more ...

Add `FloatingLiteral` to SyntaxTree

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82318