[clang-reorder-fields] Reorder leading comments (#123740)

Similarly to https://github.com/llvm/llvm-project/pull/122918, leading
comments are currently not being moved.

```
struct Foo {
// T

[clang-reorder-fields] Reorder leading comments (#123740)

Similarly to https://github.com/llvm/llvm-project/pull/122918, leading
comments are currently not being moved.

```
struct Foo {
// This one is the cool field.
int a;
int b;
};
```

becomes:

```
struct Foo {
// This one is the cool field.
int b;
int a;
};
```

but should be:

```
struct Foo {
int b;
// This one is the cool field.
int a;
};
```

show more ...

[clang][refactor] Refactor `findNextTokenIncludingComments` (#123060)

We have two copies of the same code in clang-tidy and
clang-reorder-fields, and those are extremenly similar to
`Lexer::findNe

[clang][refactor] Refactor `findNextTokenIncludingComments` (#123060)

We have two copies of the same code in clang-tidy and
clang-reorder-fields, and those are extremenly similar to
`Lexer::findNextToken`, so just add an extra agument to the latter.

---------

Co-authored-by: cor3ntin <corentinjabot@gmail.com>

show more ...

Skip escaped newlines before checking for whitespace in Lexer::getRawToken. (#117548)

The Lexer used in getRawToken is not told to keep whitespace, so when it
skips over escaped newlines, it also i

Skip escaped newlines before checking for whitespace in Lexer::getRawToken. (#117548)

The Lexer used in getRawToken is not told to keep whitespace, so when it
skips over escaped newlines, it also ignores whitespace, regardless of
getRawToken's IgnoreWhiteSpace parameter.

Instead of letting this case fall through to lexing, check
for whitespace after skipping over any escaped newlines.

show more ...

[Lex] Remove unused includes (NFC) (#116460)

Identified with misc-include-cleaner.

Remove ^^ as a token in OpenCL (#108224)

OpenCL has a reserved operator (^^), the use of which was diagnosed as
an error (735c6cdebdcd4292928079cb18a90f0dd5cd65fb). However, OpenCL
also encourages

Remove ^^ as a token in OpenCL (#108224)

OpenCL has a reserved operator (^^), the use of which was diagnosed as
an error (735c6cdebdcd4292928079cb18a90f0dd5cd65fb). However, OpenCL
also encourages working with the blocks language extension. This token
has a parsing ambiguity as a result. Consider:

unsigned x=0;
unsigned y=x^^{return 0;}();

This should result in y holding the value zero (0^0) through an
immediately invoked block call as the right-hand side of the xor
operator. However, it causes errors instead because of this reserved
token: https://godbolt.org/z/navf7jTv1

This token is still reserved in OpenCL 3.0, so we still wish to issue a
diagnostic for its use. However, we do not need to create a token for an
extension point that's been unused for about a decade. So this patch
moves the diagnostic from a parsing diagnostic to a lexing diagnostic
and no longer forms a single token. The diagnostic behavior is slightly
worse as a result, but still seems acceptable.

Part of the reason this is coming up is because WG21 is considering
using ^^ as a token for reflection, so this token may come back in the
future.

show more ...

[Clang] Warn with -Wpre-c23-compat instead of -Wpre-c++17-compat for u8 character literals in C23 (#97210)

Co-authored-by: cor3ntin <corentinjabot@gmail.com>

[Clang] Allow raw string literals in C as an extension (#88265)

This enables raw R"" string literals in C in some language modes
and adds an option to disable or enable them explicitly as an
exten

[Clang] Allow raw string literals in C as an extension (#88265)

This enables raw R"" string literals in C in some language modes
and adds an option to disable or enable them explicitly as an
extension.

Background: GCC supports raw string literals in C in `-gnuXY` modes
starting with gnu99. This pr both enables raw string literals in gnu99
mode and later in C and adds an `-f[no-]raw-string-literals` flag to override
this behaviour. The decision not to enable raw string literals in gnu89
mode, according to the GCC devs, is intentional as that mode is supposed
to be used for ‘old code’ that they don’t want to break; we’ve decided to
match GCC’s behaviour here as well.

The `-fraw-string-literals` flag can additionally be used to enable raw string
literals in modes where they aren’t enabled by default (such as c99—as
opposed to gnu99—or even e.g. C++03); conversely, the negated flag can
be used to disable them in any gnuXY modes that *do* provide them by
default, or to override a previous flag. However, we do *not* support
disabling raw string literals (or indeed either of these two options) in
C++11 mode and later, because we don’t want to just start supporting
disabling features that are actually part of the language in the general case.

This fixes #85703.

show more ...

Fix off-by-one issue found by post-commit review

[Clang] allow `` `@$ `` in raw string delimiters in C++26 (#93216)

And as an extension in older language modes.

Per https://eel.is/c++draft/lex.string#nt:d-char

Fixes #93130

bad error message on incorrect string literal #18079 (#81670)

(bad error message on incorrect string literal)

Fixed the error message for incorrect string literal

before:

```
test.cpp:1:19

bad error message on incorrect string literal #18079 (#81670)

(bad error message on incorrect string literal)

Fixed the error message for incorrect string literal

before:

```
test.cpp:1:19: error: invalid character '
' character in raw string delimiter; use PREFIX( )PREFIX to delimit raw string
char const* a = R"
^
```

now:

```
test.cpp:1:19: error: invalid newline character in raw string delimiter; use PREFIX( )PREFIX to delimit raw string
1 | char const* a = R"
| ^
```

---------

Co-authored-by: Jon Roelofs <jroelofs@gmail.com>

show more ...

[clang][NFC] Move isSimpleTypeSpecifier() from Sema to Token (#80101)

So that it can be used by clang-format.

[clang] Use StringRef::{starts,ends}_with (NFC) (#75149)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}:

[clang] Use StringRef::{starts,ends}_with (NFC) (#75149)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[HLSL] Support vector swizzles on scalars (#67700)

HLSL supports vector swizzles on scalars by implicitly converting the
scalar to a single-element vector. This syntax is a convienent way to
initi

[HLSL] Support vector swizzles on scalars (#67700)

HLSL supports vector swizzles on scalars by implicitly converting the
scalar to a single-element vector. This syntax is a convienent way to
initialize vectors based on filling a scalar value.

There are two parts of this change. The first part in the Lexer splits
numeric constant tokens when a `.x` or `.r` suffix is encountered. This
splitting is a bit hacky but allows the numeric constant to be parsed
separately from the vector element expression. There is an ambiguity
here with the `r` suffix used by fixed point types, however fixed point
types aren't supported in HLSL so this should not cause any exposable
problems (a separate issue has been filed to track validating language
options for HLSL: #67689).

The second part of this change is in Sema::LookupMemberExpr. For HLSL,
if the base type is a scalar, we implicit cast the scalar to a
one-element vector then call back to perform the vector lookup.

Fixes #56658 and #67511

show more ...

[clang] Change GetCharAndSizeSlow interface to by-value style

Instead of passing the Size by reference, assuming it is initialized,
return it alongside the expected char result as a POD.

This makes

[clang] Change GetCharAndSizeSlow interface to by-value style

Instead of passing the Size by reference, assuming it is initialized,
return it alongside the expected char result as a POD.

This makes the interface less error prone: previous interface expected
the Size reference to be initialized, and it was often forgotten,
leading to uninitialized variable usage. This patch fixes the issue.

This also generates faster code, as the returned POD (a char and an
unsigned) fits in 64 bits. The speedup according to compile time tracker
reach -O.7%, with a good number of -0.4%. Details are available on

https://llvm-compile-time-tracker.com/compare.php?from=3fe63f81fcb999681daa11b2890c82fda3aaeef5&to=fc76a9202f737472ecad4d6e0b0bf87a013866f3&stat=instructions:u

And icing on the cake, on my setup it also shaves 2kB out of
libclang-cpp :-)

This is a recommit of d8f5a18b6e587aeaa8b99707e87b652f49b160cd for

show more ...

Revert "Perf/lexer faster slow get char and size (#70543)"

This reverts commit d8f5a18b6e587aeaa8b99707e87b652f49b160cd.
Breaks build, see:
https://github.com/llvm/llvm-project/pull/70543#issuecomme

Revert "Perf/lexer faster slow get char and size (#70543)"

This reverts commit d8f5a18b6e587aeaa8b99707e87b652f49b160cd.
Breaks build, see:
https://github.com/llvm/llvm-project/pull/70543#issuecomment-1784227421

show more ...

Perf/lexer faster slow get char and size (#70543)

Co-authored-by: serge-sans-paille <sguelton@mozilla.com>

[clang] Provide an SSE4.2 implementation of identifier token lexer (#68962)

The _mm_cmpistri instruction can be used to quickly parse identifiers.

With this patch activated, clang pre-processes <

[clang] Provide an SSE4.2 implementation of identifier token lexer (#68962)

The _mm_cmpistri instruction can be used to quickly parse identifiers.

With this patch activated, clang pre-processes <iostream> 1.8% faster,
and sqlite3.c amalgametion 1.5% faster, based on time measurements and
number of executed instructions as measured by valgrind.

The introduction of an extra helper function in the regular case has no
impact on performance, see


https://llvm-compile-time-tracker.com/compare.php?from=30240e428f0ec7d4a6d1b84f9f807ce12b46cfd1&to=12bcb016cde4579ca7b75397762098c03eb4f264&stat=instructions:u

---------

Co-authored-by: serge-sans-paille <sguelton@mozilla.com>

show more ...

[clang][Lex][NFC] Make some local variables const

[Clang] Handle non-ASCII after line splicing

int a\
ス;

Failed to be parsed as a valid identifier.

Fixes #65156

Reviewed By: tahonermann

Differential Revision: https://reviews.llvm.org/D159345

[clang][NFC] Remove stray slash

Revert "[Clang] CWG1473: do not err on the lack of space after operator"""

This reverts commit f2583f3acf596cc545c8c0e3cb28e712f4ebf21b.

There is a large body of non-conforming C-like code using fo

Revert "[Clang] CWG1473: do not err on the lack of space after operator"""

This reverts commit f2583f3acf596cc545c8c0e3cb28e712f4ebf21b.

There is a large body of non-conforming C-like code using format strings
like this:

#define PRIuS "zu"
void h(size_t foo, size_t bar) {
printf("foo is %"PRIuS", bar is %"PRIuS, foo, bar);
}

Rejecting this code would be very disruptive. We could decide to do
that, but it's sufficiently disruptive that I think it requires
gathering more community consensus with an RFC, and Aaron indicated [1]
it's OK to revert for now so continuous testing systems can see past
this issue while we decide what to do.

[1] https://reviews.llvm.org/D153156#4607717

show more ...

[Lex] Preambles should contain the global module fragment.

For applications like clangd, the preamble remains an important optimization
when editing a module definition. The global module fragment i

[Lex] Preambles should contain the global module fragment.

For applications like clangd, the preamble remains an important optimization
when editing a module definition. The global module fragment is a good fit for
it as it by definition contains only preprocessor directives.
Before this patch, we would terminate the preamble immediately at the "module"
keyword.

Differential Revision: https://reviews.llvm.org/D158439

show more ...

[Clang] CWG1473: do not err on the lack of space after operator""

In addition:
1. Fix tests for CWG2521 deprecation warning.
2. Enable -Wdeprecated-literal-operator by default.

Differential Rev

[Clang] CWG1473: do not err on the lack of space after operator""

In addition:
1. Fix tests for CWG2521 deprecation warning.
2. Enable -Wdeprecated-literal-operator by default.

Differential Revision: https://reviews.llvm.org/D153156

show more ...

[C23] Rename C2x->C23 in diagnostics

This renames C2x to C23 in diagnostic identifiers and messages. The
changes were made mechanically.

[C23] Rename C2x -> C23; NFC

This does the rename for most internal uses of C2x, but does not rename
or reword diagnostics (those will be done in a follow-up).

I also updated standards references a

[C23] Rename C2x -> C23; NFC

This does the rename for most internal uses of C2x, but does not rename
or reword diagnostics (those will be done in a follow-up).

I also updated standards references and citations to the final wording
in the standard.

show more ...