[Lex] Remove unused includes (NFC) (#116460)

Identified with misc-include-cleaner.

[C++20][Modules] Allow import for a header unit after #pragma (#111662)

Summary:
`#pragma` and headers that finish with them shouldn't prevent `import
"header_unit.h"` syntax.

Test Plan: check-

[C++20][Modules] Allow import for a header unit after #pragma (#111662)

Summary:
`#pragma` and headers that finish with them shouldn't prevent `import
"header_unit.h"` syntax.

Test Plan: check-clang

show more ...

[Clang][Parse] Fix ambiguity with nested-name-specifiers that may declarative (#96364)

Consider the following:
```
template<typename T>
struct A { };

template<typename T>
int A<T>::B::* f();

[Clang][Parse] Fix ambiguity with nested-name-specifiers that may declarative (#96364)

Consider the following:
```
template<typename T>
struct A { };

template<typename T>
int A<T>::B::* f(); // error: no member named 'B' in 'A<T>'
```

Although this is clearly valid, clang rejects it because the
_nested-name-specifier_ `A<T>::` is parsed as-if it was declarative,
meaning, we parse it as-if it was the _nested-name-specifier_ in a
redeclaration/specialization. However, we don't (and can't) know whether
the _nested-name-specifier_ is declarative until we see the '`*`' token,
but at that point we have already complained that `A` has no member
named `B`! This patch addresses this bug by adding support for _fully_
unannotated _and_ unbounded tentative parsing, which allows for us to
parse past tokens without having to cache them until we reach a point
where we can guarantee to be past the construct we are disambiguating.

I don't know where the approach taken here is ideal -- alternatives are
welcome. However, the performance impact (as measured by
llvm-compile-time-tracker (https://llvm-compile-time-tracker.com/?config=Overview&stat=instructions%3Au&remote=sdkrystian)
is quite minimal (0.09%, which I plan to further improve).

show more ...

Revert "[Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros" (#99838)

Reverts llvm/llvm-project#90574

[Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros (#90574)

This PR implement [P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1), and refactor the convolute

[Clang] Implement P3034R1 Module Declarations Shouldn’t be Macros (#90574)

This PR implement [P3034R1 Module Declarations Shouldn’t be
Macros](https://wg21.link/P3034R1), and refactor the convoluted state
machines in module name lexical analysis.

---------

Signed-off-by: yronglin <yronglin777@gmail.com>
Co-authored-by: Aaron Ballman <aaron@aaronballman.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>

show more ...

[clang] Fix typo in comments

[Safe Buffers] Serialize unsafe_buffer_usage pragmas (#92031)

The commit adds serialization and de-serialization implementations for
the stored regions. Basically, the serialized representation of

[Safe Buffers] Serialize unsafe_buffer_usage pragmas (#92031)

The commit adds serialization and de-serialization implementations for
the stored regions. Basically, the serialized representation of the
regions of a PP is a (ordered) sequence of source location encodings.
For de-serialization, regions from loaded files are stored by their ASTs.
When later one queries if a loaded location L is in an opt-out
region, PP looks up the regions of the loaded AST where L is at.

(Background if helps: a pair of `#pragma clang unsafe_buffer_usage begin/end` pragmas marks a
warning-opt-out region. The begin and end locations (opt-out regions)
are stored in preprocessor instances (PP) and will be queried by the
`-Wunsafe-buffer-usage` analyzer.)

The reported issue at upstream: https://github.com/llvm/llvm-project/issues/90501
rdar://124035402

show more ...

[NFC][Clang] Improve const correctness for IdentifierInfo (#79365)

The IdentifierInfo isn't typically modified. Use 'const' wherever
possible.

[clang][Diagnostics] Highlight code snippets (#66514)

Add some primitive syntax highlighting to our code snippet output.

This adds "checkpoints" to the Preprocessor, which we can use to start lex

[clang][Diagnostics] Highlight code snippets (#66514)

Add some primitive syntax highlighting to our code snippet output.

This adds "checkpoints" to the Preprocessor, which we can use to start lexing from. When printing a code snippet, we lex from the nearest checkpoint and highlight the tokens based on their token type.

show more ...

[CLANG] Add warning when INF or NAN are used in a binary operation or as function argument in fast math mode. (#76873)

Check for operations using INF or NaN when in ffast-math mode and
generate a w

[CLANG] Add warning when INF or NAN are used in a binary operation or as function argument in fast math mode. (#76873)

Check for operations using INF or NaN when in ffast-math mode and
generate a warning.

show more ...

[clang] Change representation of CurLexerKind (#70381)

Previous representation used an enumeration combined to a switch to
dispatch to the appropriate lexer.

Use function pointer so that the dis

[clang] Change representation of CurLexerKind (#70381)

Previous representation used an enumeration combined to a switch to
dispatch to the appropriate lexer.

Use function pointer so that the dispatching is just an indirect call,
which is actually better because lexing is a costly task compared to a
function call.

This also makes the code slightly cleaner, speedup on compile time
tracker are consistent and range form -0.05% to -0.20% for NewPM-O0-g,
see


https://llvm-compile-time-tracker.com/compare.php?from=f9906508bc4f05d3950e2219b4c56f6c078a61ef&to=608c85ec1283638db949d73e062bcc3355001ce4&stat=instructions:u

Considering just the preprocessing task, preprocessing the sqlite
amalgametion takes -0.6% instructions (according to valgrind
--tool=callgrind)

---------

Co-authored-by: serge-sans-paille <sguelton@mozilla.com>
Co-authored-by: cor3ntin <corentinjabot@gmail.com>

show more ...

[Modules] Fix ModuleDeclState transition when module is used as a regular identifier (#71134)

`ModuleDeclState` is incorrectly changed to `NamedModuleImplementation`
for `struct module {}; void foo

[Modules] Fix ModuleDeclState transition when module is used as a regular identifier (#71134)

`ModuleDeclState` is incorrectly changed to `NamedModuleImplementation`
for `struct module {}; void foo(module a);`. This is mostly benign but
leads to a spurious warning after #69555.

A real world example is:
```
// pybind11.h
class module_ { ... };
using module = module_;

// tensorflow
void DefineMetricsModule(pybind11::module main_module);
// `module main_module);` incorrectly changes `ModuleDeclState` to `NamedModuleImplementation`

#include <algorithm> // spurious warning
```

show more ...

[clang][Lex][NFC] Make some local variables const

[Lex] Handle repl_input_end in Preprocessor::LexTokensUntilEOF()

This fixes many unit tests when trying to enable IncrementalExtensions
by default for testing purposes.

Differential Revision: https

[Lex] Handle repl_input_end in Preprocessor::LexTokensUntilEOF()

This fixes many unit tests when trying to enable IncrementalExtensions
by default for testing purposes.

Differential Revision: https://reviews.llvm.org/D158415

show more ...

[Lex] Introduce Preprocessor::LexTokensUntilEOF()

This new method repeatedly calls Lex() until end of file is reached
and optionally fills a std::vector of Tokens. Use it in Clang's unit
tests to av

[Lex] Introduce Preprocessor::LexTokensUntilEOF()

This new method repeatedly calls Lex() until end of file is reached
and optionally fills a std::vector of Tokens. Use it in Clang's unit
tests to avoid quite some code duplication.

Differential Revision: https://reviews.llvm.org/D158413

show more ...

[clang] NFCI: Use `FileEntryRef` in `SourceManager::overrideFileContents()`

[clang] NFCI: Use `FileEntryRef` in `ASTReader::GetHeaderFileInfo()`

This is the `ASTReader` counterpart to PR #67383.

[clang][NFC] Preprocessor only needs const LangOptions (#66874)

Re-apply "[Parse] Split incremental-extensions" (#66446)

Re-applies #65683 with a fix to always run
`Actions.ActOnEndOfTranslationUnit` regardless of incremental
processing.

Revert "[Parse] Split incremental-extensions" (#66281)

This reverts commit c2fb112021529c635cccd8bb9d846b2c64fc291d, which
breaks:
```
lldb-api.commands/expression/diagnostics.TestExprDiagnostics

Revert "[Parse] Split incremental-extensions" (#66281)

This reverts commit c2fb112021529c635cccd8bb9d846b2c64fc291d, which
breaks:
```
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
```

show more ...

[Parse] Split incremental-extensions (#65683)

The preprocessor `IncrementalProcessing` option was being used to
control whether or not to teardown the lexer or run the end of
translation unit acti

[Parse] Split incremental-extensions (#65683)

The preprocessor `IncrementalProcessing` option was being used to
control whether or not to teardown the lexer or run the end of
translation unit action. In D127284 this was merged with
`-fincremental-extensions`, which also changes top level parsing.

Split these again so that the former behavior can be achieved without
the latter (ie. to allow managing lifetime without also changing
parsing).

Resolves rdar://113406310.

show more ...

Revert "Currently the control of the eval-method is mixed with fast-math."

Setting __FLT_EVAL_METHOD__ to -1 with fast-math will set
__GLIBC_FLT_EVAL_METHOD to 2 and long double ends up being used f

Revert "Currently the control of the eval-method is mixed with fast-math."

Setting __FLT_EVAL_METHOD__ to -1 with fast-math will set
__GLIBC_FLT_EVAL_METHOD to 2 and long double ends up being used for
float_t and double_t. This creates some ABI breakage with various C libraries.
See details here: https://github.com/llvm/llvm-project/issues/60781

This reverts commit bbf0d1932a3c1be970ed8a580e51edf571b80fd5.

show more ...

[Modules] Remove -fmodules-ts

As the diagnostic message shows, we should remove -fmodules-ts flag in
clang/llvm17. Since clang/llvm16 is already branched. We can remove the
depreacared flag now.

[C++20] [Modules] [NFC] Add Preprocessor methods for named modules - for ClangScanDeps (1/4)

This patch prepares the necessary interfaces in the preprocessor part
for D137527 since we need to recogn

[C++20] [Modules] [NFC] Add Preprocessor methods for named modules - for ClangScanDeps (1/4)

This patch prepares the necessary interfaces in the preprocessor part
for D137527 since we need to recognize if we're in a module unit, the
module kinds and the module declaration and the module we're importing
in the preprocessor.

Differential Revision: https://reviews.llvm.org/D137526

show more ...

[Lex] Fix -Wunused-variable for LLVM_ENABLE_ASSERTIONS=off builds after D140179