[Clang] [NFC] Refactor AST visitors in Sema and the static analyser to use DynamicRecursiveASTVisitor (#115144)

This pr refactors all recursive AST visitors in `Sema`, `Analyze`, and
`StaticAnalysi

[Clang] [NFC] Refactor AST visitors in Sema and the static analyser to use DynamicRecursiveASTVisitor (#115144)

This pr refactors all recursive AST visitors in `Sema`, `Analyze`, and
`StaticAnalysis` to inherit from DRAV instead. This is over half of the
visitors that inherit from RAV directly.

See also #115132, #110040, #93462

LLVM Compile-Time Tracker link for this branch:
https://llvm-compile-time-tracker.com/compare.php?from=5adb5c05a2e9f31385fbba8b0436cbc07d91a44d&to=b58e589a86c06ba28d4d90613864d10be29aa5ba&stat=instructions%3Au

show more ...

[analyzer] Set and display CSA analysis entry points as notes on debugging (#84823)

When debugging CSA issues, sometimes it would be useful to have a
dedicated note for the analysis entry point, ak

[analyzer] Set and display CSA analysis entry points as notes on debugging (#84823)

When debugging CSA issues, sometimes it would be useful to have a
dedicated note for the analysis entry point, aka. the function name you
would need to pass as "-analyze-function=XYZ" to reproduce a specific
issue.
One way we use (or will use) this downstream is to provide tooling on
top of creduce to enhance to supercharge productivity by automatically
reduce cases on crashes for example.

This will be added only if the "-analyzer-note-analysis-entry-points" is
set or the "analyzer-display-progress" is on.

This additional entry point marker will be the first "note" if enabled,
with the following message: "[debug] analyzing from XYZ". They are
prefixed by "[debug]" to remind the CSA developer that this is only
meant to be visible for them, for debugging purposes.

CPP-5012

show more ...

[clang] Use StringRef::{starts,ends}_with (NFC) (#75149)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}:

[clang] Use StringRef::{starts,ends}_with (NFC) (#75149)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[clang] NFCI: Change returned AnalyzerOptions smart pointer to reference

[analyzer] Remove the unused LocalCheckers.h header

[analyzer][NFC] Remove dead code and modernize surroundings

Thanks @kazu for helping me clean these parts in D127799.

I'm leaving the dump methods, along with the unused visitor handlers and
the fo

[analyzer][NFC] Remove dead code and modernize surroundings

Thanks @kazu for helping me clean these parts in D127799.

I'm leaving the dump methods, along with the unused visitor handlers and
the forwarding methods.

The dead parts actually helped to uncover two bugs, to which I'm going
to post separate patches.

Reviewed By: martong

Differential Revision: https://reviews.llvm.org/D127836

show more ...

Reland "[analyzer] Deprecate `-analyzer-store region` flag"

I'm trying to remove unused options from the `Analyses.def` file, then
merge the rest of the useful options into the `AnalyzerOptions.def`

Reland "[analyzer] Deprecate `-analyzer-store region` flag"

I'm trying to remove unused options from the `Analyses.def` file, then
merge the rest of the useful options into the `AnalyzerOptions.def`.
Then make sure one can set these by an `-analyzer-config XXX=YYY` style
flag.
Then surface the `-analyzer-config` to the `clang` frontend;

After all of this, we can pursue the tablegen approach described
https://discourse.llvm.org/t/rfc-tablegen-clang-static-analyzer-engine-options-for-better-documentation/61488

In this patch, I'm proposing flag deprecations.
We should support deprecated analyzer flags for exactly one release. In
this case I'm planning to drop this flag in `clang-16`.

In the clang frontend, now we won't pass this option to the cc1
frontend, rather emit a warning diagnostic reminding the users about
this deprecated flag, which will be turned into error in clang-16.

Unfortunately, I had to remove all the tests referring to this flag,
causing a mass change. I've also added a test for checking this warning.

I've seen that `scan-build` also uses this flag, but I think we should
remove that part only after we turn this into a hard error.

Reviewed By: martong

Differential Revision: https://reviews.llvm.org/D126215

show more ...

Revert "[analyzer] Deprecate `-analyzer-store region` flag"

This reverts commit d50d9946d1d7e5f20881f0bc71fbd025040b1c96.
Broke check-clang, see comments on https://reviews.llvm.org/D126067

Also re

Revert "[analyzer] Deprecate `-analyzer-store region` flag"

This reverts commit d50d9946d1d7e5f20881f0bc71fbd025040b1c96.
Broke check-clang, see comments on https://reviews.llvm.org/D126067

Also revert dependent change "[analyzer] Deprecate the unused 'analyzer-opt-analyze-nested-blocks' cc1 flag"
This reverts commit 07b4a6d0461fe64e10d30029ed3d598e49ca3810.

Also revert "[analyzer] Fix buildbots after introducing a new frontend warning"
This reverts commit 90374df15ddc58d823ca42326a76f58e748f20eb.
(See https://reviews.llvm.org/rG90374df15ddc58d823ca42326a76f58e748f20eb)

show more ...

[analyzer] Deprecate `-analyzer-store region` flag

I'm trying to remove unused options from the `Analyses.def` file, then
merge the rest of the useful options into the `AnalyzerOptions.def`.
Then ma

[analyzer] Deprecate `-analyzer-store region` flag

I'm trying to remove unused options from the `Analyses.def` file, then
merge the rest of the useful options into the `AnalyzerOptions.def`.
Then make sure one can set these by an `-analyzer-config XXX=YYY` style
flag.
Then surface the `-analyzer-config` to the `clang` frontend;

After all of this, we can pursue the tablegen approach described
https://discourse.llvm.org/t/rfc-tablegen-clang-static-analyzer-engine-options-for-better-documentation/61488

In this patch, I'm proposing flag deprecations.
We should support deprecated analyzer flags for exactly one release. In
this case I'm planning to drop this flag in `clang-16`.

In the clang frontend, now we won't pass this option to the cc1
frontend, rather emit a warning diagnostic reminding the users about
this deprecated flag, which will be turned into error in clang-16.

Unfortunately, I had to remove all the tests referring to this flag,
causing a mass change. I've also added a test for checking this warning.

I've seen that `scan-build` also uses this flag, but I think we should
remove that part only after we turn this into a hard error.

Reviewed By: martong

Differential Revision: https://reviews.llvm.org/D126215

show more ...

[clang][analyzer][ctu] Make CTU a two phase analysis

This new CTU implementation is the natural extension of the normal single TU
analysis. The approach consists of two analysis phases. During the f

[clang][analyzer][ctu] Make CTU a two phase analysis

This new CTU implementation is the natural extension of the normal single TU
analysis. The approach consists of two analysis phases. During the first phase,
we do a normal single TU analysis. During this phase, if we find a foreign
function (that could be inlined from another TU) then we don’t inline that
immediately, we rather mark that to be analysed later.
When the first phase is finished then we start the second phase, the CTU phase.
In this phase, we continue the analysis from that point (exploded node)
which had been enqueued during the first phase. We gradually extend the
exploded graph of the single TU analysis with the new node that was
created by the inlining of the foreign function.

We count the number of analysis steps of the first phase and we limit the
second (ctu) phase with this number.

This new implementation makes it convenient for the users to run the
single-TU and the CTU analysis in one go, they don't need to run the two
analysis separately. Thus, we name this new implementation as "onego" CTU.

Discussion:
https://discourse.llvm.org/t/rfc-much-faster-cross-translation-unit-ctu-analysis-implementation/61728

Differential Revision: https://reviews.llvm.org/D123773

show more ...

[analyzer][ctu] Only import const and trivial VarDecls

Do import the definition of objects from a foreign translation unit if that's type is const and trivial.

Differential Revision: https://review

[analyzer][ctu] Only import const and trivial VarDecls

Do import the definition of objects from a foreign translation unit if that's type is const and trivial.

Differential Revision: https://reviews.llvm.org/D122805

show more ...

Revert "Revert "[analyzer] Prevent misuses of -analyze-function""

This reverts commit 620d99b7edc64ee87b1ce209f179305e6a919006.

Let's see if removing the two offending RUN lines makes this patch pa

Revert "Revert "[analyzer] Prevent misuses of -analyze-function""

This reverts commit 620d99b7edc64ee87b1ce209f179305e6a919006.

Let's see if removing the two offending RUN lines makes this patch pass.
Not ideal to drop tests but, it's just a debugging feature, probably not
that important.

show more ...

Revert "[analyzer] Prevent misuses of -analyze-function"

This reverts commit 841817b1ed26c1fbb709957d54c0e2751624fbf8.

Ah, it still fails on build bots for some reason.
Pinning the target triple wa

Revert "[analyzer] Prevent misuses of -analyze-function"

This reverts commit 841817b1ed26c1fbb709957d54c0e2751624fbf8.

Ah, it still fails on build bots for some reason.
Pinning the target triple was not enough.

show more ...

[analyzer] Prevent misuses of -analyze-function

Sometimes when I pass the mentioned option I forget about passing the
parameter list for c++ sources.
It would be also useful newcomers to learn about

[analyzer] Prevent misuses of -analyze-function

Sometimes when I pass the mentioned option I forget about passing the
parameter list for c++ sources.
It would be also useful newcomers to learn about this.

This patch introduces some logic checking common misuses involving
`-analyze-function`.

Reviewed-By: martong

Differential Revision: https://reviews.llvm.org/D118690

show more ...

[NFC][Analyzer] Use range based for loop.

Use range base loop loop to improve code readability.

Differential Revision: https://reviews.llvm.org/D119103

Revert "[analyzer] Prevent misuses of -analyze-function"

This reverts commit 9d6a6159730171bc0faf78d7f109d6543f4c93c2.

Exit Code: 1

Command Output (stderr):
--
/scratch/buildbot/bothome/clang-ve-n

Revert "[analyzer] Prevent misuses of -analyze-function"

This reverts commit 9d6a6159730171bc0faf78d7f109d6543f4c93c2.

Exit Code: 1

Command Output (stderr):
--
/scratch/buildbot/bothome/clang-ve-ninja/llvm-project/clang/test/Analysis/analyze-function-guide.cpp:53:21: error: CHECK-EMPTY-NOT: excluded string found in input // CHECK-EMPTY-NOT: Every top-level function was skipped.
^
<stdin>:1:1: note: found here
Every top-level function was skipped.
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Input file: <stdin>
Check file: /scratch/buildbot/bothome/clang-ve-ninja/llvm-project/clang/test/Analysis/analyze-function-guide.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
1: Every top-level function was skipped.
not:53 !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match expected
2: Pass the -analyzer-display-progress for tracking which functions are analyzed.
>>>>>>

show more ...

[analyzer] Prevent misuses of -analyze-function

Sometimes when I pass the mentioned option I forget about passing the
parameter list for c++ sources.
It would be also useful newcomers to learn about

[analyzer] Prevent misuses of -analyze-function

Sometimes when I pass the mentioned option I forget about passing the
parameter list for c++ sources.
It would be also useful newcomers to learn about this.

This patch introduces some logic checking common misuses involving
`-analyze-function`.

Reviewed-By: martong

Differential Revision: https://reviews.llvm.org/D118690

show more ...

[clang] Fix bugprone argument comments (NFC)

Identified with bugprone-argument-comment.

[analyzer] Ignore flex generated files

Some projects [1,2,3] have flex-generated files besides bison-generated
ones.
Unfortunately, the comment `"/* A lexical scanner generated by flex */"`
generate

[analyzer] Ignore flex generated files

Some projects [1,2,3] have flex-generated files besides bison-generated
ones.
Unfortunately, the comment `"/* A lexical scanner generated by flex */"`
generated by the tools is not necessarily at the beginning of the file,
thus we need to quickly skim through the file for this needle string.

Luckily, StringRef can do this operation in an efficient way.

That being said, now the bison comment is not required to be at the very
beginning of the file. This allows us to detect a couple more cases
[4,5,6].

Alternatively, we could say that we only allow whitespace characters
before matching the bison/flex header comment. That would prevent the
(probably) unnecessary string search in the buffer. However, I could not
verify that these tools would actually respect this assumption.

Additionally to this, e.g. the Twin project [1] has other non-whitespace
characters (some preprocessor directives) before the flex-generated
header comment. So the heuristic in the previous paragraph won't work
with that.
Thus, I would advocate the current implementation.

According to my measurement, this patch won't introduce measurable
performance degradation, even though we will do 2 linear scans.

I introduce the ignore-bison-generated-files and
ignore-flex-generated-files to disable skipping these files.
Both of these options are true by default.

[1]: https://github.com/cosmos72/twin/blob/master/server/rcparse_lex.cpp#L7
[2]: https://github.com/marcauresoar/make-examples/blob/22362cdcf9dd7c597b5049ce7f176621e2e9ac7a/sandbox/count-words/lexer.c#L6
[3]: https://github.com/vladcmanea/2nd-faculty-year-Formal-Languages---Automata-assignments/blob/11abdf64629d9eb741438ba69f04636769d5a374/lab1/lex.yy.c#L6

[4]: https://github.com/KritikaChoudhary/System-Software-Lab/blob/47f5b2cfe2a2738fd54eae9f8439817f6a22034e/B_yacc/1/y1.tab.h#L2
[5]: https://github.com/VirtualMonitor/VirtualMonitor/blob/71d1bf9b1e7b392a7bd0c73dc217138dc5865651/src/VBox/Additions/x11/x11include/xorg-server-1.8.0/parser.h#L2
[6]: https://github.com/bspaulding/DrawTest/blob/3f773ceb13de14275429036b9cbc5aa19e29bab9/Framework/OpenEars.framework/Versions/A/Headers/jsgf_parser.h#L2

Reviewed By: xazax.hun

Differential Revision: https://reviews.llvm.org/D114510

show more ...

[analyzer][NFC] Refactor AnalysisConsumer::getModeForDecl()

I just read this part of the code, and I found the nested ifs less
readable.

Reviewed By: martong

Differential Revision: https://reviews

[analyzer][NFC] Refactor AnalysisConsumer::getModeForDecl()

I just read this part of the code, and I found the nested ifs less
readable.

Reviewed By: martong

Differential Revision: https://reviews.llvm.org/D114441

show more ...

[analyzer] Print time taken to analyze each function
Summary: This patch is a part of an attempt to obtain more
timer data from the analyzer. In this patch, we try to use
LLVM::TimeRecord to save tim

[analyzer] Print time taken to analyze each function
Summary: This patch is a part of an attempt to obtain more
timer data from the analyzer. In this patch, we try to use
LLVM::TimeRecord to save time before starting the analysis
and to print the time that a specific function takes while
getting analyzed.

The timer data is printed along with the
-analyzer-display-progress outputs.

ANALYZE (Syntax): test.c functionName : 0.4 ms
ANALYZE (Path, Inline_Regular): test.c functionName : 2.6 ms
Authored By: RithikSharma
Reviewer: NoQ, xazax.hun, teemperor, vsavchenko
Reviewed By: NoQ
Differential Revision: https://reviews.llvm.org/D105565

show more ...

[analyzer][NFC] Display the correct function name even in crash dumps

The `-analyzer-display-progress` displayed the function name of the
currently analyzed function. It differs in C and C++. In C++

[analyzer][NFC] Display the correct function name even in crash dumps

The `-analyzer-display-progress` displayed the function name of the
currently analyzed function. It differs in C and C++. In C++, it
prints the argument types as well in a comma-separated list.
While in C, only the function name is displayed, without the brackets.
E.g.:

C++: foo(), foo(int, float)
C: foo

In crash traces, the analyzer dumps the location contexts, but the
string is not enough for `-analyze-function` in C++ mode.
This patch addresses the issue by dumping the proper function names
even in stack traces.

Reviewed By: NoQ

Differential Revision: https://reviews.llvm.org/D105708

show more ...

[analyzer] Create MacroExpansionContext member in AnalysisConsumer

Adds a `MacroExpansionContext` member to the `AnalysisConsumer` class.
Tracks macro expansions only if the `ShouldDisplayMacroExpan

[analyzer] Create MacroExpansionContext member in AnalysisConsumer

Adds a `MacroExpansionContext` member to the `AnalysisConsumer` class.
Tracks macro expansions only if the `ShouldDisplayMacroExpansions` is set.
Passes a reference down the pipeline letting AnalysisConsumers query macro
expansions during bugreport construction.

Reviewed By: martong, Szelethus

Differential Revision: https://reviews.llvm.org/D93223

show more ...

Revert "Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""""

This reverts commit 6b0ee02747ed22d41e175d15f27025183341e6f8.

Circular dep

Revert "Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""""

This reverts commit 6b0ee02747ed22d41e175d15f27025183341e6f8.

Circular dependency again.

show more ...

Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis.""""

This reverts commit b12f26733a4259c90e5f387aceb9f23c35e003b8.

Fix dead include that

Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis.""""

This reverts commit b12f26733a4259c90e5f387aceb9f23c35e003b8.

Fix dead include that looked like another missed circular dependency.

show more ...