[analyzer][NFC] Cleanup BugType lazy-init patterns (#76655)

Cleanup most of the lazy-init `BugType` legacy.
Some will be preserved, as those are slightly more complicated to
refactor.

Notice, t

[analyzer][NFC] Cleanup BugType lazy-init patterns (#76655)

Cleanup most of the lazy-init `BugType` legacy.
Some will be preserved, as those are slightly more complicated to
refactor.

Notice, that the default category for `BugType` is `LogicError`. I
omitted setting this explicitly where I could.

Please, actually have a look at the diff. I did this manually, and we
rarely check the bug type descriptions and stuff in tests, so the
testing might be shallow on this one.

show more ...

[analyzer] Clean checker options from bool to DefaultBool (NFC)

A recent review emphasized the preference to use DefaultBool instead of
bool for checker options. This change is a NFC and cleans up s

[analyzer] Clean checker options from bool to DefaultBool (NFC)

A recent review emphasized the preference to use DefaultBool instead of
bool for checker options. This change is a NFC and cleans up some of the
instances where bool was used, and could be changed to DefaultBool.

Reviewed By: steakhal

Differential Revision: https://reviews.llvm.org/D123464

show more ...

[analyzer][NFC] Change LangOptions to CheckerManager in the shouldRegister* functions

Some checkers may not only depend on language options but also analyzer options.
To make this possible this patc

[analyzer][NFC] Change LangOptions to CheckerManager in the shouldRegister* functions

Some checkers may not only depend on language options but also analyzer options.
To make this possible this patch changes the parameter of the shouldRegister*
function to CheckerManager to be able to query the analyzer options when
deciding whether the checker should be registered.

Differential Revision: https://reviews.llvm.org/D75271

show more ...

[analyzer] NFC: Introduce sub-classes for path-sensitive and basic reports.

Checkers are now required to specify whether they're creating a
path-sensitive report or a path-insensitive report by cons

[analyzer] NFC: Introduce sub-classes for path-sensitive and basic reports.

Checkers are now required to specify whether they're creating a
path-sensitive report or a path-insensitive report by constructing an
object of the respective type.

This makes BugReporter more independent from the rest of the Static Analyzer
because all Analyzer-specific code is now in sub-classes.

Differential Revision: https://reviews.llvm.org/D66572

llvm-svn: 371450

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

[analyzer] Remove the default value arg from getChecker*Option

Since D57922, the config table contains every checker option, and it's default
value, so having it as an argument for getChecker*Option

[analyzer] Remove the default value arg from getChecker*Option

Since D57922, the config table contains every checker option, and it's default
value, so having it as an argument for getChecker*Option is redundant.

By the time any of the getChecker*Option function is called, we verified the
value in CheckerRegistry (after D57860), so we can confidently assert here, as
any irregularities detected at this point must be a programmer error. However,
in compatibility mode, verification won't happen, so the default value must be
restored.

This implies something else, other than adding removing one more potential point
of failure -- debug.ConfigDumper will always contain valid values for
checker/package options!

Differential Revision: https://reviews.llvm.org/D59195

llvm-svn: 361042

show more ...

[analyzer] Emit an error rather than assert on invalid checker option input

Asserting on invalid input isn't very nice, hence the patch to emit an error
instead.

This is the first of many patches t

[analyzer] Emit an error rather than assert on invalid checker option input

Asserting on invalid input isn't very nice, hence the patch to emit an error
instead.

This is the first of many patches to overhaul the way we handle checker options.

Differential Revision: https://reviews.llvm.org/D57850

llvm-svn: 355704

show more ...

[analyzer] Enable subcheckers to possess checker options

Under the term "subchecker", I mean checkers that do not have a checker class on
their own, like unix.MallocChecker to unix.DynamicMemoryMode

[analyzer] Enable subcheckers to possess checker options

Under the term "subchecker", I mean checkers that do not have a checker class on
their own, like unix.MallocChecker to unix.DynamicMemoryModeling.

Since a checker object was required in order to retrieve checker options,
subcheckers couldn't possess options on their own.

This patch is also an excuse to change the argument order of getChecker*Option,
it always bothered me, now it resembles the actual command line argument
(checkername:option=value).

Differential Revision: https://reviews.llvm.org/D57579

llvm-svn: 355297

show more ...

[analyzer] Supply all checkers with a shouldRegister function

Introduce the boolean ento::shouldRegister##CHECKERNAME(const LangOptions &LO)
function very similarly to ento::register##CHECKERNAME. T

[analyzer] Supply all checkers with a shouldRegister function

Introduce the boolean ento::shouldRegister##CHECKERNAME(const LangOptions &LO)
function very similarly to ento::register##CHECKERNAME. This will force every
checker to implement this function, but maybe it isn't that bad: I saw a lot of
ObjC or C++ specific checkers that should probably not register themselves based
on some LangOptions (mine too), but they do anyways.

A big benefit of this is that all registry functions now register their checker,
once it is called, registration is guaranteed.

This patch is a part of a greater effort to reinvent checker registration, more
info here: D54438#1315953

Differential Revision: https://reviews.llvm.org/D55424

llvm-svn: 352277

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[analyzer][NFC] Move CheckerRegistry from the Core directory to Frontend

ClangCheckerRegistry is a very non-obvious, poorly documented, weird concept.
It derives from CheckerRegistry, and is placed

[analyzer][NFC] Move CheckerRegistry from the Core directory to Frontend

ClangCheckerRegistry is a very non-obvious, poorly documented, weird concept.
It derives from CheckerRegistry, and is placed in lib/StaticAnalyzer/Frontend,
whereas it's base is located in lib/StaticAnalyzer/Core. It was, from what I can
imagine, used to circumvent the problem that the registry functions of the
checkers are located in the clangStaticAnalyzerCheckers library, but that
library depends on clangStaticAnalyzerCore. However, clangStaticAnalyzerFrontend
depends on both of those libraries.

One can make the observation however, that CheckerRegistry has no place in Core,
it isn't used there at all! The only place where it is used is Frontend, which
is where it ultimately belongs.

This move implies that since
include/clang/StaticAnalyzer/Checkers/ClangCheckers.h only contained a single function:

class CheckerRegistry;

void registerBuiltinCheckers(CheckerRegistry &registry);

it had to re purposed, as CheckerRegistry is no longer available to
clangStaticAnalyzerCheckers. It was renamed to BuiltinCheckerRegistration.h,
which actually describes it a lot better -- it does not contain the registration
functions for checkers, but only those generated by the tblgen files.

Differential Revision: https://reviews.llvm.org/D54436

llvm-svn: 349275

show more ...

Misc typos fixes in ./lib folder

Summary: Found via `codespell -q 3 -I ../clang-whitelist.txt -L uint,importd,crasher,gonna,cant,ue,ons,orign,ned`

Reviewers: teemperor

Reviewed By: teemperor

Subs

Misc typos fixes in ./lib folder

Summary: Found via `codespell -q 3 -I ../clang-whitelist.txt -L uint,importd,crasher,gonna,cant,ue,ons,orign,ned`

Reviewers: teemperor

Reviewed By: teemperor

Subscribers: teemperor, jholewinski, jvesely, nhaehnle, whisperity, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D55475

llvm-svn: 348755

show more ...

[analyzer] Restrict AnalyzerOptions' interface so that non-checker objects have to be registered

One of the reasons why AnalyzerOptions is so chaotic is that options can be
retrieved from the comman

[analyzer] Restrict AnalyzerOptions' interface so that non-checker objects have to be registered

One of the reasons why AnalyzerOptions is so chaotic is that options can be
retrieved from the command line whenever and wherever. This allowed for some
options to be forgotten for a looooooong time. Have you ever heard of
"region-store-small-struct-limit"? In order to prevent this in the future, I'm
proposing to restrict AnalyzerOptions' interface so that only checker options
can be retrieved without special getters. I would like to make every option be
accessible only through a getter, but checkers from plugins are a thing, so I'll
have to figure something out for that.

This also forces developers who'd like to add a new option to register it
properly in the .def file.

This is done by

* making the third checker pointer parameter non-optional, and checked by an
assert to be non-null.
* I added new, but private non-checkers option initializers, meant only for
internal use,
* Renamed these methods accordingly (mind the consistent name for once with
getBooleanOption!):
- getOptionAsString -> getCheckerStringOption,
- getOptionAsInteger -> getCheckerIntegerOption
* The 3 functions meant for initializing data members (with the not very
descriptive getBooleanOption, getOptionAsString and getOptionAsUInt names)
were renamed to be overloads of the getAndInitOption function name.
* All options were in some way retrieved via getCheckerOption. I removed it, and
moved the logic to getStringOption and getCheckerStringOption. This did cause
some code duplication, but that's the only way I could do it, now that checker
and non-checker options are separated. Note that the non-checker version
inserts the new option to the ConfigTable with the default value, but the
checker version only attempts to find already existing entries. This is how
it always worked, but this is clunky and I might end reworking that too, so we
can eventually get a ConfigTable that contains the entire configuration of the
analyzer.

Differential Revision: https://reviews.llvm.org/D53483

llvm-svn: 346113

show more ...

[analyzer] Increase minimum complexity filter of the CloneChecker.

Summary:
So far we used a value of 10 which was useful for testing but produces many false-positives in real programs. The usual su

[analyzer] Increase minimum complexity filter of the CloneChecker.

Summary:
So far we used a value of 10 which was useful for testing but produces many false-positives in real programs. The usual suspicious clones we find seem to be at around a complexity value of 70 and for normal clone-reporting everything above 50 seems to be a valid normal clone for users, so let's just go with 50 for now and set this as the new default value.

This patch also explicitly sets the complexity value for the regression tests as they serve more of a regression testing/debugging purpose and shouldn't really be reported by default in real programs. I'll add more tests that reflect actual found bugs that then need to pass with the default setting in the future.

Reviewers: NoQ

Subscribers: cfe-commits, javed.absar, xazax.hun, v.g.vassilev

Differential Revision: https://reviews.llvm.org/D34178

llvm-svn: 312468

show more ...

[analyzer] Performance optimizations for the CloneChecker

Summary:
This patch aims at optimizing the CloneChecker for larger programs. Before this
patch we took around 102 seconds to analyze sqlite

[analyzer] Performance optimizations for the CloneChecker

Summary:
This patch aims at optimizing the CloneChecker for larger programs. Before this
patch we took around 102 seconds to analyze sqlite3 with a complexity value of
50. After this patch we now take 2.1 seconds to analyze sqlite3.

The biggest performance optimization is that we now put the constraint for group
size before the constraint for the complexity. The group size constraint is much
faster in comparison to the complexity constraint as it only does a simple
integer comparison. The complexity constraint on the other hand actually
traverses each Stmt and even checks the macro stack, so it is obviously not able
to handle larger amounts of incoming clones. The new order filters out all the
single-clone groups that the type II constraint generates in a faster way before
passing the fewer remaining clones to the complexity constraint. This reduced
runtime by around 95%.

The other change is that we also delay the verification part of the type II
clones back in the chain of constraints. This required to split up the
constraint into two parts - a verification and a hash constraint (which is also
making it more similar to the original design of the clone detection algorithm).
The reasoning for this is the same as before: The verification constraint has to
traverse many statements and shouldn't be at the start of the constraint chain.
However, as the type II hashing has to be the first step in our algorithm, we
have no other choice but split this constrain into two different ones. Now our
group size and complexity constrains filter out a chunk of the clones before
they reach the slow verification step, which reduces the runtime by around 8%.

I also kept the full type II constraint around - that now just calls it's two
sub-constraints - in case someone doesn't care about the performance benefits
of doing this.

Reviewers: NoQ

Reviewed By: NoQ

Subscribers: klimek, v.g.vassilev, xazax.hun, cfe-commits

Differential Revision: https://reviews.llvm.org/D34182

llvm-svn: 312222

show more ...

[analyzer] Teach CloneDetection about Qt Meta-Object Compiler to filter auto generated files

Reviewers: v.g.vassilev, teemperor

Reviewed By: teemperor

Differential Revision: https://reviews.llvm.o

[analyzer] Teach CloneDetection about Qt Meta-Object Compiler to filter auto generated files

Reviewers: v.g.vassilev, teemperor

Reviewed By: teemperor

Differential Revision: https://reviews.llvm.org/D34353

llvm-svn: 305774

show more ...

[analyzer] Teach CloneDetection about Qt Meta-Object Compiler

Reviewers: v.g.vassilev, zaks.anna, NoQ, teemperor

Reviewed By: v.g.vassilev, zaks.anna, NoQ, teemperor

Differential Revision: https:/

[analyzer] Teach CloneDetection about Qt Meta-Object Compiler

Reviewers: v.g.vassilev, zaks.anna, NoQ, teemperor

Reviewed By: v.g.vassilev, zaks.anna, NoQ, teemperor

Differential Revision: https://reviews.llvm.org/D31320

llvm-svn: 305659

show more ...

[analyzer] Reland r299544 "Add a modular constraint system to the CloneDetector"

Hopefully fix crashes by unshadowing the variable.


Original commit message:

A big part of the clone detection code

[analyzer] Reland r299544 "Add a modular constraint system to the CloneDetector"

Hopefully fix crashes by unshadowing the variable.


Original commit message:

A big part of the clone detection code is functionality for filtering clones and
clone groups based on different criteria. So far this filtering process was
hardcoded into the CloneDetector class, which made it hard to understand and,
ultimately, to extend.

This patch splits the CloneDetector's logic into a sequence of reusable
constraints that are used for filtering clone groups. These constraints
can be turned on and off and reodreder at will, and new constraints are easy
to implement if necessary.

Unit tests are added for the new constraint interface.

This is a refactoring patch - no functional change intended.

Patch by Raphael Isemann!

Differential Revision: https://reviews.llvm.org/D23418

llvm-svn: 299653

show more ...

Revert "[analyzer] Add a modular constraint system to the CloneDetector"

This reverts commit r299544.

Crashes on tests on some buildbots.

llvm-svn: 299550

[analyzer] Add a modular constraint system to the CloneDetector

A big part of the clone detection code is functionality for filtering clones and
clone groups based on different criteria. So far this

[analyzer] Add a modular constraint system to the CloneDetector

A big part of the clone detection code is functionality for filtering clones and
clone groups based on different criteria. So far this filtering process was
hardcoded into the CloneDetector class, which made it hard to understand and,
ultimately, to extend.

This patch splits the CloneDetector's logic into a sequence of reusable
constraints that are used for filtering clone groups. These constraints
can be turned on and off and reodreder at will, and new constraints are easy
to implement if necessary.

Unit tests are added for the new constraint interface.

This is a refactoring patch - no functional change intended.

Patch by Raphael Isemann!

Differential Revision: https://reviews.llvm.org/D23418

llvm-svn: 299544

show more ...

[analyzer] Re-apply r283094 "Improve CloneChecker diagnostics"

The parent commit (r283092) was reverted before and now finally landed.

llvm-svn: 283661

Revert "[analyzer] Improve CloneChecker diagnostics" as its depends on reverted r283092

This reverts commit r283094.

llvm-svn: 283182

[analyzer] Improve CloneChecker diagnostics

Highlight code clones referenced by the warning message with the help of
the extra notes feature recently introduced in r283092.

Change warning text to m

[analyzer] Improve CloneChecker diagnostics

Highlight code clones referenced by the warning message with the help of
the extra notes feature recently introduced in r283092.

Change warning text to more clang-ish. Remove suggestions from the copy-paste
error checker diagnostics, because currently our suggestions are strictly 50%
wrong (we do not know which of the two code clones contains the error), and
for that reason we should not sound as if we're actually suggesting this.
Hopefully a better solution would bring them back.

Make sure the suspicious clone pair structure always mentions
the correct variable for the second clone.

Differential Revision: https://reviews.llvm.org/D24916

llvm-svn: 283094

show more ...

[analyzer] Use faster hashing (MD5) in CloneDetector.

This replaces the old approach of fingerprinting every AST node into a string,
which avoided collisions and was simple to implement, but turned

[analyzer] Use faster hashing (MD5) in CloneDetector.

This replaces the old approach of fingerprinting every AST node into a string,
which avoided collisions and was simple to implement, but turned out to be
extremely ineffective with respect to both performance and memory.

The collisions are now dealt with in a separate pass, which no longer causes
performance problems because collisions are rare.

Patch by Raphael Isemann!

Differential Revision: https://reviews.llvm.org/D22515

llvm-svn: 279378

show more ...

[analyzer] Teach CloneDetector to find clones that look like copy-paste errors.

The original clone checker tries to find copy-pasted code that is exactly
identical to the original code, up to minor

[analyzer] Teach CloneDetector to find clones that look like copy-paste errors.

The original clone checker tries to find copy-pasted code that is exactly
identical to the original code, up to minor details.

As an example, if the copy-pasted code has all references to variable 'a'
replaced with references to variable 'b', it is still considered to be
an exact clone.

The new check finds copy-pasted code in which exactly one variable seems
out of place compared to the original code, which likely indicates
a copy-paste error (a variable was forgotten to be renamed in one place).

Patch by Raphael Isemann!

Differential Revision: https://reviews.llvm.org/D23314

llvm-svn: 279056

show more ...