Reland "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Originally commited in rG57b8a407493c34c3680e7e1e4cb82e097f43744a, bu

Reland "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Originally commited in rG57b8a407493c34c3680e7e1e4cb82e097f43744a, but
it broke the modules bot. This is solved by putting the contructors of
the CheckerManager class to the Frontend library.

Differential Revision: https://reviews.llvm.org/D75360

show more ...

Revert "[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes"

Temporarily reverting this patch because it breaks the modules build.

[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes

Its been a while since my CheckerRegistry related patches landed, allow me to

[analyzer][NFC] Tie CheckerRegistry to CheckerManager, allow CheckerManager to be constructed for non-analysis purposes

Its been a while since my CheckerRegistry related patches landed, allow me to
refresh your memory:

During compilation, TblGen turns
clang/include/clang/StaticAnalyzer/Checkers/Checkers.td into
(build directory)/tools/clang/include/clang/StaticAnalyzer/Checkers/Checkers.inc.
This is a file that contains the full name of the checkers, their options, etc.

The class that is responsible for parsing this file is CheckerRegistry. The job
of this class is to establish what checkers are available for the analyzer (even
from plugins and statically linked but non-tblgen generated files!), and
calculate which ones should be turned on according to the analyzer's invocation.

CheckerManager is the class that is responsible for the construction and storage
of checkers. This process works by first creating a CheckerRegistry object, and
passing itself to CheckerRegistry::initializeManager(CheckerManager&), which
will call the checker registry functions (for example registerMallocChecker) on
it.

The big problem here is that these two classes lie in two different libraries,
so their interaction is pretty awkward. This used to be far worse, but I
refactored much of it, which made things better but nowhere near perfect.

---

This patch changes how the above mentioned two classes interact. CheckerRegistry
is mainly used by CheckerManager, and they are so intertwined, it makes a lot of
sense to turn in into a field, instead of a one-time local variable. This has
additional benefits: much of the information that CheckerRegistry conveniently
holds is no longer thrown away right after the analyzer's initialization, and
opens the possibility to pass CheckerManager in the shouldRegister* function
rather then LangOptions (D75271).

There are a few problems with this. CheckerManager isn't the only user, when we
honor help flags like -analyzer-checker-help, we only have access to a
CompilerInstance class, that is before the point of parsing the AST.
CheckerManager makes little sense without ASTContext, so I made some changes and
added new constructors to make it constructible for the use of help flags.

Differential Revision: https://reviews.llvm.org/D75360

show more ...

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[analyzer] Don't run the analyzer for -analyzer-list-enabled-checkers

Short and sweet. Whenever I use -analyzer-list-enabled-checkers, I'm only
interested about the configuration, not about the anal

[analyzer] Don't run the analyzer for -analyzer-list-enabled-checkers

Short and sweet. Whenever I use -analyzer-list-enabled-checkers, I'm only
interested about the configuration, not about the analysis.

Differential Revision: https://reviews.llvm.org/D66714

llvm-svn: 371756

show more ...

[Driver] Use shared singleton instance of DriverOptTable

Summary:
This significantly reduces the time required to run clangd tests, by
~10%.

Should also have an effect on other tests that run comma

[Driver] Use shared singleton instance of DriverOptTable

Summary:
This significantly reduces the time required to run clangd tests, by
~10%.

Should also have an effect on other tests that run command-line parsing
multiple times inside a single invocation.

Reviewers: gribozavr, sammccall

Reviewed By: sammccall

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67163

llvm-svn: 370908

show more ...

[clang][ifs] Dropping older experimental interface stub formats.

I've been working on a new tool, llvm-ifs, for merging interface stub files
generated by clang and I've iterated on my derivative for

[clang][ifs] Dropping older experimental interface stub formats.

I've been working on a new tool, llvm-ifs, for merging interface stub files
generated by clang and I've iterated on my derivative format of TBE to a newer
format. llvm-ifs will only support the new format, so I am going to drop the
older experimental interface stubs formats in this commit to make things
simpler.

Differential Revision: https://reviews.llvm.org/D66573

llvm-svn: 369719

show more ...

[clang][ifs] New interface stubs format (llvm triple based).

After posting llvm-ifs on phabricator, I made some progress in hardening up how
I think the format for Interface Stubs should look. There

[clang][ifs] New interface stubs format (llvm triple based).

After posting llvm-ifs on phabricator, I made some progress in hardening up how
I think the format for Interface Stubs should look. There are a number of
things I think the TBE format was missing (no endianness, no info about the
Object Format because it assumes ELF), so I have added those and broken off
from being as similar to the TBE schema. In a subsequent commit I can drop the
other formats.

An example of how The format will look is as follows:

--- !experimental-ifs-v1
IfsVersion: 1.0
Triple: x86_64-unknown-linux-gnu
ObjectFileFormat: ELF
Symbols:
_Z9nothiddenv: { Type: Func }
_Z10cmdVisiblev: { Type: Func }
...

The format is still marked experimental.

Differential Revision: https://reviews.llvm.org/D66446

llvm-svn: 369715

show more ...

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
o

[Clang] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942

show more ...

[clang-ifs] Clang Interface Stubs, first version (second landing attempt).

This change reverts r363649; effectively re-landing r363626. At this point
clang::Index::CodegenNameGeneratorImpl has been

[clang-ifs] Clang Interface Stubs, first version (second landing attempt).

This change reverts r363649; effectively re-landing r363626. At this point
clang::Index::CodegenNameGeneratorImpl has been refactored into
clang::AST::ASTNameGenerator. This makes it so that the previous circular link
dependency no longer exists, fixing the previous share lib
(-DBUILD_SHARED_LIBS=ON) build issue which was the reason for r363649.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contain mangled
name info that can be used to produce a stub library. These stub libraries can
be useful for breaking up build dependencies and controlling access to a
library's internal symbols. Generation of these stubs can be invoked by:

clang -fvisibility=<visibility> -emit-interface-stubs \
-interface-stub-version=<interface format>

Notice that -fvisibility (along with use of visibility attributes) can be used
to control what symbols get generated. Currently the interface format is
experimental but there are a wide range of possibilities here.

Currently clang-ifs produces .ifs files that can be thought of as analogous to
object (.o) files, but just for the mangled symbol info. In a subsequent patch
I intend to add support for merging the .ifs files into one .ifs/.ifso file
that can be the input to something like llvm-elfabi to produce something like a
.so file or .dll (but without any of the code, just symbols).

Differential Revision: https://reviews.llvm.org/D60974

llvm-svn: 363948

show more ...

Revert D60974 "[clang-ifs] Clang Interface Stubs, first version."

This reverts commit rC363626.

clangIndex depends on clangFrontend. r363626 adds a dependency from
clangFrontend to clangIndex, whic

Revert D60974 "[clang-ifs] Clang Interface Stubs, first version."

This reverts commit rC363626.

clangIndex depends on clangFrontend. r363626 adds a dependency from
clangFrontend to clangIndex, which creates a circular dependency.

This is disallowed by -DBUILD_SHARED_LIBS=on builds:

CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
"clangFrontend" of type SHARED_LIBRARY
depends on "clangIndex" (weak)
"clangIndex" of type SHARED_LIBRARY
depends on "clangFrontend" (weak)
At least one of these targets is not a STATIC_LIBRARY. Cyclic dependencies are allowed only among static libraries.

Note, the dependency on clangIndex cannot be removed because
libclangFrontend.so is linked with -Wl,-z,defs: a shared object must
have its full direct dependencies specified on the linker command line.

In -DBUILD_SHARED_LIBS=off builds, this appears to work when linking
`bin/clang-9`. However, it can cause trouble to downstream clang library
users. The llvm build system links libraries this way:

clang main_program_object_file ... lib/libclangIndex.a ... lib/libclangFrontend.a -o exe

libclangIndex.a etc are not wrapped in --start-group.

If the downstream application depends on libclangFrontend.a but not any
other clang libraries that depend on libclangIndex.a, this can cause undefined
reference errors when the linker is ld.bfd or gold.

The proper fix is to not include clangIndex files in clangFrontend.

llvm-svn: 363649

show more ...

[clang-ifs] Clang Interface Stubs, first version.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contai

[clang-ifs] Clang Interface Stubs, first version.

Clang interface stubs (previously referred to as clang-ifsos) is a new frontend
action in clang that allows the generation of stub files that contain mangled
name info that can be used to produce a stub library. These stub libraries can
be useful for breaking up build dependencies and controlling access to a
library's internal symbols. Generation of these stubs can be invoked by:

clang -fvisibility=<visibility> -emit-interface-stubs \
-interface-stub-version=<interface format>

Notice that -fvisibility (along with use of visibility attributes) can be used
to control what symbols get generated. Currently the interface format is
experimental but there are a wide range of possibilities here.

Differential Revision: https://reviews.llvm.org/D60974

llvm-svn: 363626

show more ...

Add clang source minimizer that reduces source to directives
that might affect the dependency list for a compilation

This commit introduces a dependency directives source minimizer to clang
that min

Add clang source minimizer that reduces source to directives
that might affect the dependency list for a compilation

This commit introduces a dependency directives source minimizer to clang
that minimizes header and source files to the minimum necessary preprocessor
directives for evaluating includes. It reduces the source down to #define, #include,

The source minimizer works by lexing the input with a custom fast lexer that recognizes
the preprocessor directives it cares about, and emitting those directives in the minimized source.
It ignores source code, comments, and normalizes whitespace. It gives up and fails if seems
any directives that it doesn't recognize as valid (e.g. #define 0).

In addition to the source minimizer this patch adds a
-print-dependency-directives-minimized-source CC1 option that allows you to invoke the minimizer
from clang directly.

Differential Revision: https://reviews.llvm.org/D55463

llvm-svn: 362459

show more ...

[analyzer] List checker/plugin options in 3 categories: released, alpha, developer

Same patch as D62093, but for checker/plugin options, the only
difference being that options for alpha checkers are

[analyzer] List checker/plugin options in 3 categories: released, alpha, developer

Same patch as D62093, but for checker/plugin options, the only
difference being that options for alpha checkers are implicitly marked
as alpha.

Differential Revision: https://reviews.llvm.org/D62093

llvm-svn: 361566

show more ...

[analyzer] Hide developer-only checker/package options by default

These options are now only visible under
-analyzer-checker-option-help-developer.

Differential Revision: https://reviews.llvm.org/D

[analyzer] Hide developer-only checker/package options by default

These options are now only visible under
-analyzer-checker-option-help-developer.

Differential Revision: https://reviews.llvm.org/D61839

llvm-svn: 361561

show more ...

[analyzer] List checkers in 3 categories: released, alpha, developer

Previously, the only way to display the list of available checkers was
to invoke the analyzer with -analyzer-checker-help fronten

[analyzer] List checkers in 3 categories: released, alpha, developer

Previously, the only way to display the list of available checkers was
to invoke the analyzer with -analyzer-checker-help frontend flag. This
however wasn't really great from a maintainer standpoint: users came
across checkers meant strictly for development purposes that weren't to
be tinkered with, or those that were still in development. This patch
creates a clearer division in between these categories.

From now on, we'll have 3 flags to display the list checkers. These
lists are mutually exclusive and can be used in any combination (for
example to display both stable and alpha checkers).

-analyzer-checker-help: Displays the list for stable, production ready
checkers.

-analyzer-checker-help-alpha: Displays the list for in development
checkers. Enabling is discouraged
for non-development purposes.

-analyzer-checker-help-developer: Modeling and debug checkers. Modeling
checkers shouldn't be enabled/disabled
by hand, and debug checkers shouldn't
be touched by users.

Differential Revision: https://reviews.llvm.org/D62093

llvm-svn: 361558

show more ...

[analyzer] Add a new frontend flag to display all checker options

Add the new frontend flag -analyzer-checker-option-help to display all
checker/package options.

Differential Revision: https://revi

[analyzer] Add a new frontend flag to display all checker options

Add the new frontend flag -analyzer-checker-option-help to display all
checker/package options.

Differential Revision: https://reviews.llvm.org/D57858

llvm-svn: 361552

show more ...

[analyzer] Don't display implementation checkers under -analyzer-checker-help, but do under the new flag -analyzer-checker-help-hidden

During my work on analyzer dependencies, I created a great amou

[analyzer] Don't display implementation checkers under -analyzer-checker-help, but do under the new flag -analyzer-checker-help-hidden

During my work on analyzer dependencies, I created a great amount of new
checkers that emitted no diagnostics at all, and were purely modeling some
function or another.

However, the user shouldn't really disable/enable these by hand, hence this
patch, which hides these by default. I intentionally chose not to hide alpha
checkers, because they have a scary enough name, in my opinion, to cause no
surprise when they emit false positives or cause crashes.

The patch introduces the Hidden bit into the TableGen files (you may remember
it before I removed it in D53995), and checkers that are either marked as
hidden, or are in a package that is marked hidden won't be displayed under
-analyzer-checker-help. -analyzer-checker-help-hidden, a new flag meant for
developers only, displays the full list.

Differential Revision: https://reviews.llvm.org/D60925

llvm-svn: 359720

show more ...

[analyzer][NFC] Supply CheckerRegistry with AnalyzerOptions

Since pretty much all methods of CheckerRegistry has AnalyzerOptions as an
argument, it makes sense to just simply require it in it's cons

[analyzer][NFC] Supply CheckerRegistry with AnalyzerOptions

Since pretty much all methods of CheckerRegistry has AnalyzerOptions as an
argument, it makes sense to just simply require it in it's constructor.

Differential Revision: https://reviews.llvm.org/D56988

llvm-svn: 352279

show more ...

[analyzer] Supply all checkers with a shouldRegister function

Introduce the boolean ento::shouldRegister##CHECKERNAME(const LangOptions &LO)
function very similarly to ento::register##CHECKERNAME. T

[analyzer] Supply all checkers with a shouldRegister function

Introduce the boolean ento::shouldRegister##CHECKERNAME(const LangOptions &LO)
function very similarly to ento::register##CHECKERNAME. This will force every
checker to implement this function, but maybe it isn't that bad: I saw a lot of
ObjC or C++ specific checkers that should probably not register themselves based
on some LangOptions (mine too), but they do anyways.

A big benefit of this is that all registry functions now register their checker,
once it is called, registration is guaranteed.

This patch is a part of a greater effort to reinvent checker registration, more
info here: D54438#1315953

Differential Revision: https://reviews.llvm.org/D55424

llvm-svn: 352277

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[analyzer][NFC] Merge ClangCheckerRegistry to CheckerRegistry

Now that CheckerRegistry lies in Frontend, we can finally eliminate
ClangCheckerRegistry. Fortunately, this also provides us with a
Diag

[analyzer][NFC] Merge ClangCheckerRegistry to CheckerRegistry

Now that CheckerRegistry lies in Frontend, we can finally eliminate
ClangCheckerRegistry. Fortunately, this also provides us with a
DiagnosticsEngine, so I went ahead and removed some parameters from it's
methods.

Differential Revision: https://reviews.llvm.org/D54437

llvm-svn: 349280

show more ...

[analyzer] Prefer returns values to out-params in CheckerRegistry.cpp

Renaming collectCheckers to getEnabledCheckers
Changing the functionality to acquire all enabled checkers, rather then collect
c

[analyzer] Prefer returns values to out-params in CheckerRegistry.cpp

Renaming collectCheckers to getEnabledCheckers
Changing the functionality to acquire all enabled checkers, rather then collect
checkers for a specific CheckerOptInfo (for example, collecting all checkers for
{ "core", true }, which meant enabling all checkers from the core package, which
was an unnecessary complication).
Removing CheckerOptInfo, instead of storing whether the option was claimed via a
field, we handle errors immediately, as getEnabledCheckers can now access a
DiagnosticsEngine. Realize that the remaining information it stored is directly
accessible through AnalyzerOptions.CheckerControlList.
Fix a test with -analyzer-disable-checker -verify accidentally left in.

llvm-svn: 349274

show more ...

PTH-- Remove feature entirely-

When debugging a boost build with a modified
version of Clang, I discovered that the PTH implementation
stores TokenKind in 8 bits. However, we currently have 368
Toke

PTH-- Remove feature entirely-

When debugging a boost build with a modified
version of Clang, I discovered that the PTH implementation
stores TokenKind in 8 bits. However, we currently have 368
TokenKinds.

The result is that the value gets truncated and the wrong token
gets picked up when including PTH files. It seems that this will
go wrong every time someone uses a token that uses the 9th bit.

Upon asking on IRC, it was brought up that this was a highly
experimental features that was considered a failure. I discovered
via googling that BoostBuild (mostly Boost.Math) is the only user of
this
feature, using the CC1 flag directly. I believe that this can be
transferred over to normal PCH with minimal effort:
https://github.com/boostorg/build/issues/367

Based on advice on IRC and research showing that this is a nearly
completely unused feature, this patch removes it entirely.

Note: I considered leaving the build-flags in place and making them
emit an error/warning, however since I've basically identified and
warned the only user, it seemed better to just remove them.

Differential Revision: https://reviews.llvm.org/D54547

Change-Id: If32744275ef1f585357bd6c1c813d96973c4d8d9
llvm-svn: 348266

show more ...

Sink BuryPointer from Clang into LLVM for reuse there

llvm-svn: 347141