Refactor StaticAnalyzer to use `clang::SarifDocumentWriter`

Refactor StaticAnalyzer to use clang::SarifDocumentWriter for
serializing sarif diagnostics.

Uses clang::SarifDocumentWriter to generate

Refactor StaticAnalyzer to use `clang::SarifDocumentWriter`

Refactor StaticAnalyzer to use clang::SarifDocumentWriter for
serializing sarif diagnostics.

Uses clang::SarifDocumentWriter to generate SARIF output in the
StaticAnalyzer.

Various bugfixes are also made to clang::SarifDocumentWriter.

Summary of changes:

clang/lib/Basic/Sarif.cpp:
* Fix bug in adjustColumnPos introduced from prev move, it now uses
FullSourceLoc::getDecomposedExpansionLoc which provides the correct
location (in the presence of macros) instead of
FullSourceLoc::getDecomposedLoc.
* Fix createTextRegion so that it handles caret ranges correctly,
this should bring it to parity with the previous implementation.

clang/test/Analysis/diagnostics/Inputs/expected-sarif:
* Update the schema URL to the offical website
* Add the emitted defaultConfiguration sections to all rules
* Annotate results with the "level" property

clang/lib/StaticAnalyzer/Core/SarifDiagnostics.cpp:
* Update SarifDiagnostics class to hold a clang::SarifDocumentWriter
that it uses to convert diagnostics to SARIF.

show more ...

[clang] Convert for_each to range-based for loops (NFC)

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[analyzer] Create MacroExpansionContext member in AnalysisConsumer

Adds a `MacroExpansionContext` member to the `AnalysisConsumer` class.
Tracks macro expansions only if the `ShouldDisplayMacroExpan

[analyzer] Create MacroExpansionContext member in AnalysisConsumer

Adds a `MacroExpansionContext` member to the `AnalysisConsumer` class.
Tracks macro expansions only if the `ShouldDisplayMacroExpansions` is set.
Passes a reference down the pipeline letting AnalysisConsumers query macro
expansions during bugreport construction.

Reviewed By: martong, Szelethus

Differential Revision: https://reviews.llvm.org/D93223

show more ...

Revert "Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""""

This reverts commit 6b0ee02747ed22d41e175d15f27025183341e6f8.

Circular dep

Revert "Revert "Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""""

This reverts commit 6b0ee02747ed22d41e175d15f27025183341e6f8.

Circular dependency again.

show more ...

Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""

This reverts commit d2ddc694ff94743d9735aaf07edcaf6db8aaca04.

This still contains a circular

Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""

This reverts commit d2ddc694ff94743d9735aaf07edcaf6db8aaca04.

This still contains a circular dependency between Analysis and CrossTU:

$ grep -r include.*Analysis clang/include/clang/CrossTU
clang/include/clang/CrossTU/CrossTranslationUnit.h:
#include "clang/Analysis/CrossTUAnalysisHelper.h"
$ grep -r include.*CrossTU clang/lib/Analysis
clang/lib/Analysis/PlistHTMLPathDiagnosticConsumer.cpp:
#include "clang/CrossTU/CrossTranslationUnit.h"
clang/lib/Analysis/PlistPathDiagnosticConsumer.cpp:
#include "clang/Analysis/CrossTUAnalysisHelper.h"

show more ...

Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."

The patch introduced a cycle dependency:

clangAnalysis -> clangFrontend -> clangSema -> clangAnalysis

This re

Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."

The patch introduced a cycle dependency:

clangAnalysis -> clangFrontend -> clangSema -> clangAnalysis

This reverts commit 00ffea77ad887b576e9db82d98c97a31fee172cb.
This reverts commit ea6641085d025ca0a5cef940465ef14d0ccace02.

show more ...

Revert "Revert "Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."""

This reverts commit 41bcc05e2a4e3062eb12ac6e071bc835decc38f5.

Revert "Revert "[analyzer] NFC: Separate PathDiagnosticConsumer options from AnalyzerOptions.""

This reverts commit 10f1ca99b498347186ec74b01046ad292bde9a4c.

(cherry picked from commit c599fc738a70

Revert "Revert "[analyzer] NFC: Separate PathDiagnosticConsumer options from AnalyzerOptions.""

This reverts commit 10f1ca99b498347186ec74b01046ad292bde9a4c.

(cherry picked from commit c599fc738a70e482976c6cc0ea31bef561641279)

show more ...

clang/StaticAnalyzer: Stop using SourceManager::getBuffer

Update clang/lib/StaticAnalyzer to stop relying on a `MemoryBuffer*`,
using the `MemoryBufferRef` from `getBufferOrNone` or the
`Optional<Me

clang/StaticAnalyzer: Stop using SourceManager::getBuffer

Update clang/lib/StaticAnalyzer to stop relying on a `MemoryBuffer*`,
using the `MemoryBufferRef` from `getBufferOrNone` or the
`Optional<MemoryBufferRef>` from `getBufferOrFake`, depending on whether
there's logic for checking validity of the buffer. The change to
clang/lib/StaticAnalyzer/Core/IssueHash.cpp is potentially a
functionality change, since the logic was wrong (it checked for
`nullptr`, which was never returned by the old API), but if that was
reachable the new behaviour should be better.

Differential Revision: https://reviews.llvm.org/D89414

show more ...

Revert "[analyzer] NFC: Separate PathDiagnosticConsumer options from AnalyzerOptions."

This reverts commit fd4b3f123d6e64769881e4c6351d5bbbdac30ce3.

Revert "[analyzer] NFC: Move path diagnostic consumer implementations to libAnalysis."

This reverts commit 44b7cf2983b6a8373c99a9b254f8c3f944e03f35.

[analyzer] NFC: Separate PathDiagnosticConsumer options from AnalyzerOptions.

The AnalyzerOptions object contains too much information that's
entirely specific to the Analyzer. It is also being refe

[analyzer] NFC: Separate PathDiagnosticConsumer options from AnalyzerOptions.

The AnalyzerOptions object contains too much information that's
entirely specific to the Analyzer. It is also being referenced by
path diagnostic consumers to tweak their behavior. In order for path
diagnostic consumers to function separately from the analyzer,
make a smaller options object that only contains relevant options.

Differential Revision: https://reviews.llvm.org/D67420

show more ...

Revert "[analyzer] Change the default output type to PD_TEXT_MINIMAL in the frontend, error if an output loc is missing for PathDiagConsumers that need it"

This reverts commit fe1a3a7e8c8be33968b9a7

Revert "[analyzer] Change the default output type to PD_TEXT_MINIMAL in the frontend, error if an output loc is missing for PathDiagConsumers that need it"

This reverts commit fe1a3a7e8c8be33968b9a768666489823dabab10.

show more ...

[analyzer] Change the default output type to PD_TEXT_MINIMAL in the frontend, error if an output loc is missing for PathDiagConsumers that need it

The title and the included test file sums everythin

[analyzer] Change the default output type to PD_TEXT_MINIMAL in the frontend, error if an output loc is missing for PathDiagConsumers that need it

The title and the included test file sums everything up -- the only thing I'm
mildly afraid of is whether anyone actually depends on the weird behavior of
HTMLDiagnostics pretending to be TextDiagnostics if an output directory is not
supplied. If it is, I guess we would need to resort to tiptoeing around the
compatibility flag.

Differential Revision: https://reviews.llvm.org/D76510

show more ...

[analyzer][NFC] Move the text output type to its own file, move code to PathDiagnosticConsumer creator functions

TableGen and .def files (which are meant to be used with the preprocessor) come
with

[analyzer][NFC] Move the text output type to its own file, move code to PathDiagnosticConsumer creator functions

TableGen and .def files (which are meant to be used with the preprocessor) come
with obvious downsides. One of those issues is that generated switch-case
branches have to be identical. This pushes corner cases either to an outer code
block, or into the generated code.

Inspect the removed code in AnalysisConsumer::DigestAnalyzerOptions. You can see
how corner cases like a not existing output file, the analysis output type being
set to PD_NONE, or whether to complement the output with additional diagnostics
on stderr lay around the preprocessor generated code. This is a bit problematic,
as to how to deal with such errors is not in the hands of the users of this
interface (those implementing output types, like PlistDiagnostics etc).

This patch changes this by moving these corner cases into the generated code,
more specifically, into the called functions. In addition, I introduced a new
output type for convenience purposes, PD_TEXT_MINIMAL, which always existed
conceptually, but never in the actual Analyses.def file. This refactoring
allowed me to move TextDiagnostics (renamed from ClangDiagPathDiagConsumer) to
its own file, which it really deserved.

Also, those that had the misfortune to gaze upon Analyses.def will probably
enjoy the sight that a clang-format did on it.

Differential Revision: https://reviews.llvm.org/D76509

show more ...

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in

Avoid including FileManager.h from SourceManager.h

Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in FileManager.h and LLVM's FS headers.

The main code change here is to sink SM::createFileID into the cpp file.
I reason that this is not performance critical because it doesn't happen
on the diagnostic path, it happens along the paths of macro expansion
(could be hot) and new includes (less hot).

Saves some includes:
309 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileManager.h
272 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileSystemOptions.h
271 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/VirtualFileSystem.h
267 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/FileSystem.h
266 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Chrono.h

Differential Revision: https://reviews.llvm.org/D75406

show more ...

[clang] Replace SmallStr.str().str() with std::string conversion operator.

Use the std::string conversion operator introduced in
d7049213d0fcda691c9e79f9b41e357198d99738.

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[analyzer] Fix SARIF column locations

Differential revision: https://reviews.llvm.org/D70689

[analyzer][NFC] Fix inconsistent references to checkers as "checks"

Traditionally, clang-tidy uses the term check, and the analyzer uses checker,
but in the very early years, this wasn't the case, a

[analyzer][NFC] Fix inconsistent references to checkers as "checks"

Traditionally, clang-tidy uses the term check, and the analyzer uses checker,
but in the very early years, this wasn't the case, and code originating from the
early 2010's still incorrectly refer to checkers as checks.

This patch attempts to hunt down most of these, aiming to refer to checkers as
checkers, but preserve references to callback functions (like checkPreCall) as
checks.

Differential Revision: https://reviews.llvm.org/D67140

llvm-svn: 371760

show more ...

[analyzer] NFC: Move PathDiagnostic classes to libAnalysis.

At this point the PathDiagnostic, PathDiagnosticLocation, PathDiagnosticPiece
structures no longer rely on anything specific to Static Ana

[analyzer] NFC: Move PathDiagnostic classes to libAnalysis.

At this point the PathDiagnostic, PathDiagnosticLocation, PathDiagnosticPiece
structures no longer rely on anything specific to Static Analyzer, so we can
move them out of it for everybody to use.

PathDiagnosticConsumers are still to be handed off.

Differential Revision: https://reviews.llvm.org/D67419

llvm-svn: 371661

show more ...

Update the SARIF exporter to SARIF 2.1

This updates the SARIF exporter to produce SARIF 2.1 output. The bulk of the diffs come from two changes to SARIF:
* https://github.com/oasis-tcs/sarif-spec/is

Update the SARIF exporter to SARIF 2.1

This updates the SARIF exporter to produce SARIF 2.1 output. The bulk of the diffs come from two changes to SARIF:
* https://github.com/oasis-tcs/sarif-spec/issues/309
* https://github.com/oasis-tcs/sarif-spec/issues/179

Differential Revision: https://reviews.llvm.org/D65211

llvm-svn: 370068

show more ...

Fix a SARIF exporter crash with macro expansions

Differential Revision: https://reviews.llvm.org/D65209

llvm-svn: 370061

Fix text range end columns in SARIF to be exclusive

According to the SARIF specification, "a text region does not include the character specified by endColumn".

Differential Revision: https://revie

Fix text range end columns in SARIF to be exclusive

According to the SARIF specification, "a text region does not include the character specified by endColumn".

Differential Revision: https://reviews.llvm.org/D65206

llvm-svn: 370060

show more ...