[SpecialCaseList] Use glob by default (#74809)

https://reviews.llvm.org/D154014 addes glob support and enables it when
`#!special-case-list-v2` is the first line. This patch makes the glob
support

[SpecialCaseList] Use glob by default (#74809)

https://reviews.llvm.org/D154014 addes glob support and enables it when
`#!special-case-list-v2` is the first line. This patch makes the glob
support the default (faster than regex after
https://reviews.llvm.org/D156046) and switches to the deprecated regex
support if `#!special-case-list-v1` is the first line.

I have surveyed many ignore lists. All ignore lists I find only use
basic `*` `.` and don't use regex metacharacters such as `(` and `)`.
(As neither `src:` nor `fun:` benefits from using regex.)
They are unaffected by the transition (with a caution that regex
`src:x/a.pb.*` matches `x/axpbx` but glob `src:x/a.pb.*` doesn't).

There is no deprecating warning. If a user finds
`#!special-case-list-v1`, they shall read that the old syntax is
deprecated.

Link:
https://discourse.llvm.org/t/use-glob-instead-of-regex-for-specialcaselists/71666

show more ...

[SpecialCaseList] Add option to use Globs instead of Regex to match patterns

Add an option in `SpecialCaseList` to use Globs instead of Regex to match patterns. `GlobPattern` was extended in https:/

[SpecialCaseList] Add option to use Globs instead of Regex to match patterns

Add an option in `SpecialCaseList` to use Globs instead of Regex to match patterns. `GlobPattern` was extended in https://reviews.llvm.org/D153587 to support brace expansions which allows us to use patterns like `*/src/foo.{c,cpp}`. It turns out that most patterns only take advantage of `*` so using Regex was overkill and required lots of escaping in practice. This often led to bugs due to forgetting to escape special characters.

Since this would be a breaking change, we temporarily support Regex by default and use Globs when `#!special-case-list-v2` is the first line in the file. Users should switch to the glob format described in https://llvm.org/doxygen/classllvm_1_1GlobPattern.html. For example, `(abc|def)` should become `{abc,def}`.

See discussion in https://reviews.llvm.org/D152762 and https://discourse.llvm.org/t/use-glob-instead-of-regex-for-specialcaselists/71666.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D154014

show more ...

[clang/llvm] As part of using inclusive language within
the llvm project, migrate away from the use of blacklist and whitelist.

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

Reland 9f3fdb0d7fab: [Driver] Use VFS to check if sanitizer blacklists exist

With updates to various LLVM tools that use SpecialCastList.

It was tempting to use RealFileSystem as the default, but t

Reland 9f3fdb0d7fab: [Driver] Use VFS to check if sanitizer blacklists exist

With updates to various LLVM tools that use SpecialCastList.

It was tempting to use RealFileSystem as the default, but that makes it
too easy to accidentally forget passing VFS in clang code.

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Extend SpecialCaseList to allow users to blame matches on entries in the file.

Summary:
Extends SCL functionality to allow users to find the line number in the file the SCL is built from through Spe

Extend SpecialCaseList to allow users to blame matches on entries in the file.

Summary:
Extends SCL functionality to allow users to find the line number in the file the SCL is built from through SpecialCaseList::inSectionBlame(...).

Also removes the need to compile the SCL before use. As the matcher now contains a list of regexes to test against instead of a single regex, the regexes can be individually built on each insertion rather than one large compilation at the end of construction.

This change also fixes a bug where blank lines would cause the parser to become out-of-sync with the line number. An error on line `k` was being reported as being on line `k - num_blank_lines_before_k`.

Note: This change has a cyclical dependency on D39486. Both these changes must be submitted at the same time to avoid a build breakage.

Reviewers: vlad.tsyrklevich

Reviewed By: vlad.tsyrklevich

Subscribers: kcc, pcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D39485

llvm-svn: 317617

show more ...

Check special-case-list regex before insertion.

Summary:
Checks that the supplied regex to SpecialCaseList::Matcher::insert(..) is non-empty.

Reported by OSS-fuzz: https://bugs.chromium.org/p/oss-f

Check special-case-list regex before insertion.

Summary:
Checks that the supplied regex to SpecialCaseList::Matcher::insert(..) is non-empty.

Reported by OSS-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3688

Verified that this fixes the provided assertion failure (built with {asan, fuzzer}):
```
mitchp@mitchp2:~/llvm-build/git-fuzz$ ninja llvm-special-case-list-fuzzer[12/12] Linking CXX executable bin/llvm-special-case-list-fuzzer
mitchp@mitchp2:~/llvm-build/git-fuzz$ bin/llvm-special-case-list-fuzzer ~/Downloads/clusterfuzz-testcase-6748633157337088
INFO: Seed: 1697404507
INFO: Loaded 1 modules (18581 inline 8-bit counters): 18581 [0x9e9f60, 0x9ee7f5),
INFO: Loaded 1 PC tables (18581 PCs): 18581 [0x9ee7f8,0xa37148),
bin/llvm-special-case-list-fuzzer: Running 1 inputs 1 time(s) each.
Running: /usr/local/google/home/mitchp/Downloads/clusterfuzz-testcase-6748633157337088
Executed /usr/local/google/home/mitchp/Downloads/clusterfuzz-testcase-6748633157337088 in 0 ms
***
*** NOTE: fuzzing was not performed, you have only
*** executed the target code on a fixed set of inputs.
***
mitchp@mitchp2:~/llvm-build/git-fuzz$

```

Reviewers: kcc, vsk

Reviewed By: vsk

Subscribers: vsk, llvm-commits, vlad.tsyrklevich

Differential Revision: https://reviews.llvm.org/D39212

llvm-svn: 316537

show more ...

Add section headers to SpecialCaseLists

Summary:
Sanitizer blacklist entries currently apply to all sanitizers--there
is no way to specify that an entry should only apply to a specific
sanitizer. Th

Add section headers to SpecialCaseLists

Summary:
Sanitizer blacklist entries currently apply to all sanitizers--there
is no way to specify that an entry should only apply to a specific
sanitizer. This is important for Control Flow Integrity since there are
several different CFI modes that can be enabled at once. For maximum
security, CFI blacklist entries should be scoped to only the specific
CFI mode(s) that entry applies to.

Adding section headers to SpecialCaseLists allows users to specify more
information about list entries, like sanitizer names or other metadata,
like so:

[section1]
fun:*fun1*
[section2|section3]
fun:*fun23*

The section headers are regular expressions. For backwards compatbility,
blacklist entries entered before a section header are put into the '[*]'
section so that blacklists without sections retain the same behavior.

SpecialCaseList has been modified to also accept a section name when
matching against the blacklist. It has also been modified so the
follow-up change to clang can define a derived class that allows
matching sections by SectionMask instead of by string.

Reviewers: pcc, kcc, eugenis, vsk

Reviewed By: eugenis, vsk

Subscribers: vitalybuka, llvm-commits

Differential Revision: https://reviews.llvm.org/D37924

llvm-svn: 314170

show more ...

Re-sort #include lines for unittests. This uses a slightly modified
clang-format (https://reviews.llvm.org/D33932) to keep primary headers
at the top and handle new utility headers like 'gmock' consi

Re-sort #include lines for unittests. This uses a slightly modified
clang-format (https://reviews.llvm.org/D33932) to keep primary headers
at the top and handle new utility headers like 'gmock' consistently with
other utility headers.

No other change was made. I did no manual edits, all of this is
clang-format.

This should allow other changes to have more clear and focused diffs,
and is especially motivated by moving some headers into more focused
libraries.

llvm-svn: 304786

show more ...

Support escaping in TrigramIndex.

Summary:
This is a follow up to r288303, where I have introduced TrigramIndex
to speed up SpecialCaseList for the cases when all rules are
simple wildcards, like *h

Support escaping in TrigramIndex.

Summary:
This is a follow up to r288303, where I have introduced TrigramIndex
to speed up SpecialCaseList for the cases when all rules are
simple wildcards, like *hello*wor.d*.

Here, I add support for escaping, so that it's possible to
specify rules like *c\+\+abi*.

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D27318

llvm-svn: 288553

show more ...

Use trigrams to speed up SpecialCaseList.

Summary:
it's often the case when the rules in the SpecialCaseList
are of the form hel.o*bar. That gives us a chance to build
trigram index to quickly disca

Use trigrams to speed up SpecialCaseList.

Summary:
it's often the case when the rules in the SpecialCaseList
are of the form hel.o*bar. That gives us a chance to build
trigram index to quickly discard 99% of inputs without
running a full regex. A similar idea was used in Google Code Search
as described in the blog post:
https://swtch.com/~rsc/regexp/regexp4.html

The check is defeated, if there's at least one regex
more complicated than that. In this case, all inputs
will go through the regex. That said, the real-world
rules are often simple or can be simplied. That considerably
speeds up compiling Chromium with CFI and UBSan.

As measured on Chromium's content_message_generator.cc:

before, CFI: 44 s
after, CFI: 23 s
after, CFI, no blacklist: 23 s (~1% slower, but 3 runs were unable to show the difference)
after, regular compilation to bitcode: 23 s

Reviewers: pcc

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D27188

llvm-svn: 288303

show more ...

Try to fix some temp file leaks in SupportTests, PR18335

llvm-svn: 280443

SpecialCaseList: Add support for parsing multiple input files.

Summary:
This change allows users to create SpecialCaseList objects from
multiple local files. This is needed to implement a proper sup

SpecialCaseList: Add support for parsing multiple input files.

Summary:
This change allows users to create SpecialCaseList objects from
multiple local files. This is needed to implement a proper support
for -fsanitize-blacklist flag (allow users to specify multiple blacklists,
in addition to default blacklist, see PR22431).

DFSan can also benefit from this change, as DFSan instrumentation pass now
accepts ABI-lists both from -fsanitize-blacklist= and -mllvm -dfsan-abilist flags.

Go bindings are fixed accordingly.

Test Plan: regression test suite

Reviewers: pcc

Subscribers: llvm-commits, axw, kcc

Differential Revision: http://reviews.llvm.org/D7367

llvm-svn: 228155

show more ...

Remove support for undocumented SpecialCaseList entries.

"global-init", "global-init-src" and "global-init-type" were originally
used to blacklist entities in ASan init-order checker. However, they

Remove support for undocumented SpecialCaseList entries.

"global-init", "global-init-src" and "global-init-type" were originally
used to blacklist entities in ASan init-order checker. However, they
were never documented, and later were replaced by "=init" category.

Old blacklist entries should be converted as follows:
* global-init:foo -> global:foo=init
* global-init-src:bar -> src:bar=init
* global-init-type:baz -> type:baz=init

llvm-svn: 222401

show more ...

unique_ptrify the result of SpecialCaseList::create

llvm-svn: 216925

Convert a few more cases of direct intialization of unique_ptrs from MemoryBuffer::getMemBuffer to move initialization now that it returns by unique_ptr instead of raw pointer.

Cleanup/improvements

Convert a few more cases of direct intialization of unique_ptrs from MemoryBuffer::getMemBuffer to move initialization now that it returns by unique_ptr instead of raw pointer.

Cleanup/improvements following r216583.

llvm-svn: 216605

show more ...

Decouple llvm::SpecialCaseList text representation and its LLVM IR semantics.

Turn llvm::SpecialCaseList into a simple class that parses text files in
a specified format and knows nothing about LLVM

Decouple llvm::SpecialCaseList text representation and its LLVM IR semantics.

Turn llvm::SpecialCaseList into a simple class that parses text files in
a specified format and knows nothing about LLVM IR. Move this class into
LLVMSupport library. Implement two users of this class:
* DFSanABIList in DFSan instrumentation pass.
* SanitizerBlacklist in Clang CodeGen library.
The latter will be modified to use actual source-level information from frontend
(source file names) instead of unstable LLVM IR things (LLVM Module identifier).

Remove dependency edge from ClangCodeGen/ClangDriver to LLVMTransformUtils.

No functionality change.

llvm-svn: 212643

show more ...