[compiler-rt][ASan] Add function copying annotations (#91702)

This PR adds a `__sanitizer_copy_contiguous_container_annotations`
function, which copies annotations from one memory area to another.

[compiler-rt][ASan] Add function copying annotations (#91702)

This PR adds a `__sanitizer_copy_contiguous_container_annotations`
function, which copies annotations from one memory area to another. New
area is annotated in the same way as the old region at the beginning
(within limitations of ASan).

Overlapping case: The function supports overlapping containers, however
no assumptions should be made outside of no false positives in new
buffer area. (It doesn't modify old container annotations where it's not
necessary, false negatives may happen in edge granules of the new
container area.) I don't expect this function to be used with
overlapping buffers, but it's designed to work with them and not result
in incorrect ASan errors (false positives).

If buffers have granularity-aligned distance between them (`old_beg %
granularity == new_beg % granularity`), copying algorithm works faster.
If the distance is not granularity-aligned, annotations are copied byte
after byte.

```cpp
void __sanitizer_copy_contiguous_container_annotations(
const void *old_storage_beg_p, const void *old_storage_end_p,
const void *new_storage_beg_p, const void *new_storage_end_p) {
```

This function aims to help with short string annotations and similar
container annotations. Right now we change trait types of
`std::basic_string` when compiling with ASan and this function purpose
is reverting that change as soon as possible.


https://github.com/llvm/llvm-project/blob/87f3407856e61a73798af4e41b28bc33b5bf4ce6/libcxx/include/string#L738-L751

The goal is to not change `__trivially_relocatable` when compiling with
ASan. If this function is accepted and upstreamed, the next step is
creating a function like `__memcpy_with_asan` moving memory with ASan.
And then using this function instead of `__builtin__memcpy` while moving
trivially relocatable objects.


https://github.com/llvm/llvm-project/blob/11a6799740f824282650aa9ec249b55dcf1a8aae/libcxx/include/__memory/uninitialized_algorithms.h#L644-L646

---

I'm thinking if there is a good way to address fact that in a container
the new buffer is usually bigger than the previous one. We may add two
more arguments to the functions to address it (the beginning and the end
of the whole buffer.

Another potential change is removing `new_storage_end_p` as it's
redundant, because we require the same size.

Potential future work is creating a function `__asan_unsafe_memmove`,
which will be basically memmove, but with turned off instrumentation
(therefore it will allow copy data from poisoned area).

---------

Co-authored-by: Vitaly Buka <vitalybuka@google.com>

show more ...

[1a/3][ASan][compiler-rt] API for double ended containers

This revision is a part of a series of patches extending
AddressSanitizer C++ container overflow detection capabilities by adding
annotation

[1a/3][ASan][compiler-rt] API for double ended containers

This revision is a part of a series of patches extending
AddressSanitizer C++ container overflow detection capabilities by adding
annotations, similar to those existing in std::vector, to std::string
and std::deque collections. These changes allow ASan to detect cases
when the instrumented program accesses memory which is internally
allocated by the collection but is still not in-use (accesses before or
after the stored elements for std::deque, or between the size and
capacity bounds for std::string).

The motivation for the research and those changes was a bug, found by
Trail of Bits, in a real code where an out-of-bounds read could happen
as two strings were compared via a std::equals function that took
iter1_begin, iter1_end, iter2_begin iterators (with a custom comparison
function). When object iter1 was longer than iter2, read out-of-bounds
on iter2 could happen. Container sanitization would detect it.

This revision adds a new compiler-rt ASan sanitization API function
sanitizer_annotate_double_ended_contiguous_container necessary to
sanitize/annotate double ended contiguous containers. Note that that
function annotates a single contiguous memory buffer (for example the
std::deque's internal chunk). Such containers have the beginning of
allocated memory block, beginning of the container in-use data, end of
the container's in-use data and the end of the allocated memory block.
This also adds a new API function to verify if a double ended contiguous
container is correctly annotated
(__sanitizer_verify_double_ended_contiguous_container).

Since we do not modify the ASan's shadow memory encoding values, the
capability of sanitizing/annotating a prefix of the internal contiguous
memory buffer is limited – up to SHADOW_GRANULARITY-1 bytes may not be
poisoned before the container's in-use data. This can cause false
negatives (situations when ASan will not detect memory corruption in
those areas).

On the other hand, API function interfaces are designed to work even if
this caveat would not exist. Therefore implementations using those
functions will poison every byte correctly, if only ASan (and
compiler-rt) is extended to support it. In other words, if ASan was
modified to support annotating/poisoning of objects lying on addresses
unaligned to SHADOW_GRANULARITY (so e.g. prefixes of those blocks),
which would require changing its shadow memory encoding, this would not
require any changes in the libcxx std::string/deque code which is added
in further commits of this patch series.

If you have any questions, please email:
advenam.tacet@trailofbits.com
disconnect3d@trailofbits.com

Differential Revision: https://reviews.llvm.org/D132090

show more ...

[NFC][asan] Reformat macro

[PATCH] ASAN: Align declaration with definition of a fn

Fixes:
https://bugs.llvm.org/show_bug.cgi?id=51641

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115447

Reapply [sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)

D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to

Reapply [sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)

D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.

Reapplying with changes to the symbol names in various platform
specific code, which I missed previously.

Differential Revision: https://reviews.llvm.org/D118783

show more ...

Revert "[sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)"

This reverts commit fda29264f360820859587acdfb0ad9392c944bd6.

This breaks the sanitizer build on windows, will reapply with

Revert "[sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)"

This reverts commit fda29264f360820859587acdfb0ad9392c944bd6.

This breaks the sanitizer build on windows, will reapply with
additional changes.

show more ...

[sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)

D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more

[sanitizers] Avoid macro clash in SignalContext::WriteFlag (NFC)

D116208 may cause a macro clash on older versions of linux, where
fs.h defines a READ macro. This is resolved by switching to a more
typical casing style for non-macro symbols.

Differential Revision: https://reviews.llvm.org/D118783

show more ...

Reland "[ASan] Do not misrepresent high value address dereferences as null dereferences"

Updated: Removed offending TODO comment.

Dereferences with addresses above the 48-bit hardware addressable r

Reland "[ASan] Do not misrepresent high value address dereferences as null dereferences"

Updated: Removed offending TODO comment.

Dereferences with addresses above the 48-bit hardware addressable range
produce "invalid instruction" (instead of "invalid access") hardware
exceptions (there is no hardware address decoding logic for those bits),
and the address provided by this exception is the address of the
instruction (not the faulting address). The kernel maps the "invalid
instruction" to SEGV, but fails to provide the real fault address.

Because of this ASan lies and says that those cases are null
dereferences. This downgrades the severity of a found bug in terms of
security. In the ASan signal handler, we can not provide the real
faulting address, but at least we can try not to lie.

rdar://50366151

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D68676

> llvm-svn: 374265

llvm-svn: 374384

show more ...

Revert "[ASan] Do not misrepresent high value address dereferences as null dereferences"

As it was breaking bots running sanitizer lint check

This reverts r374265 (git b577efe4567f1f6a711ad36e1d172

Revert "[ASan] Do not misrepresent high value address dereferences as null dereferences"

As it was breaking bots running sanitizer lint check

This reverts r374265 (git b577efe4567f1f6a711ad36e1d17280dd1c4f009)

llvm-svn: 374308

show more ...

[ASan] Do not misrepresent high value address dereferences as null dereferences

Dereferences with addresses above the 48-bit hardware addressable range
produce "invalid instruction" (instead of "inv

[ASan] Do not misrepresent high value address dereferences as null dereferences

Dereferences with addresses above the 48-bit hardware addressable range
produce "invalid instruction" (instead of "invalid access") hardware
exceptions (there is no hardware address decoding logic for those bits),
and the address provided by this exception is the address of the
instruction (not the faulting address). The kernel maps the "invalid
instruction" to SEGV, but fails to provide the real fault address.

Because of this ASan lies and says that those cases are null
dereferences. This downgrades the severity of a found bug in terms of
security. In the ASan signal handler, we can not provide the real
faulting address, but at least we can try not to lie.

rdar://50366151

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D68676

llvm-svn: 374265

show more ...

[asan] clang-format parent patch

llvm-svn: 361305

[asan] Replace assignments with internal_memcpy ErrorDescription

For consistency with nearby code and to avoid interceptors during reports.

llvm-svn: 361304

[sanitizer] Implement reallocarray.

Summary:
It's a cross of calloc and realloc. Sanitizers implement calloc-like check for size
overflow.

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, #sani

[sanitizer] Implement reallocarray.

Summary:
It's a cross of calloc and realloc. Sanitizers implement calloc-like check for size
overflow.

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61108

llvm-svn: 359708

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Do not rely on that subject of ErrorAllocTypeMismatch is a heap address.

Differential Revision: https://reviews.llvm.org/D54856.

llvm-svn: 350085

[ASan] Linker-initialize static ScopedInErrorReport::current_error_.

Summary:
Static ScopedInErrorReport::current_error_ can be linker initialized to
shave one global ctor call on application startu

[ASan] Linker-initialize static ScopedInErrorReport::current_error_.

Summary:
Static ScopedInErrorReport::current_error_ can be linker initialized to
shave one global ctor call on application startup and be __asan_init-safe.

Global constructors in ASan runtime are bad because __asan_init runs
from preinit_array, before any such constructors.

Issue: https://github.com/google/sanitizers/issues/194

Reviewers: eugenis, morehouse

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48141

llvm-svn: 334748

show more ...

[Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign

Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon)

[Sanitizers] Check alignment != 0 for aligned_alloc and posix_memalign

Summary:
Move the corresponding tests to the common folder (as all of the
sanitizer allocators will support this feature soon) and add the checks
specific to aligned_alloc to ASan and LSan allocators.

Reviewers: vitalybuka

Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D47924

llvm-svn: 334316

show more ...

[ASan] NFC: make use of a new ErrorBase ctor

Summary:
Minor style changes to complement D44404:
- make use of a new ErrorBase ctor
- de-duplicate a comment about VS2013 support

Reviewers: eugenis

[ASan] NFC: make use of a new ErrorBase ctor

Summary:
Minor style changes to complement D44404:
- make use of a new ErrorBase ctor
- de-duplicate a comment about VS2013 support

Reviewers: eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45390

llvm-svn: 329586

show more ...

[ASan] Report proper ASan error on allocator failures instead of CHECK(0)-ing

Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termina

[ASan] Report proper ASan error on allocator failures instead of CHECK(0)-ing

Summary:
Currently many allocator specific errors (OOM, for example) are reported as
a text message and CHECK(0) termination, not stack, no details, not too
helpful nor informative. To improve the situation, ASan detailed errors were
defined and reported under the appropriate conditions.

Issue: https://github.com/google/sanitizers/issues/887

Reviewers: eugenis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44404

llvm-svn: 328722

show more ...

[Sanitizers] ASan: detect new/delete calls with mismatched alignment.

ASan allocator stores the requested alignment for new and new[] calls
and on delete and delete[] verifies that alignments do mat

[Sanitizers] ASan: detect new/delete calls with mismatched alignment.

ASan allocator stores the requested alignment for new and new[] calls
and on delete and delete[] verifies that alignments do match.

The representable alignments are: default alignment, 8, 16, 32, 64, 128,
256 and 512 bytes. Alignments > 512 are stored as 512, hence two
different alignments > 512 will pass the check (possibly masking the bug),
but limited memory requirements deemed to be a resonable tradeoff for
relaxed conditions.

The feature is controlled by new_delete_type_mismatch flag, the same one
protecting new/delete matching size check.

Differential revision: https://reviews.llvm.org/D38574

Issue: https://github.com/google/sanitizers/issues/799
llvm-svn: 316595

show more ...

[asan] Remove ErrorStackOverflow

Summary:
The only difference from ErrorDeadlySignal is reporting code and it lives in
sanitizer common.

Part of https://github.com/google/sanitizers/issues/637

Rev

[asan] Remove ErrorStackOverflow

Summary:
The only difference from ErrorDeadlySignal is reporting code and it lives in
sanitizer common.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D37868

llvm-svn: 313309

show more ...

[compiler-rt] Use SignalContext in ErrorStackOverflow and ErrorDeadlySignal

Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: kub

[compiler-rt] Use SignalContext in ErrorStackOverflow and ErrorDeadlySignal

Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37793

llvm-svn: 313168

show more ...

[compiler-rt] Add siginfo into SignalContext

Summary:
Information stored there is often been passed along with SignalContext.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eug

[compiler-rt] Add siginfo into SignalContext

Summary:
Information stored there is often been passed along with SignalContext.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37792

llvm-svn: 313167

show more ...

[asan] Provide bug descriptions for all reports (not just ErrorGeneric)

Differential Revision: https://reviews.llvm.org/D27012

llvm-svn: 288065

[asan] Reify ErrorGeneric

Summary:
Finish work on PR30351 (last one, after D24551, D24552, and D24554 land)

Also replace the old ReportData structure/variable with the current_error_ static
member

[asan] Reify ErrorGeneric

Summary:
Finish work on PR30351 (last one, after D24551, D24552, and D24554 land)

Also replace the old ReportData structure/variable with the current_error_ static
member of the ScopedInErrorReport class.

This has the following side-effects:
- Move ASAN_ON_ERROR(); call to the start of the destructor, instead
of in StartReporting().
- We only generate the error structure after the
ScopedInErrorReport constructor finishes, so we can't call
ASAN_ON_ERROR() during the constructor. I think this makes more
sense, since we end up never running two of the ASAN_ON_ERROR()
callback. This also works the same way as error reporting, since
we end up having a lock around it. Otherwise we could end up
with the ASAN_ON_ERROR() call for error 1, then the
ASAN_ON_ERROR() call for error 2, and then lock the mutex for
reporting error 1.
- The __asan_get_report_* functions will be able to, in the future,
provide information about other errors that aren't a "generic
error". But we might want to rethink that API, since it's too
restricted. Ideally we teach lldb about the current_error_ member of
ScopedInErrorReport.

Reviewers: vitalybuka, kcc, eugenis

Subscribers: kubabrecka, llvm-commits

Differential Revision: https://reviews.llvm.org/D24555

llvm-svn: 282107

show more ...