HWAsan: Convert tests to opaque pointers

Fixups needed in:
X86/alloca-with-calls.ll
X86/alloca.ll
alloca-with-calls.ll
atomic.ll
basic.ll
dbg-value-tag-offset.ll
kernel-alloca.ll
pro

HWAsan: Convert tests to opaque pointers

Fixups needed in:
X86/alloca-with-calls.ll
X86/alloca.ll
alloca-with-calls.ll
atomic.ll
basic.ll
dbg-value-tag-offset.ll
kernel-alloca.ll
prologue.ll
use-after-scope.ll

show more ...

[test] Remove some legacy PM tests in llvm/test/Instrumentation

hwasan: Support for outlined checks in the Linux kernel.

Add support for match-all tags and GOT-free runtime calls, which
are both required for the kernel to be able to support outlined
checks. This

hwasan: Support for outlined checks in the Linux kernel.

Add support for match-all tags and GOT-free runtime calls, which
are both required for the kernel to be able to support outlined
checks. This requires extending the access info to let the backend
know when to enable these features. To make the code easier to maintain
introduce an enum with the bit field positions for the access info.

Allow outlined checks to be enabled with -mllvm
-hwasan-inline-all-checks=0. Kernels that contain runtime support for
outlined checks may pass this flag. Kernels lacking runtime support
will continue to link because they do not pass the flag. Old versions
of LLVM will ignore the flag and continue to use inline checks.

With a separate kernel patch [1] I measured the code size of defconfig
+ tag-based KASAN, as well as boot time (i.e. time to init launch)
on a DragonBoard 845c with an Android arm64 GKI kernel. The results
are below:

code size boot time
before 92824064 6.18s
after 38822400 6.65s

[1] https://linux-review.googlesource.com/id/I1a30036c70ab3c3ee78d75ed9b87ef7cdc3fdb76

Depends on D90425

Differential Revision: https://reviews.llvm.org/D90426

show more ...

hwasan: Move fixed shadow behind opaque no-op cast as well.

This is a workaround for poor heuristics in the backend where we can
end up materializing the constant multiple times. This is particularl

hwasan: Move fixed shadow behind opaque no-op cast as well.

This is a workaround for poor heuristics in the backend where we can
end up materializing the constant multiple times. This is particularly
bad when using outlined checks because we materialize it for every call
(because the backend considers it trivial to materialize).

As a result the field containing the shadow base value will always
be set so simplify the code taking that into account.

Differential Revision: https://reviews.llvm.org/D90425

show more ...

hwasan: Move memory access checks into small outlined functions on aarch64.

Each hwasan check requires emitting a small piece of code like this:
https://clang.llvm.org/docs/HardwareAssistedAddressSa

hwasan: Move memory access checks into small outlined functions on aarch64.

Each hwasan check requires emitting a small piece of code like this:
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#memory-accesses

The problem with this is that these code blocks typically bloat code
size significantly.

An obvious solution is to outline these blocks of code. In fact, this
has already been implemented under the -hwasan-instrument-with-calls
flag. However, as currently implemented this has a number of problems:
- The functions use the same calling convention as regular C functions.
This means that the backend must spill all temporary registers as
required by the platform's C calling convention, even though the
check only needs two registers on the hot path.
- The functions take the address to be checked in a fixed register,
which increases register pressure.
Both of these factors can diminish the code size effect and increase
the performance hit of -hwasan-instrument-with-calls.

The solution that this patch implements is to involve the aarch64
backend in outlining the checks. An intrinsic and pseudo-instruction
are created to represent a hwasan check. The pseudo-instruction
is register allocated like any other instruction, and we allow the
register allocator to select almost any register for the address to
check. A particular combination of (register selection, type of check)
triggers the creation in the backend of a function to handle the check
for specifically that pair. The resulting functions are deduplicated by
the linker. The pseudo-instruction (really the function) is specified
to preserve all registers except for the registers that the AAPCS
specifies may be clobbered by a call.

To measure the code size and performance effect of this change, I
took a number of measurements using Chromium for Android on aarch64,
comparing a browser with inlined checks (the baseline) against a
browser with outlined checks.

Code size: Size of .text decreases from 243897420 to 171619972 bytes,
or a 30% decrease.

Performance: Using Chromium's blink_perf.layout microbenchmarks I
measured a median performance regression of 6.24%.

The fact that a perf/size tradeoff is evident here suggests that
we might want to make the new behaviour conditional on -Os/-Oz.
But for now I've enabled it unconditionally, my reasoning being that
hwasan users typically expect a relatively large perf hit, and ~6%
isn't really adding much. We may want to revisit this decision in
the future, though.

I also tried experimenting with varying the number of registers
selectable by the hwasan check pseudo-instruction (which would result
in fewer variants being created), on the hypothesis that creating
fewer variants of the function would expose another perf/size tradeoff
by reducing icache pressure from the check functions at the cost of
register pressure. Although I did observe a code size increase with
fewer registers, I did not observe a strong correlation between the
number of registers and the performance of the resulting browser on the
microbenchmarks, so I conclude that we might as well use ~all registers
to get the maximum code size improvement. My results are below:

Regs | .text size | Perf hit
-----+------------+---------
~all | 171619972 | 6.24%
16 | 171765192 | 7.03%
8 | 172917788 | 5.82%
4 | 177054016 | 6.89%

Differential Revision: https://reviews.llvm.org/D56954

llvm-svn: 351920

show more ...

hwasan: add -fsanitize=kernel-hwaddress flag

This patch adds -fsanitize=kernel-hwaddress flag, that essentially enables
-hwasan-kernel=1 -hwasan-recover=1 -hwasan-match-all-tag=0xff.

Differential R

hwasan: add -fsanitize=kernel-hwaddress flag

This patch adds -fsanitize=kernel-hwaddress flag, that essentially enables
-hwasan-kernel=1 -hwasan-recover=1 -hwasan-match-all-tag=0xff.

Differential Revision: https://reviews.llvm.org/D45046

llvm-svn: 330044

show more ...

hwasan: add -hwasan-match-all-tag flag

Sometimes instead of storing addresses as is, the kernel stores the address of
a page and an offset within that page, and then computes the actual address
when

hwasan: add -hwasan-match-all-tag flag

Sometimes instead of storing addresses as is, the kernel stores the address of
a page and an offset within that page, and then computes the actual address
when it needs to make an access. Because of this the pointer tag gets lost
(gets set to 0xff). The solution is to ignore all accesses tagged with 0xff.

This patch adds a -hwasan-match-all-tag flag to hwasan, which allows to ignore
accesses through pointers with a particular pointer tag value for validity.

Patch by Andrey Konovalov.

Differential Revision: https://reviews.llvm.org/D44827

llvm-svn: 329228

show more ...

[hwasan] Fix inline instrumentation.

This patch changes hwasan inline instrumentation:

Fixes address untagging for shadow address calculation (use 0xFF instead of 0x00 for the top byte).
Emits brk

[hwasan] Fix inline instrumentation.

This patch changes hwasan inline instrumentation:

Fixes address untagging for shadow address calculation (use 0xFF instead of 0x00 for the top byte).
Emits brk instruction instead of hlt for the kernel and user space.
Use 0x900 instead of 0x100 for brk immediate (0x100 - 0x800 are unavailable in the kernel).
Fixes and adds appropriate tests.

Patch by Andrey Konovalov.

Differential Revision: https://reviews.llvm.org/D43135

llvm-svn: 325711

show more ...

[hwasan] LLVM-level flags for linux kernel-compatible hwasan instrumentation.

Summary:
-hwasan-mapping-offset defines the non-zero shadow base address.
-hwasan-kernel disables calls to __hwasan_init

[hwasan] LLVM-level flags for linux kernel-compatible hwasan instrumentation.

Summary:
-hwasan-mapping-offset defines the non-zero shadow base address.
-hwasan-kernel disables calls to __hwasan_init in module constructors.
Unlike ASan, -hwasan-kernel does not force callback instrumentation.
This is controlled separately with -hwasan-instrument-with-calls.

Reviewers: kcc

Subscribers: srhines, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D42141

llvm-svn: 322785

show more ...