[msan] Fix -Wunused-variable in non-assertion builds after #124421

[msan] Handle x86_avx512_(min|max)_p[sd]_512 intrinsics (#124421)

The AVX/SSE variants are already handled heuristically (maybeHandleSimpleNomemIntrinsic via handleUnknownIntrinsic), but the AVX512

[msan] Handle x86_avx512_(min|max)_p[sd]_512 intrinsics (#124421)

The AVX/SSE variants are already handled heuristically (maybeHandleSimpleNomemIntrinsic via handleUnknownIntrinsic), but the AVX512 variants contain an additional parameter (the rounding method) which fails to match heuristically. This patch generalizes maybeHandleSimpleNomemIntrinsic to allow additional flags (ignored by MSan) and explicitly call it to handle AVX512 min/max ps/pd intrinsics.

It also updates the test added in https://github.com/llvm/llvm-project/pull/123980

show more ...

[msan] Use horizontal add to compute shadow for horizontal sub (#124835)

This improves the horizontal sub handling (from
https://github.com/llvm/llvm-project/pull/124159), by always using
horizont

[msan] Use horizontal add to compute shadow for horizontal sub (#124835)

This improves the horizontal sub handling (from
https://github.com/llvm/llvm-project/pull/124159), by always using
horizontal add for the shadow, as recommended by Vitaly.

Fixes https://github.com/llvm/llvm-project/issues/124662

show more ...

[msan][NFCI] Generalize handleIntrinsicByApplyingToShadow to allow alternative intrinsic for shadows (#124831)

https://github.com/llvm/llvm-project/pull/124159 uses
handleIntrinsicByApplyingToShado

[msan][NFCI] Generalize handleIntrinsicByApplyingToShadow to allow alternative intrinsic for shadows (#124831)

https://github.com/llvm/llvm-project/pull/124159 uses
handleIntrinsicByApplyingToShadow for horizontal add/sub, but Vitaly
recommends always using the add version to avoid false negatives for
fully uninitialized data
(https://github.com/llvm/llvm-project/issues/124662).

This patch lays the groundwork by generalizing
handleIntrinsicByApplyingToShadow to allow using a different intrinsic
(of the same type as the original intrinsic) for the shadow. Planned
work will apply it to horizontal sub.

show more ...

Reapply "[msan] Add handlers for AVX masked load/store intrinsics (#123857)"

This reverts commit b9d301cc7e4fe4c442ec15169686fa4a18f5cdfc i.e.,
relands db79fb2a91df31a07f312f8e061936927ac5c506.

I h

Reapply "[msan] Add handlers for AVX masked load/store intrinsics (#123857)"

This reverts commit b9d301cc7e4fe4c442ec15169686fa4a18f5cdfc i.e.,
relands db79fb2a91df31a07f312f8e061936927ac5c506.

I had mistakenly thought this caused a buildbot breakage (the actual
culprit was my other patch,
https://github.com/llvm/llvm-project/pull/123980, which landed at the
same time) and thus had reverted it even though AFAIK it is not broken.

show more ...

[NFC][DebugInfo] Use iterators for instruction insertion in more places (#124291)

As part of the "RemoveDIs" work to eliminate debug intrinsics, we're
replacing methods that use Instruction*'s as p

[NFC][DebugInfo] Use iterators for instruction insertion in more places (#124291)

As part of the "RemoveDIs" work to eliminate debug intrinsics, we're
replacing methods that use Instruction*'s as positions with iterators.
This patch changes some more complex call-sites, those crossing file
boundaries and where I've had to perform some minor rewrites.

show more ...

Revert "[msan] Add handlers for AVX masked load/store intrinsics (#123857)"

This reverts commit db79fb2a91df31a07f312f8e061936927ac5c506.

Reason: buildbot breakage
(https://lab.llvm.org/buildbot/#/

Revert "[msan] Add handlers for AVX masked load/store intrinsics (#123857)"

This reverts commit db79fb2a91df31a07f312f8e061936927ac5c506.

Reason: buildbot breakage
(https://lab.llvm.org/buildbot/#/builders/144/builds/16636/steps/6/logs/FAIL__LLVM__avx512-intrinsics-upgrade_ll)

show more ...

[msan] Add handlers for AVX masked load/store intrinsics (#123857)

This patch adds explicit support for AVX masked load/store intrinsics,
largely by applying the intrinsics to the shadows (but subt

[msan] Add handlers for AVX masked load/store intrinsics (#123857)

This patch adds explicit support for AVX masked load/store intrinsics,
largely by applying the intrinsics to the shadows (but subtly different
to handleIntrinsicByApplyingToShadow()).

We do not reuse the handleMaskedLoad/Store functions. The key challenge
is that the LLVM masked intrinsics require a vector of booleans, while
AVX masked intrinsics use the MSBs of a vector of integers.
X86InstCombineIntrinsic.cpp::simplifyX86MaskedLoad mentions that the x86
backend does not know how to efficiently convert from a vector of
booleans back into the AVX mask format; therefore, they (and we) do not
reduce AVX masked intrinsics into LLVM masked intrinsics.

show more ...

[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNo

[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and similar feed into
instruction insertion positions. Call-sites where that's necessary were
updated a year ago; but to ensure some type safety however, we'd like to
have all calls to getFirstNonPHI use the iterator-returning version.

This patch changes a bunch of call-sites calling getFirstNonPHI to use
getFirstNonPHIIt, which returns an iterator. All these call sites are
where it's obviously safe to fetch the iterator then dereference it. A
follow-up patch will contain less-obviously-safe changes.

We'll eventually deprecate and remove the instruction-pointer
getFirstNonPHI, but not before adding concise documentation of what
considerations are needed (very few).

---------

Co-authored-by: Stephen Tozer <Melamoto@gmail.com>

show more ...

[msan] Handle horizontal add/subtract intrinsic by applying to shadow (#124159)

Horizontal add (hadd) and subtract (hsub) are currently heuristically
handled by `maybeHandleSimpleNomemIntrinsic()`

[msan] Handle horizontal add/subtract intrinsic by applying to shadow (#124159)

Horizontal add (hadd) and subtract (hsub) are currently heuristically
handled by `maybeHandleSimpleNomemIntrinsic()` (via
`handleUnknownIntrinsic()`), which computes the shadow by bitwise OR'ing
the two operands. This has false positives for hadd/hsub shadows. For
example, suppose the shadows for the two operands are 00000000 and
11111111 respectively. The expected shadow for the result is 00001111,
but `maybeHandleSimpleNomemIntrinsic` would compute it as 11111111.

This patch handles horizontal add using
`handleIntrinsicByApplyingToShadow` (from
https://github.com/llvm/llvm-project/pull/114490), which has no false
positives for hadd/hsub: if each pair of adjacent shadow values is zero
(fully initialized), the result will be zero (fully initialized). More
generally, it is precise for hadd/hsub if at least one of the two
adjacent shadow values in each pair is zero.

It does have some false negatives for hadd/hsub: if we add/subtract two
adjacent non-zero shadow values, some bits of the result may incorrectly
be zero. We consider this an acceptable tradeoff for performance. To
make shadow propagation precise, we want the equivalent of "horizontal
OR", but this is not available. Reducing horizontal OR to (permutation
plus bitwise OR) is left as an exercise for the reader.

show more ...

[msan][NFC] Correct and clarify comment for getShadowPtrOffset()

The stated return type was incorrect; this patch corrects it. More generally, it explains how the Offset and its components fits into

[msan][NFC] Correct and clarify comment for getShadowPtrOffset()

The stated return type was incorrect; this patch corrects it. More generally, it explains how the Offset and its components fits into the overall shadow mapping calculation.

show more ...

[msan] Generalize handleIntrinsicByApplyingToShadow by adding bitcasting (#123474)

`handleIntrinsicByApplyingToShadow` (introduced in
https://github.com/llvm/llvm-project/pull/114490) requires that

[msan] Generalize handleIntrinsicByApplyingToShadow by adding bitcasting (#123474)

`handleIntrinsicByApplyingToShadow` (introduced in
https://github.com/llvm/llvm-project/pull/114490) requires that the
intrinsic supports integer-ish operands; this is not the case for all
intrinsics. This patch generalizes the function to bitcast the shadow
arguments to be the same type as the original intrinsic, thus
guaranteeing that the intrinsic exists. Additionally, it casts the
computed shadow to be an appropriate shadow type.

This function assumes that the intrinsic will handle arbitrary
bit-patterns (for example, if the intrinsic accepts floats for var1, we
assume that it works normally even if inputs are NaNs etc.).

show more ...

[IR] Replace of PointerType::get(Type) with opaque version (NFC) (#123617)

In accordance with https://github.com/llvm/llvm-project/issues/123569

In order to keep the patch at reasonable size, this

[IR] Replace of PointerType::get(Type) with opaque version (NFC) (#123617)

In accordance with https://github.com/llvm/llvm-project/issues/123569

In order to keep the patch at reasonable size, this PR only covers for
the llvm subproject, unittests excluded.

show more ...

[msan] Add debugging for handleUnknownIntrinsic (#123381)

This adds an experimental flag, msan-dump-strict-intrinsics (modeled
after msan-dump-strict-instructions), which prints out any intrinsics

[msan] Add debugging for handleUnknownIntrinsic (#123381)

This adds an experimental flag, msan-dump-strict-intrinsics (modeled
after msan-dump-strict-instructions), which prints out any intrinsics
that are heuristically handled. Additionally, MSan will print out
heuristically handled intrinsics when -debug is passed as a flag in
debug builds.

MSan's intrinsic handling can be broken down into:

1) special cases (usually highly accurate)
2) heuristic handling (sometimes erroneous)
3) not handled

This patch's -msan-dump-strict-intrinsics is intended to help debug Case
2. Case 3) (which includes all the heuristics that are not handled by
special cases nor heuristics) can be debugged using the existing
-msan-dump-strict-instructions.

show more ...

[IRBuilder] Add Align argument for CreateMaskedExpandLoad and CreateMaskedCompressStore (#122878)

This patch adds possibility to specify alignment for
llvm.masked.expandload/llvm.masked.compresssto

[IRBuilder] Add Align argument for CreateMaskedExpandLoad and CreateMaskedCompressStore (#122878)

This patch adds possibility to specify alignment for
llvm.masked.expandload/llvm.masked.compressstore intrinsics in IRBuilder
(this is mostly NFC for now since it's only used in MemorySanitizer, but
there is an intention to generate these intrinsics in the compiler
passes, e.g. in LoopVectorizer)

show more ...

[msan] Add support for avx_round_pd_256/avx_round_ps_256 (#119334)

Add support for avx_round_pd_256/avx_round_ps_256.
This is a follow-up to https://github.com/llvm/llvm-project/pull/118441

Test

[msan] Add support for avx_round_pd_256/avx_round_ps_256 (#119334)

Add support for avx_round_pd_256/avx_round_ps_256.
This is a follow-up to https://github.com/llvm/llvm-project/pull/118441

Test plan:
ninja check-all

show more ...

[msan] Support NEON vector multiplication instructions (#117944)

Approximates the shadow propagation via OR'ing.

Updates the neon_vmul.ll test introduced in
https://github.com/llvm/llvm-project/

[msan] Support NEON vector multiplication instructions (#117944)

Approximates the shadow propagation via OR'ing.

Updates the neon_vmul.ll test introduced in
https://github.com/llvm/llvm-project/pull/117935

show more ...

[Instrumentation] Fix a warning

This patch fixes:

llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp:3840:14:
error: unused variable 'NumArgOperands' [-Werror,-Wunused-variable]

[msan] Add handling for sse41_round_pd/sse41_round_ps (#118441)

Add handling for sse41_round_pd/sse41_round_ps similarly to
maybeHandleSimpleNomemIntrinsic.

Test plan: ninja check-all

[nfc][MSan] Change for-loop to ArgNo instead of drop_begin (#117553)

As discussed in
https://github.com/llvm/llvm-project/pull/109284#discussion_r1838830571
Changed for loop to use `ArgNo` instead

[nfc][MSan] Change for-loop to ArgNo instead of drop_begin (#117553)

As discussed in
https://github.com/llvm/llvm-project/pull/109284#discussion_r1838830571
Changed for loop to use `ArgNo` instead of `drop_begin` to keep loop
code consistent with other helpers.

Co-authored-by: Kamil Kashapov <kashapov@ispras.ru>

show more ...

[msan] Add 32-bit platforms support (#109284)

References https://github.com/llvm/llvm-project/issues/103057

Added `VAArgHelper` functions for platforms: ARM32, i386, RISC-V,
PowerPC32, MIPS32.

[msan] Add 32-bit platforms support (#109284)

References https://github.com/llvm/llvm-project/issues/103057

Added `VAArgHelper` functions for platforms: ARM32, i386, RISC-V,
PowerPC32, MIPS32.

ARM, RISCV and MIPS share similar conventions regarding va args.
Therefore `VAArgGenericHelper` was introduced to avoid code duplication.

---------

Co-authored-by: Kamil Kashapov <kashapov@ispras.ru>
Co-authored-by: Vitaly Buka <vitalybuka@google.com>

show more ...

[msan] Remove unnecacary zero increment (#116185)

[nfc][msan] Move VarArgGenericHelper

Part of #109284

[nfc][msan] Remove 64 from VarArg*Helper names

Part of #109284

[nfc][msan] Reorder ifs in CreateVarArgHelper

Part of #109284