[InstCombine] recognizeBSwapOrBitReverseIdiom - add vector support

Add basic vector handling to recognizeBSwapOrBitReverseIdiom/collectBitParts - this works at the element level, all vector element

[InstCombine] recognizeBSwapOrBitReverseIdiom - add vector support

Add basic vector handling to recognizeBSwapOrBitReverseIdiom/collectBitParts - this works at the element level, all vector element operations must match (splat constants etc.) and there is no cross-element support (insert/extract/shuffle etc.).

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - use generic CreateIntegerCast

Try to appease buildbots breakages due to D88578

[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191) (Reapplied)

If we're bswap'ing some bytes and zero'ing the remainder we can perform this as a bswap+mas

[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191) (Reapplied)

If we're bswap'ing some bytes and zero'ing the remainder we can perform this as a bswap+mask which helps us match 'partial' bswaps as a first step towards folding into a more complex bswap pattern.

Reapplied with early-out if recognizeBSwapOrBitReverseIdiom collects a source wider than the result type.

Differential Revision: https://reviews.llvm.org/D88578

show more ...

Revert rG3d14a1e982ad27 - "[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191)"

This reverts commit 3d14a1e982ad27111346471564d575ad5efc6419.

This is brea

Revert rG3d14a1e982ad27 - "[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191)"

This reverts commit 3d14a1e982ad27111346471564d575ad5efc6419.

This is breaking on some 2stage clang buildbots

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191)

If we're bswap'ing some bytes and zero'ing the remainder we can perform this as a bswap+mask which help

[InstCombine] recognizeBSwapOrBitReverseIdiom - support for 'partial' bswap patterns (PR47191)

If we're bswap'ing some bytes and zero'ing the remainder we can perform this as a bswap+mask which helps us match 'partial' bswaps as a first step towards folding into a more complex bswap pattern.

Differential Revision: https://reviews.llvm.org/D88578

show more ...

[InstCombine] collectBitParts - convert to use PatterMatch matchers and avoid IntegerType casts.

Make sure we're using getScalarSizeInBits instead of cast<IntegerType> to get Type bit widths.

This

[InstCombine] collectBitParts - convert to use PatterMatch matchers and avoid IntegerType casts.

Make sure we're using getScalarSizeInBits instead of cast<IntegerType> to get Type bit widths.

This is preliminary cleanup before we can start adding vector support to the bswap/bitreverse (element level) matching.

show more ...

[InstCombine] collectBitParts - use APInt directly to check for out of range bit shifts. NFCI.

[InstCombine] recognizeBSwapOrBitReverseIdiom - merge the regular/trunc+zext paths. NFCI.

There doesn't seem to be any good reason for having a separate path for when we bswap/bitreverse at a smalle

[InstCombine] recognizeBSwapOrBitReverseIdiom - merge the regular/trunc+zext paths. NFCI.

There doesn't seem to be any good reason for having a separate path for when we bswap/bitreverse at a smaller size than the destination size - so merge these to make the instruction generation a lot clearer.

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - remove unnecessary cast. NFCI.

[InstCombine] recognizeBSwapOrBitReverseIdiom - cleanup bswap/bitreverse detection loop. NFCI.

Early out if both pattern matches have failed (or we don't want them). Fix case of bit index iterator (

[InstCombine] recognizeBSwapOrBitReverseIdiom - cleanup bswap/bitreverse detection loop. NFCI.

Early out if both pattern matches have failed (or we don't want them). Fix case of bit index iterator (and avoid Wshadow issue).

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - use ArrayRef::back() helper. NFCI.

Post-commit feedback on D88316

InstCombine] collectBitParts - cleanup variable names. NFCI.

Fix a number of WShadow warnings (I was used as the instruction and index......) and fix cases to match style.

Also, replaced the Bit AP

InstCombine] collectBitParts - cleanup variable names. NFCI.

Fix a number of WShadow warnings (I was used as the instruction and index......) and fix cases to match style.

Also, replaced the Bit APInt mask check in AND instructions with a direct APInt[] bit check.

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - recognise zext(bswap(trunc(x))) patterns (PR39793)

PR39793 demonstrated an issue where we fail to recognize 'partial' bswap patterns of the lower byte

[InstCombine] recognizeBSwapOrBitReverseIdiom - recognise zext(bswap(trunc(x))) patterns (PR39793)

PR39793 demonstrated an issue where we fail to recognize 'partial' bswap patterns of the lower bytes of an integer source.

In fact, most of this is already in place collectBitParts suitably tags zero bits, so we just need to correctly handle this case by finding the zero'd upper bits and reducing the bswap pattern just to the active demanded bits.

Differential Revision: https://reviews.llvm.org/D88316

show more ...

[InstCombine] recognizeBSwapOrBitReverseIdiom - assert for correct bit providence indices. NFCI.

As suggested by @spatel on D88316

[InstCombine] collectBitParts - add fshl/fshr handling

Pulled from D87452, this is a fixed version of the collectBitParts fshl/fshr handling which as @nikic noticed wasn't checking for different pro

[InstCombine] collectBitParts - add fshl/fshr handling

Pulled from D87452, this is a fixed version of the collectBitParts fshl/fshr handling which as @nikic noticed wasn't checking for different providers or had correct bit ordering (which was hid by only testing shift amounts of bitwidth/2).

Differential Revision: https://reviews.llvm.org/D88292

show more ...

[Local] Clean up enforceKnownAlignment() (NFC)

I want to export this function, and the current API was a bit
weird: It took an additional Alignment argument that didn't really
have anything to do wi

[Local] Clean up enforceKnownAlignment() (NFC)

I want to export this function, and the current API was a bit
weird: It took an additional Alignment argument that didn't really
have anything to do with what the function does. Drop it, and
perform a max at the callsite.

Also rename it to tryEnforceAlignment().

show more ...

[NFC] EliminateDuplicatePHINodes(): small-size optimization: if there are <= 32 PHI's, O(n^2) algo is faster (geomean -0.08%)

This is functionally equivalent to the old implementation.

As per https

[NFC] EliminateDuplicatePHINodes(): small-size optimization: if there are <= 32 PHI's, O(n^2) algo is faster (geomean -0.08%)

This is functionally equivalent to the old implementation.

As per https://llvm-compile-time-tracker.com/compare.php?from=5f4e9bf6416e45eba483a4e5e263749989fdb3b3&to=4739e6e4eb54d3736e6457249c0919b30f6c855a&stat=instructions
this is a clear geomean compile-time regression-free win with overall geomean of `-0.08%`

32 PHI's appears to be the sweet spot; both the 16 and 64 performed worse:
https://llvm-compile-time-tracker.com/compare.php?from=5f4e9bf6416e45eba483a4e5e263749989fdb3b3&to=c4efe1fbbfdf0305ac26cd19eacb0c7774cdf60e&stat=instructions
https://llvm-compile-time-tracker.com/compare.php?from=5f4e9bf6416e45eba483a4e5e263749989fdb3b3&to=e4989d1c67010d3339d1a40ff5286a31f10cfe82&stat=instructions

If we have more PHI's than that, we fall-back to the original DenseSet-based implementation,
so the not-so-fast cases will still be handled.

However compile-time isn't the main motivation here.
I can name at least 3 limitations of this CSE:
1. Assumes that all PHI nodes have incoming basic blocks in the same order (can be fixed while keeping the DenseMap)
2. Does not special-handle `undef` incoming values (i don't see how we can do this with hashing)
3. Does not special-handle backedge incoming values (maybe can be fixed by hashing backedge as some magical value)

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D87408

show more ...

collectBitParts - use const references. NFCI.

Fixes clang-tidy warnings first noticed on D87452.

Fix return status of SimplifyCFG

When a switch case is folded into default's case, that's an IR change that
should be reported, update ConstantFoldTerminator accordingly.

Differential Revision: htt

Fix return status of SimplifyCFG

When a switch case is folded into default's case, that's an IR change that
should be reported, update ConstantFoldTerminator accordingly.

Differential Revision: https://reviews.llvm.org/D87142

show more ...

[NFC][Local] EliminateDuplicatePHINodes(): add STATISTIC()

[NFCI][Local] Rewrite EliminateDuplicatePHINodes to optionally check hashing invariants

EarlyCSE has a mode to verify the invariant that hash equality equals
key equality, but EliminateDuplicatePHIN

[NFCI][Local] Rewrite EliminateDuplicatePHINodes to optionally check hashing invariants

EarlyCSE has a mode to verify the invariant that hash equality equals
key equality, but EliminateDuplicatePHINodes() doesn't.

I've verified that this would have caught the stage2-stage3 mismatches
5ec2b757cc7d37ff0d03b36ee863b0962fe78108 revert has fixed,
that were introduced last time in 3e69871ab5a66fb55913a2a2f5e7f5b42899a4c9.

show more ...

[Instruction] Speculatively undo isIdenticalToWhenDefined() PHI handling changes

The stage2-stage3 differences persist even without instcombine-based
PHI CSE, so this is the only possible reason.

[InstCombine] Fix incorrect Modified status

When removing instructions from unreachable blocks, and only debug info
intrinsics were removed, InstCombine could incorrectly return a false
Modified sta

[InstCombine] Fix incorrect Modified status

When removing instructions from unreachable blocks, and only debug info
intrinsics were removed, InstCombine could incorrectly return a false
Modified status.

This is fixed by making removeAllNonTerminatorAndEHPadInstructions()
also return how many debug info intrinsics that were removed, and take
that into account.

This was caught using the check introduced by D80916.

Reviewed By: majnemer

Differential Revision: https://reviews.llvm.org/D85839

show more ...

[NFC] Move findAllocaForValue into ValueTracking.h

Differential Revision: https://reviews.llvm.org/D84616

CodeGenPrep: remove AssertingVH references before deleting dead instructions.

CodeGenPrepare keeps fairly close track of various instructions it's
seen, particularly GEPs, in maps and vectors. Howev

CodeGenPrep: remove AssertingVH references before deleting dead instructions.

CodeGenPrepare keeps fairly close track of various instructions it's
seen, particularly GEPs, in maps and vectors. However, sometimes those
instructions become dead and get removed while it's still executing.
This triggers AssertingVH references to them in an asserts build and
could lead to miscompiles in a release build (I've only seen a later
segfault though).

So this patch adds a callback to
RecursivelyDeleteTriviallyDeadInstructions which can make sure the
instruction about to be deleted is removed from CodeGenPrepare's data
structures.

show more ...