[InstCombine] Avoid repeated hash lookups (NFC) (#124243)

[NFC][DebugInfo] Use iterator moveBefore at many call-sites (#123583)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and sim

[NFC][DebugInfo] Use iterator moveBefore at many call-sites (#123583)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and similar feed into
instruction insertion positions. Call-sites where that's necessary were
updated a year ago; but to ensure some type safety however, we'd like to
have all calls to moveBefore use iterators.

This patch adds a (guaranteed dereferenceable) iterator-taking
moveBefore, and changes a bunch of call-sites where it's obviously safe
to change to use it by just calling getIterator() on an instruction
pointer. A follow-up patch will contain less-obviously-safe changes.

We'll eventually deprecate and remove the instruction-pointer
insertBefore, but not before adding concise documentation of what
considerations are needed (very few).

show more ...

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key p

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key pieces of infrastructure that must be updated to match a CmpInst
respecting samesign information. Implement this change to Cmp-matchers.

This is a preparatory step in migrating the codebase over to
CmpPredicate. Since we no functional changes are desired at this stage,
we have chosen not to migrate CmpPredicate::operator==(CmpPredicate)
calls to use CmpPredicate::getMatching(), as that would have visible
impact on tests that are not yet written: instead, we call
CmpPredicate::operator==(Predicate), preserving the old behavior, while
also inserting a few FIXME comments for follow-ups.

show more ...

[InstCombine] Optimistically allow multiple shufflevector uses in foldOpPhi (#114278)

We would like to optimize situations of the form that happen after loop
vectorization+SROA:
```
loop:
%p

[InstCombine] Optimistically allow multiple shufflevector uses in foldOpPhi (#114278)

We would like to optimize situations of the form that happen after loop
vectorization+SROA:
```
loop:
%phi = phi zeroinitializer, %interleaved

%deinterleave_a = shufflevector %phi, poison ; pick half of the lanes
%deinterleave_b = shufflevector %phi, posion ; pick remaining lanes

... %a = ... %b = ...

%interleaved = shufflevector %a, %b ; interleave lanes of a+b
```
where the interleave and de-interleave shuffle operations cancel each
other out.
This could be handled by `foldOpPhi` but does not currently work because
it does
not proceed when there are multiple uses of the `Phi` operation.

This extends `foldOpPhi` to allow multiple `shufflevector` uses when
they are
shown to simplify for all `Phi` input values.

show more ...

[InstCombine] Extend folding of aggregate construction to cases when source aggregates are partially available (#100828)

Function foldAggregateConstructionIntoAggregateReuse can fold
insertvalue(

[InstCombine] Extend folding of aggregate construction to cases when source aggregates are partially available (#100828)

Function foldAggregateConstructionIntoAggregateReuse can fold
insertvalue(phi(extractvalue(src1), extractvalue(src2)))
into
phi(src1, src2)
when we can find source aggregates in all predecessors.

This patch extends it to handle following case
insertvalue(phi(extractvalue(src1), elm2))
into
phi(src1, insertvalue(elm2))
with the condition that the predecessor without source aggregate has
only one successor.

show more ...

[InstCombine] Only fold extract element to trunc if vector `hasOneUse` (#115627)

This fixes a missed optimization caused by the `foldBitcastExtElt`
pattern interfering with other combine patterns.

[InstCombine] Only fold extract element to trunc if vector `hasOneUse` (#115627)

This fixes a missed optimization caused by the `foldBitcastExtElt`
pattern interfering with other combine patterns. In the case I was
hitting, we have IR that combines two vectors into a new larger vector
by extracting elements and inserting them into the new vector.

```llvm
define <4 x half> @bitcast_extract_insert_to_shuffle(i32 %a, i32 %b) {
%avec = bitcast i32 %a to <2 x half>
%a0 = extractelement <2 x half> %avec, i32 0
%a1 = extractelement <2 x half> %avec, i32 1
%bvec = bitcast i32 %b to <2 x half>
%b0 = extractelement <2 x half> %bvec, i32 0
%b1 = extractelement <2 x half> %bvec, i32 1
%ins0 = insertelement <4 x half> undef, half %a0, i32 0
%ins1 = insertelement <4 x half> %ins0, half %a1, i32 1
%ins2 = insertelement <4 x half> %ins1, half %b0, i32 2
%ins3 = insertelement <4 x half> %ins2, half %b1, i32 3
ret <4 x half> %ins3
}
```

With the current behavior, `InstCombine` converts each vector extract
sequence to

```llvm
%tmp = trunc i32 %a to i16
%a0 = bitcast i16 %tmp to half
%a1 = extractelement <2 x half> %avec, i32 1
```

where the extraction of `%a0` is now done by truncating the original
integer. While on it's own this is fairly reasonable, in this case it
also blocks the pattern which converts `extractelement` -
`insertelement` into shuffles which gives the overall simpler result:

```llvm
define <4 x half> @bitcast_extract_insert_to_shuffle(i32 %a, i32 %b) {
%avec = bitcast i32 %a to <2 x half>
%bvec = bitcast i32 %b to <2 x half>
%ins3 = shufflevector <2 x half> %avec, <2 x half> %bvec, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
ret <4 x half> %ins3
}
```

In this PR I fix the conflict by obeying the `hasOneUse` check even if
there is no shift instruction required. In these cases we can't remove
the vector completely, so the pattern has less benefit anyway.

Also fwiw, I think dropping the `hasOneUse` check for the 0th element
might have been a mistake in the first place. Looking at
https://github.com/llvm/llvm-project/commit/535c5d56a7bc9966036a11362d8984983a4bf090
the commit message only mentions loosening the `isDesirableIntType`
requirement and doesn't mention changing the `hasOneUse` check at all.

show more ...

[InstCombine] Fix poison safety of folding shufflevector into select (#115483)

We are allowed to fold shufflevector into select iff the condition is
guaranteed not to be poison or the RHS is a pois

[InstCombine] Fix poison safety of folding shufflevector into select (#115483)

We are allowed to fold shufflevector into select iff the condition is
guaranteed not to be poison or the RHS is a poison.
Alive2: https://alive2.llvm.org/ce/z/28zEWR

Closes https://github.com/llvm/llvm-project/issues/115465.

show more ...

[InstCombine] Do not fold `shufflevector(select)` if the select condition is a vector (#113993)

Since `shufflevector` is not element-wise, we cannot do fold it into
select when the select condition

[InstCombine] Do not fold `shufflevector(select)` if the select condition is a vector (#113993)

Since `shufflevector` is not element-wise, we cannot do fold it into
select when the select condition is a vector.
For shufflevector that doesn't change the length, it doesn't crash, but
it is still a miscompilation: https://alive2.llvm.org/ce/z/s8saCx

Fixes https://github.com/llvm/llvm-project/issues/113986.

show more ...

InstCombine: Fold shufflevector(select) and shufflevector(phi) (#113746)

- Transform `shufflevector(select(c, x, y), C)` to
`select(c, shufflevector(x, C), shufflevector(y, C))` by re-using
th

InstCombine: Fold shufflevector(select) and shufflevector(phi) (#113746)

- Transform `shufflevector(select(c, x, y), C)` to
`select(c, shufflevector(x, C), shufflevector(y, C))` by re-using
the `FoldOpIntoSelect` helper.
- Transform `shufflevector(phi(x, y), C)` to
`phi(shufflevector(x, C), shufflevector(y, C))` by re-using the
`foldOpInotPhi` helper.

show more ...

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is a

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).

show more ...

[ValueTracking] Use isSafeToSpeculativelyExecuteWithVariableReplaced() in more places (#109149)

This replaces some uses of isSafeToSpeculativelyExecute() with
isSafeToSpeculativelyExecuteWithVariab

[ValueTracking] Use isSafeToSpeculativelyExecuteWithVariableReplaced() in more places (#109149)

This replaces some uses of isSafeToSpeculativelyExecute() with
isSafeToSpeculativelyExecuteWithVariableReplaced(), in cases where we
are guarding against operand changes rather plain speculation.

I believe that this is NFC with the current implementation of the
function (as it only does something different from loads), but this
makes us more defensive against future generalizations.

show more ...

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years no

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years now, and is widely used.

show more ...

[InstCombine] Fix invalid scalarization of div

If the binop is not speculatable, and the extract index is out of
range, then scalarizing will perform the operation on a poison
operand, resulting in

[InstCombine] Fix invalid scalarization of div

If the binop is not speculatable, and the extract index is out of
range, then scalarizing will perform the operation on a poison
operand, resulting in immediate UB, instead of the previous
poison result.

Fixes https://github.com/llvm/llvm-project/issues/97053.

show more ...

[VectorUtils] Add llvm::scaleShuffleMaskElts wrapper for narrowShuffleMaskElts/widenShuffleMaskElts, NFC. (#96646)

Using the target number of vector elements, scaleShuffleMaskElts will try to use na

[VectorUtils] Add llvm::scaleShuffleMaskElts wrapper for narrowShuffleMaskElts/widenShuffleMaskElts, NFC. (#96646)

Using the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly.

Working on #58895 I didn't want to create yet another case where we have to handle both re-scaling cases.

show more ...

Revert "[IR][NFC] Update IRBuilder to use InsertPosition (#96497)"

Reverts the above commit, as it updates a common header function and
did not update all callsites:

https://lab.llvm.org/buildbot

Revert "[IR][NFC] Update IRBuilder to use InsertPosition (#96497)"

Reverts the above commit, as it updates a common header function and
did not update all callsites:

https://lab.llvm.org/buildbot/#/builders/29/builds/382

This reverts commit 6481dc57612671ebe77fe9c34214fba94e1b3b27.

show more ...

[IR][NFC] Update IRBuilder to use InsertPosition (#96497)

Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock

[IR][NFC] Update IRBuilder to use InsertPosition (#96497)

Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock
alongside a BasicBlock::iterator, using the fact that we can now get the
parent basic block from the iterator even if it points to the sentinel.
This patch removes the BasicBlock argument from each constructor or call
to setInsertPoint.

This has no functional effect, but later on as we look to remove the
`Instruction *InsertBefore` argument from instruction-creation
(discussed
[here](https://discourse.llvm.org/t/psa-instruction-constructors-changing-to-iterator-only-insertion/77845)),
this will simplify the process by allowing us to deprecate the
InsertPosition constructor directly and catch all the cases where we use
instructions rather than iterators.

show more ...

[InstCombine] Use disjoint flag for alternate binops

Check the or disjoint flag instead of the weaker MaskedValueIsZero
query.

[InstCombine] Avoid use of ConstantExpr::getShl()

Use the constant folding API instead. Use ImmConstant to make
sure it actually folds.

[InstCombine] Use m_Poison() instead of m_Undef() (NFCI)

In this case the shuffle mask checks should already guarantee a
single-source shuffle, so this is just for clarity.

[InstCombine] Use m_Poison() instead of m_Undef() (NFCI)

In this case, the isIdentityWithExtract() checks should already
guarantee that these are single-source shuffles, so this is just
for clarity.

[InstCombine] Check for poison instead of undef in single shuffle fold

Otherwise we'll convert undef to poison. Alive2 was already flagging
the existing test8 test as a miscompile.

[InstCombine] Use m_Poison instead of m_Undef (NFCI)

In this case, isIdentityWithExtract() should already ensure that
this is a single-source shuffle. This just makes things more
explicit.

[InstCombine] Check for poison instead of undef in shuffle of unop transform

Otherwise this may not actually be a single-source shuffle.

[InstCombine] Use m_Poison instead of m_Undef in some places (NFCI)

I believe that in these cases other conditions already ensure that
the second operand is not used, this is mostly for clarity.

[InstCombine] Check for poison instead of undef in splat shuffle fold

We can't canonicalize these to a splat shuffle, as doing so would
convert undef -> poison.