[LLVM][IR] Use splat syntax when printing Constant[Data]Vector. (#112548)

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

The idea behind this canonicalization is that it allows us to handle less
patterns, because we know that some will be can

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

The idea behind this canonicalization is that it allows us to handle less
patterns, because we know that some will be canonicalized away. This is
indeed very useful to e.g. know that constants are always on the right.

However, this is only useful if the canonicalization is actually
reliable. This is the case for constants, but not for arguments: Moving
these to the right makes it look like the "more complex" expression is
guaranteed to be on the left, but this is not actually the case in
practice. It fails as soon as you replace the argument with another
instruction.

The end result is that it looks like things correctly work in tests,
while they actually don't. We use the "thwart complexity-based
canonicalization" trick to handle this in tests, but it's often a
challenge for new contributors to get this right, and based on the
regressions this PR originally exposed, we clearly don't get this right
in many cases.

For this reason, I think that it's better to remove this complexity
canonicalization. It will make it much easier to write tests for
commuted cases and make sure that they are handled.

show more ...

[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)

Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elem

[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)

Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elements inside vector
splats, not undef elements.

Historically, we used to represent non-demanded elements in vectors
using undef. Nowadays, we use poison instead. As such, I believe that
support for undef in vector splats is no longer useful.

At the same time, while poison splat elements are pretty much always
safe to ignore, this is not generally the case for undef elements. We
have existing miscompiles in our tests due to this (see the
masked-merge-*.ll tests changed here) and it's easy to miss such cases
in the future, now that we write tests using poison instead of undef
elements.

I think overall, keeping support for undef elements no longer makes
sense, and we should drop it. Once this is done consistently, I think we
may also consider allowing poison in m_APInt by default, as doing that
change is much less risky than doing the same with undef.

This change involves a substantial amount of test changes. For most
tests, I've just replaced undef with poison, as I don't think there is
value in retaining both. For some tests (where the distinction between
undef and poison is important), I've duplicated tests.

show more ...

[InstCombine] Recognize more rotation patterns (#78107)

InstCombine already handles the pattern `(shl ShVal, (X & (Width - 1)))
| (lshr ShVal, ((-X) & (Width - 1)))`. Under certain circumstances, `

[InstCombine] Recognize more rotation patterns (#78107)

InstCombine already handles the pattern `(shl ShVal, (X & (Width - 1)))
| (lshr ShVal, ((-X) & (Width - 1)))`. Under certain circumstances, `X &
(Width - 1)` will be simplified to `X`. Therefore, this patch adds
support for the pattern `(shl ShVal, X) | (lshr ShVal, ((-X) & (Width -
1)))`.

Alive2: https://alive2.llvm.org/ce/z/P7JQ2V

show more ...

[InstCombine] Infer disjoint flag on Or instructions. (#72912)

The disjoint flag was recently added to IR in #72583

We already set it when we turn an add into an or. This patch sets it on Ors tha

[InstCombine] Infer disjoint flag on Or instructions. (#72912)

The disjoint flag was recently added to IR in #72583

We already set it when we turn an add into an or. This patch sets it on Ors that weren't converted from an Add.

show more ...

[InstCombine] Convert or concat to fshl if opposite or concat exists (#68502)

If there are two 'or' instructions concat variables in opposite order
and the first 'or' dominates the second one, the

[InstCombine] Convert or concat to fshl if opposite or concat exists (#68502)

If there are two 'or' instructions concat variables in opposite order
and the first 'or' dominates the second one, the second 'or' can be
optimized to fshl to rotate shift first 'or'. This can eliminate an shl
and expose more optimization opportunity for bswap/bitreverse.

show more ...

[test] Use -passes=instcombine instead of -instcombine in lots of tests. NFC

Another step moving away from the deprecated syntax of specifying
pass pipeline in opt.

Differential Revision: https://r

[test] Use -passes=instcombine instead of -instcombine in lots of tests. NFC

Another step moving away from the deprecated syntax of specifying
pass pipeline in opt.

Differential Revision: https://reviews.llvm.org/D119081

show more ...

[InstCombine] restrict funnel shift match to avoid miscompile

As noted in the post-commit discussion for:
https://reviews.llvm.org/rGabd7529625a73f405e40a63dcc446c41d51a219e

...that change exposed

[InstCombine] restrict funnel shift match to avoid miscompile

As noted in the post-commit discussion for:
https://reviews.llvm.org/rGabd7529625a73f405e40a63dcc446c41d51a219e

...that change exposed a logic hole that allows a miscompile
if the shift amount could exceed the narrow width:
https://alive2.llvm.org/ce/z/-i_CiM
https://alive2.llvm.org/ce/z/NaYz28

The restriction isn't necessary for a rotate (same operand for
both shifts), so we should adjust the matching for the shift
value as a follow-up enhancement:
https://alive2.llvm.org/ce/z/ahuuQb

show more ...

[InstCombine] add tests for funnel shift miscompile; NFC

[InstCombine] relax masking requirement for truncated funnel/rotate match

I was investigating a seemingly unrelated improvement in demanded
bits for shift-left, but that caused regressions on these

[InstCombine] relax masking requirement for truncated funnel/rotate match

I was investigating a seemingly unrelated improvement in demanded
bits for shift-left, but that caused regressions on these tests
because we were able to look through/eliminate the mask.

https://alive2.llvm.org/ce/z/Ztdr22

define i8 @src(i32 %x, i32 %y, i32 %shift) {
%and = and i32 %shift, 3
%conv = and i32 %x, 255
%shr = lshr i32 %conv, %and
%sub = sub i32 8, %and
%shl = shl i32 %y, %sub
%or = or i32 %shr, %shl
%conv2 = trunc i32 %or to i8
ret i8 %conv2
}

define i8 @tgt(i32 %x, i32 %y, i32 %shift) {
%x8 = trunc i32 %x to i8
%y8 = trunc i32 %y to i8
%shift8 = trunc i32 %shift to i8
%and = and i8 %shift8, 3
%conv2 = call i8 @llvm.fshr.i8(i8 %y8, i8 %x8, i8 %and)
ret i8 %conv2
}

declare i8 @llvm.fshr.i8(i8,i8,i8)

show more ...

[InstCombine] add tests for rotate/funnel; NFC

[InstCombine] foldSelectFunnelShift - block poison in funnel shift value

As raised by @nlopes on D90382 - if this is not a rotate then the select was blocking poison from the 'shift-by-zero' non-TVa

[InstCombine] foldSelectFunnelShift - block poison in funnel shift value

As raised by @nlopes on D90382 - if this is not a rotate then the select was blocking poison from the 'shift-by-zero' non-TVal, but a funnel shift won't - so freeze it.

show more ...

[InstCombine] foldSelectRotate - generalize to foldSelectFunnelShift

This is the last of the rotate->funnel shift InstCombine generalizations for PR46896

We still have foldGuardedRotateToFunnelShif

[InstCombine] foldSelectRotate - generalize to foldSelectFunnelShift

This is the last of the rotate->funnel shift InstCombine generalizations for PR46896

We still have foldGuardedRotateToFunnelShift to deal with in AggressiveInstCombine

Differential Revision: https://reviews.llvm.org/D90382

show more ...

[InstCombine] Add select+funnel-shift test patterns

[InstCombine] narrowFunnelShift - fold trunc/zext or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) (PR35155)

As discussed on PR35155, this extends narrowFunnelShift (recently renamed from narrowRotate)

[InstCombine] narrowFunnelShift - fold trunc/zext or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) (PR35155)

As discussed on PR35155, this extends narrowFunnelShift (recently renamed from narrowRotate) to support basic funnel shift patterns.

Unlike matchFunnelShift we don't include the computeKnownBits limitation as extracting the pattern from the zext/trunc layers should be a indicator of reasonable funnel shift codegen, in D89139 we demonstrated how to efficiently promote funnel shifts to wider types.

Differential Revision: https://reviews.llvm.org/D89542

show more ...

[InstCombine] Fix typo in narrow funnel shift test

[InstCombine] Add trunc+zext 'narrow' funnel shift tests (PR35155)

Based on the rotation equivalents in rotate.ll

Revert rG25a97c3a43d7 - "[InstCombine] visitCallInst - retain undefs in vector funnel shift amounts"

This reverts commit 25a97c3a43d7bc469ec67dd4e901a507b9b11116.

We have other constant folds that

Revert rG25a97c3a43d7 - "[InstCombine] visitCallInst - retain undefs in vector funnel shift amounts"

This reverts commit 25a97c3a43d7bc469ec67dd4e901a507b9b11116.

We have other constant folds that fold undef funnel shift amounts to 0 - so we need to be consistent.

If we end up with regressions where we lose a splat shift amount pattern we'll have to investigate other canonicalizations, but matchFunnelShift currently protects us from that.

show more ...

[InstCombine] visitCallInst - retain undefs in vector funnel shift amounts

By always performing a modulo on the shift amount constants this was causing undef amounts being replaced with zero, meanin

[InstCombine] visitCallInst - retain undefs in vector funnel shift amounts

By always performing a modulo on the shift amount constants this was causing undef amounts being replaced with zero, meaning we were losing funnel shift by splat (with undef) patterns.

Tweaked the shift amount bounds check to support (passthrough) undefs, and use Constant::mergeUndefsWith to preserve the undefs after folding.

show more ...

[InstCombine] matchFunnelShift - add support for non-uniform vectors containing undefs.

Replace m_SpecificInt with m_APIntAllowUndef to matching splats containing undefs, then use ConstantExpr::merg

[InstCombine] matchFunnelShift - add support for non-uniform vectors containing undefs.

Replace m_SpecificInt with m_APIntAllowUndef to matching splats containing undefs, then use ConstantExpr::mergeUndefsWith to merge the undefs together in the result.

The undef funnel shift amounts are getting replaced with zero later on - I'll address this in a later patch, otherwise we lose potential shift by splat value patterns.

show more ...

[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw (REAPPLIED)

If value tracking can confirm that a shift value is less than the type bitwidth then we can

[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw (REAPPLIED)

If value tracking can confirm that a shift value is less than the type bitwidth then we can more confidently fold general or(shl(a,x),lshr(b,sub(bw,x))) patterns to a funnel/rotate intrinsic pattern without causing bad codegen regressions in the backend (see D89139).

Reapplied after the shift canonicalization in rG02295e6d1a15 which removed the need to flip the shift values.

Differential Revision: https://reviews.llvm.org/D88783

show more ...

Revert rGb97093e520036f8 - "[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw"

This reverts commit b97093e520036f88c5b39e572966f1c8c387661e.

Funnel shif

Revert rGb97093e520036f8 - "[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw"

This reverts commit b97093e520036f88c5b39e572966f1c8c387661e.

Funnel shift argument commutation isn't working correctly

show more ...

[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw

If value tracking can confirm that a shift value is less than the type bitwidth then we can more confid

[InstCombine] matchFunnelShift - fold or(shl(a,x),lshr(b,sub(bw,x))) -> fshl(a,b,x) iff x < bw

If value tracking can confirm that a shift value is less than the type bitwidth then we can more confidently fold general or(shl(a,x),lshr(b,sub(bw,x))) patterns to a funnel/rotate intrinsic pattern without causing bad codegen regressions in the backend (see D89139).

Differential Revision: https://reviews.llvm.org/D88783

show more ...

[InstCombine] matchFunnelShift - support non-uniform constant vector shift amounts (PR46895)

Complete basic PR46895 fixes by refactoring D87452/D88402 to allow us to match non-uniform constant value

[InstCombine] matchFunnelShift - support non-uniform constant vector shift amounts (PR46895)

Complete basic PR46895 fixes by refactoring D87452/D88402 to allow us to match non-uniform constant values.

We still don't handle non-uniform vectors that contain undef elements, but that can wait until we have a decent generic mechanism for this.

Differential Revision: https://reviews.llvm.org/D88420

show more ...

[InstCombine] matchRotate - add support for matching general funnel shifts with constant shift amounts (PR46896)

First step towards extending the existing rotation support to full funnel shift handl

[InstCombine] matchRotate - add support for matching general funnel shifts with constant shift amounts (PR46896)

First step towards extending the existing rotation support to full funnel shift handling now that the backend legalization support has improved.

This enables us to match the shift by constant cases, which are pretty trivial to expand again if necessary.

D88420 will add non-uniform support for funnel shifts as well once its been finalized.

Differential Revision: https://reviews.llvm.org/D88834

show more ...