[IR][unittests] Replace of PointerType::getUnqual(Type) with opaque version (NFC) (#123901)

Follow up to https://github.com/llvm/llvm-project/issues/123569

[PatternMatch] Add `m_c_XorLike` matcher; NFC (#122642)

`m_c_XorLike` matches either:
`(xor L, R)`, `(xor R, L)`, or `(sub nuw R, L)` iff `R.isMask()`.

This is in preperation for dropping th

[PatternMatch] Add `m_c_XorLike` matcher; NFC (#122642)

`m_c_XorLike` matches either:
`(xor L, R)`, `(xor R, L)`, or `(sub nuw R, L)` iff `R.isMask()`.

This is in preperation for dropping the fold from:
`(sub C_Mask, X)` -> `(xor X, C_Mask)`

show more ...

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key p

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key pieces of infrastructure that must be updated to match a CmpInst
respecting samesign information. Implement this change to Cmp-matchers.

This is a preparatory step in migrating the codebase over to
CmpPredicate. Since we no functional changes are desired at this stage,
we have chosen not to migrate CmpPredicate::operator==(CmpPredicate)
calls to use CmpPredicate::getMatching(), as that would have visible
impact on tests that are not yet written: instead, we call
CmpPredicate::operator==(Predicate), preserving the old behavior, while
also inserting a few FIXME comments for follow-ups.

show more ...

[InstCombine] Extend fcmp+select folding to minnum/maxnum intrinsics (#112088)

Today, InstCombine can fold fcmp+select patterns to minnum/maxnum
intrinsics when the nnan and nsz flags are set. The

[InstCombine] Extend fcmp+select folding to minnum/maxnum intrinsics (#112088)

Today, InstCombine can fold fcmp+select patterns to minnum/maxnum
intrinsics when the nnan and nsz flags are set. The ordering of the
operands in both the fcmp and select instructions is important for the
folding to occur.

maxnum patterns:
1. (a op b) ? a : b -> maxnum(a, b), where op is one of {ogt, oge}
2. (a op b) ? b : a -> maxnum(a, b), where op is one of {ule, ult}

The second pattern is supposed to make the order of the operands in the
select instruction irrelevant. However, the pattern matching code uses
the CmpInst::getInversePredicate method to invert the comparison
predicate. This method doesn't take into account the fast-math flags,
which can lead missing the folding opportunity.

The patch extends the pattern matching code to handle unordered fcmp
instructions. This allows the folding to occur even when the select
instruction has the operands in the inverse order.

New maxnum patterns:
1. (a op b) ? a : b -> maxnum(a, b), where op is one of {ugt, uge}
2. (a op b) ? b : a -> maxnum(a, b), where op is one of {ole, olt}

The same changes are applied to the minnum intrinsic.

show more ...

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is a

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).

show more ...

[PatternMatchTest] Use APInt::getAllOnes() (NFC)

Split out from https://github.com/llvm/llvm-project/pull/80309 to
avoid assertion failures in the future.

[PatternMatch] Use `m_SpecificCmp` matchers. NFC. (#100878)

Compile-time improvement:
http://llvm-compile-time-tracker.com/compare.php?from=13996378d81c8fa9a364aeaafd7382abbc1db83a&to=861ffa4ec5f7b

[PatternMatch] Use `m_SpecificCmp` matchers. NFC. (#100878)

Compile-time improvement:
http://llvm-compile-time-tracker.com/compare.php?from=13996378d81c8fa9a364aeaafd7382abbc1db83a&to=861ffa4ec5f7bde5a194a7715593a1b5359eb581&stat=instructions:u
baseline: 803eaf29267c6aae9162d1a83a4a2ae508b440d3
```
Top 5 improvements:
stockfish/movegen.ll 2541620819 2538599412 -0.12%
minetest/profiler.cpp.ll 431724935 431246500 -0.11%
abc/luckySwap.c.ll 581173720 580581935 -0.10%
abc/kitTruth.c.ll 2521936288 2519445570 -0.10%
abc/extraUtilTruth.c.ll 1216674614 1215495502 -0.10%
Top 5 regressions:
openssl/libcrypto-shlib-sm4.ll 1155054721 1155943201 +0.08%
openssl/libcrypto-lib-sm4.ll 1155054838 1155943063 +0.08%
spike/vsm4r_vv.ll 1296430080 1297039258 +0.05%
spike/vsm4r_vs.ll 1312496906 1313093460 +0.05%
nuttx/lib_rand48.c.ll 126201233 126246692 +0.04%
Overall: -0.02112308%
```

show more ...

Recommit "[PatternMatch] Fix issue of stale reference in new `m_{I,F,}Cmp` matchers" (3rd Try) (#99292)

The first fix forgot to fixup the commutative matchers...

Revert "[PatternMatch] Fix issue of stale reference in new `m_{I,F,}Cmp` matchers" (#99062)

Reverts llvm/llvm-project#98866

It's still use-after-scope.

[PatternMatch] Fix issue of stale reference in new `m_{I,F,}Cmp` matchers (#98866)

The new matchers don't output pred. Previously we where just creating
a value on the stack and using it as a dummy

[PatternMatch] Fix issue of stale reference in new `m_{I,F,}Cmp` matchers (#98866)

The new matchers don't output pred. Previously we where just creating
a value on the stack and using it as a dummy output for the matchers,
but this results in a stale reference upon return.

To fix, this patch changes the output variable to a pointer, and
passes in `nullptr` for the matchers that don't output `pred.`

show more ...

[Pattern] Disable a new buggy test

Introduced with #98282

[Pattern] Disabled new buggy test

Introduced with #98282

[PatternMatch] Add matchers for `m_{I,F,}Cmp` and `m_{I,F,}SpecificCmp`; NFC

These matchers either take no predicate argument or match a specific
predicate respectively.

We have a lot of cases wher

[PatternMatch] Add matchers for `m_{I,F,}Cmp` and `m_{I,F,}SpecificCmp`; NFC

These matchers either take no predicate argument or match a specific
predicate respectively.

We have a lot of cases where the Pred argument is either unused and
requiring the argument reduces code clarity.

Likewise we have a lot of cases where we only pass in Pred to test
equality which the new `*Specific*` helpers can simplify.

Closes #98282

show more ...

[IR][PatternMatch] Make `m_Checked{Int,Fp}` accept `Constant *` output instead of `APInt *`

The `APInt *` version is pretty useless as any case one needs an
`APInt *` out, they could just replace wh

[IR][PatternMatch] Make `m_Checked{Int,Fp}` accept `Constant *` output instead of `APInt *`

The `APInt *` version is pretty useless as any case one needs an
`APInt *` out, they could just replace whatever they have the
`m_Checked...` lambda with direct checks on the `APInt`.

Leaving other helpers such as `m_Negative`, `m_Power2`,
etc... unchanged as the `APInt` out version is used mostly for
convenience and rarely change functionality when converted output a
`Constant *`.

Closes #91377

show more ...

[PatternMatching] Add generic API for matching constants using custom conditions

The new API is:
`m_CheckedInt(Lambda)`/`m_CheckedFp(Lambda)`
- Matches non-undef constants s.t `Lambda(el

[PatternMatching] Add generic API for matching constants using custom conditions

The new API is:
`m_CheckedInt(Lambda)`/`m_CheckedFp(Lambda)`
- Matches non-undef constants s.t `Lambda(ele)` is true for all
elements.
`m_CheckedIntAllowUndef(Lambda)`/`m_CheckedFpAllowUndef(Lambda)`
- Matches constants/undef s.t `Lambda(ele)` is true for all
elements.

The goal with these is to be able to replace the common usage of:
```
match(X, m_APInt(C)) && CustomCheck(C)
```
with
```
match(X, m_CheckedInt(C, CustomChecks);
```

The rationale if we often ignore non-splat vectors because there are
no good APIs to handle them with and its not worth increasing code
complexity for such cases.

The hope is the API creates a common method handling
scalars/splat-vecs/non-splat-vecs to essentially make this a
non-issue.

show more ...

[InstCombine] Fix poison propagation in select of bitwise fold (#89701)

We're replacing the select with the false value here, but it may be more
poisonous if m_Not contains poison elements. Fix thi

[InstCombine] Fix poison propagation in select of bitwise fold (#89701)

We're replacing the select with the false value here, but it may be more
poisonous if m_Not contains poison elements. Fix this by introducing a
m_NotForbidPoison matcher and using it here.

Fixes https://github.com/llvm/llvm-project/issues/89500.

show more ...

[IR][PatternMatch] Only accept poison in getSplatValue() (#89159)

In #88217 a large set of matchers was changed to only accept poison
values in splats, but not undef values. This is because we now

[IR][PatternMatch] Only accept poison in getSplatValue() (#89159)

In #88217 a large set of matchers was changed to only accept poison
values in splats, but not undef values. This is because we now use
poison for non-demanded vector elements, and allowing undef can cause
correctness issues.

This patch covers the remaining matchers by changing the AllowUndef
parameter of getSplatValue() to AllowPoison instead. We also carry out
corresponding renames in matchers.

As a followup, we may want to change the default for things like m_APInt
to m_APIntAllowPoison (as this is much less risky when only allowing
poison), but this change doesn't do that.

There is one caveat here: We have a single place
(X86FixupVectorConstants) which does require handling of vector splats
with undefs. This is because this works on backend constant pool
entries, which currently still use undef instead of poison for
non-demanded elements (because SDAG as a whole does not have an explicit
poison representation). As it's just the single use, I've open-coded a
getSplatValueAllowUndef() helper there, to discourage use in any other
places.

show more ...

[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)

Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elem

[PatternMatch] Do not accept undef elements in m_AllOnes() and friends (#88217)

Change all the cstval_pred_ty based PatternMatch helpers (things like
m_AllOnes and m_Zero) to only allow poison elements inside vector
splats, not undef elements.

Historically, we used to represent non-demanded elements in vectors
using undef. Nowadays, we use poison instead. As such, I believe that
support for undef in vector splats is no longer useful.

At the same time, while poison splat elements are pretty much always
safe to ignore, this is not generally the case for undef elements. We
have existing miscompiles in our tests due to this (see the
masked-merge-*.ll tests changed here) and it's easy to miss such cases
in the future, now that we write tests using poison instead of undef
elements.

I think overall, keeping support for undef elements no longer makes
sense, and we should drop it. Once this is done consistently, I think we
may also consider allowing poison in m_APInt by default, as doing that
change is much less risky than doing the same with undef.

This change involves a substantial amount of test changes. For most
tests, I've just replaced undef with poison, as I don't think there is
value in retaining both. For some tests (where the distinction between
undef and poison is important), I've duplicated tests.

show more ...

[IR] Add commutable matcher for `add nuw`; NFC

Closes #87179

[IR] Add `m_c_BitwiseLogic` in pattern match; NFC

Just a missing matcher that came up in #73362

Closes #86632

[InstCombine] Improve mask detection in `foldICmpWithLowBitMaskedVal`

Make recursive matcher that is able to detect a lot more patterns.
Proofs for all supported patterns: https://alive2.llvm.org/ce

[InstCombine] Improve mask detection in `foldICmpWithLowBitMaskedVal`

Make recursive matcher that is able to detect a lot more patterns.
Proofs for all supported patterns: https://alive2.llvm.org/ce/z/fSQ3nZ

Differential Revision: https://reviews.llvm.org/D159058

show more ...

[PatternMatch] Add m_PtrAdd() matcher (NFC)

This matches a getelementptr i8 instruction or constant expression,
with a given pointer operand and index.

[PatternMatch] Add a matching helper `m_ElementWiseBitCast`. NFC. (#80764)

This patch introduces a matching helper `m_ElementWiseBitCast`, which is
used for matching element-wise int <-> fp casts.

[PatternMatch] Add a matching helper `m_ElementWiseBitCast`. NFC. (#80764)

This patch introduces a matching helper `m_ElementWiseBitCast`, which is
used for matching element-wise int <-> fp casts.
The motivation of this patch is to avoid duplicating checks in
https://github.com/llvm/llvm-project/pull/80740 and
https://github.com/llvm/llvm-project/pull/80414.

show more ...

[llvm] Remove uses of Type::getPointerTo() (NFC)

Partial progress towards removing in-tree uses of `getPointerTo()`,
by employing the following options:

* Drop the call entirely if the sole purpose

[llvm] Remove uses of Type::getPointerTo() (NFC)

Partial progress towards removing in-tree uses of `getPointerTo()`,
by employing the following options:

* Drop the call entirely if the sole purpose of it is to support
a no-op bitcast (remove the no-op bitcast as well).

* Replace with `PointerType::get()`/`PointerType::getUnqual()`.

Also, remove no-op function `EmitBitCastOfLValueToProperType()`.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D154392

show more ...

[PatternMatch] Don't require DataLayout for m_VScale()

The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I beli

[PatternMatch] Don't require DataLayout for m_VScale()

The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I believe it is sufficient to check for the canonical
<vscale x 1 x i8> form here -- I don't think there's a need to
recognize exotic variations like <vscale x 1 x i4> as a vscale
constant representation as well.

Differential Revision: https://reviews.llvm.org/D144566

show more ...