PreISelIntrinsicLowering: Lower llvm.exp/llvm.exp2 to a loop if scalable vec arg (#117568)

Reapply [IR] Initial introduction of llvm.experimental.memset_pattern (#97583)

Relands 7ff3a9acd84654c9ec2939f45ba27f162ae7fbc3 after regenerating the
test case.

Supersedes the draft PR #94992, tak

Reapply [IR] Initial introduction of llvm.experimental.memset_pattern (#97583)

Relands 7ff3a9acd84654c9ec2939f45ba27f162ae7fbc3 after regenerating the
test case.

Supersedes the draft PR #94992, taking a different approach following
feedback:
* Lower in PreISelIntrinsicLowering
* Don't require that the number of bytes to set is a compile-time
constant
* Define llvm.memset_pattern rather than llvm.memset_pattern.inline

As discussed in the [RFC
thread](https://discourse.llvm.org/t/rfc-introducing-an-llvm-memset-pattern-inline-intrinsic/79496),
the intent is that the intrinsic will be lowered to loops, a sequence of
stores, or libcalls depending on the expected cost and availability of
libcalls on the target. Right now, there's just a single lowering path
that aims to handle all cases. My intent would be to follow up with
additional PRs that add additional optimisations when possible (e.g.
when libcalls are available, when arguments are known to be constant
etc).

show more ...

Revert "[IR] Initial introduction of llvm.experimental.memset_pattern (#97583)"

This reverts commit 7ff3a9acd84654c9ec2939f45ba27f162ae7fbc3.

Recent scheduling changes means tests need to be re-gen

Revert "[IR] Initial introduction of llvm.experimental.memset_pattern (#97583)"

This reverts commit 7ff3a9acd84654c9ec2939f45ba27f162ae7fbc3.

Recent scheduling changes means tests need to be re-generated. Reverting
to green while I do that.

show more ...

[IR] Initial introduction of llvm.experimental.memset_pattern (#97583)

Supersedes the draft PR #94992, taking a different approach following
feedback:
* Lower in PreISelIntrinsicLowering
* Don't

[IR] Initial introduction of llvm.experimental.memset_pattern (#97583)

Supersedes the draft PR #94992, taking a different approach following
feedback:
* Lower in PreISelIntrinsicLowering
* Don't require that the number of bytes to set is a compile-time
constant
* Define llvm.memset_pattern rather than llvm.memset_pattern.inline

As discussed in the [RFC
thread](https://discourse.llvm.org/t/rfc-introducing-an-llvm-memset-pattern-inline-intrinsic/79496),
the intent is that the intrinsic will be lowered to loops, a sequence of
stores, or libcalls depending on the expected cost and availability of
libcalls on the target. Right now, there's just a single lowering path
that aims to handle all cases. My intent would be to follow up with
additional PRs that add additional optimisations when possible (e.g.
when libcalls are available, when arguments are known to be constant
etc).

show more ...

[ExpandVectorPredication] Be more precise reporting changes (#102313)

This is used by PreISelIntrinsicLowering. With this change,
PreISelIntrinsicLowering does not have to assume that there were ch

[ExpandVectorPredication] Be more precise reporting changes (#102313)

This is used by PreISelIntrinsicLowering. With this change,
PreISelIntrinsicLowering does not have to assume that there were changes
just because we encountered a VP intrinsic.

show more ...

[CodeGen] Fix lower constant intrinsics for dead code (#102442)

lowerConstantIntrinsics does an RPO traveral, which doesn't reach dead
blocks. Remove the assertion that all intrinsics are lowered,

[CodeGen] Fix lower constant intrinsics for dead code (#102442)

lowerConstantIntrinsics does an RPO traveral, which doesn't reach dead
blocks. Remove the assertion that all intrinsics are lowered, because
some intrinsics might remain.

show more ...

[CodeGen] Fix PreISelLowering not reporting changes (#102184)

expandVectorPredication may change code, even if the intrinsic itself
remains in the code. Report changes whenever such an intrinsic is

[CodeGen] Fix PreISelLowering not reporting changes (#102184)

expandVectorPredication may change code, even if the intrinsic itself
remains in the code. Report changes whenever such an intrinsic is
encountered, because code could have been changed.

Another follow-up fix for #101652 to fix expensive-checks-only failure.

show more ...

[CodeGen] Allow PreISel lowering to run without TM (#102150)

Fixes #101652 after build bot failures where TM in the opt pass builder
is nullptr.

[VP] Merge ExpandVP pass into PreISelIntrinsicLowering (#101652)

Similar to #97727; avoid an extra pass over the entire IR by performing
the lowering as part of the pre-isel-intrinsic-lowering pass.

[CodeGen] Merge lowerConstantIntrinsics into pre-isel lowering (#97727)

Currently, the LowerConstantIntrinsics pass does an RPO traversal of
every function... only to find that many functions don't

[CodeGen] Merge lowerConstantIntrinsics into pre-isel lowering (#97727)

Currently, the LowerConstantIntrinsics pass does an RPO traversal of
every function... only to find that many functions don't have constant
intrinsics (is.constant, objectsize). In the CodeGen pipeline, there is
already a pre-isel intrinsic lowering pass, which iterates over
intrinsic declarations and lowers all users. Call
lowerConstantIntrinsics from this pass to avoid the extra iteration over
the entire IR and the RPO traversal.

show more ...

Reapply "[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)"

This reverts commit ac4b6b662630cd4d3bf6929f2b39ea203c0054a1.

A test change was m

Reapply "[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)"

This reverts commit ac4b6b662630cd4d3bf6929f2b39ea203c0054a1.

A test change was missing for
mlir/test/Target/LLVMIR/llvmir-intrinsics.mlir in the initial commit.

show more ...

Revert "[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)"

This reverts commit 522fd53838d577add8c19b5eccccae756fd27899 while
unexpected mlir

Revert "[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)"

This reverts commit 522fd53838d577add8c19b5eccccae756fd27899 while
unexpected mlir failures are investigated and resolved.

show more ...

[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)

Following on from the discussion in

https://discourse.llvm.org/t/rfc-introducing-an-llvm-

[Intrinsics][PreISelInstrinsicLowering] llvm.memcpy.inline length no longer needs to be constant (#98281)

Following on from the discussion in

https://discourse.llvm.org/t/rfc-introducing-an-llvm-memset-pattern-inline-intrinsic/79496
and the equivalent change for llvm.memset.inline (#95397), this removes
the requirement that the length of llvm.memcpy.inline is constant.
PreISelInstrinsicLowering will expand llvm.memcpy.inline with
non-constant lengths, while the codegen path for constant lengths is
left unaltered.

show more ...

[Intrinsics][PreISelIntrinsicLowering] llvm.memset.inline length no longer needs to be constant (#95397)

As requested in
https://discourse.llvm.org/t/rfc-introducing-an-llvm-memset-pattern-inline-i

[Intrinsics][PreISelIntrinsicLowering] llvm.memset.inline length no longer needs to be constant (#95397)

As requested in
https://discourse.llvm.org/t/rfc-introducing-an-llvm-memset-pattern-inline-intrinsic/79496
this patch removes the requirement that the length of llvm.memset.inline
is a constant, and adjusts PreISelIntrinsicLowering so it supports
expanding such the intrinsic in the case it has a non-constant length.

show more ...

Revert "[IR][NFC] Update IRBuilder to use InsertPosition (#96497)"

Reverts the above commit, as it updates a common header function and
did not update all callsites:

https://lab.llvm.org/buildbot

Revert "[IR][NFC] Update IRBuilder to use InsertPosition (#96497)"

Reverts the above commit, as it updates a common header function and
did not update all callsites:

https://lab.llvm.org/buildbot/#/builders/29/builds/382

This reverts commit 6481dc57612671ebe77fe9c34214fba94e1b3b27.

show more ...

[IR][NFC] Update IRBuilder to use InsertPosition (#96497)

Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock

[IR][NFC] Update IRBuilder to use InsertPosition (#96497)

Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock
alongside a BasicBlock::iterator, using the fact that we can now get the
parent basic block from the iterator even if it points to the sentinel.
This patch removes the BasicBlock argument from each constructor or call
to setInsertPoint.

This has no functional effect, but later on as we look to remove the
`Instruction *InsertBefore` argument from instruction-creation
(discussed
[here](https://discourse.llvm.org/t/psa-instruction-constructors-changing-to-iterator-only-insertion/77845)),
this will simplify the process by allowing us to deprecate the
InsertPosition constructor directly and catch all the cases where we use
instructions rather than iterators.

show more ...

[IRBuilder] Add CreatePtrAdd() method (NFC) (#77582)

This abstracts over the common pattern of creating a gep with i8 element
type.

Reland "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn."

This reverts commit cb62f67088aaf79493350547f74870318b71acc5.

Fixes: https://github.com/llvm/llvm-project/issues/69658

Revert "Reland "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn.""

This reverts commit 30414fc614d80a45bad4c89763a353f50d3e04d6.

Broke some buildbots.

Reland "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn."

This reverts commit cb62f67088aaf79493350547f74870318b71acc5.

Fixes: https://github.com/llvm/llvm-project/issues/69658

Revert "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn."

This reverts commit ed83797f3cbfc8fb2a1af63542f97d7ec1d5505a.

Reverting pending the investigation of https://github.com/llvm/

Revert "[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn."

This reverts commit ed83797f3cbfc8fb2a1af63542f97d7ec1d5505a.

Reverting pending the investigation of https://github.com/llvm/llvm-project/issues/69658

show more ...

[PreISelIntrinsicLowering] Use TLI for correct function

We should query the subtarget of the calling function, not of the
intrinsic.

This probably makes no functional difference (as libcalls are
un

[PreISelIntrinsicLowering] Use TLI for correct function

We should query the subtarget of the calling function, not of the
intrinsic.

This probably makes no functional difference (as libcalls are
unlikely to vary across subtargets), but fixes minor compile-time
regressions from unnecessary subtarget instantiations.

Followup to D157567.

Differential Revision: https://reviews.llvm.org/D157848

show more ...

PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions

We need a better mechanism for expressing which calls you are allowed
to emit and which calls are recognized. This

PreISelIntrinsicLowering: Check RuntimeLibcalls instead of TLI for memory functions

We need a better mechanism for expressing which calls you are allowed
to emit and which calls are recognized. This should be applied to the
17 branch.

show more ...

[llvm] Drop some typed pointer handling/bitcasts

Differential Revision: https://reviews.llvm.org/D157016

[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn.

https://clang.llvm.org/docs/AutomaticReferenceCounting.html#arc-runtime-objc-retain

rdar://79869679

Differential revision: https://re

[Intrinsics][ObjC] Mark objc_retain and friends as thisreturn.

https://clang.llvm.org/docs/AutomaticReferenceCounting.html#arc-runtime-objc-retain

rdar://79869679

Differential revision: https://reviews.llvm.org/D105671

show more ...