[VPlan] Omit zero add in VPWidenIntOrFpInductionRecipe (#119668)

I'm not sure if getStepVector was used for other things in the past
where StartIdx was non-zero, but nowadays VPWidenIntOrFpInductio

[VPlan] Omit zero add in VPWidenIntOrFpInductionRecipe (#119668)

I'm not sure if getStepVector was used for other things in the past
where StartIdx was non-zero, but nowadays VPWidenIntOrFpInductionRecipe
is the only user of it, and just passes zero to it. I presume
InstCombine was already catching this so hopefully removing this won't
affect codegen.

show more ...

[LLVM][IR] Use splat syntax when printing ConstantExpr based splats. (#116856)

This brings the printing of scalable vector constant splats inline with
their fixed length counterparts.

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

Similar to VFxUF, also add a VF VPValue to VPlan and use it to get the
runtime VF in VPWidenIntOrFpInductionRecipe. Co

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

Similar to VFxUF, also add a VF VPValue to VPlan and use it to get the
runtime VF in VPWidenIntOrFpInductionRecipe. Code for VF is only
generated if there are users of VF, to avoid unnecessary test changes.

PR: https://github.com/llvm/llvm-project/pull/95305

show more ...

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years no

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years now, and is widely used.

show more ...

[VPlan] Run VPlan optimizations on plans in native path.

Update buildVPlans (used in native path) to also run general VPlan
optimizations in another small step to align both codepaths.

[VPlan] Update HCFG builder after 72937203dd3b to fix leak.

Update buildPlainCFG to re-use the vector and latch VPBBs created as
part of the initial skeleton in 72937203dd3b.

This should fix the le

[VPlan] Update HCFG builder after 72937203dd3b to fix leak.

Update buildPlainCFG to re-use the vector and latch VPBBs created as
part of the initial skeleton in 72937203dd3b.

This should fix the leak sanitizer failure discovered by
https://lab.llvm.org/buildbot/#/builders/52/builds/619.

show more ...

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

This patch starts initial modeling of VF * UF in VPlan.
Initially, introduce a dedicated VFxUF VPValue, which is then
populated during VPla

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

This patch starts initial modeling of VF * UF in VPlan.
Initially, introduce a dedicated VFxUF VPValue, which is then
populated during VPlan::prepareToExecute. Initially, the VF * UF
applies only to the main vector loop region. Once we extend the
scope of VPlan in the future, we may want to associate different VFxUFs
with different vector loop regions (e.g. the epilogue vector loop)

This allows explicitly parameterizing recipes that rely on the
VF * UF, like the canonical induction increment. At the moment, this
mainly helps to avoid generating some duplicated calls to vscale with
scalable vectors. It should also allow using EVL as induction increments
explicitly in D99750. Referring to VF * UF is also needed in other
places that we plan to migrate to VPlan, like the minimum trip count
check during skeleton creation.

The first version creates the value for VF * UF directly in
prepareToExecute to limit the scope of the patch. A follow-on patch will
model VF * UF computation explicitly in VPlan using recipes.

Moved from Phabricator (https://reviews.llvm.org/D157322)

show more ...

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

UF * VF is loop invariant and can be computed directly in the preheader.
This prepares the code for #74761 and reduces the

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

UF * VF is loop invariant and can be computed directly in the preheader.
This prepares the code for #74761 and reduces the test changes.

show more ...

[VPlan] Support scalable vectors in outer-loop vectorization

This patch enables scalable vectors in the VPlan-native path.
If a vectorization factor is specified via loop vectorization hints,
that f

[VPlan] Support scalable vectors in outer-loop vectorization

This patch enables scalable vectors in the VPlan-native path.
If a vectorization factor is specified via loop vectorization hints,
that factor is used. If no vectorization factor is specified, but the
target preferes scalable vectorization, a scalable vectorization factor
is selected.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D157484

show more ...