[VPlan] Compute induction end values in VPlan. (#112145)

Use createDerivedIV to compute IV end values directly in VPlan, instead
of creating them up-front.

This allows updating IV users outside

[VPlan] Compute induction end values in VPlan. (#112145)

Use createDerivedIV to compute IV end values directly in VPlan, instead
of creating them up-front.

This allows updating IV users outside the loop as follow-up.

Depends on https://github.com/llvm/llvm-project/pull/110004 and
https://github.com/llvm/llvm-project/pull/109975.

PR: https://github.com/llvm/llvm-project/pull/112145

show more ...

[VPlan] Omit zero add in VPWidenIntOrFpInductionRecipe (#119668)

I'm not sure if getStepVector was used for other things in the past
where StartIdx was non-zero, but nowadays VPWidenIntOrFpInductio

[VPlan] Omit zero add in VPWidenIntOrFpInductionRecipe (#119668)

I'm not sure if getStepVector was used for other things in the past
where StartIdx was non-zero, but nowadays VPWidenIntOrFpInductionRecipe
is the only user of it, and just passes zero to it. I presume
InstCombine was already catching this so hopefully removing this won't
affect codegen.

show more ...

[LLVM][IR] Use splat syntax when printing ConstantExpr based splats. (#116856)

This brings the printing of scalable vector constant splats inline with
their fixed length counterparts.

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

Similar to VFxUF, also add a VF VPValue to VPlan and use it to get the
runtime VF in VPWidenIntOrFpInductionRecipe. Co

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

Similar to VFxUF, also add a VF VPValue to VPlan and use it to get the
runtime VF in VPWidenIntOrFpInductionRecipe. Code for VF is only
generated if there are users of VF, to avoid unnecessary test changes.

PR: https://github.com/llvm/llvm-project/pull/95305

show more ...

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years no

Move stepvector intrinsic out of experimental namespace (#98043)

This patch is moving out stepvector intrinsic from the experimental
namespace.

This intrinsic exists in LLVM for several years now, and is widely used.

show more ...

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

This patch starts initial modeling of VF * UF in VPlan.
Initially, introduce a dedicated VFxUF VPValue, which is then
populated during VPla

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

This patch starts initial modeling of VF * UF in VPlan.
Initially, introduce a dedicated VFxUF VPValue, which is then
populated during VPlan::prepareToExecute. Initially, the VF * UF
applies only to the main vector loop region. Once we extend the
scope of VPlan in the future, we may want to associate different VFxUFs
with different vector loop regions (e.g. the epilogue vector loop)

This allows explicitly parameterizing recipes that rely on the
VF * UF, like the canonical induction increment. At the moment, this
mainly helps to avoid generating some duplicated calls to vscale with
scalable vectors. It should also allow using EVL as induction increments
explicitly in D99750. Referring to VF * UF is also needed in other
places that we plan to migrate to VPlan, like the minimum trip count
check during skeleton creation.

The first version creates the value for VF * UF directly in
prepareToExecute to limit the scope of the patch. A follow-on patch will
model VF * UF computation explicitly in VPlan using recipes.

Moved from Phabricator (https://reviews.llvm.org/D157322)

show more ...

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

UF * VF is loop invariant and can be computed directly in the preheader.
This prepares the code for #74761 and reduces the

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

UF * VF is loop invariant and can be computed directly in the preheader.
This prepares the code for #74761 and reduces the test changes.

show more ...

[LV] Add users for loads to make tests more robust.

Update a few tests to add users to loads to avoid them being optimized
out by future changes. In cases the unused loads didn't matter for the
test

[LV] Add users for loads to make tests more robust.

Update a few tests to add users to loads to avoid them being optimized
out by future changes. In cases the unused loads didn't matter for the
test, remove them.

show more ...

[IRBuilder] Use canonical i64 type for insertelement index used by vector splats.

Instcombine prefers this canonical form (see getPreferredVectorIndex),
as does IRBuilder when passing the index as a

[IRBuilder] Use canonical i64 type for insertelement index used by vector splats.

Instcombine prefers this canonical form (see getPreferredVectorIndex),
as does IRBuilder when passing the index as an integer so we may as
well use the prefered form from creation.

NOTE: All test changes are mechanical with nothing else expected
beyond a change of index type from i32 to i64.

Differential Revision: https://reviews.llvm.org/D140983

show more ...

[LoopVectorize] Convert some tests to opaque pointers (NFC)

For these tests update_test_checks.py had to be rerun.

[NFC] Port all runlines for LoopVectorize pass tests to -passes syntax

[ValueTracking] Enable -branch-on-poison-as-ub by default

Now that SimpleLoopUnswitch and other transforms no longer introduce
branch on poison, enable the -branch-on-poison-as-ub option by
default.

[ValueTracking] Enable -branch-on-poison-as-ub by default

Now that SimpleLoopUnswitch and other transforms no longer introduce
branch on poison, enable the -branch-on-poison-as-ub option by
default. The practical impact of this is mostly better flag
preservation in SCEV, and some freeze instructions no longer being
necessary.

Differential Revision: https://reviews.llvm.org/D125299

show more ...

[VPlan] Model first exit values using VPLiveOut.

This patch introduces a new VPLiveOut subclass of VPUser to model
exit values explicitly. The initial version handles exit values that
are neither

[VPlan] Model first exit values using VPLiveOut.

This patch introduces a new VPLiveOut subclass of VPUser to model
exit values explicitly. The initial version handles exit values that
are neither part of induction or reduction chains nor first order
recurrence phis.

Fixes #51366, #54867, #55167, #55459

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D123537

show more ...

[VPlan] Avoid collecting scalars for SVE

This patch ensures scalars (except for uniforms) are no
longer collected (prior to LVP planning phase) for
scalable vectorization.

This is to avoid the chan

[VPlan] Avoid collecting scalars for SVE

This patch ensures scalars (except for uniforms) are no
longer collected (prior to LVP planning phase) for
scalable vectorization.

This is to avoid the chances of generating scalarized
instructions later (during LVP execute phase) as they
are not supported for scalable vectorization.

Relevant test has also been added.

Differential Revision: https://reviews.llvm.org/D121452

show more ...