[VPlan] Hook IR blocks into VPlan during skeleton creation (NFC) (#114292)

As a first step to move towards modeling the full skeleton in VPlan,
start by wrapping IR blocks created during legacy ske

[VPlan] Hook IR blocks into VPlan during skeleton creation (NFC) (#114292)

As a first step to move towards modeling the full skeleton in VPlan,
start by wrapping IR blocks created during legacy skeleton creation in
VPIRBasicBlocks and hook them into the VPlan. This means the skeleton
CFG is represented in VPlan, just before execute. This allows moving
parts of skeleton creation into recipes in the VPBBs gradually.

Note that this allows retiring some manual DT updates, as this will be
handled automatically during VPlan execution.

PR: https://github.com/llvm/llvm-project/pull/114292

show more ...

[VPlan] Compare APInt instead of getSExtValue to fix crash in unroll.

getSExtValue assumes the result fits in 64 bits, but this may not be the
case for indcutions with wider types. Instead, directly

[VPlan] Compare APInt instead of getSExtValue to fix crash in unroll.

getSExtValue assumes the result fits in 64 bits, but this may not be the
case for indcutions with wider types. Instead, directly perform the
compare on the APInt for the ConstantInt.

Fixes https://github.com/llvm/llvm-project/issues/118850.

show more ...

[Vectorize] Simplify code with DenseMap::operator[] (NFC) (#115635)

[Vectorize] Remove unused includes (NFC) (#114643)

Identified with misc-include-cleaner.

[VPlan] Introduce scalar loop header in plan, remove VPLiveOut. (#109975)

Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handl

[VPlan] Introduce scalar loop header in plan, remove VPLiveOut. (#109975)

Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handled by adding
operands to the wrapped phis in the scalar loop header.

Note that the current version only includes the scalar loop header, no
other loop blocks and also does not wrap it in a region block.

PR: https://github.com/llvm/llvm-project/pull/109975

show more ...

[LV][VPlan] Use VF VPValue in VPVectorPointerRecipe (#110974)

Refactors VPVectorPointerRecipe to use the VF VPValue to obtain the
runtime VF, similar to #95305.

Since only reverse vector pointer

[LV][VPlan] Use VF VPValue in VPVectorPointerRecipe (#110974)

Refactors VPVectorPointerRecipe to use the VF VPValue to obtain the
runtime VF, similar to #95305.

Since only reverse vector pointers require the runtime VF, the patch
sets VPUnrollPart::PartOpIndex to 1 for vector pointers and 2 for
reverse vector pointers. As a result, the generation of reverse vector
pointers is moved into a separate recipe.

show more ...

[VPlan] Remove duplicated ExtractFromEnd handling from unoll (NFC).

ExtractFromEnd is already handled earlier, remove duplicated code.

[VPlan] Run DCE after unrolling.

This cleans up a number of dead recipes after unrolling if only their
first or last parts are used. This simplifies a number of tests.

Fixes https://github.com/llvm

[VPlan] Run DCE after unrolling.

This cleans up a number of dead recipes after unrolling if only their
first or last parts are used. This simplifies a number of tests.

Fixes https://github.com/llvm/llvm-project/issues/109581.

show more ...

[VPlan] Implement unrolling as VPlan-to-VPlan transform. (#95842)

This patch implements explicit unrolling by UF as VPlan transform. In
follow up patches this will allow simplifying VPTransform st

[VPlan] Implement unrolling as VPlan-to-VPlan transform. (#95842)

This patch implements explicit unrolling by UF as VPlan transform. In
follow up patches this will allow simplifying VPTransform state (no need
to store unrolled parts) as well as recipe execution (no need to
generate code for multiple parts in an each recipe). It also allows for
more general optimziations (e.g. avoid generating code for recipes that
are uniform-across parts).

It also unifies the logic dealing with unrolled parts in a single place,
rather than spreading it out across multiple places (e.g. VPlan post
processing for header-phi recipes previously.)

In the initial implementation, a number of recipes still take the
unrolled part as additional, optional argument, if their execution
depends on the unrolled part.

The computation for start/step values for scalable inductions changed
slightly. Previously the step would be computed as scalar and then
splatted, now vscale gets splatted and multiplied by the step in a
vector mul.

This has been split off https://github.com/llvm/llvm-project/pull/94339
which also includes changes to simplify VPTransfomState and recipes'
::execute.

The current version mostly leaves existing ::execute untouched and
instead sets VPTransfomState::UF to 1.

A follow-up patch will clean up all references to VPTransformState::UF.

Another follow-up patch will simplify VPTransformState to only store a
single vector value per VPValue.

PR: https://github.com/llvm/llvm-project/pull/95842

show more ...