History log of /llvm-project/llvm/lib/Transforms/Vectorize/VPlan.cpp (Results 176 – 200 of 360)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: llvmorg-14.0.3, llvmorg-14.0.2, llvmorg-14.0.1, llvmorg-14.0.0, llvmorg-14.0.0-rc4, llvmorg-14.0.0-rc3, llvmorg-14.0.0-rc2, llvmorg-14.0.0-rc1, llvmorg-15-init, llvmorg-13.0.1, llvmorg-13.0.1-rc3, llvmorg-13.0.1-rc2, llvmorg-13.0.1-rc1, llvmorg-13.0.0, llvmorg-13.0.0-rc4
# 4e5e042d 15-Sep-2021 Igor Kirillov <Igor.Kirillov@arm.com>

[LoopVectorize] Support reductions that store intermediary result

Adds ability to vectorize loops containing a store to a loop-invariant
address as part of a reduction that isn't converted to SSA fo

[LoopVectorize] Support reductions that store intermediary result

Adds ability to vectorize loops containing a store to a loop-invariant
address as part of a reduction that isn't converted to SSA form due to
lack of aliasing info. Runtime checks are generated to ensure the store
does not alias any other accesses in the loop.

Ordered fadd reductions are not yet supported.

Differential Revision: https://reviews.llvm.org/D110235

show more ...


# e66127e6 29-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Simplify & adjust code as suggested in D123005.

Improve code as suggested in D123005. Applied separately, because the
comments where made a diff that has not been rebased to current main.


# bea69b23 20-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Initial modeling of middle block in VPlan.

This patch extends the scope of VPlan to also include the exit (aka
middle) block.

For now, the exit block remains empty, but handling of exit val

[VPlan] Initial modeling of middle block in VPlan.

This patch extends the scope of VPlan to also include the exit (aka
middle) block.

For now, the exit block remains empty, but handling of exit values will
subsequently be moved to VPlan, by adding recipes to model exit values
in the exit block.

As a first step, this will allow fixing #51366.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D123457

show more ...


# a65f2730 19-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Expand induction step in VPlan pre-header.

This patch moves SCEV expansion of steps used by
VPWidenIntOrFpInductionRecipes to the pre-header using
VPExpandSCEVRecipe. This ensures that those

[VPlan] Expand induction step in VPlan pre-header.

This patch moves SCEV expansion of steps used by
VPWidenIntOrFpInductionRecipes to the pre-header using
VPExpandSCEVRecipe. This ensures that those steps are expanded while the
CFG is in a valid state. Previously, SCEV expansion may happen during
vector body code-generation, during which the CFG may be invalid,
causing issues with SCEV expansion.

Depends on D122095.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D122096

show more ...


# 2c14cdf8 14-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Turn external defs in Value -> VPValue mapping.

This addresses an existing TODO by keeping a mapping of external IR
Value * definitions wrapped in VPValues for use in a VPlan.

Reviewed By:

[VPlan] Turn external defs in Value -> VPValue mapping.

This addresses an existing TODO by keeping a mapping of external IR
Value * definitions wrapped in VPValues for use in a VPlan.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D123700

show more ...


# 5f1eb748 10-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Place VPExpandSCEVRecipe in pre-header.

After D121624 models the pre-header in VPlan, VPExpandSCEVRecipes can be
placed there. This ensures SCEV expansion happens before modifying the
CFG du

[VPlan] Place VPExpandSCEVRecipe in pre-header.

After D121624 models the pre-header in VPlan, VPExpandSCEVRecipes can be
placed there. This ensures SCEV expansion happens before modifying the
CFG during VPlan execution, when CFG is incomplete.

Depends on D121624.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D122095

show more ...


# 256c6b0b 09-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Model pre-header explicitly.

This patch extends the scope of VPlan to also model the pre-header.
The pre-header can be used to place recipes that should be code-gen'd
outside the loop, like

[VPlan] Model pre-header explicitly.

This patch extends the scope of VPlan to also model the pre-header.
The pre-header can be used to place recipes that should be code-gen'd
outside the loop, like SCEV expansion.

Depends on D121623.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121624

show more ...


# 29fe998e 08-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Preserve debug location when creating branch.

Update createEmptyBasicBlock to preserve the debug location of the
previous terminator.


# 1ff022e2 05-Apr-2022 Florian Hahn <flo@fhahn.com>

[LV] Add vector.body block to parent loop during skeleton creation.

When creating induction resume values, SCEV queries may rely on
LoopInfo. Make sure vector.body gets added to the loop of the pre-

[LV] Add vector.body block to parent loop during skeleton creation.

When creating induction resume values, SCEV queries may rely on
LoopInfo. Make sure vector.body gets added to the loop of the pre-header
during skeleton construction.

%vector.body will be moved to the vector preheader during VPlan
execution.

Fixes #54745.

show more ...


# 1817c526 04-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Update VPInterleavedAccessInfo to use getVectorLoopRegion.

Update VPInterleavedAccessInfo to use the generic getVectorLoopRegion
helper instead of relying on the entry block being the top-m

[VPlan] Update VPInterleavedAccessInfo to use getVectorLoopRegion.

Update VPInterleavedAccessInfo to use the generic getVectorLoopRegion
helper instead of relying on the entry block being the top-most vector
loop region.

show more ...


# 8cd18927 04-Apr-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Remember previous loop and reset vector loop.

At the moment this is NFC, but will be needed once nested loops are also
modeled as regions. Preparation for D123005.


# f8101e4d 01-Apr-2022 Florian Hahn <flo@fhahn.com>

Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)"

This reverts commit 14e3650f01d158f7e4117c353927a07ceebdd504.

The issue causing the revert were fixed independently in
a08c90a4023f and 14e

Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)"

This reverts commit 14e3650f01d158f7e4117c353927a07ceebdd504.

The issue causing the revert were fixed independently in
a08c90a4023f and 14e5f9785c9c.

show more ...


# 14e3650f 31-Mar-2022 Florian Hahn <flo@fhahn.com>

Revert "Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)""

This reverts commit 8378a71b6cce611e01f42690713fd7b561ff3f30.

It looks like this patch uncovered another issue, e.g. see
https://l

Revert "Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)""

This reverts commit 8378a71b6cce611e01f42690713fd7b561ff3f30.

It looks like this patch uncovered another issue, e.g. see
https://lab.llvm.org/buildbot/#/builders/168/builds/5518

show more ...


# 8378a71b 31-Mar-2022 Florian Hahn <flo@fhahn.com>

Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)"

This reverts the revert commit 2760cdc9c6.

This version pulls in the code to create the vector loop object in VPlan
from D121624.

This is

Recommit "[LV] Remove unneeded createHeaderBranch.(NFCI)"

This reverts the revert commit 2760cdc9c6.

This version pulls in the code to create the vector loop object in VPlan
from D121624.

This is needed because otherwise existing LoopInfo verification will
fail, as a loop block doesn't have in-loop successors now that we
do not replace the branch.

Now that we do not add new loops during skeleton construction, there's
also no need to verify LI there.

show more ...


# 2760cdc9 31-Mar-2022 Florian Hahn <flo@fhahn.com>

Revert "[LV] Remove unneeded createHeaderBranch.(NFCI)"

This reverts commit 32bc83d11e19b8a8c15df81b32fde1f9f8c6156b.

This is causing bots with expensive-checks to fail. Revert while I
investigate.


# 32bc83d1 31-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Remove unneeded createHeaderBranch.(NFCI)

The only remaining use was to get the exit block of the loop. Instead of
relying on the loop, use the successor of VectorHeaderBB
(LoopMiddleBlock) dir

[LV] Remove unneeded createHeaderBranch.(NFCI)

The only remaining use was to get the exit block of the loop. Instead of
relying on the loop, use the successor of VectorHeaderBB
(LoopMiddleBlock) directly to set VPTransformState::CFG::ExitB

Depends on D121621.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121623

show more ...


# 2c494f09 31-Mar-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Remove unneeded Loop variable (NFC).

Suggested in D121623. The remaining uses of L can be replaced, reducing
the need for the variable.


# e4543af4 30-Mar-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Track current vector loop in VPTransformState (NFC).

Instead of looking up the vector loop using the header, keep track of
the current vector loop in VPTransformState. This removes the
requi

[VPlan] Track current vector loop in VPTransformState (NFC).

Instead of looking up the vector loop using the header, keep track of
the current vector loop in VPTransformState. This removes the
requirement for the vector header block being part of the loop up front.

A follow-up patch will move the code to generate the Loop object for the
vector loop to VPRegionBlock.

Depends on D121619.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121621

show more ...


# e8673f2f 30-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Do not create separate latch block in VPlan::execute.

Now that all dependencies on creating the latch block up-front have been
removed, there is no need to create it early.

Depends on D121618.

[LV] Do not create separate latch block in VPlan::execute.

Now that all dependencies on creating the latch block up-front have been
removed, there is no need to create it early.

Depends on D121618.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121619

show more ...


# 8a4077fa 30-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Pass LoopHeaderBB directly to updateDominatorTree. (NFC)

At the call site, we already know what the vector header block is. Pass
it directly.


# d1d35632 29-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Move code to place pointer induction increment to VPlan post-processing.

This patch moves the code to set the correct incoming block for the
backedge value to VPlan::execute.

When generating t

[LV] Move code to place pointer induction increment to VPlan post-processing.

This patch moves the code to set the correct incoming block for the
backedge value to VPlan::execute.

When generating the phi node, the backedge value is temporarily added
using the pre-header as incoming block. The invalid phi node will be
fixed up during VPlan::execute after main VPlan code generation.
At the same time, the backedge value is also moved to the latch.

This change removes the requirement to create the latch block up-front
for VPWidenInductionPHIRecipe::execute, which in turn will enable
modeling the pre-header in VPlan.

Depends on D121617.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121618

show more ...


# e7bf2ea9 28-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Move code to place induction increment to VPlan post-processing.

This patch moves the code to set the correct incoming block for the
backedge value to VPlan::execute.

When generating the phi n

[LV] Move code to place induction increment to VPlan post-processing.

This patch moves the code to set the correct incoming block for the
backedge value to VPlan::execute.

When generating the phi node, the backedge value is temporarily added
using the pre-header as incoming block. The invalid phi node will be
fixed up during VPlan::execute after main VPlan code generation.
At the same time, the backedge value is also moved to the latch.

This change removes the requirement to create the latch block up-front
for VPWidenIntOrFpInductionRecipe::execute, which in turn will enable
modeling the pre-header in VPlan.

As an alternative, the increment could be modeled as separate recipe,
but that would require more work and a bit of redundant code, as we need
to create the step-vector during VPWidenIntOrFpInductionRecipe::execute
anyways, to create the values for different parts.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121617

show more ...


# e47d2202 25-Mar-2022 Florian Hahn <flo@fhahn.com>

[LV] Use getVectorLoopRegion to retrieve header. (NFC)

Update all places that currently assume the entry block to the plan is
also the vector loop header to use getVectorLoopRegion instead.

getVect

[LV] Use getVectorLoopRegion to retrieve header. (NFC)

Update all places that currently assume the entry block to the plan is
also the vector loop header to use getVectorLoopRegion instead.

getVectorLoopRegion will keep doing the right thing when the pre-header
is modeled explicitly (and becomes the new entry block in the plan).

show more ...


# 46432a00 24-Mar-2022 Florian Hahn <flo@fhahn.com>

[VPlan] Add VPWidenPointerInductionRecipe.

This patch moves pointer induction handling from VPWidenPHIRecipe to its
own recipe. In the process, it adds all information required to generate
code for

[VPlan] Add VPWidenPointerInductionRecipe.

This patch moves pointer induction handling from VPWidenPHIRecipe to its
own recipe. In the process, it adds all information required to generate
code for pointer inductions without relying on Legal to access the list
of induction phis.

Alternatively VPWidenPHIRecipe could also take an optional pointer to InductionDescriptor.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D121615

show more ...


# 1b89c832 21-Mar-2022 serge-sans-paille <sguelton@redhat.com>

Cleanup includes: Transforms/Instrumentation & Transforms/Vectorize

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.or

Cleanup includes: Transforms/Instrumentation & Transforms/Vectorize

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D122181

show more ...


12345678910>>...15