[VPlan] Add assignSlot(const VPBasicBlock *) (NFC).

Factor out utility to simplify D147964 as sugested.

[VPlan] Turn Plan entry node into VPBasicBlock (NFCI).

The entry to the plan is the preheader of the vector loop and
guaranteed to be a VPBasicBlock. Make sure this is the case by
adjusting the type

[VPlan] Turn Plan entry node into VPBasicBlock (NFCI).

The entry to the plan is the preheader of the vector loop and
guaranteed to be a VPBasicBlock. Make sure this is the case by
adjusting the type.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D149005

show more ...

[VPlan] Add VPValue::isLiveIn() (NFC).

This helps to clarify checks in multiple places.

Suggested as cleanup in D147892.

Recommit "[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI)."

This reverts the revert commit 8c2276f89887d0a27298a1bbbd2181fa54bbb509.

The updated patch re-orders the getDefiningRecipe che

Recommit "[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI)."

This reverts the revert commit 8c2276f89887d0a27298a1bbbd2181fa54bbb509.

The updated patch re-orders the getDefiningRecipe check in getVPalue to avoid
a use-after-free.

Original commit message:

Before this patch, a VPlan contained 2 mappings for Values -> VPValue:
1) Value2VPValue and 2) VPExternalDefs.

This duplication is unnecessary and there are already cases where
external defs are added to Value2VPValue. This patch replaces all uses
of VPExternalDefs with Value2VPValue.

It clarifies the naming of getOrAddVPValue (to getOrAddExternalVPValue)
and addVPValue (to addExternalVPValue).

At the moment, this is NFC, but will enable additional simplifications
in D147783.

Depends on D147891.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D147892

show more ...

Revert "[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI)."

Asan detects heap-use-after-free, see D147892.

This reverts commit 4fc190351e5af901b6107d162d07e1fbca90934f.
This reverts commit

Revert "[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI)."

Asan detects heap-use-after-free, see D147892.

This reverts commit 4fc190351e5af901b6107d162d07e1fbca90934f.
This reverts commit 668045eb77628be13e448ffbb855473ffca1cc43.

show more ...

[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI).

Before this patch, a VPlan contained 2 mappings for Values -> VPValue:
1) Value2VPValue and 2) VPExternalDefs.

This duplication is unnece

[VPlan] Unify Value2VPValue and VPExternalDefs maps (NFCI).

Before this patch, a VPlan contained 2 mappings for Values -> VPValue:
1) Value2VPValue and 2) VPExternalDefs.

This duplication is unnecessary and there are already cases where
external defs are added to Value2VPValue. This patch replaces all uses
of VPExternalDefs with Value2VPValue.

It clarifies the naming of getOrAddVPValue (to getOrAddExternalVPValue)
and addVPValue (to addExternalVPValue).

At the moment, this is NFC, but will enable additional simplifications
in D147783.

Depends on D147891.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D147892

show more ...

[VPlan] Don't assign slots for external defs (NFCI).

External defs are VPValues wrapping an IR value and hence will get
printed as ir<>. We don't need to assign a slot for a VPValue number.

[VPlan] Don't add live-outs if scalar epilogue is required.

Instead of clearing live outs when a scalar epilogue is required late,
don't add live outs during VPlan construction if a scalar epilogue

[VPlan] Don't add live-outs if scalar epilogue is required.

Instead of clearing live outs when a scalar epilogue is required late,
don't add live outs during VPlan construction if a scalar epilogue is
required.

This enables more VPlan-based DCE (if the live out would be the only
user in the plan) and is a step towards removing an access of the cost
model in fixedVectorizedLoop (which is after VPlan execution).

Depends on D147468.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D147471

show more ...

[LV] Cleanup and reformatting for some debug messages. NFC

This is just some cleanup of various debug messages, pulled out of another
patch to simplify it a little.

[llvm] Use *{Map,Set}::contains (NFC)

[LV][VPlan] Fix printing TripCount liveins. NFC

The TripCount liveins would currently be printed as badref in the vplan as they
are not allocated slots in the VPSlotTracker. This patch allocates the

[LV][VPlan] Fix printing TripCount liveins. NFC

The TripCount liveins would currently be printed as badref in the vplan as they
are not allocated slots in the VPSlotTracker. This patch allocates them a slot
and adds them to the printed Live-Ins. It also makes a minor adjustment to
printing of Live-ins to reduce the empty lines when multiple Live-ins are
present.

Differential Revision: https://reviews.llvm.org/D145507

show more ...

Move global namespace cl::opt inside llvm::

[VPlan] Add vp_depth_first_deep (NFC)

Similar to vp_depth_first_shallow (D140512) add vp_depth_first_deep to
make existing code clearer and more compact.

Reviewed By: Ayal

Differential Revision: h

[VPlan] Add vp_depth_first_deep (NFC)

Similar to vp_depth_first_shallow (D140512) add vp_depth_first_deep to
make existing code clearer and more compact.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D142055

show more ...

[VPlan] Add vp_depth_first_shallow + graph traits for wrapper(NFC)

This patch adds a new VPBlockShallowTraversalWrapper struct to
provide graph traits specialization that do not traverse through
VPR

[VPlan] Add vp_depth_first_shallow + graph traits for wrapper(NFC)

This patch adds a new VPBlockShallowTraversalWrapper struct to
provide graph traits specialization that do not traverse through
VPRegionBlocks. This matches the behavior of the existing traits for
plain VPBlockBase and is a step before moving the graph traits for
VPBlockBase to traverse through VPRegionBlocks to enable cross region
support in VPDominatorTree.

Depends on D140511.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D140512

show more ...

[VPlan] Use to_vector when iterating over a temporary vector. (NFC)

[VPlan] Move GraphTraits definitions to separate header (NFC).

This reduces the size of VPlan.h and avoids future growth of the file
when the graph traits are extended in future patches.

Reviewed B

[VPlan] Move GraphTraits definitions to separate header (NFC).

This reduces the size of VPlan.h and avoids future growth of the file
when the graph traits are extended in future patches.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D140500

show more ...

[LV] Move exit cond simplification to separate transform.

This sets the stage for D133017 by moving out the code that performs
VPlan based simplifications to a separate transform that takes the
chos

[LV] Move exit cond simplification to separate transform.

This sets the stage for D133017 by moving out the code that performs
VPlan based simplifications to a separate transform that takes the
chosen VF & UF as arguments.

The main advantage is that this transform runs before any changes to
the CFG are being made. This allows using SCEV without worrying about
making queries while the IR is in an incomplete state.

Note that this patch switches the reasoning to use SCEV, but still only
simplifies loops with constant trip counts. Using SCEV here is needed to
access the backedge taken count, because the trip count IR value has not
been created yet.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D135017

show more ...

[VPlan] Add support for tracking UFs applicable to VPlan (NFC).

Explicitly track the UFs supported in a VPlan. This is needed to
allow transformations to restrict the UFs which are supported.

Discu

[VPlan] Add support for tracking UFs applicable to VPlan (NFC).

Explicitly track the UFs supported in a VPlan. This is needed to
allow transformations to restrict the UFs which are supported.

Discussed as separate improvement in D135017.

show more ...

[VPlan] Move VF and UF string generation to getName() (NFC).

The VFs and UFs may be more constrained as the plans are transformed
(e.g. see D135017 for an example).

To make sure the VFs/UFs include

[VPlan] Move VF and UF string generation to getName() (NFC).

The VFs and UFs may be more constrained as the plans are transformed
(e.g. see D135017 for an example).

To make sure the VFs/UFs included in the VPlan dump are accurate,
generate them when accessing a plan's name, rather than include them in
the name string set after initial construction.

show more ...

[NFC] Rename Function::isDebugInfoForProfiling to shouldEmit[...]

The function name was misleading - the expectation set both by the name
and by other members of Function (like isDeclaration or isIn

[NFC] Rename Function::isDebugInfoForProfiling to shouldEmit[...]

The function name was misleading - the expectation set both by the name
and by other members of Function (like isDeclaration or isIntrinsic)
would be that the function somehow would "be" "debug info for
profiling". But that's not the case - the property indicates (as the
comment over the declaration also explains) whether debug info should be
emitted (for profiling).

show more ...

Recommit "[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe."

This reverts commit bf15f1e489aa2f1ac13268c9081a992a8963eb5b.

The updated version fixes a crash by checking the induction ki

Recommit "[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe."

This reverts commit bf15f1e489aa2f1ac13268c9081a992a8963eb5b.

The updated version fixes a crash by checking the induction kind instead
of the opcode; for integer inductions, the step is always added, but the
opcode might not be set.

show more ...

Revert "[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe."

This reverts commit 0fa666ecedc3f36471c0fee925d664512e7525a8.

This triggers an assertion during AArch64 stage2 builds. Revert

Revert "[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe."

This reverts commit 0fa666ecedc3f36471c0fee925d664512e7525a8.

This triggers an assertion during AArch64 stage2 builds. Revert while I
investigate.

See https://lab.llvm.org/buildbot/#/builders/179/builds/4967/steps/11/logs/stdio

show more ...

[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe.

This patch splits off the logic to transform the canonical IV to a
a value for an induction with a different start and step. This
transf

[VPlan] Add VPDerivedIVRecipe, use for VPScalarIVStepsRecipe.

This patch splits off the logic to transform the canonical IV to a
a value for an induction with a different start and step. This
transformation only needs to be done once (independent of VF/UF) and
enables sinking of VPScalarIVStepsRecipe as follow-up.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D133758

show more ...

[VPlan] Introduce VPValue::hasDefiningRecipe helper (NFC).

This clarifies the intention of code that uses the helper.

Suggested by @Ayal during review of D136068, thanks!

[VPlan] Use recipe type to avoid getDefiningRecipe call (NFC).

Suggested by @Ayal during review of D136068, thanks!