Revert "[VPlan] Use VPValue def for VPMemoryInstructionRecipe."

It introduced a memory leak.

This reverts commit 525b085a65d30a5f2ae2af38c0be252fe8d4781b.

[VPlan] Use VPValue def for VPMemoryInstructionRecipe.

This patch turns VPMemoryInstructionRecipe into a VPValue and uses it
during VPlan construction and codegeneration instead of the plain IR
refe

[VPlan] Use VPValue def for VPMemoryInstructionRecipe.

This patch turns VPMemoryInstructionRecipe into a VPValue and uses it
during VPlan construction and codegeneration instead of the plain IR
reference where possible.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D84680

show more ...

[VPlan] Clean up uses/operands on VPBB deletion.

Update the code responsible for deleting VPBBs and recipes to properly
update users and release operands.

This is another preparation for D84680 & f

[VPlan] Clean up uses/operands on VPBB deletion.

Update the code responsible for deleting VPBBs and recipes to properly
update users and release operands.

This is another preparation for D84680 & following patches towards
enabling modeling def-use chains in VPlan.

show more ...

[VPlan] Properly update users when updating operands.

When updating operands of a VPUser, we also have to adjust the list of
users for the new and old VPValues. This is required once we start
transi

[VPlan] Properly update users when updating operands.

When updating operands of a VPUser, we also have to adjust the list of
users for the new and old VPValues. This is required once we start
transitioning recipes to become VPValues.

show more ...

[VPlan] Change recipes to inherit from VPUser instead of a member var.

Now that VPUser is not inheriting from VPValue, we can take the next
step and turn the recipes that already manage their operan

[VPlan] Change recipes to inherit from VPUser instead of a member var.

Now that VPUser is not inheriting from VPValue, we can take the next
step and turn the recipes that already manage their operands via VPUser
into VPUsers directly. This is another small step towards traversing
def-use chains in VPlan.

This is NFC with respect to the generated code, but makes the interface
more powerful.

show more ...

[VPlan] Disconnect VPValue and VPUser.

This refactors VPuser to not inherit from VPValue to facilitate
introducing operations that introduce multiple VPValues (e.g.
VPInterleaveRecipe).

Reviewed By

[VPlan] Disconnect VPValue and VPUser.

This refactors VPuser to not inherit from VPValue to facilitate
introducing operations that introduce multiple VPValues (e.g.
VPInterleaveRecipe).

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D84679

show more ...

[VPlan] Add dump() helper to VPValue & VPRecipeBase.

This provides a convenient way to print VPValues and recipes in a
debugger. In particular it saves the user from instantiating
VPSlotTracker to p

[VPlan] Add dump() helper to VPValue & VPRecipeBase.

This provides a convenient way to print VPValues and recipes in a
debugger. In particular it saves the user from instantiating
VPSlotTracker to print recipes or values.

show more ...

VPlanValue.h - reduce unnecessary includes to forward declarations. NFC.

[VPlan] Turn classes with all public members into structs (NFC).

struct should be used when all members are public:
https://llvm.org/docs/CodingStandards.html#use-of-class-and-struct-keywords

Revi

[VPlan] Turn classes with all public members into structs (NFC).

struct should be used when all members are public:
https://llvm.org/docs/CodingStandards.html#use-of-class-and-struct-keywords

Reviewers: gilr, rengolin, Ayal, hsaito

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D77865

show more ...

[VPlan] Drop redundant private: at beginning of class defs (NFC).

Default visibility for classes is private, so the private: at the top of
various class definitions is redundant.

Reviewers: gilr, r

[VPlan] Drop redundant private: at beginning of class defs (NFC).

Default visibility for classes is private, so the private: at the top of
various class definitions is redundant.

Reviewers: gilr, rengolin, Ayal, hsaito

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D77810

show more ...

[VPlan] Add & use VPValue operands for VPWidenCallRecipe (NFC).

This patch adds VPValue versions for the arguments of the call to
VPWidenCallRecipe and uses them during code-generation.

Similar to

[VPlan] Add & use VPValue operands for VPWidenCallRecipe (NFC).

This patch adds VPValue versions for the arguments of the call to
VPWidenCallRecipe and uses them during code-generation.

Similar to D76373 this reduces ingredient def-use usage by ILV as
a step towards full VPlan-based def-use relations.

Reviewers: Ayal, gilr, rengolin

Reviewed By: gilr

Differential Revision: https://reviews.llvm.org/D77655

show more ...

[VPlan] Use underlying value for printing, if available.

When the an underlying value is available, we can use its name for
printing, as discussed in D73078.

Reviewers: rengolin, hsaito, Ayal, gilr

[VPlan] Use underlying value for printing, if available.

When the an underlying value is available, we can use its name for
printing, as discussed in D73078.

Reviewers: rengolin, hsaito, Ayal, gilr

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D76200

show more ...

[VPlan] Use consecutive numbers to print VPValues instead of addresses.

Currently when printing VPValues we use the object address, which makes
it hard to distinguish VPValues as they usually are la

[VPlan] Use consecutive numbers to print VPValues instead of addresses.

Currently when printing VPValues we use the object address, which makes
it hard to distinguish VPValues as they usually are large numbers with
varying distance between them.

This patch adds a simple slot tracker, similar to the ModuleSlotTracker
used for IR values. In order to dump a VPValue or anything containing a
VPValue, a slot tracker for the enclosing VPlan needs to be created. The
existing VPlanPrinter can take care of that for the existing code. We
assign consecutive numbers to each VPValue we encounter in a reverse
post order traversal of the VPlan.

Reviewers: rengolin, hsaito, fhahn, Ayal, dorit, gilr

Reviewed By: gilr

Differential Revision: https://reviews.llvm.org/D73078

show more ...

[VPlan] Rename VPlanHCFGTransforms to VPlanTransforms (NFC).

The file is intended to gather various VPlan transformations, not only
CFG related transforms. Actually, the only transformation there is

[VPlan] Rename VPlanHCFGTransforms to VPlanTransforms (NFC).

The file is intended to gather various VPlan transformations, not only
CFG related transforms. Actually, the only transformation there is not
CFG related.

Reviewers: Ayal, gilr, hsaito, rengolin

Reviewed By: gilr

Differential Revision: https://reviews.llvm.org/D70732

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[VPlan, SLP] Add simple SLP analysis on top of VPlan.

This patch adds an initial implementation of the look-ahead SLP tree
construction described in 'Look-Ahead SLP: Auto-vectorization in the Presen

[VPlan, SLP] Add simple SLP analysis on top of VPlan.

This patch adds an initial implementation of the look-ahead SLP tree
construction described in 'Look-Ahead SLP: Auto-vectorization in the Presence
of Commutative Operations, CGO 2018 by Vasileios Porpodas, Rodrigo C. O. Rocha,
Luís F. W. Góes'.

It returns an SLP tree represented as VPInstructions, with combined
instructions represented as a single, wider VPInstruction.

This initial version does not support instructions with multiple
different users (either inside or outside the SLP tree) or
non-instruction operands; it won't generate any shuffles or
insertelement instructions.

It also just adds the analysis that builds an SLP tree rooted in a set
of stores. It does not include any cost modeling or memory legality
checks. The plan is to integrate it with VPlan based cost modeling, once
available and to only apply it to operations that can be widened.

A follow-up patch will add a support for replacing instructions in a
VPlan with their SLP counter parts.

Reviewers: Ayal, mssimpso, rengolin, mkuper, hfinkel, hsaito, dcaballe, vporpo, RKSimon, ABataev

Reviewed By: rengolin

Differential Revision: https://reviews.llvm.org/D4949

llvm-svn: 346857

show more ...

[VPlan] VPlan version of InterleavedAccessInfo.

This patch turns InterleaveGroup into a template with the instruction type
being a template parameter. It also adds a VPInterleavedAccessInfo class, w

[VPlan] VPlan version of InterleavedAccessInfo.

This patch turns InterleaveGroup into a template with the instruction type
being a template parameter. It also adds a VPInterleavedAccessInfo class, which
only contains a mapping from VPInstructions to their respective InterleaveGroup.
As we do not have access to scalar evolution in VPlan, we can re-use
convert InterleavedAccessInfo to VPInterleavedAccess info.


Reviewers: Ayal, mssimpso, hfinkel, dcaballe, rengolin, mkuper, hsaito

Reviewed By: rengolin

Differential Revision: https://reviews.llvm.org/D49489

llvm-svn: 346758

show more ...

[VPlan] Implement initial vector code generation support for simple outer loops.

Summary:
[VPlan] Implement vector code generation support for simple outer loops.

Context: Patch Series #1 for outer

[VPlan] Implement initial vector code generation support for simple outer loops.

Summary:
[VPlan] Implement vector code generation support for simple outer loops.

Context: Patch Series #1 for outer loop vectorization support in LV using VPlan. (RFC: http://lists.llvm.org/pipermail/llvm-dev/2017-December/119523.html).

This patch introduces vector code generation support for simple outer loops that are currently supported in the VPlanNativePath. Changes here essentially do the following:

- force vector code generation using explicit vectorize_width

- add conservative early returns in cost model and other places for VPlanNativePath

- add code for setting up outer loop inductions

- support for widening non-induction PHIs that can result from inner loops and uniform conditional branches

- support for generating uniform inner branches

We plan to add a handful C outer loop executable tests once the initial code generation support is committed. This patch is expected to be NFC for the inner loop vectorizer path. Since we are moving in the direction of supporting outer loop vectorization in LV, it may also be time to rename classes such as InnerLoopVectorizer.

Reviewers: fhahn, rengolin, hsaito, dcaballe, mkuper, hfinkel, Ayal

Reviewed By: fhahn, hsaito

Subscribers: dmgreen, bollu, tschuett, rkruppe, rogfer01, llvm-commits

Differential Revision: https://reviews.llvm.org/D50820

llvm-svn: 342197

show more ...

[VPlan] Reland r332654 and silence unused func warning

r332654 was reverted due to an unused function warning in
release build. This commit includes the same code with the
warning silenced.

Differe

[VPlan] Reland r332654 and silence unused func warning

r332654 was reverted due to an unused function warning in
release build. This commit includes the same code with the
warning silenced.

Differential Revision: https://reviews.llvm.org/D44338

llvm-svn: 332860

show more ...

Reverted r332654 as it has broken some buildbots and left unfixed for a long time.

The introduced problem is:
llvm.src/lib/Transforms/Vectorize/VPlanVerifier.cpp:29:13: error: unused function 'hasDu

Reverted r332654 as it has broken some buildbots and left unfixed for a long time.

The introduced problem is:
llvm.src/lib/Transforms/Vectorize/VPlanVerifier.cpp:29:13: error: unused function 'hasDuplicates' [-Werror,-Wunused-function]
static bool hasDuplicates(const SmallVectorImpl<VPBlockBase *> &VPBlockVec) {
^

llvm-svn: 332747

show more ...

[LV][VPlan] Build plain CFG with simple VPInstructions for outer loops.

Patch #3 from VPlan Outer Loop Vectorization Patch Series #1
(RFC: http://lists.llvm.org/pipermail/llvm-dev/2017-December/1195

[LV][VPlan] Build plain CFG with simple VPInstructions for outer loops.

Patch #3 from VPlan Outer Loop Vectorization Patch Series #1
(RFC: http://lists.llvm.org/pipermail/llvm-dev/2017-December/119523.html).

Expected to be NFC for the current inner loop vectorization path. It
introduces the basic algorithm to build the VPlan plain CFG (single-level
CFG, no hierarchical CFG (H-CFG), yet) in the VPlan-native vectorization
path using VPInstructions. It includes:
- VPlanHCFGBuilder: Main class to build the VPlan H-CFG (plain CFG without nested regions, for now).
- VPlanVerifier: Main class with utilities to check the consistency of a H-CFG.
- VPlanBlockUtils: Main class with utilities to manipulate VPBlockBases in VPlan.

Reviewers: rengolin, fhahn, mkuper, mssimpso, a.elovikov, hfinkel, aprantl.

Differential Revision: https://reviews.llvm.org/D44338

llvm-svn: 332654

show more ...

[LV] Model masking in VPlan, introducing VPInstructions

This patch adds a new abstraction layer to VPlan and leverages it to model the planned
instructions that manipulate masks (AND, OR, NOT), intr

[LV] Model masking in VPlan, introducing VPInstructions

This patch adds a new abstraction layer to VPlan and leverages it to model the planned
instructions that manipulate masks (AND, OR, NOT), introduced during predication.

The new VPValue and VPUser classes model how data flows into, through and out
of a VPlan, forming the vertices of a planned Def-Use graph. The new
VPInstruction class is a generic single-instruction Recipe that models a
planned instruction along with its opcode, operands and users. See
VectorizationPlan.rst for more details.

Differential Revision: https://reviews.llvm.org/D38676

llvm-svn: 318645

show more ...