[IR][unittests] Replace of PointerType::get(Type) with opaque version (NFC) (#123621)

In accordance with https://github.com/llvm/llvm-project/issues/123569

[VPlan] Make VPValue constructors protected. (NFC)

Tighten access to constructors similar to ef1260acc0. VPValues should
either be constructed by constructors of recipes defining them or should
be l

[VPlan] Make VPValue constructors protected. (NFC)

Tighten access to constructors similar to ef1260acc0. VPValues should
either be constructed by constructors of recipes defining them or should
be live-ins created by VPlan (via getOrAddLiveIn).

show more ...

[VPlan] Make VPBlock constructors private (NFC).

16d19aaed moved to manage block creation via VPlan directly, with VPlan
owning the created blocks. Follow up to make the VPBlock constructors
private

[VPlan] Make VPBlock constructors private (NFC).

16d19aaed moved to manage block creation via VPlan directly, with VPlan
owning the created blocks. Follow up to make the VPBlock constructors
private, to require creation via VPlan helpers and thus preventing
issues due to manually constructing blocks.

show more ...

[VPlan] Handle some VPInstructions in may{Read,Write}FromMemory (#120058)

This just copies the same conservative definition from mayWriteToMemory,
and enables more VPInstructions to be hoisted out i

[VPlan] Handle some VPInstructions in may{Read,Write}FromMemory (#120058)

This just copies the same conservative definition from mayWriteToMemory,
and enables more VPInstructions to be hoisted out in LICM.

I think this should give more accurate costs, and I was able to build
llvm-test-suite without the legacy-vplan cost model assertion going off.

show more ...

[VPlan] Manage created blocks directly in VPlan. (NFC) (#120918)

This patch changes the way blocks are managed by VPlan. Previously all
blocks reachable from entry would be cleaned up when a VPlan

[VPlan] Manage created blocks directly in VPlan. (NFC) (#120918)

This patch changes the way blocks are managed by VPlan. Previously all
blocks reachable from entry would be cleaned up when a VPlan is
destroyed. With this patch, each VPlan keeps track of blocks created for
it in a list and this list is then used to delete all blocks in the list
when the VPlan is destroyed. To do so, block creation is funneled
through helpers in directly in VPlan.

The main advantage of doing so is it simplifies CFG transformations, as
those do not have to take care of deleting any blocks, just adjusting
the CFG. This helps to simplify
https://github.com/llvm/llvm-project/pull/108378 and
https://github.com/llvm/llvm-project/pull/106748.

This also simplifies handling of 'immutable' blocks a VPlan holds
references to, which at the moment only include the scalar header block.

PR: https://github.com/llvm/llvm-project/pull/120918

show more ...

[VPlan] Always create initial blocks in constructor (NFC).

Update C++ unit tests to use VPlanTestBase to construct initial VPlan,
using a constructor that creates the VP blocks directly in the
const

[VPlan] Always create initial blocks in constructor (NFC).

Update C++ unit tests to use VPlanTestBase to construct initial VPlan,
using a constructor that creates the VP blocks directly in the
constructor.

Split off from and in preparation for
https://github.com/llvm/llvm-project/pull/120918.

show more ...

[VPlan] Set and use debug location for VPPredInstPHIRecipe.

Update the recipe it always set its debug location and use it during IR
generation.

[VPlan] Don't use VPlan ctor taking trip count in most unit tests (NFC).

Update tests to use constructor not passing a trip count VPValue. The
tests don't need that and are simpler as a result.

[VPlan] Remove legacy VPlan() constructors (NFC).

The constructors were retained to reduce the diff during transition.

Remove them now.

[VPlan] Hook IR blocks into VPlan during skeleton creation (NFC) (#114292)

As a first step to move towards modeling the full skeleton in VPlan,
start by wrapping IR blocks created during legacy ske

[VPlan] Hook IR blocks into VPlan during skeleton creation (NFC) (#114292)

As a first step to move towards modeling the full skeleton in VPlan,
start by wrapping IR blocks created during legacy skeleton creation in
VPIRBasicBlocks and hook them into the VPlan. This means the skeleton
CFG is represented in VPlan, just before execute. This allows moving
parts of skeleton creation into recipes in the VPBBs gradually.

Note that this allows retiring some manual DT updates, as this will be
handled automatically during VPlan execution.

PR: https://github.com/llvm/llvm-project/pull/114292

show more ...

[VPlan] Connect scalar header to CFG in VPlan unit test.

This makes sure the VPIRBasicBlock is deleted when the VPlan is
destroyed.

Fixes https://github.com/llvm/llvm-project/issues/114623.

[VPlan] Suppress leak in test (#114624)

Probably needs a better fix. #114623

[VPlan] Don't leak ScalarHeader BasicBlock in unit tests.

[VPlan] Connect scalar header to VPlan CFG in unit tests.

This ensures the VPIRBasicBlocks are deleted when the VPlan is
destroyed.

Fixes a buildbot failure with ASAN, including
https://lab.llvm.or

[VPlan] Connect scalar header to VPlan CFG in unit tests.

This ensures the VPIRBasicBlocks are deleted when the VPlan is
destroyed.

Fixes a buildbot failure with ASAN, including
https://lab.llvm.org/buildbot/#/builders/52/builds/3368

show more ...

[VPlan] Introduce scalar loop header in plan, remove VPLiveOut. (#109975)

Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handl

[VPlan] Introduce scalar loop header in plan, remove VPLiveOut. (#109975)

Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handled by adding
operands to the wrapped phis in the scalar loop header.

Note that the current version only includes the scalar loop header, no
other loop blocks and also does not wrap it in a region block.

PR: https://github.com/llvm/llvm-project/pull/109975

show more ...

[VPlan] Add VPSingleDefRecipe::dump() to resolve ambigous lookup (NFC).

This allows calling ::dump() on various sub-classes of VPSingleDefRecipe
directly, as it resolves an ambigous name lookup.

Pr

[VPlan] Add VPSingleDefRecipe::dump() to resolve ambigous lookup (NFC).

This allows calling ::dump() on various sub-classes of VPSingleDefRecipe
directly, as it resolves an ambigous name lookup.

Previously, calling VPWidenRecipe::dump() (and others), would result in

the following errors:
llvm/unittests/Transforms/Vectorize/VPlanTest.cpp:1284:19: error: member 'dump' found in multiple base classes of different types
1284 | WidenR->dump();
| ^
llvm/include/../lib/Transforms/Vectorize/VPlanValue.h:434:8: note: member found by ambiguous name lookup
434 | void dump() const;
| ^
llvm/include/../lib/Transforms/Vectorize/VPlanValue.h:108:8: note: member found by ambiguous name lookup
108 | void dump() const;
| ^
1 error generated.

show more ...

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is a

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).

show more ...

[VPlan] Introduce VPWidenIntrinsicRecipe to separate from libcall. (#110486)

This patch splits off intrinsic hanlding to a new
VPWidenIntrinsicRecipe. VPWidenIntrinsicRecipes only need access to th

[VPlan] Introduce VPWidenIntrinsicRecipe to separate from libcall. (#110486)

This patch splits off intrinsic hanlding to a new
VPWidenIntrinsicRecipe. VPWidenIntrinsicRecipes only need access to the
intrinsic ID to widen and the scalar result type (in case the intrinsic
is overloaded on the result type). It does not need access to an
underlying IR call instruction or function.

This means VPWidenIntrinsicRecipe can be created easily without access
to underlying IR.

show more ...

[VPlan] Unify mayWriteToMemory and mayHaveSideEffects logic for VPInst.

Unify logic for mayWriteToMemory and mayHaveSideEffects for
VPInstruction, with the later relying on the former. Also extend t

[VPlan] Unify mayWriteToMemory and mayHaveSideEffects logic for VPInst.

Unify logic for mayWriteToMemory and mayHaveSideEffects for
VPInstruction, with the later relying on the former. Also extend to
handle binary operators.

Split off from https://github.com/llvm/llvm-project/pull/106441

show more ...

[llvm][unittests] Don't call raw_string_ostream::flush() (NFC)

raw_string_ostream::flush() is essentially a no-op (also specified in docs).
Don't call it in tests that aren't meant to test 'raw_stri

[llvm][unittests] Don't call raw_string_ostream::flush() (NFC)

raw_string_ostream::flush() is essentially a no-op (also specified in docs).
Don't call it in tests that aren't meant to test 'raw_string_ostream' itself.

p.s. remove a few redundant calls to raw_string_ostream::str()

show more ...

[LV][EVL] Refine the constructors of EVL recipe to use call by reference. NFC (#100088)

[LV][EVL] Support in-loop reduction using tail folding with EVL. (#90184)

Following from #87816, add VPReductionEVLRecipe to describe vector
predication reduction.

Address one of TODOs from #761

[LV][EVL] Support in-loop reduction using tail folding with EVL. (#90184)

Following from #87816, add VPReductionEVLRecipe to describe vector
predication reduction.

Address one of TODOs from #76172.

show more ...

[VPlanTest] Use poison instead of undef for dummy values (NFC)

[VPlan] VPEVLBasedIVPHI is a VPSingleDefRecipe.

VPEVLBasedIVPHIRecipe inherits from VPSingleDefRecipe. Add
VPEVLBasedIVPHISC to VPSingleDefRecipe::classof to make isa/dyn_cast &
co work as expected.

[VPlan] VPEVLBasedIVPHI is a VPSingleDefRecipe.

VPEVLBasedIVPHIRecipe inherits from VPSingleDefRecipe. Add
VPEVLBasedIVPHISC to VPSingleDefRecipe::classof to make isa/dyn_cast &
co work as expected.

Split off https://github.com/llvm/llvm-project/pull/67934.

show more ...

[VPlan] Make CallInst optional for VPWidenCallRecipe (NFCI).

Replace relying on the underling CallInst for looking up the called
function and its types by instead adding the called function as opera

[VPlan] Make CallInst optional for VPWidenCallRecipe (NFCI).

Replace relying on the underling CallInst for looking up the called
function and its types by instead adding the called function as operand,
in line with how called functions are handled in CallInst.

Operand bundles, metadata and fast-math flags are optionally used if
there's an underlying CallInst.

This enables creating VPWidenCallRecipes without requiring an underlying
IR instruction.

show more ...