[LCSSA] Provide option for caller to clean up unused PHIs.

formLCSSAForInstructions is used by SCEVExpander, which tracks all
inserted instructions including LCSSA phis using asserting value
handles

[LCSSA] Provide option for caller to clean up unused PHIs.

formLCSSAForInstructions is used by SCEVExpander, which tracks all
inserted instructions including LCSSA phis using asserting value
handles. This means cleanup needs to happen in the caller.

Extend formLCSSAForInstructions to take an optional pointer to a
vector. If this argument is non-nullptr, instead of directly deleting
the phis, add them to the vector, so the caller can process them.

This should address various PPC buildbot failures, including
http://lab.llvm.org:8011/builders/clang-ppc64be-linux-lnt/builds/40567

show more ...

[LCSSA] Use IRBuilder for PHI creation.

Use IRBuilder instead PHINode::Create. This should not impact the
generated code, but IRBuilder provides a way to register callbacks for
inserted instructions

[LCSSA] Use IRBuilder for PHI creation.

Use IRBuilder instead PHINode::Create. This should not impact the
generated code, but IRBuilder provides a way to register callbacks for
inserted instructions, which is convenient for some users.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D85037

show more ...

[SCEV] don't query getSCEV() for incomplete phis

querying getSCEV() for incomplete phis leads to wrong cache value in `ExprToIVMap`,
because incomplete phis may be simplified to same value before ge

[SCEV] don't query getSCEV() for incomplete phis

querying getSCEV() for incomplete phis leads to wrong cache value in `ExprToIVMap`,
because incomplete phis may be simplified to same value before get SCEV expression.

Reviewed By: lebedev.ri, mkazantsev

Differential Revision: https://reviews.llvm.org/D77560

show more ...

[SCEVExpander] Name temporary instructions for LCSSA insertion (NFC).

Reland "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts the revert commit dc2867576886247cbe351e7c63618c09ab6af808.

It includes a fix for Polly, which uses SCEVExpander on IR th

Reland "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts the revert commit dc2867576886247cbe351e7c63618c09ab6af808.

It includes a fix for Polly, which uses SCEVExpander on IR that is not
in LCSSA form. Set PreserveLCSSA = false in that case, to ensure we do
not introduce LCSSA phis where there were none before.

show more ...

Revert "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts commit 99166fd4fb422351f131fb1265cb85d5f6c5b8da, because it
breaks the polly builders.

polly/test/Isl/CodeGen/invariant_l

Revert "[SCEVExpander] Add option to preserve LCSSA directly."

This reverts commit 99166fd4fb422351f131fb1265cb85d5f6c5b8da, because it
breaks the polly builders.

polly/test/Isl/CodeGen/invariant_load_escaping_second_scop.ll fails
because a apparently unnecessary LCSSA phi node is introduced.

Make the bots green again, while I take a closer look.

show more ...

[SCEVExpander] Add option to preserve LCSSA directly.

This patch teaches SCEVExpander to directly preserve LCSSA.

As it is currently, SCEV does not look through PHI nodes in loops,
as it might brea

[SCEVExpander] Add option to preserve LCSSA directly.

This patch teaches SCEVExpander to directly preserve LCSSA.

As it is currently, SCEV does not look through PHI nodes in loops,
as it might break LCSSA form. Once SCEVExpander can preserve
LCSSA form, it should be safe for SCEV to look through PHIs.

To preserve LCSSA form, this patch uses formLCSSAForInstructions
on operands of newly created instructions, if the definition is inside
a different loop than the new instruction.

The final value we return from expandCodeFor may also need LCSSA
phis, depending on the insert point. As no user for it exists there yet,
create a temporary instruction at the insert point, which can be passed
to formLCSSAForInstructions. This temporary instruction is removed
after LCSSA construction.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D71538

show more ...

[Analysis] TTI: Add CastContextHint for getCastInstrCost

Currently, getCastInstrCost has limited information about the cast it's
rating, often just the opcode and types. Sometimes there is a contex

[Analysis] TTI: Add CastContextHint for getCastInstrCost

Currently, getCastInstrCost has limited information about the cast it's
rating, often just the opcode and types. Sometimes there is a context
instruction as well, but it isn't trustworthy: for instance, when the
vectorizer is rating a plan, it calls getCastInstrCost with the old
instructions when, in fact, it's trying to evaluate the cost of the
instruction post-vectorization. Thus, the current system can get the
cost of certain casts incorrect as the correct cost can vary greatly
based on the context in which it's used.

For example, if the vectorizer queries getCastInstrCost to evaluate the
cost of a sext(load) with tail predication enabled, getCastInstrCost
will think it's free most of the time, but it's not always free. On ARM
MVE, a VLD2 group cannot be extended like a normal VLDR can. Similar
situations can come up with how masked loads can be extended when being
split.

To fix that, this path adds a new parameter to getCastInstrCost to give
it a hint about the context of the cast. It adds a CastContextHint enum
which contains the type of the load/store being created by the
vectorizer - one for each of the types it can produce.

Original patch by Pierre van Houtryve

Differential Revision: https://reviews.llvm.org/D79162

show more ...

[SCEVExpander] Use IRBuilderCallbackInserter to call rememberInstruction.

Currently there are plenty of instructions that SCEVExpander creates but
does not track as created. IRBuilder allows specify

[SCEVExpander] Use IRBuilderCallbackInserter to call rememberInstruction.

Currently there are plenty of instructions that SCEVExpander creates but
does not track as created. IRBuilder allows specifying a callback
whenever an instruction is inserted. Use this to call
rememberInstruction automatically for each created instruction.

There are still a few rememberInstruction calls remaining, because in
some cases Inst::Create functions are used to construct instructions.

Suggested by @lebedev.ri in D75980.

Reviewers: mkazantsev, reames, sanjoy.google, lebedev.ri

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D84326

show more ...

[SCEV][IndVarSimplify] insert point should not be block front.

Recommit after removing the unused cast instructions.

Differential Revision: https://reviews.llvm.org/D80975

Revert "[SCEV][IndVarSimplify] insert point should not be block front."

This reverts commit f1efb8bb4ba0584a9b994f3404a2c62920ce6652.

Reverted because it doesn't correctly update the pass return st

Revert "[SCEV][IndVarSimplify] insert point should not be block front."

This reverts commit f1efb8bb4ba0584a9b994f3404a2c62920ce6652.

Reverted because it doesn't correctly update the pass return status, see

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-debian/builds/9441/steps/test-check-all/logs/FAIL%3A%20LLVM%3A%3Awiden-i32-i8ptr.ll

show more ...

[SCEV][IndVarSimplify] insert point should not be block front.

The block front may be a PHI node, inserting a cast instructions like
BitCast, PtrToInt, IntToPtr among PHIs is not right.

Reviewed By

[SCEV][IndVarSimplify] insert point should not be block front.

The block front may be a PHI node, inserting a cast instructions like
BitCast, PtrToInt, IntToPtr among PHIs is not right.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D80975

show more ...

[LSR][SCEVExpander] Avoid blind cast 'Factor' to SCEVConstant in FactorOutConstant.

Summary:
In SCEVExpander FactorOutConstant(), when GEP indexing into/over scalable vector,
it is legal for the 'Fa

[LSR][SCEVExpander] Avoid blind cast 'Factor' to SCEVConstant in FactorOutConstant.

Summary:
In SCEVExpander FactorOutConstant(), when GEP indexing into/over scalable vector,
it is legal for the 'Factor' in a MulExpr to be the size of a scalable vector
instead of a compile-time constant.

Current upstream crash with the test attached.

Reviewers: efriedma, sdesmalen, sanjoy.google, mkazantsev

Reviewed By: efriedma

Subscribers: hiraditya, javed.absar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80973

show more ...

[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).

SCEVExpander modifies the underlying function so it is more suitable in
Transforms/Utils, rather than Analysis. This allows using o

[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).

SCEVExpander modifies the underlying function so it is more suitable in
Transforms/Utils, rather than Analysis. This allows using other
transform utils in SCEVExpander.

This patch was originally committed as b8a3c34eee06, but broke the
modules build, as LoopAccessAnalysis was using the Expander.

The code-gen part of LAA was moved to lib/Transforms recently, so this
patch can be landed again.

Reviewers: sanjoy.google, efriedma, reames

Reviewed By: sanjoy.google

Differential Revision: https://reviews.llvm.org/D71537

show more ...

[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).

SCEVExpander modifies the underlying function so it is more suitable in
Transforms/Utils, rather than Analysis. This allows using o

[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).

SCEVExpander modifies the underlying function so it is more suitable in
Transforms/Utils, rather than Analysis. This allows using other
transform utils in SCEVExpander.

Reviewers: sanjoy.google, efriedma, reames

Reviewed By: sanjoy.google

Differential Revision: https://reviews.llvm.org/D71537

show more ...