[DebugInfo][RemoveDIs] Handle DPValues at remaining dbg.value using sites (#73788)

This patch updates the last few places in LLVM using findDbgValues that
don't also collect and handle DPValue obje

[DebugInfo][RemoveDIs] Handle DPValues at remaining dbg.value using sites (#73788)

This patch updates the last few places in LLVM using findDbgValues that
don't also collect and handle DPValue objects. This largely involves
instcombine and mem2reg changes, and are largely mechanical, calling
existing utilities on collections of DPValues instead of just
DbgValuesInsts.

A variety of tests have had RemoveDIs RUN lines added to them to cover
these behaviours. We have some technical debt of the instcombine sinking
code for DPValues not being implemented yet, so I've left FIXME stubs
indicating that we intend to cover tests with RemoveDIs but haven't yet.

show more ...

[JumpThreading] Remove LVI printer flag (#73426)

This patch removes the -print-lvi-after-jump-threading flag now that we
can print everything in the LVI cache using the print<lazy-value-info>
pass.

[DebugInfo][RemoveDIs] Instrument inliner for non-instr debug-info (#72884)

With intrinsics representing debug-info, we just clone all the
intrinsics when inlining a function and don't think about

[DebugInfo][RemoveDIs] Instrument inliner for non-instr debug-info (#72884)

With intrinsics representing debug-info, we just clone all the
intrinsics when inlining a function and don't think about it any
further. With non-instruction debug-info however we need to be a bit
more careful and manually move the debug-info from one place to another.
For the most part, this means keeping a "cursor" during block cloning of
where we last copied debug-info from, and performing debug-info copying
whenever we successfully clone another instruction.

There are several utilities in LLVM for doing this, all of which now
need to manually call cloneDebugInfo. The testing story for this is not
well covered as we could rely on normal instruction-cloning mechanisms
to do all the hard stuff. Thus, I've added a few tests to explicitly
test dbg.value behaviours, ahead of them becoming not-instructions.

show more ...

[DebugInfo][RemoveDIs] Instrument jump-threading to update DPValues (#73127)

This patch makes jump-threading handle non-instruction debug-info stored
in DPValues in the same way that it updates dbg

[DebugInfo][RemoveDIs] Instrument jump-threading to update DPValues (#73127)

This patch makes jump-threading handle non-instruction debug-info stored
in DPValues in the same way that it updates dbg.values nowadays. This
involves re-targetting their operands as with dbg.values getting moved
from one block to another, and manually cloning them when duplicating
blocks. The SSAUpdater class also grows some functions for SSA-updating
DPValues in the same way as dbg.values.

All of this is largely covered by existing debug-info tests, except for
the cloning of DPValues attached to elidable instructions and branches,
where I've added a test to thread-debug-info.ll. Where previously we
could rely on dbg.values being copied and cloned as normal instructions
are, as we need to explicitly perform that operation now I've added some
explicit testing for it.

show more ...

Add setBranchWeigths convenience function. NFC (#72446)

Add `setBranchWeights` convenience function to ProfDataUtils.h and use
it where appropriate.

[JumpThreading] Don't phi translate past loop phi (#70664)

When evaluating comparisons in predecessors, phi operands are translated
into the predecessor. If the translation is across a backedge, th

[JumpThreading] Don't phi translate past loop phi (#70664)

When evaluating comparisons in predecessors, phi operands are translated
into the predecessor. If the translation is across a backedge, this
means that the two operands of the icmp will be from two different loop
iterations, resulting in incorrect simplification.

Fix this by not performing the phi translation for phis in loop headers.

Note: This is not a complete fix. If the
jump-threading-across-loop-headers option is enabled, the LoopHeaders
variable does not get populated. Additional changes will be needed to
fix that case.

Related to https://github.com/llvm/llvm-project/issues/70651.

show more ...

[llvm] Use llvm::erase_if (NFC)

Use BlockFrequency type in more places (NFC) (#68266)

The `BlockFrequency` class abstracts `uint64_t` frequency values. Use it
more consistently in various APIs and disable implicit conversion to

Use BlockFrequency type in more places (NFC) (#68266)

The `BlockFrequency` class abstracts `uint64_t` frequency values. Use it
more consistently in various APIs and disable implicit conversion to
make usage more consistent and explicit.

- Use `BlockFrequency Freq` parameter for `setBlockFreq`,
`getProfileCountFromFreq` and `setBlockFreqAndScale` functions.
- Return `BlockFrequency` in `getEntryFreq()` functions.
- While on it change some `const BlockFrequency& Freq` parameters to
plain `BlockFreqency Freq`.
- Mark `BlockFrequency(uint64_t)` constructor as explicit.
- Add missing `BlockFrequency::operator!=`.
- Remove `uint64_t BlockFreqency::getMaxFrequency()`.
- Add `BlockFrequency BlockFrequency::max()` function.

show more ...

[JumpThreading] Avoid use of ConstantExpr::getCast()

Use the constant folding API instead.

JumpThreading: Propagate branch weights in tryToUnfoldSelectInCurrBB (#66116)

Propagate "branch_weights" metadata whe turning a select into a
conditional branch in tryToUnfoldSelectInCurrBB

[NFC][RemoveDIs] Prefer iterator-insertion over instructions

Continuing the patch series to get rid of debug intrinsics [0], instruction
insertion needs to be done with iterators rather than instruc

[NFC][RemoveDIs] Prefer iterator-insertion over instructions

Continuing the patch series to get rid of debug intrinsics [0], instruction
insertion needs to be done with iterators rather than instruction pointers,
so that we can communicate information in the iterator class. This patch
adds an iterator-taking insertBefore method and converts various call sites
to take iterators. These are all sites where such debug-info needs to be
preserved so that a stage2 clang can be built identically; it's likely that
many more will need to be changed in the future.

At this stage, this is just changing the spelling of a few operations,
which will eventually become signifiant once the debug-info bearing
iterator is used.

[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939

Differential Revision: https://reviews.llvm.org/D152537

show more ...

[JumpThreading] Invalidate LVI after `combineMetadataForCSE`.

[IR] Treat callbr as special terminator (PR64215)

isLegalToHoistInto() currently return true for callbr instructions.
That means that a callbr with one successor will be considered a
proper loop pre

[IR] Treat callbr as special terminator (PR64215)

isLegalToHoistInto() currently return true for callbr instructions.
That means that a callbr with one successor will be considered a
proper loop preheader, which may result in instructions that use
the callbr return value being hoisted past it.

Fix this by adding callbr to isExceptionTerminator (with a rename
to isSpecialTerminator), which also fixes similar assumptions in
other places.

Fixes https://github.com/llvm/llvm-project/issues/64215.

Differential Revision: https://reviews.llvm.org/D158609

show more ...

TTI: Pass function to hasBranchDivergence in a few passes

https://reviews.llvm.org/D152033

Revert "Revert "InstSimplify: Require instruction be parented""

This reverts commit 0c03f48480f69b854f86d31235425b5cb71ac921.

Going to fix forward size regression instead due to more dependent patc

Revert "Revert "InstSimplify: Require instruction be parented""

This reverts commit 0c03f48480f69b854f86d31235425b5cb71ac921.

Going to fix forward size regression instead due to more dependent patches needing to be reverted otherwise.

show more ...

Revert "InstSimplify: Require instruction be parented"

This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Causes large binary size regressions, see comments on https://reviews.llvm.org/r

Revert "InstSimplify: Require instruction be parented"

This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Causes large binary size regressions, see comments on https://reviews.llvm.org/rG1536e299e63d7788f38117b0212ca50eb76d7a3b.

show more ...

Revert "Revert "InstSimplify: Require instruction be parented""

This reverts commit 00264eac4d0938ae8a0826da38e4777be269124c.

Reason: caused a bunch of bots to break

Revert "InstSimplify: Require instruction be parented"

This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Reason: causes a regression in the inliner (see https://crbug.com/1454531 and ht

Revert "InstSimplify: Require instruction be parented"

This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Reason: causes a regression in the inliner (see https://crbug.com/1454531 and https://reviews.llvm.org/rG1536e299e63d7788f38117b0212ca50eb76d7a3b#1217141)

show more ...

InstSimplify: Require instruction be parented

Unlike every other analysis and transform, simplifyInstruction
permitted operating on instructions which are not inserted
into a function. This created

InstSimplify: Require instruction be parented

Unlike every other analysis and transform, simplifyInstruction
permitted operating on instructions which are not inserted
into a function. This created an edge case no other code needs
to really worry about, and limited transforms in cases that
can make use of the context function. Only the inliner and a handful
of other utilities were making use of this, so just fix up these
edge cases. Results in some IR ordering differences since
cloned blocks are inserted eagerly now. Plus some additional
simplifications trigger (e.g. some add 0s now folded out that
previously didn't).

show more ...

Remove several no longer needed includes. NFCI

Mostly removing includes of InitializePasses.h and Pass.h in
passes that no longer has support for the legacy PM.

[Passes] Remove some legacy passes

DFAJumpThreading
JumpThreading
LibCallsShrink
LoopVectorize
SLPVectorizer
DeadStoreElimination
AggressiveDCE
CorrelatedValuePropagation
IndVarSimplify

These are p

[Passes] Remove some legacy passes

DFAJumpThreading
JumpThreading
LibCallsShrink
LoopVectorize
SLPVectorizer
DeadStoreElimination
AggressiveDCE
CorrelatedValuePropagation
IndVarSimplify

These are part of the optimization pipeline, of which the legacy version is deprecated and being removed.

show more ...

[JT] Always create BPI/BFI when running in legacy PM

This is wasteful, but only affects the legacy pass manager. Otherwise
a1b78fb929fccf96acaa0212cf68fee82298e747 would crash JT when running
with t

[JT] Always create BPI/BFI when running in legacy PM

This is wasteful, but only affects the legacy pass manager. Otherwise
a1b78fb929fccf96acaa0212cf68fee82298e747 would crash JT when running
with that PM. There are still a few users of the legacy PM out there
that are reluctant to migrate, numba in this case.

No test as we don't test legacy PM anymore.

show more ...

[JT][CT] Preserve exisiting BPI/BFI during JumpThreading

Currently, JT creates and updates local instances of BPI\BFI. As a result global ones have to be invalidated if JT made any changes.
In fact,

[JT][CT] Preserve exisiting BPI/BFI during JumpThreading

Currently, JT creates and updates local instances of BPI\BFI. As a result global ones have to be invalidated if JT made any changes.
In fact, JT doesn't use any information from BPI/BFI for the sake of the transformation itself. It only creates BPI/BFI to keep them up to date. But since it updates local copies (besides cases when it updates profile metadata) it just waste of time.

Current patch is a rework of D124439. D124439 makes one step and replaces local copies with global ones retrieved through AnalysisPassManager. Here we do one more step and don't create BPI/BFI if the only reason of creation is to keep BPI/BFI up to date. Overall logic is the following. If there is cached BPI/BFI then update it along the transformations. If there is no existing BPI/BFI, then create it only if it is required to update profile metadata.

Please note if BPI/BFI exists on exit from JT (either cached or created) it is always up to date and no reason to invalidate it.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D136827

show more ...

[DebugInfo] Make debug intrinsics to track cloned values in JumpThreading

This patch causes debug value intrinsics outside of cloned blocks in the
Jump Threading pass to correctly point towards any

[DebugInfo] Make debug intrinsics to track cloned values in JumpThreading

This patch causes debug value intrinsics outside of cloned blocks in the
Jump Threading pass to correctly point towards any derived values. If it cannot,
it kills them.

Reviewed By: probinson, StephenTozer

Differential Revision: https://reviews.llvm.org/D140404

show more ...

Revert "[JT][CT] Preserve exisiting BPI/BFI during JumpThreading"

This reverts commit 26e7cb24cb5dfa560683064d37f560558f00aa67.