[NFC][DebugInfo] Use iterator moveBefore at many call-sites (#123583)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and sim

[NFC][DebugInfo] Use iterator moveBefore at many call-sites (#123583)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and similar feed into
instruction insertion positions. Call-sites where that's necessary were
updated a year ago; but to ensure some type safety however, we'd like to
have all calls to moveBefore use iterators.

This patch adds a (guaranteed dereferenceable) iterator-taking
moveBefore, and changes a bunch of call-sites where it's obviously safe
to change to use it by just calling getIterator() on an instruction
pointer. A follow-up patch will contain less-obviously-safe changes.

We'll eventually deprecate and remove the instruction-pointer
insertBefore, but not before adding concise documentation of what
considerations are needed (very few).

show more ...

[LoopSink] Exit loop finding BBs to sink into early when possible (NFC) (#101115)

As noted in the comments, findBBsToSinkInto is
O(UseBBs.size() * ColdLoopBBs.size())

A very large function wi

[LoopSink] Exit loop finding BBs to sink into early when possible (NFC) (#101115)

As noted in the comments, findBBsToSinkInto is
O(UseBBs.size() * ColdLoopBBs.size())

A very large function with a huge loop was incurring a high compile time
in this code. The size of the ColdLoopBBs set was over 14K. There is a
limit on the size of the UseBBs set, but not the ColdLoopBBs (and adding
a limit for the latter actually slowed down some later passes).

This change exits the loop early once we detect that there is no further
refinement possible for the BBsToSinkInto set. This is possible because
the ColdLoopBBs set is sorted in ascending magnitude of frequency.

This cut down the LoopSinkPass time by around 33% (78s to just over
50s).

show more ...

[NewPM] Remove LoopSinkLegacy Pass (#72811)

This pass isn't used anywhere and thus has no test coverage. For these
reasons, remove it.

Use BlockFrequency type in more places (NFC) (#68266)

The `BlockFrequency` class abstracts `uint64_t` frequency values. Use it
more consistently in various APIs and disable implicit conversion to

Use BlockFrequency type in more places (NFC) (#68266)

The `BlockFrequency` class abstracts `uint64_t` frequency values. Use it
more consistently in various APIs and disable implicit conversion to
make usage more consistent and explicit.

- Use `BlockFrequency Freq` parameter for `setBlockFreq`,
`getProfileCountFromFreq` and `setBlockFreqAndScale` functions.
- Return `BlockFrequency` in `getEntryFreq()` functions.
- While on it change some `const BlockFrequency& Freq` parameters to
plain `BlockFreqency Freq`.
- Mark `BlockFrequency(uint64_t)` constructor as explicit.
- Add missing `BlockFrequency::operator!=`.
- Remove `uint64_t BlockFreqency::getMaxFrequency()`.
- Add `BlockFrequency BlockFrequency::max()` function.

show more ...

[LoopSink] Don't sort BBs if there is only 1 of them (NFC)

This was trigerred by the debug check when comp(a, a) was called.
On line 216 it's checked that LoopBlockNumber should contain
all if there

[LoopSink] Don't sort BBs if there is only 1 of them (NFC)

This was trigerred by the debug check when comp(a, a) was called.
On line 216 it's checked that LoopBlockNumber should contain
all if there are more than 1. NFC for end users

Differential Revision: https://reviews.llvm.org/D157954

show more ...

[LoopSink] Allow sinking to PHI-use (2nd attempt)

This change allows sinking defs from loop preheader with PHI-use into loop body. Loop sink can now see through PHI-use and select incoming blocks of

[LoopSink] Allow sinking to PHI-use (2nd attempt)

This change allows sinking defs from loop preheader with PHI-use into loop body. Loop sink can now see through PHI-use and select incoming blocks of value being used as candidate sink destination.

It makes loop sink more effective so more LICM can be undone if proven unprofitable with profile info. It addresses the motivating case in D87551, without resorting to profile guided LICM which breaks canonicalization.

This is the 2nd attempt after D152772.

show more ...

Revert "[LoopSink] Allow sinking to PHI-use"

This reverts commit 54711a6a5872d5f97da4c0a1bd7e58d0546ca701.

The commit is causing a clang crash: https://reviews.llvm.org/D152772#4437254

[LoopSink] Allow sinking to PHI-use

This change allows sinking defs from loop preheader with PHI-use into loop body. Loop sink can now see through PHI-use and select incoming blocks of value being u

[LoopSink] Allow sinking to PHI-use

This change allows sinking defs from loop preheader with PHI-use into loop body. Loop sink can now see through PHI-use and select incoming blocks of value being used as candidate sink destination.

It makes loop sink more effective so more LICM can be undone if proven unprofitable with profile info. It addresses the motivating case in D87551, without resorting to profile guided LICM which breaks canonicalization.

Differential Revision: https://reviews.llvm.org/D152772

show more ...

[LICM] Require MSSA in SinkAndHoistLICMFlags (NFC)

Nowadays MSSA is required for LICM/LoopSink, so drop the checks
for whether its available or not.

[LoopSink] Don't fetch analyses without profile data (NFCI)

The loop sink pass only does something if the function has
profile data. Move the check for that before analyses are fetched,
to avoid com

[LoopSink] Don't fetch analyses without profile data (NFCI)

The loop sink pass only does something if the function has
profile data. Move the check for that before analyses are fetched,
to avoid computing things like BFI or MSSA unnecessarily.

show more ...

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955

show more ...

Recommit "[SCEV] Support clearing Block/LoopDispositions for a single value."

This reverts commit 92f698f01fa0bb8967233a6391ada2d51849fa68.

The updated version of the patch includes handling for no

Recommit "[SCEV] Support clearing Block/LoopDispositions for a single value."

This reverts commit 92f698f01fa0bb8967233a6391ada2d51849fa68.

The updated version of the patch includes handling for non-SCEVable
types. A test case has been added in ec86e9a99bca802.

show more ...

Revert "[SCEV] Support clearing Block/LoopDispositions for a single value."

This reverts commit 9e931439ddb9b6b8f655940b9d8ed6db50c2a7e2.

This commit causes a crash when TSan, e.g. with
https://lab

Revert "[SCEV] Support clearing Block/LoopDispositions for a single value."

This reverts commit 9e931439ddb9b6b8f655940b9d8ed6db50c2a7e2.

This commit causes a crash when TSan, e.g. with
https://lab.llvm.org/buildbot/#/builders/70/builds/28309/steps/10/logs/stdio

Reverting while I extract a reproducer and submit a fix.

show more ...

[SCEV] Support clearing Block/LoopDispositions for a single value.

Extend forgetBlockAndLoopDisposition to allow clearing information for a
single value. This can be useful when only a single value

[SCEV] Support clearing Block/LoopDispositions for a single value.

Extend forgetBlockAndLoopDisposition to allow clearing information for a
single value. This can be useful when only a single value is changed,
e.g. because the instruction is moved.

We also need to clear the cached values for all SCEV users, because they
may depend on the starting value's disposition.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D134614

show more ...

[SCEV][NFC] Remove unused parameter from forgetLoopDispositions

Let's be honest about it, we don't drop loop dispositions for
particular loops. Remove the parameter that misleadingly makes
it appare

[SCEV][NFC] Remove unused parameter from forgetLoopDispositions

Let's be honest about it, we don't drop loop dispositions for
particular loops. Remove the parameter that misleadingly makes
it apparent that we do.

show more ...

[Transforms] Fix comment typos (NFC)

[LICM] Pass MemorySSAUpdater by referene (NFC)

Make it clearer that this is a required dependency.

[LoopSink] Require MemorySSA

This makes MemorySSA in LoopSink required, and removes the AST-based
implementation, as well as the related support code in LICM.

Differential Revision: https://reviews

[LoopSink] Require MemorySSA

This makes MemorySSA in LoopSink required, and removes the AST-based
implementation, as well as the related support code in LICM.

Differential Revision: https://reviews.llvm.org/D123288

show more ...

[LoopSink] Use MemorySSA with legacy pass manager

LoopSink with the legacy pass manager still uses AST, because we
can't compute MemorySSA conditionally. I think now that the legacy
pass manager wil

[LoopSink] Use MemorySSA with legacy pass manager

LoopSink with the legacy pass manager still uses AST, because we
can't compute MemorySSA conditionally. I think now that the legacy
pass manager will be removed soon(TM) we don't need to care about
compile-time impact here anymore. Additionally, since MemorySSA is
no longer eagerly optimized, the impact is actually not that high
anymore (~0.2% geomean regression on CTMark).

This just makes legacy PM and new PM behavior line up -- as a
followup I'll drop these options entirely and make MemorySSA use
mandatory.

Differential Revision: https://reviews.llvm.org/D123216

show more ...

[Transform] Apply clang-tidy fixes for readability-redundant-smartptr-get (NFC)

[LoopSink] Do not try to sink phi nodes.

Skip phi nodes in the preheader. They may not be considered loop
invariant by the assertion below.

Reviewed By: reames

Differential Revision: https://revie

[LoopSink] Do not try to sink phi nodes.

Skip phi nodes in the preheader. They may not be considered loop
invariant by the assertion below.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D121010

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120741

show more ...

[Transforms] Use make_early_inc_range (NFC)

[llvm] Use set_is_subset (NFC)