Reapply "[DomTreeUpdater] Move critical edge splitting code to updater" (#119547)

This relands commit #115111.
Use traditional way to update post dominator tree, i.e. break critical
edge splitting

Reapply "[DomTreeUpdater] Move critical edge splitting code to updater" (#119547)

This relands commit #115111.
Use traditional way to update post dominator tree, i.e. break critical
edge splitting into insert, insert, delete sequence.
When splitting critical edges, the post dominator tree may change its
root node, and `setNewRoot` only works in normal dominator tree...
See

https://github.com/llvm/llvm-project/blob/6c7e5827eda26990e872eb7c3f0d7866ee3c3171/llvm/include/llvm/Support/GenericDomTree.h#L684-L687

show more ...

Revert "[DomTreeUpdater] Move critical edge splitting code to updater" (#119512)

Reverts llvm/llvm-project#115111 Causes #119511

[DomTreeUpdater] Move critical edge splitting code to updater (#115111)

Support critical edge splitting in dominator tree updater. Continue the
work in #100856.

Compile time check:
https://llvm

[DomTreeUpdater] Move critical edge splitting code to updater (#115111)

Support critical edge splitting in dominator tree updater. Continue the
work in #100856.

Compile time check:
https://llvm-compile-time-tracker.com/compare.php?from=87c35d782795b54911b3e3a91a5b738d4d870e55&to=42b3e5623a9ab4c3648564dc0926b36f3b438a3a&stat=instructions%3Au

show more ...

[MachineLoopInfo] Fix getLoopID to handle multi latches. (#106195)

This patch also fixed `CodegenPrepare` to preserve loop metadata when
merging blocks.

This fixes issue #102632

[MachineLoopInfo] Fix assertion failure on undef use operands (#100137)

Fixes #100115

Revert "[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)"

This reverts commit c5e5088033fed170068d818c54af6862e449b545.

Causes large compile-time regressions.

[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)

Summary:
- Remove wrappers in `MachineDominatorTree`.
- Remove `MachineDominatorTree` update code in
`MachineBasicBlo

[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)

Summary:
- Remove wrappers in `MachineDominatorTree`.
- Remove `MachineDominatorTree` update code in
`MachineBasicBlock::SplitCriticalEdge`.
- Use `MachineDomTreeUpdater` in passes which call
`MachineBasicBlock::SplitCriticalEdge` and preserve
`MachineDominatorTreeWrapperPass` or CFG analyses.

Commit abea99f65a97248974c02a5544eaf25fc4240056 introduced related
methods in 2014. Now we have SemiNCA based dominator tree in 2017 and
dominator tree updater, the solution adopted here seems a bit outdated.

show more ...

[CodeGen][NewPM] Port `machine-loops` to new pass manager (#97793)

- Add `MachineLoopAnalysis`.
- Add `MachineLoopPrinterPass`.
- Convert to `MachineLoopInfoWrapperPass` in legacy pass manager.

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree v

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.

show more ...

[CodeGen] Remove unused lambda capture (NFC)

llvm-project/llvm/lib/CodeGen/MachineLoopInfo.cpp:215:14:
error: lambda capture 'Reg' is not used [-Werror,-Wunused-lambda-capture]
[this, Reg](con

[CodeGen] Remove unused lambda capture (NFC)

llvm-project/llvm/lib/CodeGen/MachineLoopInfo.cpp:215:14:
error: lambda capture 'Reg' is not used [-Werror,-Wunused-lambda-capture]
[this, Reg](const MachineInstr &MI) { return this->contains(&MI); });
~~^~~
1 error generated.

show more ...

[MachineLICM] Give opportunity to analyze physregs for invariance. (#84779)

At the moment MachineLoopInfo has a very simple way to determine if a
use of a physical register will be invariant: if it

[MachineLICM] Give opportunity to analyze physregs for invariance. (#84779)

At the moment MachineLoopInfo has a very simple way to determine if a
use of a physical register will be invariant: if it is not a constant
value or if it's not an ignorable use, then it's not considered
invariant.

From a compile-time performance perspective this makes a lot of sense,
but it limits code that uses implicit physical registers from being
hoisted until the later MachineLICM pass (after register allocation),
which has a lot fewer opportunities to hoist.

For AArch64 SME we use an implicit physical register ($vg) to avoid
rematerialization beyond certain instructions. Doing this led to
regressions because simple expressions were no longer hoisted by Early
MachineLICM.

This patch adds some extra checks to 'isLoopInvariant' to see if any of
the defs are found in the loop. If not, we can considered it loop
invariant.

We expect the impact on compile-time to be negligible because there is
an incentive for users to reduce the need for the smstart/smstop
instructions that define $vg. In either case, we've put the
functionality under a target interface to limit this only to specific
registers.

show more ...

[MachineLICM] Hoist COPY instruction only when user can be hoisted (#81735)

befa925acac8fd6a9266e introduced preliminary hoisting of COPY
instructions when the user of the COPY is inside the same l

[MachineLICM] Hoist COPY instruction only when user can be hoisted (#81735)

befa925acac8fd6a9266e introduced preliminary hoisting of COPY
instructions when the user of the COPY is inside the same loop. That
optimization appeared to be too aggressive and hoisted too many COPY's
greatly increasing register pressure causing performance regressions for
AMDGPU target.

This is intended to fix the regression by hoisting COPY instruction only
if either:
- User of COPY can be hoisted (other args are invariant)
or
- Hoisting COPY doesn't bring high register pressure

show more ...

[MachineBlockPlacement][X86] Use max of MDAlign and TLIAlign to align Loops. (#71026)

This patch added backend consumption on a new loop metadata:
!1 = !{!"llvm.loop.align", i32 64}
which is gener

[MachineBlockPlacement][X86] Use max of MDAlign and TLIAlign to align Loops. (#71026)

This patch added backend consumption on a new loop metadata:
!1 = !{!"llvm.loop.align", i32 64}
which is generated from clang's new loop attribute:
[[clang::code_align()]]
clang patch: #70762

show more ...

[LoopInfo] Move generic LoopInfo into own files

This commit splits the generic part of `LoopInfo` into separate files.
These new `GenericLoopInfo` files are located in `llvm/Support` to be inline
wi

[LoopInfo] Move generic LoopInfo into own files

This commit splits the generic part of `LoopInfo` into separate files.
These new `GenericLoopInfo` files are located in `llvm/Support` to be inline
with `GenericDomTree`.

Furthermore, this change ensures that MLIR's Bazel build does not have
to link against `LLVMAnalysis` just to use these template headers.

Depends on D148219

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D148235

show more ...

[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFC

Use isPhysical/isVirtual methods.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D141715

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-in

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121681

show more ...

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https:/

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https://reviews.llvm.org/D121169

show more ...

Cleanup codegen includes

after: 1061034926
before: 1063332844

Differential Revision: https://reviews.llvm.org/D121169

[AMDGPU] MachineLICM cannot hoist VALU

MachineLoop::isLoopInvariant() returns false for all VALU
because of the exec use. Check TII::isIgnorableUse() to
allow hoisting.

That unfortunately results i

[AMDGPU] MachineLICM cannot hoist VALU

MachineLoop::isLoopInvariant() returns false for all VALU
because of the exec use. Check TII::isIgnorableUse() to
allow hoisting.

That unfortunately results in higher register consumption
since MachineLICM does not adequately estimate pressure.
Therefor I think it shall only be enabled after D107677 even
though it does not depend on it.

Differential Revision: https://reviews.llvm.org/D107859

show more ...

[ARM] Allow findLoopPreheader to return headers with multiple loop successors

The findLoopPreheader function will currently not find a preheader if it
branches to multiple different loop headers. Th

[ARM] Allow findLoopPreheader to return headers with multiple loop successors

The findLoopPreheader function will currently not find a preheader if it
branches to multiple different loop headers. This patch adds an option
to relax that, allowing ARMLowOverheadLoops to process more loops
successfully. This helps with WhileLoopStart setup instructions that can
branch/fallthrough to the low overhead loop and to branch to a separate
loop from the same preheader (but I don't believe it is possible for
both loops to be low overhead loops).

Differential Revision: https://reviews.llvm.org/D102747

show more ...

[MachineLoop] New helper isLoopInvariant()

This factors out code from MachineLICM that determines whether an instruction
is loop-invariant, which is a generally useful function. Thus this allows to

[MachineLoop] New helper isLoopInvariant()

This factors out code from MachineLICM that determines whether an instruction
is loop-invariant, which is a generally useful function. Thus this allows to
use that helper elsewhere too.

Differential Revision: https://reviews.llvm.org/D94082

show more ...

Revert "Include static prof data when collecting loop BBs"

This reverts commit 129c911efaa492790c251b3eb18e4db36b55cbc5.

Due to an internal benchmark regression.

Include static prof data when collecting loop BBs

Summary:
If the programmer adds static profile data to a branch---i.e. uses
"__builtin_expect()" or similar---then we should honor it. Otherwise,
"_

Include static prof data when collecting loop BBs

Summary:
If the programmer adds static profile data to a branch---i.e. uses
"__builtin_expect()" or similar---then we should honor it. Otherwise,
"__builtin_expect()" is ignored in crucial situations. So we trust that
the programmer knows what they're doing until proven wrong.

Subscribers: hiraditya, JDevlieghere, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74809

show more ...

Sink all InitializePasses.h includes

This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of reco

Sink all InitializePasses.h includes

This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of recompilation.

I found this fact by looking at this table, which is sorted by the
number of times a file was changed over the last 100,000 git commits
multiplied by the number of object files that depend on it in the
current checkout:
recompiles touches affected_files header
342380 95 3604 llvm/include/llvm/ADT/STLExtras.h
314730 234 1345 llvm/include/llvm/InitializePasses.h
307036 118 2602 llvm/include/llvm/ADT/APInt.h
213049 59 3611 llvm/include/llvm/Support/MathExtras.h
170422 47 3626 llvm/include/llvm/Support/Compiler.h
162225 45 3605 llvm/include/llvm/ADT/Optional.h
158319 63 2513 llvm/include/llvm/ADT/Triple.h
140322 39 3598 llvm/include/llvm/ADT/StringRef.h
137647 59 2333 llvm/include/llvm/Support/Error.h
131619 73 1803 llvm/include/llvm/Support/FileSystem.h

Before this change, touching InitializePasses.h would cause 1345 files
to recompile. After this change, touching it only causes 550 compiles in
an incremental rebuild.

Reviewers: bkramer, asbirlea, bollu, jdoerfert

Differential Revision: https://reviews.llvm.org/D70211

show more ...

[PGO][PGSO] SizeOpts changes.

Summary:
(Split of off D67120)

SizeOpts/MachineSizeOpts changes for profile guided size optimization.

(A second try after previously committed as r375254 and reverted

[PGO][PGSO] SizeOpts changes.

Summary:
(Split of off D67120)

SizeOpts/MachineSizeOpts changes for profile guided size optimization.

(A second try after previously committed as r375254 and reverted as r375375.)

Subscribers: mgorny, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69409

show more ...