Reapply "[DomTreeUpdater] Move critical edge splitting code to updater" (#119547)

This relands commit #115111.
Use traditional way to update post dominator tree, i.e. break critical
edge splitting

Reapply "[DomTreeUpdater] Move critical edge splitting code to updater" (#119547)

This relands commit #115111.
Use traditional way to update post dominator tree, i.e. break critical
edge splitting into insert, insert, delete sequence.
When splitting critical edges, the post dominator tree may change its
root node, and `setNewRoot` only works in normal dominator tree...
See

https://github.com/llvm/llvm-project/blob/6c7e5827eda26990e872eb7c3f0d7866ee3c3171/llvm/include/llvm/Support/GenericDomTree.h#L684-L687

show more ...

Revert "[DomTreeUpdater] Move critical edge splitting code to updater" (#119512)

Reverts llvm/llvm-project#115111 Causes #119511

[DomTreeUpdater] Move critical edge splitting code to updater (#115111)

Support critical edge splitting in dominator tree updater. Continue the
work in #100856.

Compile time check:
https://llvm

[DomTreeUpdater] Move critical edge splitting code to updater (#115111)

Support critical edge splitting in dominator tree updater. Continue the
work in #100856.

Compile time check:
https://llvm-compile-time-tracker.com/compare.php?from=87c35d782795b54911b3e3a91a5b738d4d870e55&to=42b3e5623a9ab4c3648564dc0926b36f3b438a3a&stat=instructions%3Au

show more ...

[AMDGPU] Remove unused includes (NFC) (#116154)

Identified with misc-include-cleaner.

AMDGPU/NewPM: Port SILowerI1Copies to new pass manager (#102663)

Revert "[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)"

This reverts commit c5e5088033fed170068d818c54af6862e449b545.

Causes large compile-time regressions.

[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)

Summary:
- Remove wrappers in `MachineDominatorTree`.
- Remove `MachineDominatorTree` update code in
`MachineBasicBlo

[CodeGen] Remove `applySplitCriticalEdges` in `MachineDominatorTree` (#97055)

Summary:
- Remove wrappers in `MachineDominatorTree`.
- Remove `MachineDominatorTree` update code in
`MachineBasicBlock::SplitCriticalEdge`.
- Use `MachineDomTreeUpdater` in passes which call
`MachineBasicBlock::SplitCriticalEdge` and preserve
`MachineDominatorTreeWrapperPass` or CFG analyses.

Commit abea99f65a97248974c02a5544eaf25fc4240056 introduced related
methods in 2014. Now we have SemiNCA based dominator tree in 2017 and
dominator tree updater, the solution adopted here seems a bit outdated.

show more ...

[CodeGen][NewPM] Split `MachinePostDominators` into a concrete analysis result (#95113)

`MachinePostDominators` version of #94571.

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree v

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.

show more ...

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#80003)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 p

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#80003)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 phis and select
them as lane mask phis in same way SILowerI1Copies select VReg_1 phis.
Note that divergent i1 phis include phis created by LCSSA and all cases
of uses outside of cycle are actually covered by "lowering LCSSA phis".
GlobalISel lane masks are registers with sgpr register class and S1 LLT.

TODO: General goal is that instructions created in this pass are fully
instruction-selected so that selection of lane mask phis is not split
across multiple passes.

patch 3 from: https://github.com/llvm/llvm-project/pull/73337

show more ...

Revert "AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis" (#79274)

Reverts llvm/llvm-project#78482

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#78482)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 p

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#78482)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 phis and select
them as lane mask phis in same way SILowerI1Copies select VReg_1 phis.
Note that divergent i1 phis include phis created by LCSSA and all cases
of uses outside of cycle are actually covered by "lowering LCSSA phis".
GlobalISel lane masks are registers with sgpr register class and S1 LLT.

TODO: General goal is that instructions created in this pass are fully
instruction-selected so that selection of lane mask phis is not split
across multiple passes.

patch 3 from: https://github.com/llvm/llvm-project/pull/73337

show more ...

Revert "AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis" (#78468)

Reverts llvm/llvm-project#76145

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#76145)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 p

AMDGPU/GlobalISelDivergenceLowering: select divergent i1 phis (#76145)

Implement PhiLoweringHelper for GlobalISel in DivergenceLoweringHelper.
Use machine uniformity analysis to find divergent i1 phis and select
them as lane mask phis in same way SILowerI1Copies select VReg_1 phis.
Note that divergent i1 phis include phis created by LCSSA and all cases
of uses outside of cycle are actually covered by "lowering LCSSA phis".
GlobalISel lane masks are registers with sgpr register class and S1 LLT.

TODO: General goal is that instructions created in this pass are fully
instruction-selected so that selection of lane mask phis is not split
across multiple passes.

patch 3 from: https://github.com/llvm/llvm-project/pull/73337

show more ...

AMDGPU: refactor phi lowering from SILowerI1Copies (NFCI) (#75349)

Make abstract class PhiLoweringHelper and expose it for use in
GlobalISel path.
SILowerI1Copies implements PhiLoweringHelper as V

AMDGPU: refactor phi lowering from SILowerI1Copies (NFCI) (#75349)

Make abstract class PhiLoweringHelper and expose it for use in
GlobalISel path.
SILowerI1Copies implements PhiLoweringHelper as Vreg1LoweringHelper and
it is equivalent to SILowerI1Copies.
Notable change that createLaneMaskReg now clones attributes from
register that has lane mask attributes instead of creating register with
lane mask register class. This is because lane masks have
different(more) attributes in GlobalISel.

patch 2 from: https://github.com/llvm/llvm-project/pull/73337

show more ...

AMDGPU/SILowerI1Copies process phi incomings in specific order (#72375)

When merging lane masks, value from block that is always visited first
(PrevReg in buildMergeLaneMasks) needs to exist becaus

AMDGPU/SILowerI1Copies process phi incomings in specific order (#72375)

When merging lane masks, value from block that is always visited first
(PrevReg in buildMergeLaneMasks) needs to exist because we do on-the-fly
constant folding. For PrevReg to exist, basic block that should contain
PrevReg definition must be processed first. Sort the incomings such that
incoming values that dominate other incoming values are processed first.

Sorting of phi incomings makes no changes for phis created by SDAG
because SDAG adds phi incomings as it selects basic blocks in reversed
post order traversal.

This change is required by upcoming lane mask merging implementation
for GlobalISel that leaves phi incomings as they are in IR.

show more ...

AMDGPU/SILowerI1Copies: refactor phi incoming handling [NFC] (#72374)

Incoming block, incoming Register and updated incoming Register that
correspond to the same incoming of a phi are kept on same

AMDGPU/SILowerI1Copies: refactor phi incoming handling [NFC] (#72374)

Incoming block, incoming Register and updated incoming Register that
correspond to the same incoming of a phi are kept on same index in
different vectors.
Use structure with fields: block, register and updated register instead.

show more ...

[AMDGPU] SILowerI1Copies: clear kill flags on COPY (#65883)

Clear kill flags on COPY source as it will be reused.

AMDGPU: Factor out hasDivergentBranch(). NFC

This is helpful for detecting whether a block ends with divergent branch
in passes before lowering the pseudo control flow instructions.

Differential Re

AMDGPU: Factor out hasDivergentBranch(). NFC

This is helpful for detecting whether a block ends with divergent branch
in passes before lowering the pseudo control flow instructions.

Differential Revision: https://reviews.llvm.org/D133184

show more ...

[AMDGPU] Return better Changed status from SILowerI1Copies

Differential Revision: https://reviews.llvm.org/D119946

AMDGPU: Treat IMPLICIT_DEF like a constant lanemask source

This is partially a workaround. SILowerI1Copies does not understand
unstructured loops. This would result in inserting instructions to
merg

AMDGPU: Treat IMPLICIT_DEF like a constant lanemask source

This is partially a workaround. SILowerI1Copies does not understand
unstructured loops. This would result in inserting instructions to
merge a mask register in the same block where it was defined in an
unstructured loop.

show more ...

[Target] Use llvm::append_range (NFC)

Revert "[Target] Use llvm::append_range (NFC)"

This reverts commit cc7a23828657f35f706343982cf96bb6583d4d73.

The X86WinEHState.cpp hunk seems to break certain builds.

[Target] Use llvm::append_range (NFC)

[NFC][AMDGPU] Split AMDGPUSubtarget.h to R600 and GCN subtargets

... to reduce headers dependency.

Reviewed By: rampitec, arsenm

Differential Revision: https://reviews.llvm.org/D95036