[StructurizeCFG] Use `poison` instead of `undef` as placeholder [NFC] (#119137)

[StructurizeCFG] Refactor insertConditions. NFC. (#115476)

This just makes it more obvious that having Parent as the single
predecessor is a special case, instead of checking for it in the middle

[StructurizeCFG] Refactor insertConditions. NFC. (#115476)

This just makes it more obvious that having Parent as the single
predecessor is a special case, instead of checking for it in the middle
of a loop that finds the nearest common dominator of multiple
predecessors.

show more ...

[StructurizeCFG] Remove one SSAUpdater::AddAvailableValue. NFCI. (#115472)

[StructurizeCFG] Introduce struct PredInfo. NFC. (#115457)

This just provides a neater encapsulation of the info about the
predicate for an edge, rather than ValueWeightPair aka std::pair.

[Scalar] Remove unused includes (NFC) (#114645)

Identified with misc-include-cleaner.

Reapply "StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)" (#114347)

This reverts commit be40c723ce2b7bf2690d22039d74d21b2bd5b7cf.

[AMDGPU][StructurizeCFG] Maintain branch MD_prof metadata (#109813)

Currently `StructurizeCFG` drops branch_weight metadata .
This metadata can be generated from user annotations in the source code

[AMDGPU][StructurizeCFG] Maintain branch MD_prof metadata (#109813)

Currently `StructurizeCFG` drops branch_weight metadata .
This metadata can be generated from user annotations in the source code
like:

```cpp
if (...) [[likely]] {
}
```

show more ...

[StructurizeCFG] Avoid repeated hash lookups (NFC) (#107797)

StructurizeCFG: Add SkipUniformRegions pass parameter to new PM version (#102812)

Keep respecting the old cl::opt for now.

Revert "StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)"

This reverts commit c62e2a2a4ed69d53a3c6ca5c24ee8d2504d6ba2b.

Since it caused regression in HIP buildbot:

https:

Revert "StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)"

This reverts commit c62e2a2a4ed69d53a3c6ca5c24ee8d2504d6ba2b.

Since it caused regression in HIP buildbot:

https://lab.llvm.org/buildbot/#/builders/123/builds/3282

show more ...

StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)

After investigating more while-break cases, I think we should try to
optimize
the way we reconstruct phi nodes. Previousl

StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)

After investigating more while-break cases, I think we should try to
optimize
the way we reconstruct phi nodes. Previously, we reconstruct each phi
nodes separately, but this is not optimal. For example:

```
header:
%v.1 = phi float [ %v, %entry ], [ %v.2, %latch ]
br i1 %cc, label %if, label %latch

if:
%v.if = fadd float %v.1, 1.0
br i1 %cc2, label %latch, label %exit

latch:
%v.2 = phi float [ %v.if, %if ], [ %v.1, %header ]
br i1 %cc3, label %exit, label %header

exit:
%v.3 = phi float [ %v.2, %latch ], [ %v.if, %if ]
```

For this case, we have different copies of value `v`, but there is at
most one copy of value `v` alive at any program point shown above.

The existing ssa reconstruction will use the incoming values from the
old deleted phi. Below is a possible output after ssa reconstruction.

```
header:
%v.1 = phi float [ %v, %entry ], [ %v.loop, %Flow1 ]
br i1 %cc, label %if, label %flow

if:
%v.if = fadd float %v.1, 1.0
br label %flow

flow:
%v.exit.if = phi float [ %v.if, %if ], [ undef, %header ]
%v.latch = phi float [ %v.if, %if ], [ %v.1, %header ]

latch:
br label %flow1

flow1:
%v.loop = phi float [ %v.latch, %latch ], [ undef, %Flow ]
%v.exit = phi float [ %v.latch, %latch ], [ %v.exit.if, %Flow ]

exit:
%v.3 = phi float [ %v.exit, %flow1 ]
```

If we look closely, in order to reconstruct `v.1` `v.2` `v.3`, we are
having two simultaneous copies of `v` alive at `flow` and `flow1`.
We highly depend on register coalescer to coalesce them together.
But register coalescer may not always be able to coalesce them
because of the complexity in the chain of phi.

On the other side, now that we have only one copy of `v` alive at any
program point before the transform, why not simplify the phi network
as much as we can? Look at the incoming values of these PHIs:
```
header if latch
v.1: -- -- v.2
v.2: v.1 v.if --
v.3: -- v.if v.2
```
If we let them share the same incoming values for these three different
incoming blocks, then we would have only one copy of alive `v` at any
program point after ssa reconstruction. Something like:

```
header:
%v.1 = phi float [ %v, %entry ], [ %v.2, %Flow1 ]
br i1 %cc, label %if, label %flow

if:
%v.if = fadd float %v.1, 1.0
br label %flow

flow:
%v.2 = phi float [ %v.if, %if ], [ %v.1, %header ]

latch:
br label %flow1

flow1:
...

exit:
%v.3 = phi float [ %v.2, %flow1 ]
```

show more ...

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, re

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, replacing the
current `getParent()->getDataLayout()` pattern.

show more ...

[LowerSwitch] Don't let pass manager handle the dependency (#68662)

Some passes has limitation that only support simple terminators:
branch/unreachable/return. Right now, they ask the pass manager

[LowerSwitch] Don't let pass manager handle the dependency (#68662)

Some passes has limitation that only support simple terminators:
branch/unreachable/return. Right now, they ask the pass manager to add
LowerSwitch pass to eliminate `switch`. Let's manage such kind of pass
dependency by ourselves. Also add the assertion in the related passes.

show more ...

[StructurizeCFG] Use poison instead of undef as placeholder [NFC]
These are used to create branch instructions. The condition is patched later

[StructurizeCFG] Correctly depend on UniformityAnalysis

Small oversight in https://reviews.llvm.org/D145688 - the pass' dependency was not updated to reflect the change to UA.

Also, change Divergen

[StructurizeCFG] Correctly depend on UniformityAnalysis

Small oversight in https://reviews.llvm.org/D145688 - the pass' dependency was not updated to reflect the change to UA.

Also, change DivergenceAnalysis to UniformityAnalysis in a comment. That way, StructurizeCFG only refers to UA and not DA anymore.

show more ...

[StructurizeCFG] Use UniformityAnalysis instead of DivergenceAnalysis

Depends on D145572

Reviewed By: foad, sameerds

Differential Revision: https://reviews.llvm.org/D145688

[StructurizeCFG][DebugInfo] Avoid use-after-free

Reviewed By: dstuttard

Differential Revision: https://reviews.llvm.org/D137408

[StructurizeCFG][DebugInfo] Maintain DILocations in the branches created by StructurizeCFG

Make StructurizeCFG preserve the debug locations of the branch instructions it introduces.

Differential Re

[StructurizeCFG][DebugInfo] Maintain DILocations in the branches created by StructurizeCFG

Make StructurizeCFG preserve the debug locations of the branch instructions it introduces.

Differential Revision: https://reviews.llvm.org/D135967

show more ...

[StructurizeCFG] Remove imposible case and replace by assert

In addition, replace outdated XFAIL test by a new one.

Differential Revision: https://reviews.llvm.org/D134439

StructurizeCFG: Set Undef for non-predecessors in setPhiValues()

During structurization process, we may place non-predecessor blocks
between the predecessors of a block in the structurized CFG. Take

StructurizeCFG: Set Undef for non-predecessors in setPhiValues()

During structurization process, we may place non-predecessor blocks
between the predecessors of a block in the structurized CFG. Take
the typical while-break case as an example:
```
/---A(v=...)
| / \
^ B C
| \ /|
\---L |
\ /
E (r = phi (v:C)...)
```
After structurization, the CFG would be look like:
```
/---A
| |\
| | C
| |/
| F1
^ |\
| | B
| |/
| F2
| |\
| | L
\ |/
\--F3
|
E
```
We can see that block B is placed between the predecessors(C/L) of E.
During phi reconstruction, to achieve the same sematics as before, we
are reconstructing the PHIs as:
F1: v1 = phi (v:C), (undef:A)
F3: r = phi (v1:F2), ...
But this is also saying that `v1` would be live through B, which is not
quite necessary. The idea in the change is to say the incoming value
from B is Undef for the PHI in E. With this change, the reconstructed
PHI would be:
F1: v1 = phi (v:C), (undef:A)
F2: v2 = phi (v1:F1), (undef:B)
F3: r = phi (v2:F2), ...

Reviewed by: sameerds

Differential Revision: https://reviews.llvm.org/D132450

show more ...

StructurizeCFG: prefer reduced number of live values

The instruction simplification will try to simplify the affected phis.
In some cases, this might extend the liveness of values. For example:

B

StructurizeCFG: prefer reduced number of live values

The instruction simplification will try to simplify the affected phis.
In some cases, this might extend the liveness of values. For example:

BB0:
| \
| BB1
| /
BB2:phi (BB0, v), (BB1, undef)

The phi in BB2 will be simplified to v as v dominates BB2, but this is
increasing the number of active values in BB1. By setting CanUseUndef
to false, we will not simplify the phi in this way, this would help
register pressure. This is mandatory for the later change to help
reducing VGPR pressure for AMDGPU.

Reviewed by: foad, sameerds

Differential Revision: https://reviews.llvm.org/D132449

show more ...

[Scalar] Qualify auto in range-based for loops (NFC)

Identified with readability-qualified-auto.

[llvm] Qualify auto (NFC)

Identified with readability-qualified-auto.

Revert "[StructurizeCFG] Improve basic block ordering"

This reverts commit f1b05a0a2bbbea160002be709f8a1c59de366761.

Need to revert to due to issues identified with testing. The
transformation is i

Revert "[StructurizeCFG] Improve basic block ordering"

This reverts commit f1b05a0a2bbbea160002be709f8a1c59de366761.

Need to revert to due to issues identified with testing. The
transformation is incorrect for blocks that contain convergent
instructions.

show more ...

[StructurizeCFG] Improve basic block ordering

StructurizeCFG linearizes the successors of branching basic block
by adding Flow blocks to record the true/false path for branches
and back edges. This

[StructurizeCFG] Improve basic block ordering

StructurizeCFG linearizes the successors of branching basic block
by adding Flow blocks to record the true/false path for branches
and back edges. This patch reduces the number of Phi values needed
to capture the control flow path by improving the basic block
ordering.

Previously, StructurizeCFG adds loop exit blocks outside of the
loop. StructurizeCFG sets a boolean value to indicate the path
taken, and all exit block live values extend to after the loop.
For loops with a large number of exits blocks, this creates a
huge number of values that are maintained, which increases
compilation time and register pressure. This is problem
especially with ASAN, which adds early exits to blocks with
unreachable instructions for each instrumented check in the loop.

In specific cases, this patch reduces the number of values needed
after the loop by moving the exit block into the loop. This is
done for blocks that have a single predecessor and single successor
by moving the block to appear just after the predecessor.

Differential Revision: https://reviews.llvm.org/D123231

show more ...