[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneede

[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneeded use of a
template.

show more ...

[MISched] Unify the way to specify scheduling direction (#119518)

For pre-ra scheduling, we use two options `-misched-topdown` and
`-misched-bottomup` to force the direction.

While for post-ra sche

[MISched] Unify the way to specify scheduling direction (#119518)

For pre-ra scheduling, we use two options `-misched-topdown` and
`-misched-bottomup` to force the direction.

While for post-ra scheduling, we use `-misched-postra-direction`
with enumerated values (`topdown`, `bottomup` and `bidirectional`).

This is not unified and adds some mental burdens. Here we replace
these two options `-misched-topdown` and `-misched-bottomup` with
`-misched-prera-direction` with the same enumerated values.

To avoid the condition of `getNumOccurrences() > 0`, we add a new
enum value `Unspecified` and make it the default initial value.

These options are hidden, so we needn't keep the compatibility.

show more ...

[MISched] Compare right next cluster node (#116584)

We support bottom-up and bidirectonal postra scheduling now, but we
only compare successive next cluster node as if we are doing topdown
schedulin

[MISched] Compare right next cluster node (#116584)

We support bottom-up and bidirectonal postra scheduling now, but we
only compare successive next cluster node as if we are doing topdown
scheduling. This makes load/store clustering and macro fusions wrong.

This patch makes sure that we can get the right cluster node by the
scheduling direction.

show more ...

[Sched] Skip MemOp with unknown size when clustering (#118443)

In #83875, we changed the type of `Width` to `LocationSize`. To get
the clsuter bytes, we use `LocationSize::getValue()` to calculate
t

[Sched] Skip MemOp with unknown size when clustering (#118443)

In #83875, we changed the type of `Width` to `LocationSize`. To get
the clsuter bytes, we use `LocationSize::getValue()` to calculate
the value.

But when `Width` is an unknown size `LocationSize`, an assertion
"Getting value from an unknown LocationSize!" will be triggered.

This patch simply skips MemOp with unknown size to fix this issue
and keep the logic the same as before.

This issue was found when implementing software pipeliner for
RISC-V in #117546. The pipeliner may clone some memory operations
with `BeforeOrAfterPointer` size.

show more ...

[MISched] Use right boundary when trying latency heuristics (#116592)

We may do bottom-up or bidirectional scheduling but previously we
assume we are doing top-down scheduling, which may cause some

[MISched] Use right boundary when trying latency heuristics (#116592)

We may do bottom-up or bidirectional scheduling but previously we
assume we are doing top-down scheduling, which may cause some issues.

show more ...

[MISched] Add a hook to override PostRA scheduling policy (#115455)


PostRA scheduling supports different directions now, but we can
only specify it via command line options.

This patch adds a new

[MISched] Add a hook to override PostRA scheduling policy (#115455)


PostRA scheduling supports different directions now, but we can
only specify it via command line options.

This patch adds a new hook `overridePostRASchedPolicy` for targets
to override PostRA scheduling policy.

Note that some options like tracking register pressure won't take
effect in PostRA scheduling.

show more ...

[CodeGen][MISched] Set DumpDirection after initPolicy (#115112)


Previously we set the dump direction according to command line
options, but we may override the scheduling direction in `initPolicy`

[CodeGen][MISched] Set DumpDirection after initPolicy (#115112)


Previously we set the dump direction according to command line
options, but we may override the scheduling direction in `initPolicy`
and this results in mismatch between dump and actual policy.

Here we simply set the dump direction after initializing the policy.

show more ...

llvm-reduce: Don't print verifier failed machine functions (#109673)

This produces far too much terminal output, particularly for the
instruction reduction. Since it doesn't consider the liveness o

llvm-reduce: Don't print verifier failed machine functions (#109673)

This produces far too much terminal output, particularly for the
instruction reduction. Since it doesn't consider the liveness of of
the instructions it's deleting, it produces quite a lot of verifier
errors.

show more ...

[ExtendLifetimes] Implement llvm.fake.use to extend variable lifetimes (#86149)

This patch is part of a set of patches that add an `-fextend-lifetimes`
flag to clang, which extends the lifetimes of

[ExtendLifetimes] Implement llvm.fake.use to extend variable lifetimes (#86149)

This patch is part of a set of patches that add an `-fextend-lifetimes`
flag to clang, which extends the lifetimes of local variables and
parameters for improved debuggability. In addition to that flag, the
patch series adds a pragma to selectively disable `-fextend-lifetimes`,
and an `-fextend-this-ptr` flag which functions as `-fextend-lifetimes`
for this pointers only. All changes and tests in these patches were
written by Wolfgang Pieb (@wolfy1961), while Stephen Tozer (@SLTozer)
has handled review and merging. The extend lifetimes flag is intended to
eventually be set on by `-Og`, as discussed in the RFC
here:

https://discourse.llvm.org/t/rfc-redefine-og-o1-and-add-a-new-level-of-og/72850

This patch implements a new intrinsic instruction in LLVM,
`llvm.fake.use` in IR and `FAKE_USE` in MIR, that takes a single operand
and has no effect other than "using" its operand, to ensure that its
operand remains live until after the fake use. This patch does not emit
fake uses anywhere; the next patch in this sequence causes them to be
emitted from the clang frontend, such that for each variable (or this) a
fake.use operand is inserted at the end of that variable's scope, using
that variable's value. This patch covers everything post-frontend, which
is largely just the basic plumbing for a new intrinsic/instruction,
along with a few steps to preserve the fake uses through optimizations
(such as moving them ahead of a tail call or translating them through
SROA).

Co-authored-by: Stephen Tozer <stephen.tozer@sony.com>

show more ...

[CodeGen] Construct SmallVector with ArrayRef (NFC) (#101841)

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use `std::unique_ptr` instead of raw pointer for `LICalc`, so
destructor and default move constructor can handle it correctly.

This would be the last analysis required by `PHIElimination`.

show more ...

[CodeGen][NewPM] Port `SlotIndexes` to new pass manager (#97941)

- Add `SlotIndexesAnalysis`.
- Add `SlotIndexesPrinterPass`.
- Use `SlotIndexesWrapperPass` in legacy pass.

[CodeGen][NewPM] Port `machine-loops` to new pass manager (#97793)

- Add `MachineLoopAnalysis`.
- Add `MachineLoopPrinterPass`.
- Convert to `MachineLoopInfoWrapperPass` in legacy pass manager.

[llvm][CodeGen] Avoid 'raw_string_ostream::str' (NFC) (#97318)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference

[llvm][CodeGen] Avoid 'raw_string_ostream::str' (NFC) (#97318)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.

Work towards TODO comment to remove `raw_string_ostream::str()`.

show more ...

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree v

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.

show more ...

[MISched][NFC] Add documentation comment in pickNode for ReadyQueue maintenence (#92976)

I had some trouble understanding why `removeReady` removed nodes from
the Pending queue, since my intuition

[MISched][NFC] Add documentation comment in pickNode for ReadyQueue maintenence (#92976)

I had some trouble understanding why `removeReady` removed nodes from
the Pending queue, since my intuition told me that the Pending queue did
not represent a node that was ready. I took a deeper look and found that
pickOnlyNode and pickNodeFromQueue only picked nodes from the Available
queue too.

I found that need to nodes from the Available and Pending queues that
correspond to the opposite direction that we ended up choosing from
(IsTopNode vs !IsTopNode).

It took me a little longer than I would have liked to understand this
fact, so I figured that I would add a comment in the code that makes it
clear for future readers.

show more ...

MachineScheduler: Add parameter name comments

[mi-sched] Suppress register pressure with i64. (#88256)

Machine scheduler will suppress register pressure when the scheduling
window is too small, but now it doesn't consider i64 register type,
a

[mi-sched] Suppress register pressure with i64. (#88256)

Machine scheduler will suppress register pressure when the scheduling
window is too small, but now it doesn't consider i64 register type,
and this MR extends it into i64 register type, so architecture like
RISCV64 that only supports i64 interger register will have the same
behavior like RISCV32.

show more ...

MachineScheduler: Simplify usage of TargetInstrInfo

[CodeGen][MISched] Add misched post-regalloc bidirectional scheduling (#77138)

This PR is stacked on #76186.

This PR keeps the default strategy as top-down since that is what
existing targets ex

[CodeGen][MISched] Add misched post-regalloc bidirectional scheduling (#77138)

This PR is stacked on #76186.

This PR keeps the default strategy as top-down since that is what
existing targets expect. It can be enabled using
`-misched-postra-direction=bidirectional`.

It is up to targets to decide whether they would like to enable this
option for themselves.

show more ...

[Codegen] Make Width in getMemOperandsWithOffsetWidth a LocationSize. (#83875)

This is another part of #70452 which makes getMemOperandsWithOffsetWidth
use a LocationSize for Width, as opposed to t

[Codegen] Make Width in getMemOperandsWithOffsetWidth a LocationSize. (#83875)

This is another part of #70452 which makes getMemOperandsWithOffsetWidth
use a LocationSize for Width, as opposed to the unsigned it currently
uses. The advantages on it's own are not super high if
getMemOperandsWithOffsetWidth usually uses known sizes, but if the
values can come from an MMO it can help be more accurate in case they
are Unknown (and in the future, scalable).

show more ...

[CodeGen][MISched] Add misched post-regalloc bottom-up scheduling

There is the possibility that the bottom-up direction will lead to
performance improvements on certain targets, as this is certainly

[CodeGen][MISched] Add misched post-regalloc bottom-up scheduling

There is the possibility that the bottom-up direction will lead to
performance improvements on certain targets, as this is certainly the case for
the pre-regalloc GenericScheduler. This patch will give people the
opportunity to experiment for their sub-targets. However, this patch
keeps the top-down approach as the default for the PostGenericScheduler
since that is what subtargets expect today.

show more ...

[CodeGen][MISched] dumpSched direction depends on field in DAG.

This is a precommit to supporting post reg-alloc bottom up scheduling.
We'd like to have post-ra scheduling direction that can be diff

[CodeGen][MISched] dumpSched direction depends on field in DAG.

This is a precommit to supporting post reg-alloc bottom up scheduling.
We'd like to have post-ra scheduling direction that can be different from
pre-ra direction. The current dumpSchedule function is changed in this
patch to support the fact that the post-ra and pre-ra directions will
depend on different command line options.

show more ...

[Sched] Add MacroFusion mutation if fusions are not empty (#72227)

We can get the fusions list by `getMacroFusions` and if it is not
empty, then we will add the MacroFusion mutation automatically.

[llvm] Move CodeGenTypes library to its own directory (#79444)

Finally addresses https://reviews.llvm.org/D148769#4311232 :)

No behavior change.