[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneede

[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneeded use of a
template.

show more ...

[CodeGen] Use Register/MCRegister::isPhysical. NFC

[MachinePipeliner] Add an abstract layer to manipulate Data Dependenc… (#109918)

…e Graph

In MachinePipeliner, a DAG class is used to represent the Data
Dependence Graph. Data Dependence Graph g

[MachinePipeliner] Add an abstract layer to manipulate Data Dependenc… (#109918)

…e Graph

In MachinePipeliner, a DAG class is used to represent the Data
Dependence Graph. Data Dependence Graph generally contains cycles, so
it's not appropriate to use DAG classes. In fact, some "hacks" are used
to express back-edges in the current implementation. This patch adds a
new class to provide a better interface for manipulating dependencies.
Our approach is as follows:

- To build the graph, we use the ScheduleDAGInstrs class as it is,
because it has powerful functions and the current implementation depends
heavily on it.
- After the graph construction is finished (i.e., during scheduling), we
use the new class DataDependenceGraph to manipulate the dependencies.

Since we don't change the dependencies during scheduling, the new class
only provides functions to read them. Also, this patch is just a
refactoring, i.e., scheduling results should not change with or without
this patch.

show more ...

[MachinePipeliner] Remove unused private field MF

[MachinePipeliner] Skip reserved registers when computing register pressure (#120694)

We used to skip fixed registers, but fixed registers are not enough
because there are some runtime unusable regi

[MachinePipeliner] Skip reserved registers when computing register pressure (#120694)

We used to skip fixed registers, but fixed registers are not enough
because there are some runtime unusable registers like registers
reserved by `-ffixed-xxx` options.

Here we change to use reserved registers so that the estimated
pressure is more accurate.

show more ...

[MachinePipeliner] Use `RegisterClassInfo::getRegPressureSetLimit` (#119827)

`RegisterClassInfo::getRegPressureSetLimit` is a wrapper of
`TargetRegisterInfo::getRegPressureSetLimit` with some logics

[MachinePipeliner] Use `RegisterClassInfo::getRegPressureSetLimit` (#119827)

`RegisterClassInfo::getRegPressureSetLimit` is a wrapper of
`TargetRegisterInfo::getRegPressureSetLimit` with some logics to
adjust the limit by removing reserved registers.

It seems that we shouldn't use
`TargetRegisterInfo::getRegPressureSetLimit`
directly, just like the comment "This limit must be adjusted
dynamically for reserved registers" said.

Thus we should use `RegisterClassInfo::getRegPressureSetLimit` and
remove replicated code.

Separate from https://github.com/llvm/llvm-project/pull/118787

show more ...

[NFC] Remove some unnecessary semicolons

All inside LLVM_DEBUG, some of which have been cleaned up by adding block
scopes to allow them to format more nicely.

[CodeGen] Remove unused includes (NFC) (#115996)

Identified with misc-include-cleaner.

[CodeGen] Avoid repeated hash lookups (NFC) (#110203)

[MachinePipeliner] Fix incorrect use of getPressureSets. (#109179)

The code was passing a physical register directly to getPressureSets
which expects a register unit.

Fix this by looping over th

[MachinePipeliner] Fix incorrect use of getPressureSets. (#109179)

The code was passing a physical register directly to getPressureSets
which expects a register unit.

Fix this by looping over the register units and calling getPressureSets
for each of them.

Found while trying to add a RegisterUnit class to stop storing register
units in `Register`. 0 is a valid register unit but not a valid
Register.

show more ...

[MachinePipeliner] Make Recurrence MII More Accurate (#105475)

Current RecMII calculation is bigger than it needs to be. The
calculation was refined in this patch.

[MachinePipeliner] Fix instruction order with physical register (#99264)

dependencies in same cycle

Dependency checks were insufficient when reordering instructions with
physical register depend

[MachinePipeliner] Fix instruction order with physical register (#99264)

dependencies in same cycle

Dependency checks were insufficient when reordering instructions with
physical register dependencies (i.e. Anti/Output dependencies). This
could result in generating incorrect code.

show more ...

[llvm][CodeGen] Added a new restriction for II by pragma in window scheduler (#99448)

Added a new restriction for window scheduling.
Window scheduling is disabled when llvm.loop.pipeline.initiation

[llvm][CodeGen] Added a new restriction for II by pragma in window scheduler (#99448)

Added a new restriction for window scheduling.
Window scheduling is disabled when llvm.loop.pipeline.initiationinterval
is set.

show more ...

[CodeGen] Use range-based for loops (NFC) (#98706)

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use `std::unique_ptr` instead of raw pointer for `LICalc`, so
destructor and default move constructor can handle it correctly.

This would be the last analysis required by `PHIElimination`.

show more ...

[CodeGen][NewPM] Port `machine-loops` to new pass manager (#97793)

- Add `MachineLoopAnalysis`.
- Add `MachineLoopPrinterPass`.
- Convert to `MachineLoopInfoWrapperPass` in legacy pass manager.

Reapply "[MachinePipeliner] Fix constraints aren't considered in cert… (#97259)

…ain cases" (#97246)

This reverts commit e6a961dbef773b16bda2cebc4bf9f3d1e0da42fc.

There is no difference from t

Reapply "[MachinePipeliner] Fix constraints aren't considered in cert… (#97259)

…ain cases" (#97246)

This reverts commit e6a961dbef773b16bda2cebc4bf9f3d1e0da42fc.

There is no difference from the original change. I re-ran the failed
test and it passed. So the failure wasn't caused by this change.
test result: https://lab.llvm.org/buildbot/#/builders/176/builds/585

show more ...

Revert "[MachinePipeliner] Fix constraints aren't considered in certain cases" (#97246)

Reverts llvm/llvm-project#95356

Due to ppc64le test failures caught by the LLVM Buildbot.
https://lab.llvm

Revert "[MachinePipeliner] Fix constraints aren't considered in certain cases" (#97246)

Reverts llvm/llvm-project#95356

Due to ppc64le test failures caught by the LLVM Buildbot.
https://lab.llvm.org/buildbot/#/builders/176/builds/576

show more ...

[MachinePipeliner] Fix constraints aren't considered in certain cases (#95356)

when scheduling

When scheduling an instruction, if both any predecessors and any
successors of the instruction are

[MachinePipeliner] Fix constraints aren't considered in certain cases (#95356)

when scheduling

When scheduling an instruction, if both any predecessors and any
successors of the instruction are already scheduled, `SchedStart` isn't
taken into account. It may result generating incorrect code. This patch
fixes the problem. Also, this patch merges `SchedStart` into
`EarlyStart` (same for `SchedEnd`).

Fixes https://github.com/llvm/llvm-project/issues/93936

show more ...

[llvm][CodeGen] Add a new software pipeliner 'Window Scheduler' (#84443)

This commit implements the Window Scheduler as described in the RFC:

https://discourse.llvm.org/t/rfc-window-scheduling-al

[llvm][CodeGen] Add a new software pipeliner 'Window Scheduler' (#84443)

This commit implements the Window Scheduler as described in the RFC:

https://discourse.llvm.org/t/rfc-window-scheduling-algorithm-for-machinepipeliner-in-llvm/74718

This Window Scheduler implements the window algorithm designed by
Steven Muchnick in the book "Advanced Compiler Design And
Implementation",
with some improvements:

1. Copy 3 times of the loop kernel and construct the corresponding DAG
to identify dependencies between MIs;
2. Use heuristic algorithm to obtain a set of window offsets.

The window algorithm is equivalent to modulo scheduling algorithm with a
stage of 2. It is mainly applied in targets where hardware resource
conflicts are severe, and the SMS algorithm often fails in such cases.
On our own DSA, this window algorithm typically can achieve a
performance
improvement of over 10%.

Co-authored-by: Kai Yan <aklkaiyan@tencent.com>
Co-authored-by: Ran Xiao <lennyxiao@tencent.com>

---------

Co-authored-by: Kai Yan <aklkaiyan@tencent.com>
Co-authored-by: Ran Xiao <lennyxiao@tencent.com>

show more ...

[ModuloSchedule][AArch64] Implement modulo variable expansion for pipelining (#65609)

Modulo variable expansion is a technique that resolves overlap of
variable lifetimes by unrolling. The existing

[ModuloSchedule][AArch64] Implement modulo variable expansion for pipelining (#65609)

Modulo variable expansion is a technique that resolves overlap of
variable lifetimes by unrolling. The existing implementation solves it
by making a copy by move instruction for processors with ordinary
registers such as Arm and x86. This method may result in a very large
number of move instructions, which can cause performance problems.

Modulo variable expansion is enabled by specifying -pipeliner-mve-cg. A
backend must implement some newly defined interfaces in
PipelinerLoopInfo. They were implemented for AArch64.

Discourse thread:
https://discourse.llvm.org/t/implementing-modulo-variable-expansion-for-machinepipeliner

show more ...

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree v

[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis result (#94571)

Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.

show more ...

[AArch64] Improve scheduling latency into Bundles (#86310)

By default the scheduling info of instructions into a BUNDLE are given a
latency of 0 as they operate on the implicit register of the bund

[AArch64] Improve scheduling latency into Bundles (#86310)

By default the scheduling info of instructions into a BUNDLE are given a
latency of 0 as they operate on the implicit register of the bundle.
This modifies that for AArch64 so that the latency is adjusted to use
the latency from the instruction in the bundle instead. This essentially
assumes that the bundled instructions are executed in a single cycle,
which for AArch64 is probably OK considering they are mostly used for
MOVPFX bundles, where this can help create slightly better scheduling
especially for in-order cores.

show more ...

Replace copy with a reference. (#87975)

Reapply "[CodeGen] Fix register pressure computation in MachinePipeli… (#87312)

…ner (#87030)"

Fix broken test.

This reverts commit b8ead2198f27924f91b90b6c104c1234ccc8972e.