[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneede

[CodeGen] Rename RegisterMaskPair to VRegMaskOrUnit. NFC (#123799)

This holds a physical register unit or virtual register and mask.

While I was here I've used emplace_back and removed an unneeded use of a
template.

show more ...

[AMDGPU] Optionally Use GCNRPTrackers during scheduling (#93090)

This adds the ability to use the GCNRPTrackers during scheduling. These trackers have several advantages over the generic trackers: 1

[AMDGPU] Optionally Use GCNRPTrackers during scheduling (#93090)

This adds the ability to use the GCNRPTrackers during scheduling. These trackers have several advantages over the generic trackers: 1. global live-thru trackers, 2. subregister based RP deltas, and 3. flexible vreg -> PressureSet mappings.

This feature is off-by-default to ease with the roll-out process. In particular, when using the optional trackers, the scheduler will still maintain the generic trackers leading to unnecessary compile time.

show more ...

[AMDGPU] Constrain use LiveMask by the operand's LaneMask for RP calculation.

For speculative RP queries, recede may calculate inaccurate masks for subreg uses. Previously, the calculation would loo

[AMDGPU] Constrain use LiveMask by the operand's LaneMask for RP calculation.

For speculative RP queries, recede may calculate inaccurate masks for subreg uses. Previously, the calculation would look at any live lane for the use at the position of the MI in the LIS. This also adds lanes for any subregs which are live at but not used by the instruction. By constraining against the getSubRegIndexLaneMask for the operand's subreg, we are sure to not pick up on these extra lanes.

For current clients of recede, this is not an issue. This is because 1. the current clients do not violate the program order in the LIS, and 2. the change to RP is based on the difference between previous mask and new mask. Since current clients are not exposed to this issue, this patch is sort of NFC.

Co-authored-by: Valery Pykhtin Valery.Pykhtin@amd.com
Change-Id: Iaed80271226b2587297e6fb78fe081afec1a9275

show more ...

[AMDGPU] Qualify auto. NFC. (#110878)

Generated automatically with:
$ clang-tidy -fix -checks=-*,llvm-qualified-auto $(find
lib/Target/AMDGPU/ -type f)

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use

[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)

- Add `LiveIntervalsAnalysis`.
- Add `LiveIntervalsPrinterPass`.
- Use `LiveIntervalsWrapperPass` in legacy pass manager.
- Use `std::unique_ptr` instead of raw pointer for `LICalc`, so
destructor and default move constructor can handle it correctly.

This would be the last analysis required by `PHIElimination`.

show more ...

[AMDGPU] Prefer lower total register usage in regions with spilling

Change-Id: Ia5c434b0945bdcbc357c5e06c3164118fc91df25

[AMDGPU] Fix GCNUpwardRPTracker: max register pressure on defs. (#74422)

Treat a defined register as fully live "at" the instruction and update maximum pressure accordingly. Fixes #3786.

[AMDGPU] Add live-through register set printing to GCNRegPressurePrinter pass. (#71096)

Add live-through register set printing, assuming live-through register
is in live-in and live-out sets, has n

[AMDGPU] Add live-through register set printing to GCNRegPressurePrinter pass. (#71096)

Add live-through register set printing, assuming live-through register
is in live-in and live-out sets, has no redefinitions but may have uses
in the block.

show more ...

[AMDGPU] Fix GCNUpwardRPTracker. (#71186)

Fixed:

1. Maximum register pressure calculation at the instruction level.
Previously max RP included both def and use of registers of an
instruction.

[AMDGPU] Fix GCNUpwardRPTracker. (#71186)

Fixed:

1. Maximum register pressure calculation at the instruction level.
Previously max RP included both def and use of registers of an
instruction. Now maximum RP includes _uses_ and _early-clobber defs_.

2. Uses were incorrectly tracked and this resulted in a mismatch of
live-in set reported by LiveIntervals and tracked live reg set when the
beginning of the block is reached.

Interface has changed, moveMaxPressure becomes deprecated and
getMaxPressure, resetMaxPressure functions are added. reset function
seem now more consistent.

show more ...

[AMDGPU] GCNRegPressurePrinter pass to print GCNRegPressure values for testing. (#70031)

Using GCNDownwardRPTracker or GCNUpwardRPTracker the pass collects register pressure values for a function an

[AMDGPU] GCNRegPressurePrinter pass to print GCNRegPressure values for testing. (#70031)

Using GCNDownwardRPTracker or GCNUpwardRPTracker the pass collects register pressure values for a function and prints these values next to instructions. Output can be used to generate Filecheck rules in mir tests.

show more ...

[AMDGPU] Tweak tuple weight calculation. NFC. (#66490)

This just makes it more obvious that GCNRegPressure does not actually
use pressure sets.

[AMDGPU] Make use of MachineInstr::all_defs and all_uses. NFCI.

[AMDGPU] Speedup GCNDownwardRPTracker::advanceBeforeNext

The function makes liveness tests for the entire live register set for every instruction it passes by.
This becomes very slow on high RP regi

[AMDGPU] Speedup GCNDownwardRPTracker::advanceBeforeNext

The function makes liveness tests for the entire live register set for every instruction it passes by.
This becomes very slow on high RP regions such as ASAN enabled code.

Instead only uses of last tracked instruction should be tested and this greatly improves compilation time.

This patch revealed few bugs in SIFormMemoryClauses and PreRARematStage::sinkTriviallyRematInsts which should
be fixed first.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D136267

show more ...

Revert "[AMDGPU] Speedup GCNDownwardRPTracker::advanceBeforeNext"

This reverts commit 2d09bec169277fb5a341249afacff532c7511756.

It was causing assertion failures in some out-of-tree tests.

[AMDGPU] Speedup GCNDownwardRPTracker::advanceBeforeNext

The function makes liveness tests for the entire live register set for every instruction it passes by.
This becomes very slow on high RP regi

[AMDGPU] Speedup GCNDownwardRPTracker::advanceBeforeNext

The function makes liveness tests for the entire live register set for every instruction it passes by.
This becomes very slow on high RP regions such as ASAN enabled code.

Instead only uses of last tracked instruction should be tested and this greatly improves compilation time.

This patch revealed few bugs in SIFormMemoryClauses and PreRARematStage::sinkTriviallyRematInsts which should
be fixed first.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D136267

show more ...

[AMDGPU] Fix GCNDownwardRPTracker::advanceBeforeNext at the end of MBB

The problem with GCNDownwardRPTracker::advanceBeforeNext is that it doesn't allow to get register pressure
after the last instr

[AMDGPU] Fix GCNDownwardRPTracker::advanceBeforeNext at the end of MBB

The problem with GCNDownwardRPTracker::advanceBeforeNext is that it doesn't allow to get register pressure
after the last instruction in a MBB.

However when we track RP through the boundary of a MBB we need the state that is after the last instruction
of the MBB and before the first instruction of the successor MBB. Currently we stop traking RP in the state
'at' the last instruction of the MBB which is incorrect.

This patch fixes 27 lit tests with EXPENSIVE_CHECKS enabled.

Reviewed By: rampitec, arsenm

Differential Revision: https://reviews.llvm.org/D136927

show more ...

Revert "[AMDGPU] Add EXPENSIVE_CHECK into GCNRPTracker::reset"

This reverts commit fecf067db40ffa1a6d5d665769c90cd29547f502.

The change introduces 420 test failures with EXPENSIVE_CHECK in AMDGPU w

Revert "[AMDGPU] Add EXPENSIVE_CHECK into GCNRPTracker::reset"

This reverts commit fecf067db40ffa1a6d5d665769c90cd29547f502.

The change introduces 420 test failures with EXPENSIVE_CHECK in AMDGPU which I don't want to disable.
Going to fix the failures and recommit the check.

show more ...

[AMDGPU] Add EXPENSIVE_CHECK into GCNRPTracker::reset

This would check if passed in live-ins registers match those calculated using LIS.

This check currently breaks 420 lit tests when enabled.

Rev

[AMDGPU] Add EXPENSIVE_CHECK into GCNRPTracker::reset

This would check if passed in live-ins registers match those calculated using LIS.

This check currently breaks 420 lit tests when enabled.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D136860

show more ...

[AMDGPU] Refactor debug printing routines for GCNRPTracker

Use Printable to enhance syntax, remove duplication, unify.

Reviewed By: arsenm, rampitec

Differential Revision: https://reviews.llvm.org

[AMDGPU] Refactor debug printing routines for GCNRPTracker

Use Printable to enhance syntax, remove duplication, unify.

Reviewed By: arsenm, rampitec

Differential Revision: https://reviews.llvm.org/D136704

show more ...

[NFC] Use Register instead of unsigned

[AMDGPU] Make vector superclasses allocatable

The combined vector register classes with both
VGPRs and AGPRs are currently unallocatable.
This patch turns them into allocatable as a
prerequisite to

[AMDGPU] Make vector superclasses allocatable

The combined vector register classes with both
VGPRs and AGPRs are currently unallocatable.
This patch turns them into allocatable as a
prerequisite to enable copy between VGPR and
AGPR registers during regalloc.

Also, added the missing AV register classes from
192b to 1024b.

Reviewed By: rampitec

Differential Revision: https://reviews.llvm.org/D109300

show more ...

[AMDGPU] gfx90a support

Differential Revision: https://reviews.llvm.org/D96906

AMDGPU: Fix dbg_value handling when forming soft clause bundles

DBG_VALUES placed between memory instructions would change
codegen. Skip over these and re-insert them after the bundle instead
of giv

AMDGPU: Fix dbg_value handling when forming soft clause bundles

DBG_VALUES placed between memory instructions would change
codegen. Skip over these and re-insert them after the bundle instead
of giving up on bundling.

show more ...

[NFC][AMDGPU] Split AMDGPUSubtarget.h to R600 and GCN subtargets

... to reduce headers dependency.

Reviewed By: rampitec, arsenm

Differential Revision: https://reviews.llvm.org/D95036

[Target] Use llvm::find_if (NFC)