[AMDGPU] SILoadStoreOptimizer: tweak API of CombineInfo::setMI. NFC.

Change CombineInfo::setMI to take a reference to the
SILoadStoreOptimizer instance, for easy access to common fields like
TII and

[AMDGPU] SILoadStoreOptimizer: tweak API of CombineInfo::setMI. NFC.

Change CombineInfo::setMI to take a reference to the
SILoadStoreOptimizer instance, for easy access to common fields like
TII and STM.

Differential Revision: https://reviews.llvm.org/D118366

show more ...

[AMDGPU] SILoadStoreOptimizer: Allow merging across a swizzled access

Swizzled accesses are not merged, but there is no particular reason not
to merge two instructions if any of the intervening inst

[AMDGPU] SILoadStoreOptimizer: Allow merging across a swizzled access

Swizzled accesses are not merged, but there is no particular reason not
to merge two instructions if any of the intervening instructions happens
to be a swizzled access.

This moves the check for swizzled accesses out of checkAndPrepareMerge
into collectMergeableInsts where I think it makes more sense.

Differential Revision: https://reviews.llvm.org/D118267

show more ...

[AMDGPU] SILoadStoreOptimizer: Remove redundant check for volatile

SILoadStoreOptimizer::collectMergeableInsts already ends the current
block if it sees a volatile (or ordered) memory access, so the

[AMDGPU] SILoadStoreOptimizer: Remove redundant check for volatile

SILoadStoreOptimizer::collectMergeableInsts already ends the current
block if it sees a volatile (or ordered) memory access, so there is no
need to check for them again when scanning the instructions between two
pairing candidates in a block.

Differential Revision: https://reviews.llvm.org/D118266

show more ...

[AMDGPU] Simplify SILoadStoreOptimizer::getSubRegIdxs. NFC.

[llvm] Use nullptr instead of 0 (NFC)

Identified with modernize-use-nullptr.

[AMDGPU] Make vector superclasses allocatable

The combined vector register classes with both
VGPRs and AGPRs are currently unallocatable.
This patch turns them into allocatable as a
prerequisite to

[AMDGPU] Make vector superclasses allocatable

The combined vector register classes with both
VGPRs and AGPRs are currently unallocatable.
This patch turns them into allocatable as a
prerequisite to enable copy between VGPR and
AGPR registers during regalloc.

Also, added the missing AV register classes from
192b to 1024b.

Reviewed By: rampitec

Differential Revision: https://reviews.llvm.org/D109300

show more ...

[AMDGPU][NFC] Fix typos

Differential Revision: https://reviews.llvm.org/D113672

Fix building with GCC 12:

Fixes: https://bugs.llvm.org/show_bug.cgi?id=52380

Differential Revision: https://reviews.llvm.org/D112990

[AMDGPU] Add merging into S_BUFFER_LOAD_DWORDX8_IMM

Extend SILoadStoreOptimizer to merge into DWORDX8 variant of S_BUFFER_LOAD.

Merging into DWORDX2 and DWORDX4 variants is handled already.

Differ

[AMDGPU] Add merging into S_BUFFER_LOAD_DWORDX8_IMM

Extend SILoadStoreOptimizer to merge into DWORDX8 variant of S_BUFFER_LOAD.

Merging into DWORDX2 and DWORDX4 variants is handled already.

Differential Revision: https://reviews.llvm.org/D108909

show more ...

[AMDGPU] Make BVH isel consistent with other MIMG opcodes

Suffix opcodes with _gfx10.
Remove direct references to architecture specific opcodes.
Add a BVH flag and apply this to diassembly.
Fix a nu

[AMDGPU] Make BVH isel consistent with other MIMG opcodes

Suffix opcodes with _gfx10.
Remove direct references to architecture specific opcodes.
Add a BVH flag and apply this to diassembly.
Fix a number of disassembly errors on gfx90a target caused by
previous incorrect BVH detection code.

Reviewed By: rampitec

Differential Revision: https://reviews.llvm.org/D108117

show more ...

AMDGPU: Fix SILoadStoreOptimizer for gfx90a

This was hardcoding the register class to use for the newly created
pointer registers, violating the aligned VGPR requirement.

[AMDGPU] Fixed msan failure with uninitialized value

[AMDGPU] Use single cache policy operand

Replace individual operands GLC, SLC, and DLC with a single cache_policy
bitmask operand. This will reduce the number of operands in MIR and I hope
the amoun

[AMDGPU] Use single cache policy operand

Replace individual operands GLC, SLC, and DLC with a single cache_policy
bitmask operand. This will reduce the number of operands in MIR and I hope
the amount of code. These operands are mostly 0 anyway.

Additional advantage that parser will accept these flags in any order unlike
now.

Differential Revision: https://reviews.llvm.org/D96469

show more ...

AMDGPU: Add even aligned VGPR/AGPR register classes

gfx90a operations require even aligned registers, but this was
previously achieved by reserving registers inside the full class.

Ideally this wou

AMDGPU: Add even aligned VGPR/AGPR register classes

gfx90a operations require even aligned registers, but this was
previously achieved by reserving registers inside the full class.

Ideally this would be captured in the static instruction definitions
for the operands, and we would have different instructions per
subtarget. The hackiest part of this is we need to manually reassign
AGPR register classes after instruction selection (we get away without
this for VGPRs since those types are actually registered for legal
types).

show more ...

[AMDGPU] Fixed msan build

LoadStoreOptimizer was using uninitialized SCC value for
instructions where it is unsupported.

[AMDGPU] gfx90a support

Differential Revision: https://reviews.llvm.org/D96906

[AMDGPU] Better selection of base offset when merging DS reads/writes

When merging a pair of DS reads or writes needs to materialize the base
offset in a vgpr, choose a value that is aligned to as h

[AMDGPU] Better selection of base offset when merging DS reads/writes

When merging a pair of DS reads or writes needs to materialize the base
offset in a vgpr, choose a value that is aligned to as high a power of
two as possible. This maximises the chance that different pairs can use
the same base offset, in which case the base offset registers can be
commoned up by MachineCSE.

Differential Revision: https://reviews.llvm.org/D96421

show more ...

[AMDGPU] Fix comments in SILoadStoreOptimizer::offsetsCanBeCombined

[NFC][AMDGPU] Split AMDGPUSubtarget.h to R600 and GCN subtargets

... to reduce headers dependency.

Reviewed By: rampitec, arsenm

Differential Revision: https://reviews.llvm.org/D95036

[NFC][AMDGPU] Reduce include files dependency.

Reviewed By: rampitec

Differential Revision: https://reviews.llvm.org/D93813

[AMDGPU] gfx1030 RT support

Differential Revision: https://reviews.llvm.org/D87782

[AMDGPU] Remove uses of Register::isPhysicalRegister/isVirtualRegister

... in favour of the isPhysical/isVirtual methods.

AMDGPU: Rename add/sub with carry out instructions

The hardware has created a real mess in the naming for add/sub, which
have been renamed basically every generation. Switch the carry out
pseudos to

AMDGPU: Rename add/sub with carry out instructions

The hardware has created a real mess in the naming for add/sub, which
have been renamed basically every generation. Switch the carry out
pseudos to have the gfx9/gfx10 names. We were using the original SI/CI
v_add_i32/v_sub_i32 names. Later targets reintroduced these names as
carryless instructions with a saturating clamp bit, which we do not
define. Do this rename so we can unambiguously add these missing
instructions.

The carry-in versions should also be renamed, but at least those had a
consistent _u32 name to begin with. The 16-bit instructions were also
renamed, but aren't ambiguous.

This does regress assembler error message quality in some cases. In
mismatched wave32/wave64 situations, this will switch from
"unsupported instruction" to "invalid operand", with the error
pointing at the wrong position. I couldn't quite follow how the
assembler selects these, but the previous behavior seemed accidental
to me. It looked like there was a partial attempt to handle this which
was never completed (i.e. there is an AMDGPUOperand::isBoolReg but it
isn't used for anything).

show more ...

SILoadStoreOptimizer: add support for GFX10 image instructions

GFX10 image instructions use one or more address operands starting at
vaddr0, instead of a single vaddr operand, to allow for NSA forms

SILoadStoreOptimizer: add support for GFX10 image instructions

GFX10 image instructions use one or more address operands starting at
vaddr0, instead of a single vaddr operand, to allow for NSA forms.

Differential Revision: https://reviews.llvm.org/D81675

show more ...

AMDGPU: Use IsSSA property check instead of asserting on isSSA

Also fix an SSA violation in a test the MIRParser/verifier fails to
catch. It's illegal to define a subregister in SSA. For the purpose

AMDGPU: Use IsSSA property check instead of asserting on isSSA

Also fix an SSA violation in a test the MIRParser/verifier fails to
catch. It's illegal to define a subregister in SSA. For the purpose of
the test, it just needs to define the super-register to use the
subregister in the use operand.

show more ...