[aarch64][win] Update Called Globals info when updating Call Site info (#122762)

Fixes the "use after poison" issue introduced by #121516 (see
<https://github.com/llvm/llvm-project/pull/121516#issue

[aarch64][win] Update Called Globals info when updating Call Site info (#122762)

Fixes the "use after poison" issue introduced by #121516 (see
<https://github.com/llvm/llvm-project/pull/121516#issuecomment-2585912395>).

The root cause of this issue is that #121516 introduced "Called Global"
information for call instructions modeling how "Call Site" info is
stored in the machine function, HOWEVER it didn't copy the
copy/move/erase operations for call site information.

The fix is to rename and update the existing copy/move/erase functions
so they also take care of Called Global info.

show more ...

[LiveRangeEdit] Remove any MemoryOperand on MI when converting it to KILL. (#114407)

When LiveRangeEdit::eliminateDeadDef() converts an MI to a KILL instruction,
it should also call dropMemRefs() i

[LiveRangeEdit] Remove any MemoryOperand on MI when converting it to KILL. (#114407)

When LiveRangeEdit::eliminateDeadDef() converts an MI to a KILL instruction,
it should also call dropMemRefs() in order to erase any MachineMemOperand
present.

This was discovered in testing as the MachineVerifier does not accept an MMO
without the corresponding MI mayLoad/mayStore flag, which the KILL opcode
lacks.

show more ...

[CodeGen] Remove AA parameter of isSafeToMove (#100691)

This `AA` parameter is not used and for most uses they just pass
a nullptr.

The use of `AA` was removed since 8d0383e.

LiveRangeEdit: Replace setIsDead with an assert (#92964)

I noticed this was possibly buggy with implicit operands with the same
dest register, and should maybe be using addRegisterDead. However, th

LiveRangeEdit: Replace setIsDead with an assert (#92964)

I noticed this was possibly buggy with implicit operands with the same
dest register, and should maybe be using addRegisterDead. However, this
is never called in a situation where the operand wasn't already marked
dead. This is eliminateDeadDef, implying the def was already known to be
dead.

Add an assert to detect inconsistencies in dead flags. This was
apparently added in 9a16d655c71826bef98b7d6e9590e4494ac0e1a9.

show more ...

[CodeGen] Use range-based for loops (NFC)

LiveRangeEdit: Clear all dead flags when rematerializing (#73933)

It's allowed to rematerialize instructions with implicit-defs of the
same register as the single explicit def. If this happened, it

LiveRangeEdit: Clear all dead flags when rematerializing (#73933)

It's allowed to rematerialize instructions with implicit-defs of the
same register as the single explicit def. If this happened, it was only
clearing the dead flags on the one main result.

show more ...

Reapply "[CodeGen]Allow targets to use target specific COPY instructions for live range splitting"

This reverts commit a496c8be6e638ae58bb45f13113dbe3a4b7b23fd.

The workaround in c26dfc81e254c78dc2

Reapply "[CodeGen]Allow targets to use target specific COPY instructions for live range splitting"

This reverts commit a496c8be6e638ae58bb45f13113dbe3a4b7b23fd.

The workaround in c26dfc81e254c78dc23579cf3d1336f77249e1f6 should work
around the underlying problem with SUBREG_TO_REG.

show more ...

Revert "[CodeGen]Allow targets to use target specific COPY instructions for live range splitting"

And dependent commits.

Details in D150388.

This reverts commit 825b7f0ca5f2211ec3c93139f98d1e24048

Revert "[CodeGen]Allow targets to use target specific COPY instructions for live range splitting"

And dependent commits.

Details in D150388.

This reverts commit 825b7f0ca5f2211ec3c93139f98d1e24048c225c.
This reverts commit 7a98f084c4d121244ef7286bc6503b6a181d446e.
This reverts commit b4a62b1fa546312d882fa12dfdcd015177d66826.
This reverts commit b7836d856206ec39509d42529f958c920368166b.

No conflicts in the code, few tests had conflicts in autogenerated CHECKs:
llvm/test/CodeGen/Thumb2/mve-float32regloops.ll
llvm/test/CodeGen/AMDGPU/fix-frame-reg-in-custom-csr-spills.ll

Reviewed By: alexfh

Differential Revision: https://reviews.llvm.org/D156381

show more ...

[CodeGen]Allow targets to use target specific COPY instructions for live range splitting

Replacing D143754. Right now the LiveRangeSplitting during register allocation uses
TargetOpcode::COPY instru

[CodeGen]Allow targets to use target specific COPY instructions for live range splitting

Replacing D143754. Right now the LiveRangeSplitting during register allocation uses
TargetOpcode::COPY instruction for splitting. For AMDGPU target that creates a
problem as we have both vector and scalar copies. Vector copies perform a copy over
a vector register but only on the lanes(threads) that are active. This is mostly sufficient
however we do run into cases when we have to copy the entire vector register and
not just active lane data. One major place where we need that is live range splitting.

Allowing targets to use their own copy instructions(if defined) will provide a lot of
flexibility and ease to lower these pseudo instructions to correct MIR.

- Introduce getTargetCopyOpcode() virtual function and use if to generate copy in Live range
splitting.
- Replace necessary MI.isCopy() checks with TII.isCopyInstr() in register allocator pipeline.

Reviewed By: arsenm, cdevadas, kparzysz

Differential Revision: https://reviews.llvm.org/D150388

show more ...

InlineSpiller: Consider copy bundles when looking for snippet copies

This was looking for full copies produced by SplitKit, but SplitKit
introduces copy bundles if not all lanes are live. The scan f

InlineSpiller: Consider copy bundles when looking for snippet copies

This was looking for full copies produced by SplitKit, but SplitKit
introduces copy bundles if not all lanes are live. The scan for uses
needs to look at bundles, not individual instructions.

This is a prerequisite to avoiding some redundant spills due to
subregisters which will help avoid an allocation failure in a future
patch.

show more ...

LiveRangeEdit: Use Register

[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFC

Use isPhysical/isVirtual methods.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D141715

[CodeGen] Use cloneVirtualRegister in LiveIntervals and LiveRangeEdit

It is needed to invoke the delegate methods effectively whenever a
virtual register is cloned from an existing register of the s

[CodeGen] Use cloneVirtualRegister in LiveIntervals and LiveRangeEdit

It is needed to invoke the delegate methods effectively whenever a
virtual register is cloned from an existing register of the same class.

Reviewed By: qcolombet

Differential Revision: https://reviews.llvm.org/D138517

show more ...

[NFC] Use Register instead of unsigned for variables that receive a Register object

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D139451

[LiveRangeEdit] Add a statistic variable for rematerialization

Add a statistic variable for rematerialization.

Differential Revision: https://reviews.llvm.org/D134907

[RegisterCoalescer] Use LiveRangeEdit to handle rematerialization

This patch uses the API provided by LiveRangeEdit to handle rematerialization.
It will make future maintenance and improvement more

[RegisterCoalescer] Use LiveRangeEdit to handle rematerialization

This patch uses the API provided by LiveRangeEdit to handle rematerialization.
It will make future maintenance and improvement more easier.

No functional change.

Differential Revision: https://reviews.llvm.org/D133610

show more ...

Fix subrange liveness checking at rematerialization

This patch fixes an issue where an instruction reading a whole register would be moved during register allocation into a spot where one of the sub

Fix subrange liveness checking at rematerialization

This patch fixes an issue where an instruction reading a whole register would be moved during register allocation into a spot where one of the subregisters was dead.

The code to check whether an instruction can be rematerialized at a given point or not was already checking for subranges to ensure that subregisters are live, but only when the instruction being moved was using a subregister, this patch changes that so the subranges are checked even when the moved instruction uses the full register.

This patch also adds a case to the original test for the subrange checking that trigger the issue described above.

The original subrange checking code was introduced in this revision: https://reviews.llvm.org/D115278

And I've encountered this issue on AMDGPUs while working with DPC++: https://github.com/intel/llvm/issues/6209

Essentially the greedy register allocator attempts to move the following instruction:

```
%3961:vreg_64 = V_LSHLREV_B64_e64 3, %3078:vreg_64, implicit $exec
```

From `@3440` into the body of a loop `@16312`, but `%3078` has the following live ranges:

```
%3078 [2224r,2240r:0)[2240r,3488B:1)[16192B,38336B:1) 0@2224r 1@2240r L0000000000000003 [2224r,3440r:0) 0@2224r L000000000000000C [2240r,3488B:0)[16192B,38336B:0) 0@2240r
```

So `@16312e` `%3078.sub1` is alive but `%3078.sub0` is dead, so this instruction being moved there leads to invalid memory accesses as `3078.sub0` ends up being trashed and the result of this instruction is used as part of an address calculation for a load.

On the original ticket this issue showed up on gfx906 and gfx90a but not on gfx908, this turned out to be because on gfx908 instead of moving the shift instruction into the loop, its value is spilled into an ACC register, gfx906 doesn't have ACC registers and for gfx90a ACC registers are used like regular vector registers and so aren't used for spilling.

With this patch the original application from the DPC++ ticket works properly on gfx906, and the result of the shift instruction is correctly spilled instead of moving the instruction in the loop.

Original Author: npmiller

Reviewed by: rampitec

Submitted by: rampitec

Differential Revision: https://reviews.llvm.org/D131884

show more ...

RegAllocGreedy: Fix subranges when rematerializing dead subreg defs

This would create a new interval missing the subrange and hit this
verifier error:

*** Bad machine code: Live interval for subreg

RegAllocGreedy: Fix subranges when rematerializing dead subreg defs

This would create a new interval missing the subrange and hit this
verifier error:

*** Bad machine code: Live interval for subreg operand has no subranges ***
- function: test_remat_subreg_def
- basic block: %bb.0 (0xa568758) [0B;128B)
- instruction: 32B dead undef %4.sub0:vreg_64 = V_MOV_B32_e32 2, implicit $exec

show more ...

CodeGen: Remove AliasAnalysis from regalloc

This was stored in LiveIntervals, but not actually used for anything
related to LiveIntervals. It was only used in one check for if a load
instruction is

CodeGen: Remove AliasAnalysis from regalloc

This was stored in LiveIntervals, but not actually used for anything
related to LiveIntervals. It was only used in one check for if a load
instruction is rematerializable. I also don't think this was entirely
correct, since it was implicitly assuming constant loads are also
dereferenceable.

Remove this and rely only on the invariant+dereferenceable flags in
the memory operand. Set the flag based on the AA query upfront. This
should have the same net benefit, but has the possible disadvantage of
making this AA query nonlazy.

Preserve the behavior of assuming pointsToConstantMemory implying
dereferenceable for now, but maybe this should be changed.

show more ...

[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated comments

Check subrange liveness at rematerialization

LiveRangeEdit::allUsesAvailableAt checks that VNI at use is the same
as at the original use slot. However, the VNI can be the same while
a specific subra

Check subrange liveness at rematerialization

LiveRangeEdit::allUsesAvailableAt checks that VNI at use is the same
as at the original use slot. However, the VNI can be the same while
a specific subrange needed for use can be dead at the new index.

This patch adds subrange liveness check if there is a subreg use.

Fixes: SWDEV-312810

Differential Revision: https://reviews.llvm.org/D115278

show more ...

[llvm] Use range-based for loops (NFC)

[CodeGen] Use llvm::is_contained (NFC)

[CodeGen] Use MachineInstr::operands (NFC)

[llvm] Use pop_back_val (NFC)