[CodeGen] Add nneg and disjoint flags (#86650)

MachineInstr learned the new flags.

[CodeGen] Update for scalable MemoryType in MMO (#70452)

Remove getSizeOrUnknown call when MachineMemOperand is created. For Scalable
TypeSize, the MemoryType created becomes a scalable_vector.

[CodeGen] Update for scalable MemoryType in MMO (#70452)

Remove getSizeOrUnknown call when MachineMemOperand is created. For Scalable
TypeSize, the MemoryType created becomes a scalable_vector.

2 MMOs that have scalable memory access can then use the updated BasicAA that
understands scalable LocationSize.

Original Patch by Harvin Iriawan
Co-authored-by: David Green <david.green@arm.com>

show more ...

[CodeGen] Use LocationSize for MMO getSize (#84751)

This is part of #70452 that changes the type used for the external
interface of MMO to LocationSize as opposed to uint64_t. This means the
const

[CodeGen] Use LocationSize for MMO getSize (#84751)

This is part of #70452 that changes the type used for the external
interface of MMO to LocationSize as opposed to uint64_t. This means the
constructors take LocationSize, and convert ~UINT64_C(0) to
LocationSize::beforeOrAfter(). The getSize methods return a
LocationSize.

This allows us to be more precise with unknown sizes, not accidentally
treating them as unsigned values, and in the future should allow us to
add proper scalable vector support but none of that is included in this
patch. It should mostly be an NFC.

Global ISel is still expected to use the underlying LLT as it needs, and
are not expected to see unknown sizes for generic operations. Most of
the changes are hopefully fairly mechanical, adding a lot of getValue()
calls and protecting them with hasValue() where needed.

show more ...

[Codegen] Change getSpillSize/getReloadSize to LocationSize. NFC (#82636)

This is a small part of #70452, attempting to take a small simpler part
of it in isolation to simplify what remains. It cha

[Codegen] Change getSpillSize/getReloadSize to LocationSize. NFC (#82636)

This is a small part of #70452, attempting to take a small simpler part
of it in isolation to simplify what remains. It changes the getSpillSize,
getFoldedSpillSize, getRestoreSize and getFoldedRestoreSize methods to return
optional<LocationSize> instead of unsigned. The code is intended to be the
same, keeping the optional<> to specify when there was no size found, with some
minor adjustments to make sure that unknown (~UINT64_C(0)) sizes are handled
sensibly. Hopefully as more unsigned's are converted to LocationSize's the use
of ~UINT64_C(0) can be cleaned up too.

show more ...

[llvm] Move CodeGenTypes library to its own directory (#79444)

Finally addresses https://reviews.llvm.org/D148769#4311232 :)

No behavior change.

MachineVerifier: Reject extra non-register operands on instructions (#73758)

We were allowing extra immediate arguments, and only bothering to check
if registers were implicit or not.

Also conso

MachineVerifier: Reject extra non-register operands on instructions (#73758)

We were allowing extra immediate arguments, and only bothering to check
if registers were implicit or not.

Also consolidate extra operand checks in verifier, to make this
testable. We had 3 different places checking if you were trying to build
an instruction with more operands than allowed by the definition. We had
an assertion in addOperand, a direct check in the MIRParser to avoid the
assertion, and the machine verifier checks. Remove the assert and parser
check so the verifier can provide a consistent verification experience,
which will also handle instructions modified in place.

show more ...

[InlineAsm] Steal a bit to denote a register is foldable (#70738)


When using the inline asm constraint string "rm" (or "g"), we generally
would like the compiler to choose "r", but it is permitted

[InlineAsm] Steal a bit to denote a register is foldable (#70738)


When using the inline asm constraint string "rm" (or "g"), we generally
would like the compiler to choose "r", but it is permitted to choose "m"
if there's register pressure. This is distinct from "r" in which the
register is not permitted to be spilled to the stack.

The decision of which to use must be made at some point. Currently, the
instruction selection frameworks (ISELs) make the choice, and the
register allocators had better be able to handle the result.

Steal a bit from Storage when using register operands to disambiguate
between the two cases. Add helpers/getters/setters, and print in MIR
when such a register is foldable.

The getter will later be used by the register allocation frameworks (and
asserted by the ISELs) while the setters will be used by the instruction
selection frameworks.

Link: https://github.com/llvm/llvm-project/issues/20571

show more ...

[MachineInstr] add insert method for variadic instructions (#67699)

As alluded to in #20571, it would be nice if we could mutate operand
lists of MachineInstr's more safely. Add an insert method tha

[MachineInstr] add insert method for variadic instructions (#67699)

As alluded to in #20571, it would be nice if we could mutate operand
lists of MachineInstr's more safely. Add an insert method that together
with removeOperand allows for easier splicing of operands.

Splitting this patch off early to get feedback; I need to either:
- mutate an INLINEASM{_BR} MachinInstr's MachineOperands from being
registers (physical or virtual) to memory
(MachineOperandType::MO_FrameIndex). These are not 1:1 operand
replacements, but N:M operand replacements. i.e. we need to
update 2 MachineOperands into the middle of the operand list to 5 (at
least for x86_64).
- copy, modify, write a new MachineInstr which has its relevant operands
replaced.

Either approaches are hazarded by existing references to either the
operands being moved, or the instruction being removed+replaced. For my
purposes in regalloc, either seem to work for me, so hopefully reviewers
can help me determine which approach is preferable. The second would
involve no new methods on MachineInstr.

One question I had while looking at this was: "why does MachineInstr
have BOTH a NumOperands member AND a MCInstrDesc member that itself has
a NumOperands member? How many operands can a MachineInstr have? Do I
need to update BOTH (keeping them in sync)?" FWICT, only "variadic"
MachineInstrs have MCInstrDesc with NumOperands (of the MCInstrDesc) set
to zero. If the MCInstrDesc's NumOperands is non-zero, then the
NumOperands
on the MachineInstr itself cannot exceed this value (IIUC) else an
assert will
be triggered.

For most non-psuedo instructions (or at least non-varidic instructions),
insert is less likely to be useful.

To run the newly added unittest:
$ pushd llvm/build; ninja CodeGenTests; popd
$ ./llvm/build/unittests/CodeGen/CodeGenTests \
--gtest_filter=MachineInstrTest.SpliceOperands

This is meant to mirror `MCInst::insert`.

show more ...

[X86, Peephole] Enable FoldImmediate for X86

Enable FoldImmediate for X86 by implementing X86InstrInfo::FoldImmediate.

Also enhanced peephole by deleting identical instructions after FoldImmediate.

[X86, Peephole] Enable FoldImmediate for X86

Enable FoldImmediate for X86 by implementing X86InstrInfo::FoldImmediate.

Also enhanced peephole by deleting identical instructions after FoldImmediate.

Differential Revision: https://reviews.llvm.org/D151848

show more ...

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66264)

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

This reverts commit ee643b706be2b6bef9980b25cc9cc988dab94bb5.

Fi

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66264)

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

This reverts commit ee643b706be2b6bef9980b25cc9cc988dab94bb5.

Fix up build failures in targets I missed in #66003

Kept as 3 commits for reviewers to see better what's changed. Will
squash when
merging.

- reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)
- fix all the targets I missed in #66003
- fix off by one found by llvm/test/CodeGen/SystemZ/inline-asm-addr.ll

show more ...

Revert "[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)"

This reverts commit 2ca4d136124d151216aac77a0403dcb5c5835bcd.

Also revert the followup, "[InlineAsm] fix botched merge conflict r

Revert "[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)"

This reverts commit 2ca4d136124d151216aac77a0403dcb5c5835bcd.

Also revert the followup, "[InlineAsm] fix botched merge conflict resolution"

This reverts commit 8b9bf3a9f715ee5dce96eb1194441850c3663da1.

There were SystemZ and Mips build errors, too many to fix forward.

show more ...

[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

Similar to
commit 2fad6e69851e ("[InlineAsm] wrap Kind in enum class NFC")

Fix the TODOs added in
commit 93bd428742f9 ("[InlineAsm] refact

[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

Similar to
commit 2fad6e69851e ("[InlineAsm] wrap Kind in enum class NFC")

Fix the TODOs added in
commit 93bd428742f9 ("[InlineAsm] refactor InlineAsm class NFC
(#65649)")

show more ...

[InlineAsm] refactor InlineAsm class NFC (#65649)

I would like to steal one of these bits to denote whether a kind may be
spilled by the register allocator or not, but I'm afraid to touch of any
thi

[InlineAsm] refactor InlineAsm class NFC (#65649)

I would like to steal one of these bits to denote whether a kind may be
spilled by the register allocator or not, but I'm afraid to touch of any
this code using bitwise operands.

Make flags a first class type using bitfields, rather than launder data
around via `unsigned`.

show more ...

Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables

The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables

The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

* The address of the branch instruction that uses the jump table.
* The address of the jump table.
* The "base" address that the values in the jump table are relative to.
* The type of each entry (absolute pointer, a relative integer, a relative integer that is shifted).

Together this information can be used by debuggers and binary analysis tools to understand what an jump table indirect branch is doing and where it might jump to.

Documentation for the symbol can be found in the Microsoft PDB library dumper: https://github.com/microsoft/microsoft-pdb/blob/0fe89a942f9a0f8e061213313e438884f4c9b876/cvdump/dumpsym7.cpp#L5518

This change adds support to LLVM to emit the `S_ARMSWITCHTABLE` debug symbol as well as to dump it out (for testing purposes).

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D149367

show more ...

[InlineAsm] wrap Kind in enum class NFC

Should add some minor type safety to the use of this information, since
there's quite a bit of metadata being laundered through an `unsigned`.

I'm looking to

[InlineAsm] wrap Kind in enum class NFC

Should add some minor type safety to the use of this information, since
there's quite a bit of metadata being laundered through an `unsigned`.

I'm looking to potentially add more bitfields to that `unsigned`, but I
find InlineAsm's big ol' bag of enum values and usage of `unsigned`
confusing, type-unsafe, and un-ergonomic. These can probably be better
abstracted.

I think the lack of static_cast outside of InlineAsm indicates the prior
code smell fixed here.

Reviewed By: qcolombet

Differential Revision: https://reviews.llvm.org/D159242

show more ...

Revert "Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables"

This reverts commit 8d0c3db388143f4e058b5f513a70fd5d089d51c3.

Causes crashes, see comments in https://reviews.llvm.org/D14

Revert "Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables"

This reverts commit 8d0c3db388143f4e058b5f513a70fd5d089d51c3.

Causes crashes, see comments in https://reviews.llvm.org/D149367.

Some follow-up fixes are also reverted:

This reverts commit 636269f4fca44693bfd787b0a37bb0328ffcc085.
This reverts commit 5966079cf4d4de0285004eef051784d0d9f7a3a6.
This reverts commit e7294dbc85d24a08c716d9babbe7f68390cf219b.

show more ...

Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables

The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

Emit the CodeView `S_ARMSWITCHTABLE` debug symbol for jump tables

The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

* The address of the branch instruction that uses the jump table.
* The address of the jump table.
* The "base" address that the values in the jump table are relative to.
* The type of each entry (absolute pointer, a relative integer, a relative integer that is shifted).

Together this information can be used by debuggers and binary analysis tools to understand what an jump table indirect branch is doing and where it might jump to.

Documentation for the symbol can be found in the Microsoft PDB library dumper: https://github.com/microsoft/microsoft-pdb/blob/0fe89a942f9a0f8e061213313e438884f4c9b876/cvdump/dumpsym7.cpp#L5518

This change adds support to LLVM to emit the `S_ARMSWITCHTABLE` debug symbol as well as to dump it out (for testing purposes).

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D149367

show more ...

AMDGPU: Check for implicit defs before constant folding instruction

Can't delete the constant folded instruction if scc is used.

Fixes #63986

https://reviews.llvm.org/D157504

[DebugInfo] Fix crash when printing malformed DBG machine instructions

MachineVerifier does not check that DBG_VALUE, DBG_VALUE_LIST and
DBG_INSTR_REF have the expected number of operands, so printi

[DebugInfo] Fix crash when printing malformed DBG machine instructions

MachineVerifier does not check that DBG_VALUE, DBG_VALUE_LIST and
DBG_INSTR_REF have the expected number of operands, so printing them
(e.g. with -print-after-all) should not crash.

Differential Revision: https://reviews.llvm.org/D156226

show more ...

[MachineInst] Bump NumOperands back up to 24bits

In https://reviews.llvm.org/D149445, it was lowered from 32 to 16bits, which
broke an internal project of ours. The relevant code being compiled is a

[MachineInst] Bump NumOperands back up to 24bits

In https://reviews.llvm.org/D149445, it was lowered from 32 to 16bits, which
broke an internal project of ours. The relevant code being compiled is a fairly
large nested switch that results in a PHI node with 65k+ operands, which can't
easily be turned into a table for perf reasons.

This change unifies `NumOperands`, `Flags`, and `AsmPrinterFlags` into a packed
7-byte struct, which `CapOperands` can follow as the 8th byte, rounding it up
to a nice alignment before the `Info` field.

rdar://111217742&109362033

Differential revision: https://reviews.llvm.org/D153791

show more ...

[SDAG] Preserve unpredictable metadata, teach X86CmovConversion to respect this metadata

Sometimes an developer would like to have more control over cmov vs branch. We have unpredictable metadata in

[SDAG] Preserve unpredictable metadata, teach X86CmovConversion to respect this metadata

Sometimes an developer would like to have more control over cmov vs branch. We have unpredictable metadata in LLVM IR, but currently it is ignored by X86 backend. Propagate this metadata and avoid cmov->branch conversion in X86CmovConversion for cmov with this metadata.

Example:

```
int MaxIndex(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is converted to branch by X86CmovConversion
if (a[i] > a[t]) t = i;
}
return t;
}

int MaxIndex2(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is preserved
if (__builtin_unpredictable(a[i] > a[t])) t = i;
}
return t;
}
```

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D118118

show more ...

[DebugInfo][InstrRef] Prettyprint metadata

Some metadata prettyprinting, including variable prettyprinting and
debug line info comments, is currently only supported for `DBG_VALUE`.
This allows `DBG

[DebugInfo][InstrRef] Prettyprint metadata

Some metadata prettyprinting, including variable prettyprinting and
debug line info comments, is currently only supported for `DBG_VALUE`.
This allows `DBG_INSTR_REF` can be printed in the same way.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D150620

show more ...

[MachineInst] Switch NumOperands to 16bits

Decrease NumOperands from 32 to 16bits (matches MCInstrDesc) so we can use saved bits to extend Flags (https://reviews.llvm.org/D118118).

Reviewed By: bar

[MachineInst] Switch NumOperands to 16bits

Decrease NumOperands from 32 to 16bits (matches MCInstrDesc) so we can use saved bits to extend Flags (https://reviews.llvm.org/D118118).

Reviewed By: barannikov88

Differential Revision: https://reviews.llvm.org/D149445

show more ...

Restore CodeGen/LowLevelType from `Support`

This is rework of;
- D30046 (LLT)

Since I have introduced `llvm-min-tblgen` as D146352, `llvm-tblgen`
may depend on `CodeGen`.

`LowLevlType.h` origina

Restore CodeGen/LowLevelType from `Support`

This is rework of;
- D30046 (LLT)

Since I have introduced `llvm-min-tblgen` as D146352, `llvm-tblgen`
may depend on `CodeGen`.

`LowLevlType.h` originally belonged to `CodeGen`. Almost all userse are
still under `CodeGen` or `Target`. I think `CodeGen` is the right place
to put `LowLevelType.h`.

`MachineValueType.h` may be moved as well. (later, D149024)

I have made many modules depend on `CodeGen`. It is consistent but
inefficient. It will be split out later, D148769

Besides, I had to isolate MVT and LLT in modmap, since
`llvm::PredicateInfo` clashes between `TableGen/CodeGenSchedule.h`
and `Transforms/Utils/PredicateInfo.h`.
(I think better to introduce namespace llvm::TableGen)

Depends on D145937, D146352, and D148768.

Differential Revision: https://reviews.llvm.org/D148767

show more ...

[GlobalISel][NFC] Add MachineInstr::getFirst[N]{Regs,LLTs}() helpers to extract regs & types.

These reduce the typing and clutter from:
Register Dst = MI.getOperand(0).getReg();
Register Src1 = MI.g

[GlobalISel][NFC] Add MachineInstr::getFirst[N]{Regs,LLTs}() helpers to extract regs & types.

These reduce the typing and clutter from:
Register Dst = MI.getOperand(0).getReg();
Register Src1 = MI.getOperand(1).getReg();
Register Src2 = MI.getOperand(2).getReg();
Register Src3 = MI.getOperand(3).getReg();
LLT DstTy = MRI.getType(Dst);
... etc etc

To just:
auto [Dst, Src1, Src2, Src3] = MI.getFirst4Regs();
auto [DstTy, Src1Ty, Src2Ty, Src3Ty] = MI.getFirst4LLTs();

Or even more concise:
auto [Dst, DstTy, Src1, Src1Ty, Src2, Src2Ty, Src3, Src3Ty] =
MI.getFirst4RegLLTs();

Differential Revision: https://reviews.llvm.org/D144687

show more ...