[GlobalISel] Add G_LLROUND

Basically the same as G_LROUND. Handles the llvm.llround family of intrinsics.

Also add a helper function to the MachineVerifier for checking if all of the
(virtual regis

[GlobalISel] Add G_LLROUND

Basically the same as G_LROUND. Handles the llvm.llround family of intrinsics.

Also add a helper function to the MachineVerifier for checking if all of the
(virtual register) operands of an instruction are scalars. Seems like a useful
thing to have.

Differential Revision: https://reviews.llvm.org/D108429

show more ...

[GlobalISel] Add a G_LROUND instruction

Meant to represent the `@llvm.lround.*` family.

Add the opcode, docs, and verification.

Differential Revision: https://reviews.llvm.org/D108417

[AArch64][GlobalISel] Add G_VECREDUCE fewerElements support for full scalarization.

For some reductions like G_VECREDUCE_OR on AArch64, we need to scalarize
completely if the source is <= 64b. This

[AArch64][GlobalISel] Add G_VECREDUCE fewerElements support for full scalarization.

For some reductions like G_VECREDUCE_OR on AArch64, we need to scalarize
completely if the source is <= 64b. This change adds support for that in
the legalizer. If the source has a pow-2 num elements, then we can do
a tree reduction using the scalar operation in the individual elements.
Otherwise, we just create a sequential chain of operations.

For AArch64, we only need to scalarize if the input is <64b. If it's great than
64b then we can first do a fewElements step to 64b, taking advantage of vector
instructions until we reach the point of scalarization.

I also had to relax the verifier checks for reductions because the intrinsics
support <1 x EltTy> types, which we lower to scalars for GlobalISel.

Differential Revision: https://reviews.llvm.org/D108276

show more ...

[GlobalISel] Add G_ISNAN

Add a generic opcode equivalent to the `llvm.isnan` intrinsic +
MachineVerifier support for it.

We need an opcode here because we may want target-specific lowering later on

[GlobalISel] Add G_ISNAN

Add a generic opcode equivalent to the `llvm.isnan` intrinsic +
MachineVerifier support for it.

We need an opcode here because we may want target-specific lowering later on.

Differential Revision: https://reviews.llvm.org/D108222

show more ...

[NFC] Rename AttributeList::hasFnAttribute() -> hasFnAttr()

This is more consistent with similar methods.

[MachineVerifier] Make INSERT_SUBREG diagnostic respect operand 2 subregs

This came out of post-commit review: https://reviews.llvm.org/D105953#inline-1012919

Thanks uabelho!

[MachineVerifier] Diagnose invalid INSERT_SUBREGs

Differential revision: https://reviews.llvm.org/D105953

Revert "[MachineVerifier] Diagnose invalid INSERT_SUBREGs"

This reverts commit dd57ba1a17b93dbe211d04cb2d4de5f6dc898d60.

It broke some tests: http://45.33.8.238/linux/51314/step_12.txt

[MachineVerifier] Diagnose invalid INSERT_SUBREGs

Differential revision: https://reviews.llvm.org/D105953

[GISel] Support llvm.memcpy.inline

Differential revision: https://reviews.llvm.org/D105072

GlobalISel: Relax verification of physical register copy types

This was picking a concrete size for a physical register, and
enforcing exact match on the virtual register's type size. Some
targets a

GlobalISel: Relax verification of physical register copy types

This was picking a concrete size for a physical register, and
enforcing exact match on the virtual register's type size. Some
targets add multiple types to a register class, and some are smaller
than the full bit width. For example x86 adds f32 to 128-bit xmm
registers, and AMDGPU adds i16/f16 to 32-bit registers.

It might be better to represent these cases as a copy of the full
register and an extraction of the subpart, but a lot of code assumes
you can directly copy. This will help fix the current usage of the DAG
calling convention infrastructure which is incompatible with how
GlobalISel is now using it.

The API is somewhat cumbersome here, but I just mirrored the existing
functions, except now with LLTs (and allow returning null on failure,
unlike the MVT version). I think the concept of selecting register
classes based on type is flawed to begin with, but I'm trying to keep
this compatible with the existing handling.

show more ...

[CSSPGO] Exclude pseudo probe from slotindex verification.

MachineVerifier: Continue reporting errors for copies

This was skipping verification of later copies, but generally the
verifier tries to report as many things wrong as possible in the
function.

[Greedy RA] Add a check to MachineVerifier

If Virtual Register is alive in landing pad its def must be
before the call causing the exception or it should be statepoint instruction itself and
in this

[Greedy RA] Add a check to MachineVerifier

If Virtual Register is alive in landing pad its def must be
before the call causing the exception or it should be statepoint instruction itself and
in this case def actually means the relocation of gc pointer and is alive in
landing pad.

The test shows the triggering this check for an option under development
use-registers-for-gc-values-in-landing-pad which is off by default until
it is functionally correct.

Reviewers: reames, void, jyknight, nickdesaulniers, efriedma, arsenm, rnk
Reviewed By: rnk
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D100525

show more ...

[GlobalISel] Add G_ROTR and G_ROTL opcodes for rotates.

Differential Revision: https://reviews.llvm.org/D99383

[AArch64][GlobalISel] Emit bzero on Darwin

Darwin platforms for both AArch64 and X86 can provide optimized `bzero()`
routines. In this case, it may be preferable to use `bzero` in place of a
memset

[AArch64][GlobalISel] Emit bzero on Darwin

Darwin platforms for both AArch64 and X86 can provide optimized `bzero()`
routines. In this case, it may be preferable to use `bzero` in place of a
memset of 0.

This adds a G_BZERO generic opcode, similar to G_MEMSET et al. This opcode can
be generated by platforms which may want to use bzero.

To emit the G_BZERO, this adds a pre-legalize combine for AArch64. The
conditions for this are largely a port of the bzero case in
`AArch64SelectionDAGInfo::EmitTargetCodeForMemset`.

The only difference in comparison to the SelectionDAG code is that, when
compiling for minsize, this will fire for all memsets of 0. The original code
notes that it's not beneficial to do this for small memsets; however, using
bzero here will save a mov from wzr. For minsize, I think that it's preferable
to prioritise omitting the mov.

This also fixes a bug in the libcall legalization code which would delete
instructions which could not be legalized. It also adds a check to make sure
that we actually get a libcall name.

Code size improvements (Darwin):

- CTMark -Os: -0.0% geomean (-0.1% on pairlocalalign)
- CTMark -Oz: -0.2% geomean (-0.5% on bullet)

Differential Revision: https://reviews.llvm.org/D99358

show more ...

[GlobalISel] Add G_SBFX + G_UBFX (bitfield extraction opcodes)

There is a bunch of similar bitfield extraction code throughout *ISelDAGToDAG.

E.g, ARMISelDAGToDAG, AArch64ISelDAGToDAG, and AMDGPUIS

[GlobalISel] Add G_SBFX + G_UBFX (bitfield extraction opcodes)

There is a bunch of similar bitfield extraction code throughout *ISelDAGToDAG.

E.g, ARMISelDAGToDAG, AArch64ISelDAGToDAG, and AMDGPUISelDAGToDAG all contain
code that matches a bitfield extract from an and + right shift.

Rather than duplicating code in the same way, this adds two opcodes:

- G_UBFX (unsigned bitfield extract)
- G_SBFX (signed bitfield extract)

They work like this

```
%x = G_UBFX %y, %lsb, %width
```

Where `lsb` and `width` are

- The least-significant bit of the extraction
- The width of the extraction

This will extract `width` bits from `%y`, starting at `lsb`. G_UBFX zero-extends
the result, while G_SBFX sign-extends the result.

This should allow us to use the combiner to match the bitfield extraction
patterns rather than duplicating pattern-matching code in each target.

Differential Revision: https://reviews.llvm.org/D98464

show more ...

[M68k][MIR](2/8) Changes in the target-independent MIR part

- Add new callback in `TargetInstrInfo` --
`isPCRelRegisterOperandLegal` -- to query whether pc-rel
register MachineOperand is legal

[M68k][MIR](2/8) Changes in the target-independent MIR part

- Add new callback in `TargetInstrInfo` --
`isPCRelRegisterOperandLegal` -- to query whether pc-rel
register MachineOperand is legal.
- Add new function to search DebugLoc in a reverse ordering

Authors: myhsu, m4yers, glaubitz

Differential Revision: https://reviews.llvm.org/D88386

show more ...

GlobalISel: Verify G_CONCAT_VECTORS has at least 2 sources

[GlobalISel] Add G_ASSERT_SEXT

This adds a G_ASSERT_SEXT opcode, similar to G_ASSERT_ZEXT. This instruction
signifies that an operation was already sign extended from a smaller type.

This is useful

[GlobalISel] Add G_ASSERT_SEXT

This adds a G_ASSERT_SEXT opcode, similar to G_ASSERT_ZEXT. This instruction
signifies that an operation was already sign extended from a smaller type.

This is useful for functions with sign-extended parameters.

E.g.

```
define void @foo(i16 signext %x) {
...
}
```

This adds verifier, regbankselect, and instruction selection support for
G_ASSERT_SEXT equivalent to G_ASSERT_ZEXT.

Differential Revision: https://reviews.llvm.org/D96890

show more ...

[llvm] Use llvm::is_contained (NFC)

[GlobalISel] Simpler verification of G_SEXT_INREG and G_ASSERT_ZEXT

There's no need to call verifyVectorElementMatch since we already know
that the source and destination types are identical.

Diffe

[GlobalISel] Simpler verification of G_SEXT_INREG and G_ASSERT_ZEXT

There's no need to call verifyVectorElementMatch since we already know
that the source and destination types are identical.

Differential Revision: https://reviews.llvm.org/D96589

show more ...

[GlobalISel] Add G_ASSERT_ZEXT

This adds a generic opcode which communicates that a type has already been
zero-extended from a narrower type.

This is intended to be similar to AssertZext in Selecti

[GlobalISel] Add G_ASSERT_ZEXT

This adds a generic opcode which communicates that a type has already been
zero-extended from a narrower type.

This is intended to be similar to AssertZext in SelectionDAG.

For example,

```
%x_was_extended:_(s64) = G_ASSERT_ZEXT %x, 16
```

Signifies that the top 48 bits of %x are known to be 0.

This is useful in cases like this:

```
define i1 @zeroext_param(i8 zeroext %x) {
%cmp = icmp ult i8 %x, -20
ret i1 %cmp
}
```

In AArch64, `%x` must use a 32-bit register, which is then truncated to a 8-bit
value.

If we know that `%x` is already zero-ed out in the relevant high bits, we can
avoid the truncate.

Currently, in GISel, this looks like this:

```
_zeroext_param:
and w8, w0, #0xff ; We don't actually need this!
cmp w8, #236
cset w0, lo
ret
```

While SDAG does not produce the truncation, since it knows that it's
unnecessary:

```
_zeroext_param:
cmp w0, #236
cset w0, lo
ret
```

This patch

- Adds G_ASSERT_ZEXT
- Adds MIRBuilder support for it
- Adds MachineVerifier support for it
- Documents it

It also puts G_ASSERT_ZEXT into its own class of "hint instruction." (There
should be a G_ASSERT_SEXT in the future, maybe a G_ASSERT_ALIGN as well.)

This allows us to skip over hints in the legalizer etc. These can then later
be selected like COPY instructions or removed.

Differential Revision: https://reviews.llvm.org/D95564

show more ...

[CodeGen] Use llvm::append_range (NFC)

[llvm] Use llvm::all_of (NFC)