AMDGPU/GlobalISel: Calculate isKnownNeverNaN for fminnum and fmaxnum

Implements same logis as in SelectionDAG.
G_FMINNUM_IEEE and G_FMAXNUM_IEEE are never SNaN by definition and
never NaN when one o

AMDGPU/GlobalISel: Calculate isKnownNeverNaN for fminnum and fmaxnum

Implements same logis as in SelectionDAG.
G_FMINNUM_IEEE and G_FMAXNUM_IEEE are never SNaN by definition and
never NaN when one operand is known non-NaN and other known non-SNaN.
G_FMINNUM and G_FMAXNUM are never NaN/SNaN when one of the operands
is known non-NaN/SNaN.

Differential Revision: https://reviews.llvm.org/D91716

show more ...

AMDGPU/GlobalISel: Check values of constants in isKnownNeverNaN

Differential Revision: https://reviews.llvm.org/D91714

Recommit "[GlobalISel] Walk through hints in getDefIgnoringCopies et al"

Recommit of 4580acf6752ea3cc884657b5aa3e174bed86fc8c

`Opc = DefMI->getOpcode()` was in the wrong place.

Revert "[GlobalISel] Walk through hints in getDefIgnoringCopies et al"

This reverts commit 4580acf6752ea3cc884657b5aa3e174bed86fc8c.

Reverting while looking into some test failures.

[GlobalISel] Walk through hints in getDefIgnoringCopies et al

Treat hint instructions like G_ASSERT_ZEXT like COPY instructions in helpers
which walk through copies.

This ensures that instructions

[GlobalISel] Walk through hints in getDefIgnoringCopies et al

Treat hint instructions like G_ASSERT_ZEXT like COPY instructions in helpers
which walk through copies.

This ensures that instructions like G_ASSERT_ZEXT won't impact any optimizations
that rely on these helpers.

Differential Revision: https://reviews.llvm.org/D95577

show more ...

[AMDGPU][GlobalISel] Avoid selecting S_PACK with constants

If constants are hidden behind G_ANYEXT we can treat them same way as G_SEXT.
For that purpose we extend getConstantVRegValWithLookThrough

[AMDGPU][GlobalISel] Avoid selecting S_PACK with constants

If constants are hidden behind G_ANYEXT we can treat them same way as G_SEXT.
For that purpose we extend getConstantVRegValWithLookThrough with option
to handle G_ANYEXT same way as G_SEXT.

Differential Revision: https://reviews.llvm.org/D92219

show more ...

[GlobalISel] Add missing operand update when copy is required

When constraining an operand register using constrainOperandRegClass(),
the function may emit a COPY in case the provided register class

[GlobalISel] Add missing operand update when copy is required

When constraining an operand register using constrainOperandRegClass(),
the function may emit a COPY in case the provided register class does
not match the current operand register class. However, the operand
itself is not updated to make use of the COPY, thereby resulting in
incorrect code. This patch fixes that bug by updating the machine
operand accordingly.

Reviewed By: dsanders

Differential Revision: https://reviews.llvm.org/D91244

show more ...

GlobalISel: Add isKnownToBeAPowerOfTwo helper function

GlobalISel: Return APInt from getConstantVRegVal

Returning int64_t was arbitrarily limiting for wide integer types, and
the functions should handle the full generality of the IR.

Also changes the f

GlobalISel: Return APInt from getConstantVRegVal

Returning int64_t was arbitrarily limiting for wide integer types, and
the functions should handle the full generality of the IR.

Also changes the full form which returns the originally defined
vreg. Add another wrapper for the common case of just immediately
converting to int64_t (arguably this would be useful for the full
return value case as well).

One possible issue with this change is some of the existing uses did
break without conversion to getConstantVRegSExtVal, and it's possible
some without adequate test coverage are now broken.

show more ...

[GlobalISel] Add matchers for specific constants and a matcher for negations

It's fairly common to need matchers for a specific constant value, or for
common idioms like finding a negated register.

[GlobalISel] Add matchers for specific constants and a matcher for negations

It's fairly common to need matchers for a specific constant value, or for
common idioms like finding a negated register.

Add

- `m_SpecificICst`, which returns true when matching a specific value..
- `m_ZeroInt`, which returns true when an integer 0 is matched.
- `m_Neg`, which returns when a register is negated.

Also update a few places which use idioms related to the new matchers.

Differential Revision: https://reviews.llvm.org/D91397

show more ...

GlobalISel: Directly expose getDefSrcRegIgnoringCopies utility

It's useful to get both the instruction and register at the same time.

AMDGPU/GlobalISel Check for NoNaNsFPMath in isKnownNeverSNaN

Check for NoNaNsFPMath function attribute in isKnownNeverSNaN.
Function attributes are in held in 'TargetMachine.Options'.
Among other th

AMDGPU/GlobalISel Check for NoNaNsFPMath in isKnownNeverSNaN

Check for NoNaNsFPMath function attribute in isKnownNeverSNaN.
Function attributes are in held in 'TargetMachine.Options'.
Among other things, this allows selection of some patterns imported
in D87351 since G_FCANONICALIZE is not generated when isKnownNeverSNaN
returns true in lowerFMinNumMaxNum.

However we notice some incorrect results since function attributes are
not correctly written in TargetMachine.Options when next function is
processed. Take a look at @v_test_no_global_nnans_med3_f32_pat0_srcmod0,
it has "no-nans-fp-math"="false" but TargetMachine.Options still has it
set to true since first function in test file had this attribute set to
true. This will be fixed in D87511.

Differential Revision: https://reviews.llvm.org/D87456

show more ...

[GlobalISel] Rewrite the elide-br-by-swapping-icmp-ops combine to do less.

This combine previously tried to take sequences like:
%cond = G_ICMP pred, a, b
G_BRCOND %cond, %truebb
G_BR %falsebb

[GlobalISel] Rewrite the elide-br-by-swapping-icmp-ops combine to do less.

This combine previously tried to take sequences like:
%cond = G_ICMP pred, a, b
G_BRCOND %cond, %truebb
G_BR %falsebb
%truebb:
...
%falsebb:
...

and by inverting the compare predicate and swapping branch targets, delete the
G_BR and instead have a single conditional branch to the falsebb. Since in an
earlier patch we have a combine to fold not(icmp) into just an inverted icmp,
we don't need this combine to do as much. This patch instead generalizes the
combine by just looking for:
G_BRCOND %cond, %truebb
G_BR %falsebb
%truebb:
...
%falsebb:
...

and then inverting the condition using a not (xor). The xor can be folded away
in a separate combine. This change also lets us avoid some optimization code
in the IRTranslator.

I also think that deleting G_BRs in the combiner is unnecessary. That's
something that targets can decide to do at selection time and could simplify
generic code in future.

Differential Revision: https://reviews.llvm.org/D86664

show more ...

GlobalISel/Utils.h - remove unused includes. NFCI.

Twine is unused, and TargetLowering can be reduced to a forward declaration and moved to Utils.cpp

Revert "Revert "[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)" (and dependent patch "Optimize away a Not feeding a brcond by using tbz instead of tbnz.")"

This reverts commit

Revert "Revert "[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)" (and dependent patch "Optimize away a Not feeding a brcond by using tbz instead of tbnz.")"

This reverts commit 8693ddc74371dedc742c9f3d3e4eda1da72c13ea.

Re-committing with the test requiring asserts.

show more ...

Revert "[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)" (and dependent patch "Optimize away a Not feeding a brcond by using tbz instead of tbnz.")

This reverts commit 8ad8f484

Revert "[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)" (and dependent patch "Optimize away a Not feeding a brcond by using tbz instead of tbnz.")

This reverts commit 8ad8f484b63ca507417b58c9016d2761f2b1a1a8. It causes crashes when running `ninja check-llvm-codegen-aarch64-globalisel`, e.g.
http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/24132/steps/test-stage1-compiler/logs/stdio.
Note that the crash does not seem to reproduce in debug builds.

5ded4442520d3dbb1aa72e6fe03cddef8828c618 depends on this, so revert that too.

show more ...

[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)

This is needed for an upcoming change to how we translate conditional branches
which might generate these.

Differential Revisio

[GlobalISel] Fold xor(cmp(pred, _, _), 1) -> cmp(inverse(pred), _, _)

This is needed for an upcoming change to how we translate conditional branches
which might generate these.

Differential Revision: https://reviews.llvm.org/D86383

show more ...

GlobalISel: Remove unnecessary check for copy type

COPY isn't allowed to change the type, but can mix no type with type.

GlobalISel: Remove unnecessary llvm::

TableGen/GlobalISel: Partially handle immAllOnesV/immAllZerosV

These should really match either G_BUILD_VECTOR or
G_BUILD_VECTOR_TRUNC, but there doesn't seem to be an existing
mechanism for matchin

TableGen/GlobalISel: Partially handle immAllOnesV/immAllZerosV

These should really match either G_BUILD_VECTOR or
G_BUILD_VECTOR_TRUNC, but there doesn't seem to be an existing
mechanism for matching alternative opcodes. There is GIM_SwitchOpcode,
but it seems to assume it's oly only used for matcher optimization.

I could also omit any opcode check and rely on the matcher directly
checking the opcode, but the table optimizer currently assumes there
has to be an opcode check.

Also doesn't try to handle undef elements like the DAG version.

show more ...

GlobalISel: Add utilty for getting function argument live ins

Get the argument register and ensure there's a copy to the virtual
register. AMDGPU and AArch64 have similarish code to get the livein
v

GlobalISel: Add utilty for getting function argument live ins

Get the argument register and ensure there's a copy to the virtual
register. AMDGPU and AArch64 have similarish code to get the livein
value, and I also want to use this in multiple places.

This is a bit more aggressive about setting the register class than
the original function, but that's probably OK.

I think we're missing a few verifier checks for function live ins. I
noticed AArch64's calling convention code is not actually adding
liveins to functions, only the entry block (which apparently might not
matter that much?). There should probably be a verifier check that
entry block live ins are also live into the function. We also might
need a verifier check that the copy to the livein virtual register is
in the entry block.

show more ...

GlobalISel: Handle llvm.localescape

This one is pretty easy and shrinks the list of unhandled
intrinsics. I'm not sure how relevant the insert point is. Using the
insert position of EntryBuilder wil

GlobalISel: Handle llvm.localescape

This one is pretty easy and shrinks the list of unhandled
intrinsics. I'm not sure how relevant the insert point is. Using the
insert position of EntryBuilder will place this after
constants. SelectionDAG seems to end up emitting these after argument
copies and before anything else, but I don't think it really
matters. This also ends up emitting these in the opposite order from
SelectionDAG, but I don't think that matters either.

This also needs a fix to stop the later passes dropping this as a dead
instruction. DeadMachineInstructionElim's version of isDead special
cases LOCAL_ESCAPE for some reason, and I'm not sure why it's excluded
from MachineInstr::isLabel (or why isDead doesn't check it).

I also noticed DeadMachineInstructionElim never considers inline asm
as dead, but GlobalISel will drop asm with no constraints.

show more ...

[AArch64][GlobalISel] Selection support for vector DUP[X]lane instructions.

In future, we'd like to use the perfect-shuffle mechanism to deal with these
shuffle permutations. For now, this improves

[AArch64][GlobalISel] Selection support for vector DUP[X]lane instructions.

In future, we'd like to use the perfect-shuffle mechanism to deal with these
shuffle permutations. For now, this improves performance by avoiding the
super-expensive const-pool load + tbl instruction.

Differential Revision: https://reviews.llvm.org/D84866

show more ...

GlobalISel: Rewrite getLCMType

Try to make the behavior more consistent with getGCDType, and bias
towards returning something closer to the source type whenever there's
an ambiguity.

GlobalISel: Handle more cases in getGCDType

Try harder to find a canonical unmerge type when trying to cover the
desired target type. Handle finding a compatible unmerge type for two
vectors with di

GlobalISel: Handle more cases in getGCDType

Try harder to find a canonical unmerge type when trying to cover the
desired target type. Handle finding a compatible unmerge type for two
vectors with different element types. This will return the largest
multiple of the source vector element that will evenly divide the
target vector type.

Also make the handling mixing scalars and vectors, and prefer the
source element type as the unmerge target type.

show more ...