Revert "Reland "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG" (#123632)"

There's a regression with one of the bootstrap builds for x86.
I'll revert this while

Revert "Reland "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG" (#123632)"

There's a regression with one of the bootstrap builds for x86.
I'll revert this while I investigate.

This reverts commit 4df6d3df24ae9cff07c70c96a1663cbba6e1dca5.

show more ...

Reland "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG" (#123632)

This PR aims to reland work done by @arsenm which was previously
reverted due to some tangenti

Reland "RegisterCoalescer: Add implicit-def of super register when coalescing SUBREG_TO_REG" (#123632)

This PR aims to reland work done by @arsenm which was previously
reverted due to some tangentially related scheduler issues as discussed
on #76416.

This PR cherry-picks the original commit (0e46b49de433), and adds
another patch on top with the following changes:

* The code in `updateRegDefsUses` now updates subranges when
subreg-liveness-tracking is enabled.

* When adding an implicit-def operand for the super-register,
the code in `reMaterializeTrivialDef` which tries to remove
undefined subranges should now take into account that the lanes
from the super-reg are no longer undefined.

Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com>

show more ...

Emit constrained atan2 intrinsic for clang builtin (#113636)

This change is part of this proposal:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294

- `Builtins.td` - Add f16 suppor

Emit constrained atan2 intrinsic for clang builtin (#113636)

This change is part of this proposal:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294

- `Builtins.td` - Add f16 support for libm atan2 builtin
- `CGBuiltin.cpp` - Emit constraint atan2 intrinsic for clang builtin
- `clang/test/CodeGenCXX/builtin-calling-conv.cpp` - Use erff instead of
atan2 for clang builtin to lib call calling convention check, now that
atan2 maps to an intrinsic.
- add atan2 cases to llvm.experimental.constrained tests for more
backends: ARM, PowerPC, RISCV, SystemZ.
- LangRef.rst: add llvm.experimental.constrained.atan2, revise
llvm.atan2 description.

Last part of Implement the atan2 HLSL Function. Fixes #70096.

show more ...

[PowerPC] Add phony subregisters to cover the high half of the VSX registers. (#94628)

On PowerPC there are 128 bit VSX registers. These registers are half
overlapped with 64 bit floating point reg

[PowerPC] Add phony subregisters to cover the high half of the VSX registers. (#94628)

On PowerPC there are 128 bit VSX registers. These registers are half
overlapped with 64 bit floating point registers (FPR). The 64 bit half
of the VXS register that does not overlap with the FPR does not overlap
with any other register class. The FPR are the only subregisters of the
VSX registers but they do not fully cover the 128 bit super register.
This leads to incorrect lane masks being created.

This patch adds phony registers for the other half of the VSX registers
in order to fully cover them and to make sure that the lane masks are
not the same for the VSX and the floating point register.

show more ...

[clang] Reland Add tanf16 builtin and support for tan constrained intrinsic (#94559)

Relanding this PR now that
https://github.com/llvm/llvm-project/pull/90503 has merged. with `FTAN`
landing in

[clang] Reland Add tanf16 builtin and support for tan constrained intrinsic (#94559)

Relanding this PR now that
https://github.com/llvm/llvm-project/pull/90503 has merged. with `FTAN`
landing in
[TargetLoweringBase.cpp:L1021](https://github.com/llvm/llvm-project/blob/main/llvm/lib/CodeGen/TargetLoweringBase.cpp#L1020C23-L1021C63
) There is now a llvm tan intrinsic 32\64\128 Expand case for all llvm
backends.

In LLVM, the `llvm.experimental.constrained.cos` and
`llvm.experimental.constrained.sin` intrinsics are used for performing
cosine and sine calculations with additional constraints on
floating-point operations. This behavior is expected for all
floating-point math intrinsics. This change adds these constraints for
the `tan` intrinsic.

- `Builtins.td` - replace TanF128 with F16F128MathTemplate
- `CGBuiltin.cpp` - map existing tan builtins to `tan` and
`constrained_tan` intrinsic
- `ConstrainedOps.def` map tan and constrained_tan to an ISDOpcode.

resolves #91421

---------

Co-authored-by: Farzon Lotfi <farzon@farzon.com>

show more ...

[PowerPC] Define SchedModel for Power8

PowerPC subtargets prior to Power9 use the 'legacy' itinerary way to
provide scheduling information. This patch re-writes the tablegen file
to define the sched

[PowerPC] Define SchedModel for Power8

PowerPC subtargets prior to Power9 use the 'legacy' itinerary way to
provide scheduling information. This patch re-writes the tablegen file
to define the scheduling information in the new SchedModel way, which
can bring improvements to some benchmarks.

Reviewed By: shchenz

Differential Revision: https://reviews.llvm.org/D154488

show more ...

[PowerPC] Fix the xxperm swap requirements

This patch is to fix the xxperm vector operand swap condition so that the
single-use operand is in V2 to prevent copying, it also fixes the subtarget
condi

[PowerPC] Fix the xxperm swap requirements

This patch is to fix the xxperm vector operand swap condition so that the
single-use operand is in V2 to prevent copying, it also fixes the subtarget
condition to exploit the xpperm.

Reviewed By: stefanp

Differential Revision: https://reviews.llvm.org/D146632

show more ...

[PowerPC] remove XXSWAPD after load from CP which is a splat value

If the value from constant-pool is a splat value of vector type, do not
need swap after load from constant-pool.

Reviewed By: shch

[PowerPC] remove XXSWAPD after load from CP which is a splat value

If the value from constant-pool is a splat value of vector type, do not
need swap after load from constant-pool.

Reviewed By: shchenz

Differential Revision: https://reviews.llvm.org/D139491

show more ...

[PowerPC] Fix vperm codegen

Commit rG934d5fa2b8672695c335deed0e19d0e777c98403 changed the vperm codegen
for cases that vperm is not replaced by xxperm, this patch is to revert that.

Reviewed By: st

[PowerPC] Fix vperm codegen

Commit rG934d5fa2b8672695c335deed0e19d0e777c98403 changed the vperm codegen
for cases that vperm is not replaced by xxperm, this patch is to revert that.

Reviewed By: stefanp

Differential Revision: https://reviews.llvm.org/D138736

show more ...

[PowerPC] Exploit xxperm, check for dead vectors and substitute vperm with xxperm

vperm instruction requires the data to be in the Altivec registers, if one of
the vector operands is not used after

[PowerPC] Exploit xxperm, check for dead vectors and substitute vperm with xxperm

vperm instruction requires the data to be in the Altivec registers, if one of
the vector operands is not used after this vperm instruction then it can be
substituted by xxperm which doubles the number of available registers.

Reviewed By: stefanp

Differential Revision: https://reviews.llvm.org/D133700

show more ...

[PowerPC] make expensive mflr be away from its user in the function prologue

mflr is kind of expensive on Power version smaller than 10, so we should
schedule the store for the mflr's def away from

[PowerPC] make expensive mflr be away from its user in the function prologue

mflr is kind of expensive on Power version smaller than 10, so we should
schedule the store for the mflr's def away from mflr.

In epilogue, the expensive mtlr has no user for its def, so it doesn't
matter that the load and the mtlr are back-to-back.

Reviewed By: RolandF

Differential Revision: https://reviews.llvm.org/D137423

show more ...

[PowerPC] emit VSX instructions instead of VMX instructions for vector loads and stores

This patch changes the PowerPC backend to generate VSX load/store instructions
for all vector loads/stores on

[PowerPC] emit VSX instructions instead of VMX instructions for vector loads and stores

This patch changes the PowerPC backend to generate VSX load/store instructions
for all vector loads/stores on Power8 and earlier (LE) instead of VMX
load/store instructions. The reason for this change is because VMX instructions
require the vector to be 16-byte aligned. So, a vector load/store will fail with
VMX instructions if the vector is misaligned. Also, `gcc` generates VSX
instructions in this situation which allow for unaligned access but require a
swap instruction after loading/before storing. This is not an issue for BE
because we already emit VSX instructions since no swap is required. And this is
not an issue on Power9 and up since we have access to `lxv[x]`/`stxv[x]` which
allow for unaligned access and do not require swaps.

This patch also delays the VSX load/store for LE combines until after
LegalizeOps to prioritize other load/store combines.

Reviewed By: #powerpc, stefanp

Differential Revision: https://reviews.llvm.org/D127309

show more ...

[PowerPC] Add missed VSX shuffles instead of Altivec ones

VSX introduced some permute instructions that are direct
replacements for Altivec ones except they can target all
the VSX registers. We have

[PowerPC] Add missed VSX shuffles instead of Altivec ones

VSX introduced some permute instructions that are direct
replacements for Altivec ones except they can target all
the VSX registers. We have added code generation for most
of these but somehow missed the low/hi word merges (XXMRG[LH]W).
This caused some additional spills on some large
computationally intensive code.

This patch simply adds the missed patterns.

show more ...

[PowerPC] Respect rounding mode in the back end

Currently, the floating point instructions that depend on
rounding mode are correctly marked in the PPC back end with
an implicit use of the RM regist

[PowerPC] Respect rounding mode in the back end

Currently, the floating point instructions that depend on
rounding mode are correctly marked in the PPC back end with
an implicit use of the RM register. Similarly, instructions
that explicitly define the register are marked with an
implicit def of the same register. So for the most part,
RM-using code won't be moved across RM-setting instructions.

However, calls are not marked as RM-setting instructions so
code can be moved across calls. This is generally desired,
but so is the ability to turn off this behaviour with an
appropriate option - and -frounding-math really should be
that option.

This patch provides a set of call instructions (for direct
and indirect calls) that are marked with an implicit def of
the RM register. These will be used for calls that are marked
with the strictfp attribute.

Differential revision: https://reviews.llvm.org/D111433

show more ...

[PowerPC] Update Refactored Load/Store Implementation, XForm VSX Patterns, and Tests

This patch includes the following updates to the load/store refactoring effort introduced in D93370:
- Update va

[PowerPC] Update Refactored Load/Store Implementation, XForm VSX Patterns, and Tests

This patch includes the following updates to the load/store refactoring effort introduced in D93370:
- Update various VSX patterns that use to "force" an XForm, to instead just XForm.
This allows the ability for the patterns to compute the most optimal addressing
mode (and to produce a DForm instruction when possible)
- Update pattern and test case for the LXVD2X/STXVD2X intrinsics
- Update LIT test cases that use to use the XForm instruction to use the DForm instruction

Differential Revision: https://reviews.llvm.org/D95115

show more ...

[PowerPC] Split tests for constrained intrinsics

The test CodeGen/PowerPC/vector-constrained-fp-intrinsics.ll checks code
generation for constrained floating point intrinsics. Many test cases in
it

[PowerPC] Split tests for constrained intrinsics

The test CodeGen/PowerPC/vector-constrained-fp-intrinsics.ll checks code
generation for constrained floating point intrinsics. Many test cases in
it were implemented using operations on constants. Constant folding of
constrained intrinsics would make these test cases almost useless,
because they would check only constant loading.

To keep the tests useful, operations on constants were replaced with
operations on function parameters.

Differential Revision: https://reviews.llvm.org/D103259

show more ...

[PowerPC] Canonicalize shuffles on big endian targets as well

Extend shuffle canonicalization and conversion of shuffles fed by vectorized
scalars to big endian subtargets. For big endian subtargets

[PowerPC] Canonicalize shuffles on big endian targets as well

Extend shuffle canonicalization and conversion of shuffles fed by vectorized
scalars to big endian subtargets. For big endian subtargets, loads and direct
moves of scalars into vector registers put the data in the correct element for
SCALAR_TO_VECTOR if the data type is 8 bytes wide. However, if the data type is
narrower, the value still ends up in the wrong place - althouth a different
wrong place than on little endian targets.

This patch extends the combine that keeps values where they are if they feed a
shuffle to big endian targets.

Differential revision: https://reviews.llvm.org/D100478

show more ...

[PowerPC] Use correct node to get a super register from a subreg

The VSX tablegen file has some rather eggregious uses of
COPY_TO_REGCLASS even in situations where it needs to use
SUBREG_TO_REG. Whi

[PowerPC] Use correct node to get a super register from a subreg

The VSX tablegen file has some rather eggregious uses of
COPY_TO_REGCLASS even in situations where it needs to use
SUBREG_TO_REG. While this produces correct code, it often doesn't
allow the register coalescer to coalesce copies and the resulting
code ends up being suboptimal. This patch just changes over
patterns that should use SUBREG_TO_REG.

show more ...

[PowerPC] [FPEnv] Disable strict FP mutation by default

22a0edd0 introduced a config IsStrictFPEnabled, which controls the
strict floating point mutation (transforming some strict-fp operations
into

[PowerPC] [FPEnv] Disable strict FP mutation by default

22a0edd0 introduced a config IsStrictFPEnabled, which controls the
strict floating point mutation (transforming some strict-fp operations
into non-strict in ISel). This patch disables the mutation by default
since we've finished PowerPC strict-fp enablement in backend.

Reviewed By: uweigand

Differential Revision: https://reviews.llvm.org/D87222

show more ...

[PowerPC] Fix STRICT_FRINT/STRICT_FNEARBYINT lowering

In standard C library, both rint and nearbyint returns rounding result
in current rounding mode. But nearbyint never raises inexact exception.
O

[PowerPC] Fix STRICT_FRINT/STRICT_FNEARBYINT lowering

In standard C library, both rint and nearbyint returns rounding result
in current rounding mode. But nearbyint never raises inexact exception.
On PowerPC, x(v|s)r(d|s)pic may modify FPSCR XX, raising inexact
exception. So we can't select constrained fnearbyint into xvrdpic.

One exception here is xsrqpi, which will not raise inexact exception, so
fnearbyint f128 is okay here.

Reviewed By: uweigand

Differential Revision: https://reviews.llvm.org/D87220

show more ...

[PowerPC] Support constrained vector fp/int conversion

This patch makes these operations legal, and add necessary codegen
patterns.

There's still some issue similar to D77033 for conversion from v1

[PowerPC] Support constrained vector fp/int conversion

This patch makes these operations legal, and add necessary codegen
patterns.

There's still some issue similar to D77033 for conversion from v1i128
type. But normal type tests synced in vector-constrained-fp-intrinsic
are passed successfully.

Reviewed By: uweigand

Differential Revision: https://reviews.llvm.org/D83654

show more ...

[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics

tryLatency compares two sched candidates. For the top zone it prefers
the one with lesser depth, but only if that depth is grea

[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics

tryLatency compares two sched candidates. For the top zone it prefers
the one with lesser depth, but only if that depth is greater than the
total latency of the instructions we've already scheduled -- otherwise
its latency would be hidden and there would be no stall.

Unfortunately it only tests the depth of one of the candidates. This can
lead to situations where the TopDepthReduce heuristic does not kick in,
but a lower priority heuristic chooses the other candidate, whose depth
*is* greater than the already scheduled latency, which causes a stall.

The fix is to apply the heuristic if the depth of *either* candidate is
greater than the already scheduled latency.

All this also applies to the BotHeightReduce heuristic in the bottom
zone.

Differential Revision: https://reviews.llvm.org/D72392

show more ...

[PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

We currently miss a number of opportunities to emit single-instruction
VMRG[LH][BHW] instructions for shuffles on little

[PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

We currently miss a number of opportunities to emit single-instruction
VMRG[LH][BHW] instructions for shuffles on little endian subtargets. Although
this in itself is not a huge performance opportunity since loading the permute
vector for a VPERM can always be pulled out of loops, producing such merge
instructions is useful to downstream optimizations.
Since VPERM is essentially opaque to all subsequent optimizations, we want to
avoid it as much as possible. Other permute instructions have semantics that can
be reasoned about much more easily in later optimizations.

This patch does the following:
- Canonicalize shuffles so that the first element comes from the first vector
(since that's what most of the mask matching functions want)
- Switch the elements that come from splat vectors so that they match the
corresponding elements from the other vector (to allow for merges)
- Adds debugging messages for when a shuffle is matched to a VPERM so that
anyone interested in improving this further can get the info for their code

Differential revision: https://reviews.llvm.org/D77448

show more ...

[PowerPC] Support constrained rounding operations

This patch adds handling of constrained FP intrinsics about round,
truncate and extend for PowerPC target, with necessary tests.

Reviewed By: steve

[PowerPC] Support constrained rounding operations

This patch adds handling of constrained FP intrinsics about round,
truncate and extend for PowerPC target, with necessary tests.

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D64193

show more ...

[PowerPC] Add missing handling for half precision

The fix for PR39865 took care of some of the handling for half precision
but it missed a number of issues that still exist. This patch fixes the
rem

[PowerPC] Add missing handling for half precision

The fix for PR39865 took care of some of the handling for half precision
but it missed a number of issues that still exist. This patch fixes the
remaining issues that cause crashes in the PPC back end.

Fixes: https://bugs.llvm.org/show_bug.cgi?id=45776

Differential revision: https://reviews.llvm.org/D79283

show more ...