Revert "[AArch64] Enable subreg liveness tracking by default."

This reverts commit 9c319d5bb40785c969d2af76535ca62448dfafa7.

Some issues were discovered with the bootstrap builds, which
seem like t

Revert "[AArch64] Enable subreg liveness tracking by default."

This reverts commit 9c319d5bb40785c969d2af76535ca62448dfafa7.

Some issues were discovered with the bootstrap builds, which
seem like they were caused by this commit. I'm reverting to investigate.

show more ...

[AArch64] Enable subreg liveness tracking by default.

Internal testing didn't flag up any functional- or performance regressions.

[AArch64][GlobalISel] Update and cleanup a number of gisel tests. NFC

Mostly removing unnecessary -global-isel-abort=2 or adding fallback messages

[GlobalISel] Combine unmerge(unmerge()) if the result is legal. (#109606)

This attempts to fold:
```
%1:_(<2 x s32>), %2:_(<2 x s32>) = G_UNMERGE_VALUES %0:_(<4 x s32>)
%3:_(s32), %4:_(s32) =

[GlobalISel] Combine unmerge(unmerge()) if the result is legal. (#109606)

This attempts to fold:
```
%1:_(<2 x s32>), %2:_(<2 x s32>) = G_UNMERGE_VALUES %0:_(<4 x s32>)
%3:_(s32), %4:_(s32) = G_UNMERGE_VALUES %1
```
Into a single UNMERGE:
```
%3:_(s32), %4:_(s32), %5:_(s32), %6:_(s32) = G_UNMERGE_VALUES %0
```

This transform already exists, this patch alters it to occur when the
result UNMERGE is considered legal.

It does not try to transform where the result would be extracting a
subelement from a vector at the moment, as the code is not setup to
handle it.
```
%1:_(s32), %2:_(s32) = G_UNMERGE_VALUES %0:_(<2 x s32>)
%3:_(s16), %4:_(s16) = G_UNMERGE_VALUES %1
```

This helps us reduce the amount of legalization artefacts, especially
from widened vectors padded with undef.

show more ...

[AArch64] Remove EXT instr before UZP when extracting elements from vector (#91328)

Assembly generated for getting odd/even elements from vector contained
extra EXT instruction. This was due to way

[AArch64] Remove EXT instr before UZP when extracting elements from vector (#91328)

Assembly generated for getting odd/even elements from vector contained
extra EXT instruction. This was due to way llvm constructs DAGs when
vector_shuffling from larger type to smaller. This patch optimises DAG
in these situations, allowing for correct assembly to be emitted.

show more ...

Move several vector intrinsics out of experimental namespace (#88748)

This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from th

Move several vector intrinsics out of experimental namespace (#88748)

This patch is moving out following intrinsics:
* vector.interleave2/deinterleave2
* vector.reverse
* vector.splice

from the experimental namespace.

All these intrinsics exist in LLVM for more than a year now, and are
widely used, so should not be considered as experimental.

show more ...

[GlobalISel] Add support for interleave and deinterleave intrinsics to IRTranslator (#85199)

This patch adds support for the @llvm.experimental.vector.{interleave2,
deinterleave2} intrinsics to IRT

[GlobalISel] Add support for interleave and deinterleave intrinsics to IRTranslator (#85199)

This patch adds support for the @llvm.experimental.vector.{interleave2,
deinterleave2} intrinsics to IRTranslator for fixed-width vector types.
They are lowered to vector shuffles, in roughly the same manner as
SelectionDAG.

show more ...

[AArch64] Update generic sched model to A510

Refresh of the generic scheduling model to use A510 instead of A55.
Main benefits are to the little core, and introducing SVE scheduling information.

[AArch64] Update generic sched model to A510

Refresh of the generic scheduling model to use A510 instead of A55.
Main benefits are to the little core, and introducing SVE scheduling information.
Changes tested on various OoO cores, no performance degradation is seen.

Differential Revision: https://reviews.llvm.org/D156799

show more ...

[AArch64] Remove 64bit->128bit vector insert lowering

The AArch64 backend, during lowering, will convert an 64bit vector insert to a
128bit vector:

vector_insert %dreg, %v, %idx
=>
%qreg = insert_s

[AArch64] Remove 64bit->128bit vector insert lowering

The AArch64 backend, during lowering, will convert an 64bit vector insert to a
128bit vector:

vector_insert %dreg, %v, %idx
=>
%qreg = insert_subvector undef, %dreg, 0
%ins = vector_insert %qreg, %v, %idx
EXTRACT_SUBREG %ins, dsub

This creates a bit of mess in the DAG, and the EXTRACT_SUBREG being a machine
nodes makes it difficult to simplify. This patch removes that, treating the
64bit vector insert as legal and handling them with extra tablegen patterns.

The end result is a simpler DAG that is easier to write tablegen patterns for.

Differential Revision: https://reviews.llvm.org/D144550

show more ...

[IR] Add new intrinsics interleave and deinterleave vectors

This patch adds 2 new intrinsics:

; Interleave two vectors into a wider vector
<vscale x 4 x i64> @llvm.vector.interleave2.nxv2i64(<v

[IR] Add new intrinsics interleave and deinterleave vectors

This patch adds 2 new intrinsics:

; Interleave two vectors into a wider vector
<vscale x 4 x i64> @llvm.vector.interleave2.nxv2i64(<vscale x 2 x i64> %even, <vscale x 2 x i64> %odd)

; Deinterleave the odd and even lanes from a wider vector
{<vscale x 2 x i64>, <vscale x 2 x i64>} @llvm.vector.deinterleave2.nxv2i64(<vscale x 4 x i64> %vec)

The main motivator for adding these intrinsics is to support vectorization of
complex types using scalable vectors.

The intrinsics are kept simple by only supporting a stride of 2, which makes
them easy to lower and type-legalize. A stride of 2 is sufficient to handle
complex types which only have a real/imaginary component.

The format of the intrinsics matches how `shufflevector` is used in
LoopVectorize. For example:

using cf = std::complex<float>;

void foo(cf * dst, int N) {
for (int i=0; i<N; ++i)
dst[i] += cf(1.f, 2.f);
}

For this loop, LoopVectorize:
(1) Loads a wide vector (e.g. <8 x float>)
(2) Extracts odd lanes using shufflevector (leading to <4 x float>)
(3) Extracts even lanes using shufflevector (leading to <4 x float>)
(4) Performs the addition
(5) Interleaves the two <4 x float> vectors into a single <8 x float> using
shufflevector
(6) Stores the wide vector.

In this example, we can 1-1 replace shufflevector in (2) and (3) with the
deinterleave intrinsic, and replace the shufflevector in (5) with the
interleave intrinsic.

The SelectionDAG nodes might be extended to support higher strides (3, 4, etc)
as well in the future.

Similar to what was done for vector.splice and vector.reverse, the intrinsic
is lowered to a shufflevector when the type is fixed width, so to benefit from
existing code that was written to recognize/optimize shufflevector patterns.

Note that this approach does not prevent us from adding new intrinsics for other
strides, or adding a more generic shuffle intrinsic in the future. It just solves
the immediate problem of being able to vectorize loops with complex math.

Reviewed By: paulwalker-arm

Differential Revision: https://reviews.llvm.org/D141924

show more ...