[RISCV][TTI] Model the cost of insert/extractelt when the vector split into multiple register group and idx exceed single group. (#118401)

This patch implements the cost when the size of the vector

[RISCV][TTI] Model the cost of insert/extractelt when the vector split into multiple register group and idx exceed single group. (#118401)

This patch implements the cost when the size of the vector need to split
into multiple groups and the index exceed single vector group.

For extract element, we need to store split vectors to stack and load
the target element.
For insert element, we need to store split vectors to stack and store
the target element and load vectors back.

After this patch, the cost of insert/extract element will close to the
generated assembly.

show more ...

[RISCV][TTI] Adjust cost for extract/insert element when VLEN is known (#108595)

If we know an exact VLEN, then the index is effectively modulo the
number of elements in a single vector register. O

[RISCV][TTI] Adjust cost for extract/insert element when VLEN is known (#108595)

If we know an exact VLEN, then the index is effectively modulo the
number of elements in a single vector register. Our lowering performs
this subvector optimization.

A bit of context. This change may look a bit strange on it's own given
we are currently *not* scaling insert/extract cost by LMUL. This costing
decision needs to change, but is very intertwined with SLP
profitability, and is thus a bit hard to adjust. I'm hoping that
https://github.com/llvm/llvm-project/pull/108419 will let me start to
untangle this. This change is basically a case of finding a subset I can
tackle before other dependencies are in place which does no real harm in
the meantime.

show more ...

[RISCV][TTI] Model cost for insert/extract into illegal types (#106440)

We'd previously just deferred to the base implementation, but that more
or less always returns 1. This underestimates the cos

[RISCV][TTI] Model cost for insert/extract into illegal types (#106440)

We'd previously just deferred to the base implementation, but that more
or less always returns 1. This underestimates the cost of the
insert/extract, biases the SLP vectorizer towards forming illegally
typed vectors, and underestimates the cost of scalarized operations
(like unaligned scatter/gather).

show more ...

[RISCV] Add cost model coverage for insert/extract element w/ 2^N - 1 types

We currently return costs which are too low for these.

Recommit "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

Changes in Recommit:
Add an additional check on sign/zero extend to the same type.

Original message:
Use the destinati

Recommit "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

Changes in Recommit:
Add an additional check on sign/zero extend to the same type.

Original message:
Use the destination data type to measure the LMUL size for
latency/throughput cost

show more ...

Revert "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

This reverts commit 7545c635729a2055a429c5decd26a619a8d6e74b as it's
failing on the Linux bots.

[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)

Use the destination data type to measure the LMUL size for
latency/throughput cost

[RISCV][CostModel] Estimate cost of Extract/InsertElement with non-constant index when vector instructions are not available (#67334)

This patch fixes the compilation time issue of matrix-types-spec

[RISCV][CostModel] Estimate cost of Extract/InsertElement with non-constant index when vector instructions are not available (#67334)

This patch fixes the compilation time issue of matrix-types-spec test
from test-suite.

Reproduction of the problem:
```
clang++ -DNDEBUG --target=riscv64-linux-gnu --sysroot=<sysroot path> --gcc-toolchain=<gcc path> -O2 -fenable-matrix <test-suite-path>/SingleSource/UnitTests/matrix-types-spec.cpp
```

On my machine, compilation takes 50.44s. In comparison, the same test
with RVV (-march=rv64gcv) compiles in 3.06s, and for x86-64 target it
takes 1.71s. It turns out that the main issue is unrolling of loop in
multiplySpec function, that has extractelements with non-constant index:
```
for.body9.i: ; preds = %for.body9.i, %for.cond6.preheader.i
%indvars.iv.i92 = phi i64 [ 0, %for.cond6.preheader.i ], [ %indvars.iv.next.i93, %for.body9.i ]
%Elt.033.i = phi double [ 0.000000e+00, %for.cond6.preheader.i ], [ %80, %for.body9.i ]
%77 = mul nuw nsw i64 %indvars.iv.i92, 25
%78 = add nuw nsw i64 %77, %indvars.iv39.i91
%matrixext.i = extractelement <475 x double> %62, i64 %78
%79 = add nuw nsw i64 %indvars.iv.i92, %74
%matrixext13.i = extractelement <209 x double> %73, i64 %79
%80 = tail call double @llvm.fmuladd.f64(double %matrixext.i, double %matrixext13.i, double %Elt.033.i)
%indvars.iv.next.i93 = add nuw nsw i64 %indvars.iv.i92, 1
%exitcond.not.i94 = icmp eq i64 %indvars.iv.next.i93, 19
br i1 %exitcond.not.i94, label %for.cond.cleanup8.i, label %for.body9.i, !llvm.loop !21
```

When RVV is supported, extractelement/insertelement with non-constant
index can be lowered quite efficiently with vslidedown/vslideup;
otherwise it's lowered via stack memory operations, i.e. for
extractelement each vector element is stored on stack and then the
needed element is loaded back; for insertelement is stores all vector
elements, rewrites the required element value and then loads vector
back. Currently the cost of such expensive operation is estimated as
zero, so loop unroll processes the loop more aggresively. The proper
estimation of cost (in a way like in X86 target) prohibits unrolling of
this loop and fixes compilation time (2.77s on my machine).

show more ...

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[RISCV] Improve cost model test coverage for insert/extract element

In particular, high LMULs, constant offsets within high LMUL, and types which require splitting. Note that most of these are way

[RISCV] Improve cost model test coverage for insert/extract element

In particular, high LMULs, constant offsets within high LMUL, and types which require splitting. Note that most of these are way off with current lowering.

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[RISCV][NFC] Add test coverage for insertelement/extractelement of widen vector type.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D135534

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or extract the first element, and use vslide to move it. But for mask vector or i64 vector in i32 target, we need special instructions to make it.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D133007

show more ...

[RISCV][test] Add test for the cost model of vector insert/extract element.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D133005