[RISCV][CostModel] Add cost for fabs/fsqrt of type bf16/f16 (#118608)

[TTI][RISCV] Implement cost of some intrinsics with LMUL (#117874)

Intrinsics include:
sadd_sat/ssub_sat/uadd_sat/usub_sat/fabs/fsqrt/cttz/ctlz/ctpop

[Test] Rename the test function name suffix. NFC (#114504)

[RISCV][TTI] Reduce cost of a build_vector pattern (#108419)

This change is actually two related changes, but they're very hard to
meaningfully separate as the second balances the first, and yet do

[RISCV][TTI] Reduce cost of a build_vector pattern (#108419)

This change is actually two related changes, but they're very hard to
meaningfully separate as the second balances the first, and yet doesn't
do much good on it's own.

First, we can reduce the cost of a build_vector pattern. Our current
costing for this defers to generic insertelement costing which isn't
unreasonable, but also isn't correct. While inserting N elements
requires N-1 slides and N vmv.s.x, doing the full build_vector only
requires N vslide1down. (Note there are other cases that our build
vector lowering can do more cheaply, this is simply the easiest upper
bound which appears to be "good enough" for SLP costing purposes.)

Second, we need to tell SLP that calls don't preserve vector registers.
Without this, SLP will vectorize scalar code which performs e.g. 4 x
float @exp calls as two <2 x float> @exp intrinsic calls. Oddly, the
costing works out that this is in fact the optimal choice - except that
we don't actually have a <2 x float> @exp, and unroll during DAG. This
would be fine (or at least cost neutral) except that the libcall for the
scalar @exp blows all vector registers. So the net effect is we added a
bunch of spills that SLP had no idea about. Thankfully, AArch64 has a
similiar problem, and has taught SLP how to reason about spill cost once
the right TTI hook is implemented.

Now, for some implications...

The SLP solution for spill costing has some inaccuracies. In particular,
it basically just guesses whether a intrinsic will be lowered to a call
or not, and can be wrong in both directions. It also has no mechanism to
differentiate on calling convention.

This has the effect of making partial vectorization (i.e. starting in
scalar) more profitable. In practice, the major effect of this is to
make it more like SLP will vectorize part of a tree in an intersecting
forrest, and then vectorize the remaining tree once those uses have been
removed.

This has the effect of biasing us slightly away from strided, or indexed
loads during vectorization - because the scalar cost is more accurately
modeled, and these instructions look relevatively less profitable.

show more ...

[RISCV] Add zvfhmin cost model test coverage. NFC

This adds tests coverage for zvfhmin and halfs in general in the cost
model tests.

Some existing half tests were split into separate functions so t

[RISCV] Add zvfhmin cost model test coverage. NFC

This adds tests coverage for zvfhmin and halfs in general in the cost
model tests.

Some existing half tests were split into separate functions so that if
the check prefixes diverge it won't affect the rest of the non-half
instructions.

Whilst we're here, also remove the redundant
-riscv-vector-bits-min=128 and declares.

show more ...

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[RISCV] Add vector legalization for fmaximum/fminimum.

Reviewed By: fakepaper56

Differential Revision: https://reviews.llvm.org/D156937

[RISCV] Add lowering for scalar fmaximum/fminimum.

Unlike fmaxnum and fminnum, these operations propagate nan and
consider -0.0 to be less than +0.0.

Without Zfa, we don't have a single instruction

[RISCV] Add lowering for scalar fmaximum/fminimum.

Unlike fmaxnum and fminnum, these operations propagate nan and
consider -0.0 to be less than +0.0.

Without Zfa, we don't have a single instruction for this. The
lowering I've used forces the other input to nan if one input
is a nan. If both inputs are nan, they get swapped. Then use
the fmax or fmin instruction.

New ISD nodes are needed because fmaxnum/fminnum to not define
the order of -0.0 and +0.0.

This lowering ensures the snans are quieted though that is probably not
required in default environment). Also ensures non-canonical nans
are canonicalized, though I'm also not sure that's needed.

Another option could be to use fmax/fmin and then overwrite the
result based on the inputs being nan, but I'm not sure we can do
that with any less code.

Future work will handle nonans FMF, and handling the case where
we can prove the input isn't nan.

This does fix the crash in #64022, but we need to do more work
to avoid scalarization.

Reviewed By: fakepaper56

Differential Revision: https://reviews.llvm.org/D156069

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[RISCV][CostModel] Add half type support for the cost model of sqrt/fabs

1. Refactor for costs of sqrt/fabs
2. Add half type support for the cost model of sqrt/fabs

Reviewed By: craig.topper

Diffe

[RISCV][CostModel] Add half type support for the cost model of sqrt/fabs

1. Refactor for costs of sqrt/fabs
2. Add half type support for the cost model of sqrt/fabs

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D132908

show more ...

[RISCV] Add missing vscale x 1 cost model entries and tests.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D136411

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or extract the first element, and use vslide to move it. But for mask vector or i64 vector in i32 target, we need special instructions to make it.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D133007

show more ...

[RISCV][CostModel] Add test coverage for all the vectorizable binary intrinsics