[RISCV][Test] Add test for vp float arithmetic ops. NFC (#114516)

[RISCV][TTI] Add vp.fneg intrinsic cost with functionalOP (#114378)

[RISCV] Mark math functions as expanded for zvfhmin/zvfbfmin (#112508)

For regular floating point types we mark these as expanded on scalable
vectors so they're not legal in the cost model, so this

[RISCV] Mark math functions as expanded for zvfhmin/zvfbfmin (#112508)

For regular floating point types we mark these as expanded on scalable
vectors so they're not legal in the cost model, so this does the same
for f16 w/ zvfhmin and bf16.

show more ...

[RISCV] Fix bf16 cost model tests. NFC

These were inadvertently changed in #112393

[ValueTypes][RISCV] Add v1bf16 type (#111112)

When trying to add RISC-V fadd reduction cost model tests for bf16, I
noticed a crash when the vector was of <1 x bfloat>.

It turns out that this wa

[ValueTypes][RISCV] Add v1bf16 type (#111112)

When trying to add RISC-V fadd reduction cost model tests for bf16, I
noticed a crash when the vector was of <1 x bfloat>.

It turns out that this was being scalarized because unlike f16/f32/f64,
there's no v1bf16 value type, and the existing cost model code assumed
that the legalized type would always be a vector.

This adds v1bf16 to bring bf16 in line with the other fp types.

It also adds some more RISC-V bf16 reduction tests which previously
crashed, including tests to ensure that SLP won't emit fadd/fmul
reductions for bf16 or f16 w/ zvfhmin after #111000.

show more ...

[RISCV] Handle zvfhmin and zvfbfmin promotion to f32 in half arith costs (#108361)

Arithmetic half or bfloat ops on zvfhmin and zvfbfmin respectively will
be promoted and carried out in f32, so thi

[RISCV] Handle zvfhmin and zvfbfmin promotion to f32 in half arith costs (#108361)

Arithmetic half or bfloat ops on zvfhmin and zvfbfmin respectively will
be promoted and carried out in f32, so this updates
getArithmeticInstrCost to check for this.

show more ...

[RISCV] Add zvfbfmin arithmetic cost model test coverage. NFC

[RISCV][TTI] Reduce cost of a build_vector pattern (#108419)

This change is actually two related changes, but they're very hard to
meaningfully separate as the second balances the first, and yet do

[RISCV][TTI] Reduce cost of a build_vector pattern (#108419)

This change is actually two related changes, but they're very hard to
meaningfully separate as the second balances the first, and yet doesn't
do much good on it's own.

First, we can reduce the cost of a build_vector pattern. Our current
costing for this defers to generic insertelement costing which isn't
unreasonable, but also isn't correct. While inserting N elements
requires N-1 slides and N vmv.s.x, doing the full build_vector only
requires N vslide1down. (Note there are other cases that our build
vector lowering can do more cheaply, this is simply the easiest upper
bound which appears to be "good enough" for SLP costing purposes.)

Second, we need to tell SLP that calls don't preserve vector registers.
Without this, SLP will vectorize scalar code which performs e.g. 4 x
float @exp calls as two <2 x float> @exp intrinsic calls. Oddly, the
costing works out that this is in fact the optimal choice - except that
we don't actually have a <2 x float> @exp, and unroll during DAG. This
would be fine (or at least cost neutral) except that the libcall for the
scalar @exp blows all vector registers. So the net effect is we added a
bunch of spills that SLP had no idea about. Thankfully, AArch64 has a
similiar problem, and has taught SLP how to reason about spill cost once
the right TTI hook is implemented.

Now, for some implications...

The SLP solution for spill costing has some inaccuracies. In particular,
it basically just guesses whether a intrinsic will be lowered to a call
or not, and can be wrong in both directions. It also has no mechanism to
differentiate on calling convention.

This has the effect of making partial vectorization (i.e. starting in
scalar) more profitable. In practice, the major effect of this is to
make it more like SLP will vectorize part of a tree in an intersecting
forrest, and then vectorize the remaining tree once those uses have been
removed.

This has the effect of biasing us slightly away from strided, or indexed
loads during vectorization - because the scalar cost is more accurately
modeled, and these instructions look relevatively less profitable.

show more ...

[RISCV] Add zvfhmin cost model test coverage. NFC

This adds tests coverage for zvfhmin and halfs in general in the cost
model tests.

Some existing half tests were split into separate functions so t

[RISCV] Add zvfhmin cost model test coverage. NFC

This adds tests coverage for zvfhmin and halfs in general in the cost
model tests.

Some existing half tests were split into separate functions so that if
the check prefixes diverge it won't affect the rest of the non-half
instructions.

Whilst we're here, also remove the redundant
-riscv-vector-bits-min=128 and declares.

show more ...

[RISCV] Don't cost vector arithmetic fp ops as cheaper than scalar (#99594)

I was comparing some SPEC CPU 2017 benchmarks across rva22u64 and
rva22u64_v, and noticed that in a few cases that rva22u

[RISCV] Don't cost vector arithmetic fp ops as cheaper than scalar (#99594)

I was comparing some SPEC CPU 2017 benchmarks across rva22u64 and
rva22u64_v, and noticed that in a few cases that rva22u64_v was
considerably slower.

One of them was 519.lbm_r, which has a large loop that was being
unprofitably vectorized. It has an if/else in the loop which requires
large amounts of predication when vectorized, but despite the loop
vectorizer taking this into account the vector cost came out as cheaper
than the scalar.

It looks like the reason for this is because we cost scalar floating
point ops as 2, but their vector equivalents as 1 (for LMUL 1). This
comes from how we use BasicTTIImpl for scalars which treats floats as
twice as expensive as integers.

This patch doubles the cost of vector floating point arithmetic ops so
that they're at least as expensive as their scalar counterparts, which
gives a 13% speedup on 519.lbm_r at -O3 on the spacemit-x60.

Fixes #62576 (the last point there about scalar fsub/fmul)

show more ...

Recommit [RISCV][TTI] Support fdiv/udiv/sdiv/srem/urem in getArithmeticInstrCost (#89170)

Insert a break to fix the implicit-fallthrough caught by sanitizer.

Original commit message:

This patch ma

Recommit [RISCV][TTI] Support fdiv/udiv/sdiv/srem/urem in getArithmeticInstrCost (#89170)

Insert a break to fix the implicit-fallthrough caught by sanitizer.

Original commit message:

This patch made following changes:
1. Support ISD FDIV/UDIV/SDIV/UREM/SREM
2. Classify instructions which cost the same

show more ...

Revert "[RISCV][TTI] Support fdiv/udiv/sdiv/srem/urem in getArithmeticInstrCost (#89170)"

This reverts commit ed16e7aac44f2024b45d8c6c9dc2817d77d0ea97.

[RISCV][TTI] Support fdiv/udiv/sdiv/srem/urem in getArithmeticInstrCost (#89170)

This patch made following changes:
1. Support ISD FDIV/UDIV/SDIV/UREM/SREM
2. Classify instructions which cost the

[RISCV][TTI] Support fdiv/udiv/sdiv/srem/urem in getArithmeticInstrCost (#89170)

This patch made following changes:
1. Support ISD FDIV/UDIV/SDIV/UREM/SREM
2. Classify instructions which cost the same

show more ...

[RISCV] Remove -riscv-v-fixed-length-vector-lmul-max from arith tests (#89886)

This patch splits off from #89170 to clean up the tests.

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[RISCV] Incorporate LMUL into costs for arithmetic and shuffles

This reuses the routine implemented in 0e6f0b7 to implement several existing TODOs. Many of the operations scale linearly with LMUL; t

[RISCV] Incorporate LMUL into costs for arithmetic and shuffles

This reuses the routine implemented in 0e6f0b7 to implement several existing TODOs. Many of the operations scale linearly with LMUL; this change represents that in the cost model.

Differential Revision: https://reviews.llvm.org/D139039

show more ...

[RISCV] Add cost model for integer and float vector arithmetic instructions.

This patch implements getArithmeticInstrCost for RISCV, supports cost
model for integer and float vector arithmetic instr

[RISCV] Add cost model for integer and float vector arithmetic instructions.

This patch implements getArithmeticInstrCost for RISCV, supports cost
model for integer and float vector arithmetic instructions.

Differential Revision: https://reviews.llvm.org/D133552 (Original patch by jacquesguan. Subset by me with todos added.)

show more ...

[RISCV] Add missing vscale x 1 cost model entries and tests.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D136411

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or extract the first element, and use vslide to move it. But for mask vector or i64 vector in i32 target, we need special instructions to make it.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D133007

show more ...

[RISCV][NFC] Add cost model tests of llvm.fmuladd

Reviewed By: benshi001

Differential Revision: https://reviews.llvm.org/D132922

[RISCV][NFC] Add cost model coverage for fp arithmetic instructions.

This patch adds cost model coverage for fp arithmetic instructions. Some is not exact, I am working on a revision to implement th

[RISCV][NFC] Add cost model coverage for fp arithmetic instructions.

This patch adds cost model coverage for fp arithmetic instructions. Some is not exact, I am working on a revision to implement that.

Differential Revision: https://reviews.llvm.org/D132537

show more ...