[RISCV] Cost ordered bf16/f16 w/ zvfhmin reductions as invalid (#114250)

In #111000 we removed promotion of fadd/fmul reductions for bf16 and f16
without zvfh, and marked the cost as invalid to pre

[RISCV] Cost ordered bf16/f16 w/ zvfhmin reductions as invalid (#114250)

In #111000 we removed promotion of fadd/fmul reductions for bf16 and f16
without zvfh, and marked the cost as invalid to prevent the vectorizers
from emitting them. However it inadvertently didn't change the cost for
ordered reductions, so this moves the check earlier to fix this.

This also uses BasicTTIImpl instead which now assigns a valid but
expensive cost for fixed-length vectors, which reflects how codegen will
actually scalarize them.

show more ...

[RISCV] Add cost model tests for scalable FP reductions. NFC

There are already some in reduce-scalable-fp.ll but this makes it a
bit easier to see the difference alongside their fixed-length
counter

[RISCV] Add cost model tests for scalable FP reductions. NFC

There are already some in reduce-scalable-fp.ll but this makes it a
bit easier to see the difference alongside their fixed-length
counterparts.

show more ...

[ValueTypes][RISCV] Add v1bf16 type (#111112)

When trying to add RISC-V fadd reduction cost model tests for bf16, I
noticed a crash when the vector was of <1 x bfloat>.

It turns out that this wa

[ValueTypes][RISCV] Add v1bf16 type (#111112)

When trying to add RISC-V fadd reduction cost model tests for bf16, I
noticed a crash when the vector was of <1 x bfloat>.

It turns out that this was being scalarized because unlike f16/f32/f64,
there's no v1bf16 value type, and the existing cost model code assumed
that the legalized type would always be a vector.

This adds v1bf16 to bring bf16 in line with the other fp types.

It also adds some more RISC-V bf16 reduction tests which previously
crashed, including tests to ensure that SLP won't emit fadd/fmul
reductions for bf16 or f16 w/ zvfhmin after #111000.

show more ...

[SDAG][RISCV] Don't promote VP_REDUCE_{FADD,FMUL} (#111000)

In https://reviews.llvm.org/D153848, promotion was added for a variety
of f16 ops with zvfhmin, including VP reductions.

However I don

[SDAG][RISCV] Don't promote VP_REDUCE_{FADD,FMUL} (#111000)

In https://reviews.llvm.org/D153848, promotion was added for a variety
of f16 ops with zvfhmin, including VP reductions.

However I don't believe it's correct to promote f16 fadd or fmul
reductions to f32 since we need to round the intermediate results.

Today if we lower @llvm.vp.reduce.fadd.nxv1f16 on RISC-V, we'll get two
different results depending on whether we compiled with +zvfh or
+zvfhmin, for example with a 3 element reduction:

; v9 = [0.1563, 5.97e-8, 0.00006104]

; zvfh
vsetivli x0, 3, e16, m1, ta, ma
vmv.v.i v8, 0
vfredosum.vs v8, v9, v8
vfmv.f.s fa0, v8
; fa0 = 0.1563

; zvfhmin
vsetivli x0, 3, e16, m1, ta, ma
vfwcvt.f.f.v v10, v9
vsetivli x0, 3, e32, m1, ta, ma
vmv.v.i v8, 0
vfredosum.vs v8, v10, v8
vfmv.f.s fa0, v8
fcvt.h.s fa0, fa0
; fa0 = 0.1564

This same thing happens with reassociative reductions e.g. vfredusum.vs,
and this also applies for bf16.

I couldn't find anything in the LangRef for reductions that suggest the
excess precision is allowed. There may be something we can do in Clang
with -fexcess-precision=fast, but I haven't looked into this yet.

I presume the same precision issue occurs with fmul, but not with
fmin/fmax/fminimum/fmaximum.

I can't think of another way of lowering these other than scalarizing,
and we can't scalarize scalable vectors, so this just removes the
promotion and adjusts the cost model to return an invalid cost. (It
looks like we also don't currently cost fmul reductions, so presumably
they also have an invalid cost?)

I think this should be enough to stop the loop vectorizer or SLP from
emitting these intrinsics.

show more ...

[RISCV] Remove -riscv-v-vector-bits-min from cost model tests. NFC

It looks like they were added to prevent fixed length vectors from
being expanded, but that's no longer the case today:
https://rev

[RISCV] Remove -riscv-v-vector-bits-min from cost model tests. NFC

It looks like they were added to prevent fixed length vectors from
being expanded, but that's no longer the case today:
https://reviews.llvm.org/D121447#3376520

show more ...

[RISCV][CostModel] Refine Arithmetic reduction costs (#79103)

This patch is split off from #77342

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, ReductionOp, V

[RISCV][CostModel] Refine Arithmetic reduction costs (#79103)

This patch is split off from #77342

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, ReductionOp, VMV.X}
- Add SplitCost
Unordered reduction chain a series of VADD/VFADD/... which scales with
LMUL.
Ordered reductions chain a series of VFREDOSUMs.
- Use MVT to estimate VL.

show more ...

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[CostModel][RISCV] Model code size cost for reduction

Since code-size cost doesn't scale linearly with LMUL,
this change is to separate it from throughput.

Reviewed By: reames

Differential Revisio

[CostModel][RISCV] Model code size cost for reduction

Since code-size cost doesn't scale linearly with LMUL,
this change is to separate it from throughput.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D142068

show more ...

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79483 soon

show more ...

[RISCV] Refresh two autogened tests to avoid future whitespace diffs [nfc]

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector spec. For f32
and f64 types, the Zve32f, Zve64f, Zve64d, and V explicitly control
the availablity of floating point types in vectors.

In order to make our compiler compliant, we either need to remove all support
for half in vectors or we need an extension to control it.

Draft spec here https://github.com/riscv/riscv-v-spec/pull/780

Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D121345

show more ...

[RISCV] Add basic code modeling for fixed length vector reduction.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D121447