[RISCV][CostModel] Correct the cost of some reductions (#118072)

Reductions include: and/or/max/min

Recommit "[RISCV] Refine cost on Min/Max reduction (#79402)" (#86480)

This is recommitted as the test and fix for
llvm.vector.reduce.fmaximum/fminimum are covered in #80553 and #80697

Recommit "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

Changes in Recommit:
Add an additional check on sign/zero extend to the same type.

Original message:
Use the destinati

Recommit "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

Changes in Recommit:
Add an additional check on sign/zero extend to the same type.

Original message:
Use the destination data type to measure the LMUL size for
latency/throughput cost

show more ...

Revert "[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)"

This reverts commit 7545c635729a2055a429c5decd26a619a8d6e74b as it's
failing on the Linux bots.

[RISCV][TTI] Scale the cost of the sext/zext with LMUL (#86617)

Use the destination data type to measure the LMUL size for
latency/throughput cost

Revert "[RISCV] Refine cost on Min/Max reduction" (#80340)

Reverts llvm/llvm-project#79402. Crash reported. On closer inspection,
this patch does not handle Intrinsic::maximum and Intrinsic::minimu

Revert "[RISCV] Refine cost on Min/Max reduction" (#80340)

Reverts llvm/llvm-project#79402. Crash reported. On closer inspection,
this patch does not handle Intrinsic::maximum and Intrinsic::minimum.

show more ...

[RISCV] Refine cost on Min/Max reduction (#79402)

This patch is split off from #77342, and follows #79103

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, Reduct

[RISCV] Refine cost on Min/Max reduction (#79402)

This patch is split off from #77342, and follows #79103

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, ReductionOp, VMV.X}
- Add SplitCost which chains a series of VMAX/VMIN/... which scales with
LMUL.
- Use MVT to estimate VL.

show more ...

[RISCV][CostModel] Refine Arithmetic reduction costs (#79103)

This patch is split off from #77342

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, ReductionOp, V

[RISCV][CostModel] Refine Arithmetic reduction costs (#79103)

This patch is split off from #77342

- Correct for CodeSize cost that 1 instruction is not included. 3 is
from {VMV.S, ReductionOp, VMV.X}
- Add SplitCost
Unordered reduction chain a series of VADD/VFADD/... which scales with
LMUL.
Ordered reductions chain a series of VFREDOSUMs.
- Use MVT to estimate VL.

show more ...

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[CostModel][RISCV] Model code size cost for reduction

Since code-size cost doesn't scale linearly with LMUL,
this change is to separate it from throughput.

Reviewed By: reames

Differential Revisio

[CostModel][RISCV] Model code size cost for reduction

Since code-size cost doesn't scale linearly with LMUL,
this change is to separate it from throughput.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D142068

show more ...

[RISCV] Use VScaleForTuning in costing of operations whose cost depends on VL

On known hardware, reductions, gather, and scatter operations have execution latencies which correlated with the vector

[RISCV] Use VScaleForTuning in costing of operations whose cost depends on VL

On known hardware, reductions, gather, and scatter operations have execution latencies which correlated with the vector length (VL) of the operation. Most other operations (e.g. simply arithmetic) don't correlated in this way, and instead essentially fixed cost as VL varies.

When I'd implemented initial scalable cost model support for reductions, gather, and scatter operations, I had used an upper bound on the statically unknown VL. The argument at the time was that this prevented falsely low costs, and biased the vectorizer away from generating bad (on some hardware) code. Unfortunately, practical experience shows we were a bit too effective at that goal, and the high costs defacto prevents vectorization using these constructs at all.

This patch reverses course, and ties the returned cost not to the maximum possible VL, but the VL which would correspond to VScaleForTuning. This parameter is the same one the vectorizer uses when normalizing loop costs, so the term effectively cancels out. The result is that the vectorizer now sees these constructs as comparable in cost to their fixed length variants.

This does introduce the possibility of the cost for these operations being a significant under estimate on platforms where actual VLEN is far from that implied by VScaleForTuning. On such platforms, we might make poor heuristic choices. Probably not in LV itself (due to the cancellation mentioned above), but possibly during e.g. lowering. I'm not currently aware of any concrete examples of this, but this patch does open a concern which did not previously exist.

Previously, we had the problem of overestimating costs causing the same problem on machines much closer to default values for vscale for tuning. With this patch, we still have that problem potentially if vscale for tuning is set high (manually), and then the code is run on a narrow VLEN machine.

Differential Revision: https://reviews.llvm.org/D131519

show more ...

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79483 soon

show more ...

[RISCV] Cost model for scalable reductions

This extends the existing cost model for reductions for scalable vectors.

The existing cost model assumes that reductions are roughly logarithmic in cost

[RISCV] Cost model for scalable reductions

This extends the existing cost model for reductions for scalable vectors.

The existing cost model assumes that reductions are roughly logarithmic in cost for unordered variants and linear for ordered ones. This change keeps that same basic model, and extends it out to the maximum number of elements a scalable vector could possibly have.

This results in costs which aren't terribly high for unordered reductions, but are for ordered ones. This seems about right; we want to strongly bias away from using scalable ordered reductions if the cost might be linear in VL.

Differential Revision: https://reviews.llvm.org/D127447

show more ...

[RISCV] Use common prefixes to reduce duplication in cost model tests

[RISCV] Add cost model test coverage of scalable reductions