[RISCV][VLOPT] Add support for checkUsers when UserMI is a Single-Width Integer Reduction (#120345)

Reductions are weird because for some operands, they are vector
registers but only read the first

[RISCV][VLOPT] Add support for checkUsers when UserMI is a Single-Width Integer Reduction (#120345)

Reductions are weird because for some operands, they are vector
registers but only read the first lane. For these operands, we do not
need to check to make sure the EEW and EMUL ratios match. The EEWs,
however, do need to match.

show more ...

[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional schedu

[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional scheduling and tracking register
pressure.

Disclaimer: I haven't tested it on many cores, maybe we should make
some options being features. I believe downstreams must have tried
this before, so feedbacks are welcome.

show more ...

[RISCV] Remove redundant +zfh from +zvfh[min] tests. NFC

In the vast majority of f16 tests we don't end up emitting any scalar
code that needs +zfh, so remove it.

[RISCV] Coverage for a few missed vector idioms

[RISCV] Don't look through EXTRACT_ELEMENT in lowerScalarInsert if the element types are different. (#78668)

If the element type of the vector we're extracting from doesn't match the type we're
ins

[RISCV] Don't look through EXTRACT_ELEMENT in lowerScalarInsert if the element types are different. (#78668)

If the element type of the vector we're extracting from doesn't match the type we're
inserting into, we can't directly insert or extract the subvector.

show more ...

[RISCV] Remove -riscv-v-vector-bits-min in tests. NFC (#65404)

V implies Zvl128b, but a lot of the fixed vector tests also redundantly
specify -riscv-v-vector-bits-min=128. This patch removes them

[RISCV] Remove -riscv-v-vector-bits-min in tests. NFC (#65404)

V implies Zvl128b, but a lot of the fixed vector tests also redundantly
specify -riscv-v-vector-bits-min=128. This patch removes them where
there isn't another minimum vlen being tested for, and for cases where
Zve* is being used Zvl128b was added to maintain the old test diff (and
because an awkward vlen probably isn't interesting to test for). Other
places where -risc-v-vector-bits-min were being used were replaced with
Zvl.

show more ...

[RISCV] Use the first element of source as the start value of reduction.

Previously when llvm.reduce.* lowered, riscv backend created scalar vector with
netural element as start value. For llvm.redu

[RISCV] Use the first element of source as the start value of reduction.

Previously when llvm.reduce.* lowered, riscv backend created scalar vector with
netural element as start value. For llvm.reduce.and/or/min/max/fmax/fmin, we
could use the first element of source as the start value. It's benefit for RVV
since we could just use source vector as start vector.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D155929

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[RISCV] Remove one use restriction on the scalar from combineBinOpToReduce.

If multiple reductions use the neutral start value, this prevented
us from optimizing any of them.

This increases the num

[RISCV] Remove one use restriction on the scalar from combineBinOpToReduce.

If multiple reductions use the neutral start value, this prevented
us from optimizing any of them.

This increases the number of scalar->vector operations but reduces
the number of scalar operations.

Note, we were already inconsistent about the one use check when we
peeked through INSERT_SUBVECTORs.

Reviewed By: fakepaper56

Differential Revision: https://reviews.llvm.org/D149181

show more ...

[RISCV] Add test case showing failure to fold (fadd (reduce -0.0, X), Y) due to one use check. NFC"

If the -0.0 has multiple uses we won't fold the reduction.

[RISCV] Ensure extract_vector_elt has a single use in combineBinOpToReduce.

Without this, the original reduction will not be removed and we'll
end up with a second reduction. Reductions are expensiv

[RISCV] Ensure extract_vector_elt has a single use in combineBinOpToReduce.

Without this, the original reduction will not be removed and we'll
end up with a second reduction. Reductions are expensive operations so
we should avoid that.

show more ...

[RISCV] Add test case showing duplicated reduction due to missing one use check. NFC

We don't check that the extract_vector_elt has one use in combineBinOpToReduce.

[RISCV][InsertVSETVLI] Split out demanded property for zero/non-zero of VL

The scalar move instructions (vmv.s.x, and fvmv.s.f) depend solely on whether the VL is 0 or non-zero. By tracking the fact

[RISCV][InsertVSETVLI] Split out demanded property for zero/non-zero of VL

The scalar move instructions (vmv.s.x, and fvmv.s.f) depend solely on whether the VL is 0 or non-zero. By tracking the fact we only demand the zeroness and not the whole VL value, we can allow changing VL over a scalar move. This helps to eliminate vsetvli toggles.

Differential Revision: https://reviews.llvm.org/D140157

show more ...

[RISCV] Reuse VL (if non-zero) when building single element vector for start of reduction chain

This is an alternative patch on a path to D137530.

The basic problem being tackled here is that we ne

[RISCV] Reuse VL (if non-zero) when building single element vector for start of reduction chain

This is an alternative patch on a path to D137530.

The basic problem being tackled here is that we need to place a scalar into lane 0 of a vector register before our reduction instructions. Since we only care about lane 0 of the vector, we can use any VL >= 1 provided that the total amount of work performed matches the work performed for a VL=1.

This change does not contain the logic from D137530 to perform the insert at the original VT, and then extract down to LMUL1. That turns out to be a good choice, as discussion in this review has indicated there are issues around LMUL2 and above with our representation of vmv.s.x. We'd also need to be careful with the splat logic for the same reasons.

The only potentially concerning codegen change I spot here is that we stop using a broadcast load (for VL=1) and instead do a scalar load and insert. I think this is probably reasonable; if reviewers disagree, I can investigate using a broadcast load which writes to the undef lanes. If we want to do that, we should do it for VECTOR_INSERT_ELT as well, so that'll end up as it's own patch series.

Differential Revision: https://reviews.llvm.org/D139656

show more ...

[RISCV][InsertVSETVLI] Default to MA not MU

This changes the default value used for mask policy from mask undisturbed to mask agnostic. In hardware, there may be a minor preference for ta/ma, but si

[RISCV][InsertVSETVLI] Default to MA not MU

This changes the default value used for mask policy from mask undisturbed to mask agnostic. In hardware, there may be a minor preference for ta/ma, but since this is only going to apply to instructions which don't use the mask policy bit, this is functionally mostly a nop. The main value is to make future changes to using MA when legal for masked instructions easier to review by reducing test churn.

The prior code was motivated by a desire to minimize state transitions between masked and unmasked code. This patch achieves the same effect using the demanded field logic (landed in afb45ff), and there are no regressions I spotted in the test diffs. (Given the size, I have only been able to skim.) I do want to call out that regressions are possible here; the demanded analysis only works on a block local scope right now, so e.g. a tight loop mixing masked and unmasked computation might see an extra vsetvli or two.

Differential Revision: https://reviews.llvm.org/D133803

show more ...

[RISCV][NFC] Refactor fadd test to match the code.

Change fadd test case in D122563 to match the fold base case.

Differential Revision: https://reviews.llvm.org/D132722

[RISCV] Add DAGCombine to fold base operation and reduction.

Transform (<bop> x, (reduce.<bop> vec, splat(neutral_element))) to
(reduce.<bop> vec, splat (x)).

Reviewed By: craig.topper

Differentia

[RISCV] Add DAGCombine to fold base operation and reduction.

Transform (<bop> x, (reduce.<bop> vec, splat(neutral_element))) to
(reduce.<bop> vec, splat (x)).

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D122563

show more ...