[RISCV] Lower fixed-length mgather/mscatter for zvfhmin/zvfbfmin (#114945)

In preparation for allowing zvfhmin and zvfbfmin in
isLegalElementTypeForRVV, this lowers fixed-length masked gathers and

[RISCV] Lower fixed-length mgather/mscatter for zvfhmin/zvfbfmin (#114945)

In preparation for allowing zvfhmin and zvfbfmin in
isLegalElementTypeForRVV, this lowers fixed-length masked gathers and
scatters

We need to mark f16 and bf16 as legal in isLegalMaskedGatherScatter
otherwise ScalarizeMaskedMemIntrin will just scalarize them, but we can
move this back into isLegalElementTypeForRVV afterwards.

The scalarized codegen required #114938, #114927 and #114915 to not
crash.

show more ...

[RISCV] Remove -riscv-v-vector-bits-min from cost model tests. NFC

It looks like they were added to prevent fixed length vectors from
being expanded, but that's no longer the case today:
https://rev

[RISCV] Remove -riscv-v-vector-bits-min from cost model tests. NFC

It looks like they were added to prevent fixed length vectors from
being expanded, but that's no longer the case today:
https://reviews.llvm.org/D121447#3376520

show more ...

[RISCV][TTI] Model cost for insert/extract into illegal types (#106440)

We'd previously just deferred to the base implementation, but that more
or less always returns 1. This underestimates the cos

[RISCV][TTI] Model cost for insert/extract into illegal types (#106440)

We'd previously just deferred to the base implementation, but that more
or less always returns 1. This underestimates the cost of the
insert/extract, biases the SLP vectorizer towards forming illegally
typed vectors, and underestimates the cost of scalarized operations
(like unaligned scatter/gather).

show more ...

[TTI] getCommonMaskedMemoryOpCost - consistently use getScalarizationOverhead instead of ExtractElement costs for address/mask extraction. (#87771)

These aren't unknown extraction indices, we will b

[TTI] getCommonMaskedMemoryOpCost - consistently use getScalarizationOverhead instead of ExtractElement costs for address/mask extraction. (#87771)

These aren't unknown extraction indices, we will be extracting every address/mask element in sequence.

show more ...

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and

[RISCV][CostModel] Add getCFInstrCost RISC-V implementation (#65599)

This patch implements getCFInstrCost TTI hook that mostly affects
LoopVectorizer decisions. It sets zero cost for PHI nodes and zero
throughput cost for branches (assuming that branches are likely to
be predicted). The implementation is similar to X86/AArch64/PowerPC
targets and reduces loop cost by excluding induction PHIs/loop latch
branches, which in turn leads to selecting smaller vectorization
factor.

show more ...

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: h

[RISCV] Upgrade Zvfh version to 1.0 and move out of experimental state.

This has been ratified according to https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions

Differential Revision: https://reviews.llvm.org/D155668

show more ...

[CostModel] Convert some tests to opaque pointers (NFC)

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or

[RISCV] Add cost model for vector insert/extract element.

This patch adds cost model for vector insert/extract element instructions. In RVV, we could use vector scalar move instruction to insert or extract the first element, and use vslide to move it. But for mask vector or i64 vector in i32 target, we need special instructions to make it.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D133007

show more ...

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79

[CostModel] Update RUN -passes=* to double quotes to appease update scripts on windows

DOS really doesn't like `` quotes to be used in command lines

Some prep work as I'm intending to resurrect D79483 soon

show more ...

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector

[RISCV] Add +experimental-zvfh extension to cover half types in vectors.

Currently we allow half types in vectors if the scalar Zfh extension
is enabled. This behavior is not inline with the vector spec. For f32
and f64 types, the Zve32f, Zve64f, Zve64d, and V explicitly control
the availablity of floating point types in vectors.

In order to make our compiler compliant, we either need to remove all support
for half in vectors or we need an extension to control it.

Draft spec here https://github.com/riscv/riscv-v-spec/pull/780

Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D121345

show more ...

[test] Migrate '-analyze -cost-model' to '-passes=print<cost-model>'

[RISCV] Remove experimental prefix from rvv-related extensions.

Extensions affected: +v, +zve*, +zvl*

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117860

[RISCV] update zfh and zfhmin extention to v1.0

`zfh` and `zfhmin` have been ratified, with version 1.0.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D117098

[NFC][llvm] Inclusive language: reword uses of sanity test and check

Part of continuing work to use more inclusive language. Reworded uses
of sanity check and sanity test in llvm/test/

[RISCV] Expand unaligned fixed-length vector memory accesses

RVV vectors must be aligned to their element types, so anything less is
unaligned.

For regular loads and stores, our custom-lowering of

[RISCV] Expand unaligned fixed-length vector memory accesses

RVV vectors must be aligned to their element types, so anything less is
unaligned.

For regular loads and stores, our custom-lowering of fixed-length
vectors meant that we opted out of LegalizeDAG's built-in unaligned
expansion. This patch adds that logic in to our custom lower function.

For masked intrinsics, we declare that anything unaligned is not legal,
leaving the ScalarizeMaskedMemIntrin pass to do the expansion for us.

Note that neither of these methods can handle the expansion of
scalable-vector memory ops, so those cases are left alone by this patch.
Scalable loads and stores already go through expansion by default but
hit an assertion, and scalable masked intrinsics will silently generate
incorrect code. It may be prudent to return an error in both of these
cases.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D102493

show more ...

[RISCV][NFC] Correct alignment in scatter/gather tests

This lays the groundwork for changes to alignment in D102493 to be more
apparent.

[RISCV] Reorder checks in RISCVTTIImpl::getGatherScatterOpCost to avoid calling getMinRVVVectorSizeInBits() when V extension is not enabled.

getMinRVVVectorSizeInBits() asserts if the V extension is

[RISCV] Reorder checks in RISCVTTIImpl::getGatherScatterOpCost to avoid calling getMinRVVVectorSizeInBits() when V extension is not enabled.

getMinRVVVectorSizeInBits() asserts if the V extension isn't
enabled. So check that gather/scatter is legal first since it
already contains a check for V extension being enabled. It
also already checks getMinRVVVectorSizeInBits for fixed length
vectors so we don't need a check in getGatherScatterOpCost.

show more ...

[RISCV] Add basic cost modelling for fixed vector gather/scatter.

Reviewed By: frasercrmck

Differential Revision: https://reviews.llvm.org/D99142