[RISCV][ISEL] Lowering to load-acquire/store-release for RISCV Zalasr (#82914)

Lowering to load-acquire/store-release for RISCV Zalasr.

Currently uses the psABI lowerings for WMO load-acquire/sto

[RISCV][ISEL] Lowering to load-acquire/store-release for RISCV Zalasr (#82914)

Lowering to load-acquire/store-release for RISCV Zalasr.

Currently uses the psABI lowerings for WMO load-acquire/store-release
(which are identical to A.7). These are incompatable with the A.6
lowerings currently used by LLVM. This should be OK for now since Zalasr
is behind the enable experimental extensions flag, but needs to be fixed
before it is removed from that.

For TSO, it uses the standard Ztso mappings except for lowering seq_cst
loads/store to load-acquire/store-release, I had Andrea review that.

show more ...

Reapply "[RISCV] Remove experimental from Ztso. (#96465)"

This was reverted in f985a8826bfa4ca3d23e654185de35e30ea6dc79. Since that,
the default WMO lowering has moved to A67 compatible, the ABI at

Reapply "[RISCV] Remove experimental from Ztso. (#96465)"

This was reverted in f985a8826bfa4ca3d23e654185de35e30ea6dc79. Since that,
the default WMO lowering has moved to A67 compatible, the ABI attribute
emission has landed (off by default), and the LLD change to merge said
attributes have landed. Our ztso lowering is believed to also be A67
compatible, and no known issues remain.

Original commit message:

Ztso 1.0 was ratified in January 2023.
Documentation:
https://github.com/riscv/riscv-isa-manual/blob/main/src/ztso-st-ext.adoc

show more ...

Reapply "[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)" (#90267)

With the tag merging in place, we can safely change the default for
+seq-cst-trailing-fence to the def

Reapply "[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)" (#90267)

With the tag merging in place, we can safely change the default for
+seq-cst-trailing-fence to the default, according to the recommendation
in
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-atomic.adoc

This patch changes the default for the feature flag, and moves to more
consistent naming with respect to existing features.

This was reverted with https://github.com/llvm/llvm-project/pull/84597,
because ld.bfd would segfault with unknown riscv attributes. Now that
attributes emission is guarded with a backend flag,
`--riscv-abi-attributes`, this should be safe to reland, since it won't
introduce abi tags unless the user opts into them.

show more ...

Revert "[RISCV] Remove experimental from Ztso. (#96465)"

This reverts commit 9cd6ef4b8a5c843ef491437c765d4cb2ff2f8fe3. See
discussion on review thread.

[RISCV] Remove experimental from Ztso. (#96465)

Ztso 1.0 was ratified in January 2023.
Documentation:
https://github.com/riscv/riscv-isa-manual/blob/main/src/ztso-st-ext.adoc

Revert "[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)"

This reverts commit 733b271db793ce30c504a1b5c4ae7a8775b0a6a2.

Reverting in order to revert the companion patch a

Revert "[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)"

This reverts commit 733b271db793ce30c504a1b5c4ae7a8775b0a6a2.

Reverting in order to revert the companion patch adding the atomics ABI
ELF attributes due to the reported incompatibility with GNU ld.
https://github.com/llvm/llvm-project/pull/84597#issuecomment-2079128332

show more ...

[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)

With the tag merging in place, we can safely change the default for
+seq-cst-trailing-fence to the default, according to

[llvm][RISCV] Enable trailing fences for seq-cst stores by default (#87376)

With the tag merging in place, we can safely change the default for
+seq-cst-trailing-fence to the default, according to the recommendation
in

https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-atomic.adoc

This tag changes the default for the feature flag, and moves to more
consistent naming with respect to existing features.

show more ...

[RISCV] Omit "@plt" in assembly output "call foo@plt" (#72467)

R_RISCV_CALL/R_RISCV_CALL_PLT distinction is not necessary and
R_RISCV_CALL has been deprecated. Since https://reviews.llvm.org/D132530

[RISCV] Omit "@plt" in assembly output "call foo@plt" (#72467)

R_RISCV_CALL/R_RISCV_CALL_PLT distinction is not necessary and
R_RISCV_CALL has been deprecated. Since https://reviews.llvm.org/D132530
`call foo` assembles to R_RISCV_CALL_PLT. The `@plt` suffix is not
useful and can be removed now (matching AArch64 and PowerPC).

GNU assembler assembles `call foo` to RISCV_CALL_PLT since 2022-09
(70f35d72ef04cd23771875c1661c9975044a749c).

Without this patch, unconditionally changing MO_CALL to MO_PLT could
create `jump .L1@plt, a0`, which is invalid in LLVM integrated assembler
and GNU assembler.

show more ...

[RISC-V] Add proposed mapping for Ztso

Currently LLVM emits Ztso code for fences, loads, and stores (behind an
experimental flag) [1]. This patch updates the mapping and implements
support for LR/SC

[RISC-V] Add proposed mapping for Ztso

Currently LLVM emits Ztso code for fences, loads, and stores (behind an
experimental flag) [1]. This patch updates the mapping and implements
support for LR/SC and AMO ops. This updated mapping is compatible with
the RVWMO ABI present in the psABI. Additional context can be found in
the psABI pull request [2].

[1] https://reviews.llvm.org/D143076
[2] https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/391

Differential Revision: https://reviews.llvm.org/D155517

show more ...

[RISCV] Strengthen atomic ordering for sequentially consistent stores

This is a similar change to one proposed for GCC:
https://inbox.sourceware.org/gcc-patches/20230414170942.1695672-1-patrick@rivo

[RISCV] Strengthen atomic ordering for sequentially consistent stores

This is a similar change to one proposed for GCC:
https://inbox.sourceware.org/gcc-patches/20230414170942.1695672-1-patrick@rivosinc.com/

The changes in this patch are based on the proposal by Hans Boehm to more
closely match the intended semantics for sequentially consistent stores
and to allow some platforms to avoid an ABI break when switching to more
performant atomic instructions. Platforms that have already compiled
code using the existing mappings will also have more time to gradually
replace that code in preparation of the switch.

Further details can be found in the psABI proposal:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/378.

This patch implements a mapping that is stronger than the one outlined in table
A.6 of the RISC-V unprivileged spec to be future compatible with table A.7 of
the same document. The related discussion can be found at
https://lists.riscv.org/g/tech-unprivileged/topic/risc_v_memory_model_topics/92916241

The major change to RISC-V code generation is that we will now emit a trailing
fence for sequentially consistent stores.

The new code sequence should have the following form:
```
fence rw,w; s{b|h|w|d}; fence rw,rw;
```

Other changes and optimizations like using amoswap will be handled separately.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D149486

show more ...

[RISCV] Implement a proposed mapping for Ztso

This change implements a proposed lowering from LLVM's memory model to the TSO memory model defined by the Ztso extension. Selecting the proposed mappin

[RISCV] Implement a proposed mapping for Ztso

This change implements a proposed lowering from LLVM's memory model to the TSO memory model defined by the Ztso extension. Selecting the proposed mapping turns out to be an involved conversation that really didn't fit within a review description, so let me refer you to https://github.com/preames/public-notes/blob/master/riscv-tso-mappings.rst. This review implements the WMO compatible variant (the proposed one in that document).

Ztso is currently accepted as an experimental extension in LLVM. Despite the fact the extension was recently ratified, I think we need to leave it as experimental until we have wide agreement on the chosen mapping for ABI purposes.

I need to note that the current in-tree implementation defaults to generating WMO compatible fences. This is entirely compatible with the proposed mapping in this patch, but is unfortunately not compatible with the major alternative. The in tree implementation is explicitly experimental so the impact of this is limited, but it is worth calling out that if settle on the alternative we will have a minor ABI break. My apologies for not calling this out in the original patch; I had not realized at the time that one of our realistic choices for mappings wouldn't be WMO compatible.

This patch only contains the changes for load/store and fence. That is, it does not change the lowering for atomicrmw operations. This is a sound thing to do under the proposed mapping since the existing WMO mappings remain compatible. I do plan to change these; I'm just working incrementally.

Differential Revision: https://reviews.llvm.org/D143076

show more ...

[RISCV] Add -experimental-ztso test coverage

For the moment, this is the same as the default memory model.

[RISCV] Convert some tests to opaque pointers (NFC)

[RISCV] Generate pseudo instruction li

Add an alias of `addi [x], zero, imm` to generate pseudo
instruction li, which makes assembly mush more readable.
For existed tests, users can update them by r

[RISCV] Generate pseudo instruction li

Add an alias of `addi [x], zero, imm` to generate pseudo
instruction li, which makes assembly mush more readable.
For existed tests, users can update them by running script
`llvm/utils/update_llc_test_checks.py`.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D112692

show more ...

[RISCV][NFC] Regenerate RISCV CodeGen tests

Regenerated using:

./llvm/utils/update_llc_test_checks.py -u llvm/test/CodeGen/RISCV/*.ll

This has added comments to spill-related instructions and adde

[RISCV][NFC] Regenerate RISCV CodeGen tests

Regenerated using:

./llvm/utils/update_llc_test_checks.py -u llvm/test/CodeGen/RISCV/*.ll

This has added comments to spill-related instructions and added @plt to
some symbols.

Differential Revision: https://reviews.llvm.org/D92841

show more ...

[RISCV] Add codegen support for RV64A

In order to support codegen RV64A, this patch:
* Introduces masked atomics intrinsics for atomicrmw operations and cmpxchg
that use the i64 type. These are ul

[RISCV] Add codegen support for RV64A

In order to support codegen RV64A, this patch:
* Introduces masked atomics intrinsics for atomicrmw operations and cmpxchg
that use the i64 type. These are ultimately lowered to masked operations
using lr.w/sc.w, but we need to use these alternate intrinsics for RV64
because i32 is not legal
* Modifies RISCVExpandPseudoInsts.cpp to handle PseudoAtomicLoadNand64 and
PseudoCmpXchg64
* Modifies the AtomicExpandPass hooks in RISCVTargetLowering to sext/trunc as
needed for RV64 and to select the i64 intrinsic IDs when necessary
* Adds appropriate patterns to RISCVInstrInfoA.td
* Updates test/CodeGen/RISCV/atomic-*.ll to show RV64A support

This ends up being a fairly mechanical change, as the logic for RV32A is
effectively reused.

Differential Revision: https://reviews.llvm.org/D53233

llvm-svn: 351422

show more ...

[RISCV][NFC] Add CHECK lines for atomic operations on RV64I

As or RV32I, we include these for completeness. Committing now to make it
easier to review the RV64A patch.

llvm-svn: 350962

[RISCV] atomic_store_nn have a different layout to regular store

We cannot directy reuse the patterns of StPat because for some reason the store
DAG node and the atomic_store_nn DAG nodes put the pt

[RISCV] atomic_store_nn have a different layout to regular store

We cannot directy reuse the patterns of StPat because for some reason the store
DAG node and the atomic_store_nn DAG nodes put the ptr and the value in
different positions. Currently we attempt to store the address to an address
formed by the value.

Differential Revision: https://reviews.llvm.org/D51217

llvm-svn: 340722

show more ...

[RISCV] Add codegen support for atomic load/stores with RV32A

Fences are inserted according to table A.6 in the current draft of version 2.3
of the RISC-V Instruction Set Manual, which incorporates

[RISCV] Add codegen support for atomic load/stores with RV32A

Fences are inserted according to table A.6 in the current draft of version 2.3
of the RISC-V Instruction Set Manual, which incorporates the memory model
changes and definitions contributed by the RISC-V Memory Consistency Model
task group.

Instruction selection failures will now occur for 8/16/32-bit atomicrmw and
cmpxchg operations when targeting RV32IA until lowering for these operations
is added in a follow-on patch.

Differential Revision: https://reviews.llvm.org/D47589

llvm-svn: 334591

show more ...

[RISCV] Codegen support for atomic operations on RV32I

This patch adds lowering for atomic fences and relies on AtomicExpandPass to
lower atomic loads/stores, atomic rmw, and cmpxchg to __atomic_* l

[RISCV] Codegen support for atomic operations on RV32I

This patch adds lowering for atomic fences and relies on AtomicExpandPass to
lower atomic loads/stores, atomic rmw, and cmpxchg to __atomic_* libcalls.

test/CodeGen/RISCV/atomic-* are modelled on the exhaustive
test/CodeGen/PPC/atomics-regression.ll, and will prove more useful once RV32A
codegen support is introduced.

Fence mappings are taken from table A.6 in the current draft of version 2.3 of
the RISC-V Instruction Set Manual, which incorporates the memory model changes
and definitions contributed by the RISC-V Memory Consistency Model task group.

Differential Revision: https://reviews.llvm.org/D47587

llvm-svn: 334590

show more ...