[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional schedu

[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional scheduling and tracking register
pressure.

Disclaimer: I haven't tested it on many cores, maybe we should make
some options being features. I believe downstreams must have tried
this before, so feedbacks are welcome.

show more ...

[RISCV] Pass f32/f64 directly without a bitcast for Zfinx/Zdinx. (#107464)

With Zfinx/Zdinx, f32/f64 are legal types for a GPR, we don't need a
bitcast.

This avoids turning fneg/fabs into bitwis

[RISCV] Pass f32/f64 directly without a bitcast for Zfinx/Zdinx. (#107464)

With Zfinx/Zdinx, f32/f64 are legal types for a GPR, we don't need a
bitcast.

This avoids turning fneg/fabs into bitwise operations purely because of
these bitcasts. If the bitwise operations are faster for some reason on
a Zfinx CPU, then that seems like it should be done for all fneg/fabs,
not just ones near function arguments/returns.

I don't have much interest in Zfinx, this just makes the code more
similar to what I proposed for Zhinx in #107446.

show more ...

[RISCV] Don't cost Fmv for Zfinx in isFPImmLegal. (#107361)

There is no Fmv with Zfinx.

Revert "Revert "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)" and more..."

This reverts commit 16bd10a38730fed27a3bf111076b8ef7a7e7b3ee.

Re-applies:
b3c55b707110084a9f5

Revert "Revert "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)" and more..."

This reverts commit 16bd10a38730fed27a3bf111076b8ef7a7e7b3ee.

Re-applies:
b3c55b707110084a9f50a16aade34c3be6fa18da - "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)"
8e2f6495c0bac1dd6ee32b6a0d24152c9c343624 - "[DAGCombiner] Do not always fold FREEZE over BUILD_VECTOR (#85932)"
73472c5996716cda0dbb3ddb788304e0e7e6a323 - "[SelectionDAG] Treat CopyFromReg as freezing the value (#85932)"

with a fix in DAGCombiner::visitFREEZE.

show more ...

Revert "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)" and more...

This reverts:
b3c55b707110084a9f50a16aade34c3be6fa18da - "[SelectionDAG] Handle more opcodes in canCreateUn

Revert "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)" and more...

This reverts:
b3c55b707110084a9f50a16aade34c3be6fa18da - "[SelectionDAG] Handle more opcodes in canCreateUndefOrPoison (#84921)"
(because it updates a test case that I don't know how to resolve the conflict for)
8e2f6495c0bac1dd6ee32b6a0d24152c9c343624 - "[DAGCombiner] Do not always fold FREEZE over BUILD_VECTOR (#85932)"
73472c5996716cda0dbb3ddb788304e0e7e6a323 - "[SelectionDAG] Treat CopyFromReg as freezing the value (#85932)"

Due to a test suite failure on AArch64 when compiling for SVE.
https://lab.llvm.org/buildbot/#/builders/197/builds/13955

clang: ../llvm/llvm/include/llvm/CodeGen/ValueTypes.h:307: MVT llvm::EVT::getSimpleVT() const: Assertion `isSimple() && "Expected a SimpleValueType!"' failed.

show more ...

[SelectionDAG] Treat CopyFromReg as freezing the value (#85932)

The description of CopyFromReg in ISDOpcodes.h says that the input
valus is defined outside the scope of the current SelectionDAG. I
t

[SelectionDAG] Treat CopyFromReg as freezing the value (#85932)

The description of CopyFromReg in ISDOpcodes.h says that the input
valus is defined outside the scope of the current SelectionDAG. I
think that means that we basically can treat it as a FREEZE in the
sense that it can be seen as neither being undef nor poison.

Being able to fold freeze(CopyFromReg) into CopyFromReg seems
useful to avoid regressions if we start to introduce freeze
instruction in DAGCombiner/foldBoolSelectToLogic, e.g. to solve
https://github.com/llvm/llvm-project/issues/84653

Things _not_ dealt with in this patch:
- Depending on calling convention an input argument can be passed
also on the stack and not in a register. If it is allowed to treat
an argument received in a register as not being poison, then I think
we want to treat arguments received on the stack the same way. But
then we need to attribute load instructions, or add explicit FREEZE
when lowering formal arguments.
- A common pattern is that there is an AssertZext or AssertSext just
after CopyFromReg. I think that if we treat CopyFromReg as never
being poison, then it should be allowed to fold
(freeze(AssertZext(CopyFromReg))) -> AssertZext(CopyFromReg))

show more ...

[RISCV] Add integer RISCVISD::SELECT_CC to canCreateUndefOrPoison and isGuaranteedNotToBeUndefOrPoison. (#84693)

Integer RISCVISD::SELECT_CC doesn't create poison. If none of the,
operands are pois

[RISCV] Add integer RISCVISD::SELECT_CC to canCreateUndefOrPoison and isGuaranteedNotToBeUndefOrPoison. (#84693)

Integer RISCVISD::SELECT_CC doesn't create poison. If none of the,
operands are poison, the result is not poison.

This allows ISD::FREEZE to be hoisted above RISCVISD::SELECT_CC.

show more ...

[SelectionDAG] Allow FREEZE to be hoisted before FP SETCC. (#84358)

No nans/infs in SelectionDAG is complicated. Hopefully I've captured
all of the cases. I've only applied to ConsiderFlags to the

[SelectionDAG] Allow FREEZE to be hoisted before FP SETCC. (#84358)

No nans/infs in SelectionDAG is complicated. Hopefully I've captured
all of the cases. I've only applied to ConsiderFlags to the SDNodeFlags
since those are the only ones that will be droped by hoisting. The
condition code and TargetOptions would still be in effect.

Recovers some regression from #84232.

show more ...

[RISCV] Insert a freeze before converting select to AND/OR. (#84232)

Select blocks poison, but AND/OR do not. We need to insert a freeze
to block poison propagation.

This creates suboptimal code

[RISCV] Insert a freeze before converting select to AND/OR. (#84232)

Select blocks poison, but AND/OR do not. We need to insert a freeze
to block poison propagation.

This creates suboptimal codegen which I will try to fix with other
patches. I'm prioritizing the correctness fix since we have 2 bug reports.

Fixes #84200 and #84350

show more ...

[RISCV] Omit "@plt" in assembly output "call foo@plt" (#72467)

R_RISCV_CALL/R_RISCV_CALL_PLT distinction is not necessary and
R_RISCV_CALL has been deprecated. Since https://reviews.llvm.org/D132530

[RISCV] Omit "@plt" in assembly output "call foo@plt" (#72467)

R_RISCV_CALL/R_RISCV_CALL_PLT distinction is not necessary and
R_RISCV_CALL has been deprecated. Since https://reviews.llvm.org/D132530
`call foo` assembles to R_RISCV_CALL_PLT. The `@plt` suffix is not
useful and can be removed now (matching AArch64 and PowerPC).

GNU assembler assembles `call foo` to RISCV_CALL_PLT since 2022-09
(70f35d72ef04cd23771875c1661c9975044a749c).

Without this patch, unconditionally changing MO_CALL to MO_PLT could
create `jump .L1@plt, a0`, which is invalid in LLVM integrated assembler
and GNU assembler.

show more ...

[RISCV][NFC] Add comments and tests for frint case of performFP_TO_INT_SATCombine. (#76014)

performFP_TO_INT_SATCombine could also serve pattern (fp_to_int_sat
(frint X)).

Revert "[CodeGen] Really renumber slot indexes before register allocation (#67038)"

This reverts commit 2501ae58e3bb9a70d279a56d7b3a0ed70a8a852c.

Reverted due to various buildbot failures.

[CodeGen] Really renumber slot indexes before register allocation (#67038)

PR #66334 tried to renumber slot indexes before register allocation, but
the numbering was still affected by list entries

[CodeGen] Really renumber slot indexes before register allocation (#67038)

PR #66334 tried to renumber slot indexes before register allocation, but
the numbering was still affected by list entries for instructions which
had been erased. Fix this to make the register allocator's live range
length heuristics even less dependent on the history of how instructions
have been added to and removed from SlotIndexes's maps.

show more ...

[RISCV][CodeGen] Support Zfinx codegen

This patch was split from D122918 . Co-Author: @liaolucy @realqhc

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D148874

[TargetLowering] Don't use ISD::SELECT_CC in expandFP_TO_INT_SAT.

This function gets called for vectors and ISD::SELECT_CC was never
intended to support vectors. Some updates were made to support
it

[TargetLowering] Don't use ISD::SELECT_CC in expandFP_TO_INT_SAT.

This function gets called for vectors and ISD::SELECT_CC was never
intended to support vectors. Some updates were made to support
it when this function started getting used for vectors.

Overall, using separate ISD::SETCC and ISD::SELECT looks like an
improvement even for scalar.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D149481

show more ...

[RISCV] Move compressible registers to the beginning of the FP allocation order.

We don't have very many compressible FP instructions, just load and store.
These instruction require the FP register

[RISCV] Move compressible registers to the beginning of the FP allocation order.

We don't have very many compressible FP instructions, just load and store.
These instruction require the FP register to be f8-f15.

This patch changes the FP allocation order to prioritize f10-f15 first.
These are also the FP argument registers. So I allocated them in reverse
order starting at f15 to avoid taking the first argument registers.
This appears to match gcc allocation order.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D146488

show more ...

[RISCV] Don't use constantpool for floating-point value if the value can be easily constructed by integer sequence and a floating-point move.

In addition, this commit does the following combine

vfm

[RISCV] Don't use constantpool for floating-point value if the value can be easily constructed by integer sequence and a floating-point move.

In addition, this commit does the following combine

vfmv.v.f + fmv.[dhw].x -> vmv.v.x
vfmv.s.f + fmv.[dhw].x -> vmv.s.x
vfmerge.vfm + fmv.[dhw].x -> vmerge.vxm

Differential Revision: https://reviews.llvm.org/D142953

show more ...

[RISCV] Inline scalar ceil/floor/trunc/rint/round/roundeven.

This avoids the call overhead as well as the the save/restore of
fflags and the snan handling in the libm function.

The save/restore of

[RISCV] Inline scalar ceil/floor/trunc/rint/round/roundeven.

This avoids the call overhead as well as the the save/restore of
fflags and the snan handling in the libm function.

The save/restore of fflags and snan handling are needed to be
correct for -ftrapping-math. I think we can ignore them in the
default environment.

The inline sequence will generate an invalid exception for nan
and an inexact exception if fractional bits are discarded.

I've used a custom inserter to explicitly create the control flow
around the float->int->float conversion.

We can probably avoid the final fsgnj after the conversion for
no signed zeros FMF, but I'll leave that for future work.

Note the comparison constant is slightly different than glibc uses.
They use 1<<53 for double, I'm using 1<<52. I believe either are valid.
Numbers >= 1<<52 can't have any fractional bits. It's ok to do the
float->int->float conversion on numbers between 1<<53 and 1<<52 since
they will all fit in 64. We only have a problem if the double can't fit
in i64

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D136508

show more ...

[RISCV] Match (select C, -1, X)->(or -C, X) during lowerSelect

Same with (select C, X, -1), (select C, 0, X), and (select C, X, 0).

There's a DAGCombine after we turn the select into select_cc, but

[RISCV] Match (select C, -1, X)->(or -C, X) during lowerSelect

Same with (select C, X, -1), (select C, 0, X), and (select C, X, 0).

There's a DAGCombine after we turn the select into select_cc, but
that may introduce a setcc that didn't previously exist. We could
add more DAGCombines to remove the extra setcc, but this seemed lower
effort.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D135833

show more ...

[RISCV] Use branchless form for selects with 0 in either arm

Continuing the theme of adding branchless lowerings for simple selects, this time handle the 0 arm case. This is very common for various

[RISCV] Use branchless form for selects with 0 in either arm

Continuing the theme of adding branchless lowerings for simple selects, this time handle the 0 arm case. This is very common for various umin idioms, etc..

Differential Revision: https://reviews.llvm.org/D135600

show more ...

[RISCV] Use branchless form for selects with -1 in either arm

We can lower these as an or with the negative of the condition value. This appears to result in significantly less branch-y code on mult

[RISCV] Use branchless form for selects with -1 in either arm

We can lower these as an or with the negative of the condition value. This appears to result in significantly less branch-y code on multiple common idioms (as seen in tests).

Differential Revision: https://reviews.llvm.org/D135316

show more ...

[RISCV] Use check-prefixes to reduce check lines

Reviewed By: frasercrmck

Differential Revision: https://reviews.llvm.org/D125083

[RISCV] Implement a basic version of AArch64RedundantCopyElimination pass.

Using AArch64's original implementation for reference, this patch
implements a pass to remove unneeded copies of X0. This p

[RISCV] Implement a basic version of AArch64RedundantCopyElimination pass.

Using AArch64's original implementation for reference, this patch
implements a pass to remove unneeded copies of X0. This pass runs
after register allocation and looks to see if a register is implied
to be 0 by a branch in the predecessor basic block.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D118160

show more ...

[RISCV] Add DAG combine to fold (fp_to_int_sat (ffloor X)) -> (select X == nan, 0, (fcvt X, rdn))

Similar for ceil, trunc, round, and roundeven. This allows us to use
static rounding modes to avoid

[RISCV] Add DAG combine to fold (fp_to_int_sat (ffloor X)) -> (select X == nan, 0, (fcvt X, rdn))

Similar for ceil, trunc, round, and roundeven. This allows us to use
static rounding modes to avoid a libcall.

This is similar to D116771, but for the saturating conversions.

This optimization is done for AArch64 as isel patterns.
RISCV doesn't have instructions for ceil/floor/trunc/round/roundeven
so the operations don't stick around until isel to enable a pattern
match. Thus I've implemented a DAG combine.

I'm only handling saturating to i64 or i32. This could be extended
to other sizes in the future.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D116864

show more ...