[TTI][RISCV] Unconditionally break critical edges to sink ADDI (#108889)

This looks like a rather weird change, so let me explain why this isn't
as unreasonable as it looks. Let's start with the pr

[TTI][RISCV] Unconditionally break critical edges to sink ADDI (#108889)

This looks like a rather weird change, so let me explain why this isn't
as unreasonable as it looks. Let's start with the problem it's solving.

```
define signext i32 @overlap_live_ranges(ptr %arg, i32 signext %arg1) { bb:
%i = icmp eq i32 %arg1, 1
br i1 %i, label %bb2, label %bb5

bb2: ; preds = %bb
%i3 = getelementptr inbounds nuw i8, ptr %arg, i64 4
%i4 = load i32, ptr %i3, align 4
br label %bb5

bb5: ; preds = %bb2, %bb
%i6 = phi i32 [ %i4, %bb2 ], [ 13, %bb ]
ret i32 %i6
}
```

Right now, we codegen this as:

```
li a3, 1
li a2, 13
bne a1, a3, .LBB0_2
lw a2, 4(a0)
.LBB0_2:
mv a0, a2
ret
```

In this example, we have two values which must be assigned to a0 per the
ABI (%arg, and the return value). SelectionDAG ensures that all values
used in a successor phi are defined before exit the predecessor block.
This creates an ADDI to materialize the immediate in the entry block.

Currently, this ADDI is not sunk into the tail block because we'd have
to split a critical edges to do so. Note that if our immediate was
anything large enough to require two instructions we *would* split this
critical edge.

Looking at other targets, we notice that they don't seem to have this
problem. They perform the sinking, and tail duplication that we don't.
Why? Well, it turns out for AArch64 that this is entirely an accident of
the existance of the gpr32all register class. The immediate is
materialized into the gpr32 class, and then copied into the gpr32all
register class. The existance of that copy puts us right back into the
two instruction case noted above.

This change essentially just bypasses this emergent behavior aspect of
the aarch64 behavior, and implements the same "always sink immediates"
behavior for RISCV as well.

show more ...

[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional schedu

[RISCV] Enable bidirectional scheduling and tracking register pressure (#115445)


This is based on other targets like PPC/AArch64 and some experiments.

This PR will only enable bidirectional scheduling and tracking register
pressure.

Disclaimer: I haven't tested it on many cores, maybe we should make
some options being features. I believe downstreams must have tried
this before, so feedbacks are welcome.

show more ...

[RISCV] Add coverage for select C, C1, C2 where (C1-C2)*[0,1] is cheap

[RISC-V][ISel] Remove redundant czero.eqz like 'czero.eqz a0, a0, a0' (#90208)

In RISC-V ISel, the instruction `czero.eqz a0, a0, a0` is meaningless.
This patch does the following folds in ISel:
`

[RISC-V][ISel] Remove redundant czero.eqz like 'czero.eqz a0, a0, a0' (#90208)

In RISC-V ISel, the instruction `czero.eqz a0, a0, a0` is meaningless.
This patch does the following folds in ISel:
```
czero_eqz x, (setcc x, 0, ne) -> x
czero_nez x, (setcc x, 0, eq) -> x
```

---------

Signed-off-by: Zhijin Zeng <zhijin.zeng@spacemit.com>

show more ...

[RISCV][SDAG] Improve codegen of select with constants if zicond is available (#82456)

This patch uses `add + czero.eqz/nez` to lower select with constants if
zicond is available.
```
(select c,

[RISCV][SDAG] Improve codegen of select with constants if zicond is available (#82456)

This patch uses `add + czero.eqz/nez` to lower select with constants if
zicond is available.
```
(select c, c1, c2) -> (add (czero_nez c2 - c1, c), c1)
(select c, c1, c2) -> (add (czero_eqz c1 - c2, c), c2)
```
The above code sequence is suggested by [RISCV Optimization
Guide](https://riscv-optimization-guide-riseproject-c94355ae3e6872252baa952524.gitlab.io/riscv-optimization-guide.html#_avoid_branches_using_conditional_moves).

show more ...

[RISCV][SDAG] Fold `select c, ~x, x` into `xor -c, x` (#82462)

This patch lowers select of constants if `TrueV == ~FalseV`.
Address the comment in
https://github.com/llvm/llvm-project/pull/82456#d

[RISCV][SDAG] Fold `select c, ~x, x` into `xor -c, x` (#82462)

This patch lowers select of constants if `TrueV == ~FalseV`.
Address the comment in
https://github.com/llvm/llvm-project/pull/82456#discussion_r1496881603.

show more ...

[RISCV] Graduate Zicond to non-experimental (#79811)

The Zicond extension was ratified in the last few months, with no
changes that affect the LLVM implementation. Although there's surely
more tun

[RISCV] Graduate Zicond to non-experimental (#79811)

The Zicond extension was ratified in the last few months, with no
changes that affect the LLVM implementation. Although there's surely
more tuning that could be done about when to select Zicond or not, there
are no known correctness issues. Therefore, we should mark support as
non-experimental.

show more ...

[TargetLowering][RISCV] Introduce shouldFoldSelectWithSingleBitTest and RISC-V implement. (#72978)

DAGCombiner folds (select_cc seteq (and x, y), 0, 0, A) to (and (sra
(shl x)) A) where y has a sin

[TargetLowering][RISCV] Introduce shouldFoldSelectWithSingleBitTest and RISC-V implement. (#72978)

DAGCombiner folds (select_cc seteq (and x, y), 0, 0, A) to (and (sra
(shl x)) A) where y has a single bit set. Previously, DAGCombiner relies
on `shouldAvoidTransformToShift` to decide when to do the combine, but
`shouldAvoidTransformToShift` is only about shift cost. This patch
introuduces a specific hook to decide when to do the combine and disable
the combine when Zicond enabled and AndMask <= 1024.

show more ...

[RISCV] Strip W suffix from ADDIW (#68425)

The motivation of this change is simply to reduce test duplication. As
can be seen in the (massive) test delta, we have many tests whose output
differ on

[RISCV] Strip W suffix from ADDIW (#68425)

The motivation of this change is simply to reduce test duplication. As
can be seen in the (massive) test delta, we have many tests whose output
differ only due to the use of addi on rv32 vs addiw on rv64 when the
high bits are don't care.

As an aside, we don't need to worry about the non-zero immediate
restriction on the compressed variants because we're not directly
forming the compressed variants. If we happen to get a zero immediate
for the ADDI, then either a later optimization will strip the useless
instruction or the encoder is responsible for not compressing the
instruction.

show more ...

[RISCV] Generalize 'tryFoldSelectIntOp` to other operations.

Currently, only `SUB`, `ADD`, `OR` and `XOR` are covered. This patch
adds `AND`, `SHL`, `SRA`, `SRL`.

Reviewed By: craig.topper
Differen

[RISCV] Generalize 'tryFoldSelectIntOp` to other operations.

Currently, only `SUB`, `ADD`, `OR` and `XOR` are covered. This patch
adds `AND`, `SHL`, `SRA`, `SRL`.

Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D155344

show more ...

[RISCV] Use RISCVISD::CZERO_EQZ/CZERO_NEZ for XVentanaCondOps.

This makes Zicond and XVentanaCondOps use the same code path.
The instructions have identical semantics.

Reviewed By: wangpc

Differen

[RISCV] Use RISCVISD::CZERO_EQZ/CZERO_NEZ for XVentanaCondOps.

This makes Zicond and XVentanaCondOps use the same code path.
The instructions have identical semantics.

Reviewed By: wangpc

Differential Revision: https://reviews.llvm.org/D155391

show more ...

[RISCV] Introduce RISCVISD::CZERO_{EQZ,NEZ} nodes produce them when zicond is present in lowerSELECT

This patch is a step towards altering how we handle the emission of
condops. Marking ISD::SELECT

[RISCV] Introduce RISCVISD::CZERO_{EQZ,NEZ} nodes produce them when zicond is present in lowerSELECT

This patch is a step towards altering how we handle the emission of
condops. Marking ISD::SELECT as legal is a major change in the codegen
path, and gives few options for maintaining the old codegen path when
it is believed to be better (e.g. a better branchless sequence is
possible using non-zicond instructions, or the branch-based sequence is
preferable).

This removes the existing SelectionDAG patterns and moves the logic into
lowerSELECT. Along some small codegen changes you'll note a few minor
regressions in the generated code quality - this are due to the fact
that by lowering the SELECT node early we miss out on combines that
would kick in later when setcc condcodes that aren't natively supported
have been expanded (thus exposing opportunities for optimisation by
performing logical negation and swapping truev/falsev). I've opted to
split out work that addresses these into follow-on patches (especially
as zicond is still 'experimental').

matchSetCC is a straight-forward translation from the version in
RISCVISelDAGToDAG. Ideally, in the future it can be converted to a
helper shared between both files.

Differential Revision: https://reviews.llvm.org/D155083

show more ...

[RISCV] Removed the requirement of XLenVT for performSELECTCombine.

Reviewed By: Craig Topper

Differential Revision: https://reviews.llvm.org/D153044

[RISCV] Add codegen for the experimental zicond extension

This directly matches the codegen for xventanacondops with vt.maskcn =>
czero.nez and vt.maskc => czero.eqz. An additional difference is tha

[RISCV] Add codegen for the experimental zicond extension

This directly matches the codegen for xventanacondops with vt.maskcn =>
czero.nez and vt.maskc => czero.eqz. An additional difference is that
zicond is available on RV32 in addition to RV64 (xventanacondops is RV64
only).

Differential Revision: https://reviews.llvm.org/D147147

show more ...

[RISCV][test] Update CHECK lines in condops related tests in preparation for Zicond codegen

Prefixes like 'CONDOPS' referring to the xventanacondops extension are
going to be confusing once zicond i

[RISCV][test] Update CHECK lines in condops related tests in preparation for Zicond codegen

Prefixes like 'CONDOPS' referring to the xventanacondops extension are
going to be confusing once zicond is added to the mix.

show more ...

[RISCV] Add more XVentanaCondOps patterns.

Add patterns with seteq/setne conditions.

We don't have instructions for seteq/setne except for comparing
with zero and need to emit an ADDI or XOR before

[RISCV] Add more XVentanaCondOps patterns.

Add patterns with seteq/setne conditions.

We don't have instructions for seteq/setne except for comparing
with zero and need to emit an ADDI or XOR before a seqz/snez to
compare other values.

The select ISD node takes a 0/1 value for the condition, but the
VT_MASKC(N) instructions check all XLen bits for zero or non-zero.
We can use this to avoid the seqz/snez in many cases.

This is pretty ridiculous number of patterns. I wonder if we could
use some ComplexPatterns to merge them, but I'd like to do that as
a follow up and focus on correctness of the result in this patch.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D140421

show more ...

[RISCV] Add DAG combine to fold (select C, (add X, Y), Y) -> (add (select C, X, 0), Y).

Similar for sub, or, and xor. These are all operations that have 0
as a neutral value. This is based on a simi

[RISCV] Add DAG combine to fold (select C, (add X, Y), Y) -> (add (select C, X, 0), Y).

Similar for sub, or, and xor. These are all operations that have 0
as a neutral value. This is based on a similar tranform in InstCombine.

This allows us to remove some XVentanaCondOps patterns and
some code from DAGCombine for RISCVISD::SELECT_CC.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D140465

show more ...

[RISCV] Add more test cases to select.ll. NFC

These are test for select (and (x , 0x1) == 0), (z ^ y), y ) and select (and (x , 0x1) == 0), (z | y), y )

These can be made branchless by using ((x-1)

[RISCV] Add more test cases to select.ll. NFC

These are test for select (and (x , 0x1) == 0), (z ^ y), y ) and select (and (x , 0x1) == 0), (z | y), y )

These can be made branchless by using ((x-1) & z ) ^ y.

show more ...

[RISCV] Match neg (and x, 1) to two shifts to improve codesize

The negate operation is never compressible (as the destination and rs1 register must differ). The two shift versions will be equal size

[RISCV] Match neg (and x, 1) to two shifts to improve codesize

The negate operation is never compressible (as the destination and rs1 register must differ). The two shift versions will be equal size if the input GPR is reused, or smaller if this is the only use of the input.

For clarity, the operation being performed is (select (low-bit-of x), -1, 0).

Differential Revision: https://reviews.llvm.org/D140319

show more ...

[RISCV] Add codegen support for RISCV XVentanaCondOps Extension

This patch adds codegen support for part of XVentanaCondOps extension.
This extension is designed to reduce the number of branches in

[RISCV] Add codegen support for RISCV XVentanaCondOps Extension

This patch adds codegen support for part of XVentanaCondOps extension.
This extension is designed to reduce the number of branches in
the generated RISCV assembly by replacing branches with conditional
move instructions as defined by XVentanaCondOps specification.

The specification for XVentanaCondOps extension can be found at:
https://github.com/ventanamicro/ventana-custom-extensions/releases/download/v1.0.1/ventana-custom-extensions-v1.0.1.pdf

Co-authored-by: Mikhail Gudim <mgudim@ventanamicro.com>

Differential Revision: https://reviews.llvm.org/D139394

show more ...

[RISCV] Teach getRegAllocationHints about compressible SRAI/SRLI.

Similar to previous patches for ADDI/ADDIW/SLLI/ADD, but restricted
to only cases where the register is x8-x15(GPRC reg class).

I'v

[RISCV] Teach getRegAllocationHints about compressible SRAI/SRLI.

Similar to previous patches for ADDI/ADDIW/SLLI/ADD, but restricted
to only cases where the register is x8-x15(GPRC reg class).

I've restricted it so that we can be precise about whether the
resulting instruction would be compressible. Changing the register
allocation may make some other instruction not compressible so we
should try to be accurate.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D138740

show more ...

[RISCV] Add codegen coverage for select idioms which might benefit from XVentanaCondOps

[RISCV] Branchless lowering for select (and (x , 0x1) == 0), y, (z ^ y) ) and select (and (x , 0x1) == 0), y, (z | y) )

This code is directly ported from the X86 backend which applies the same rewri

[RISCV] Branchless lowering for select (and (x , 0x1) == 0), y, (z ^ y) ) and select (and (x , 0x1) == 0), y, (z | y) )

This code is directly ported from the X86 backend which applies the same rewrite (along with several others). Planning on looking more closely at the other branchless variants from x86 to see if any are worth porting in future changes.

Motivation here is the coremark crc8 routine from https://github.com/eembc/coremark/blob/main/core_util.c#L165. This patch significantly reduces the number of unpredictable branches in the workload.

Differential Revision: https://reviews.llvm.org/D134881

show more ...

[RISCV] Add test coverage for upcoming select lowering optimization

Test copied from X86 backend since I'm going to be taking the code from there too.