[RISCV] Convert AVLs with vlenb to VLMAX where possible (#97800)

Given an AVL that's computed from vlenb, if it's equal to VLMAX then we
can replace it with the VLMAX sentinel value.

The main mo

[RISCV] Convert AVLs with vlenb to VLMAX where possible (#97800)

Given an AVL that's computed from vlenb, if it's equal to VLMAX then we
can replace it with the VLMAX sentinel value.

The main motiviation is to be able to express an EVL of VLMAX in VP
intrinsics whilst emitting vsetvli a0, zero, so that we can replace
llvm.riscv.masked.strided.{load,store} with their VP counterparts.

This is done in RISCVVectorPeephole (previously RISCVFoldMasks, renamed
to account for the fact that it no longer just folds masks) instead of
SelectionDAG since there are multiple places places where VP nodes are
lowered that would have need to have been handled.

This also avoids doing it in RISCVInsertVSETVLI as it's much harder to
lookup the value of the AVL, and in RISCVVectorPeephole we can take
advantage of DeadMachineInstrElim to remove any leftover
PseudoReadVLENBs.

show more ...

[RISCV][LoopIdiomVectorize] Support VP intrinsics in LoopIdiomVectorize (#94082)

Teach LoopIdiomVectorize to use VP intrinsics to replace the byte
compare loops. Right now only RISC-V uses LoopIdio

[RISCV][LoopIdiomVectorize] Support VP intrinsics in LoopIdiomVectorize (#94082)

Teach LoopIdiomVectorize to use VP intrinsics to replace the byte
compare loops. Right now only RISC-V uses LoopIdiomVectorize of this
style.

show more ...

[RISCV] Move Machine Copy Propagation Pass before Branch relaxation pass (#97261)

Machine Copy Propagation Pass may enlarge branch relaxation distance by
breaking generation of compressed insts. Th

[RISCV] Move Machine Copy Propagation Pass before Branch relaxation pass (#97261)

Machine Copy Propagation Pass may enlarge branch relaxation distance by
breaking generation of compressed insts. This commit moves Machine Copy
Propagation Pass before Branch relaxation pass so the results of Branch
relaxation pass won't be affected by Machine Copy Propagation Pass.

show more ...

Reland "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94149)

- Fix build with `EXPENSIVE_CHECKS`
- Remove unused `PassName::ID` to resolve warning
- Mark `~SelectionDAGISel` virt

Reland "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94149)

- Fix build with `EXPENSIVE_CHECKS`
- Remove unused `PassName::ID` to resolve warning
- Mark `~SelectionDAGISel` virtual so AArch64 backend can work properly

show more ...

Revert "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94146)

This reverts commit de37c06f01772e02465ccc9f538894c76d89a7a1 to
de37c06f01772e02465ccc9f538894c76d89a7a1

It still b

Revert "[NewPM][CodeGen] Port selection dag isel to new pass manager" (#94146)

This reverts commit de37c06f01772e02465ccc9f538894c76d89a7a1 to
de37c06f01772e02465ccc9f538894c76d89a7a1

It still breaks EXPENSIVE_CHECKS build. Sorry.

show more ...

[NewPM][CodeGen] Port selection dag isel to new pass manager (#83567)

Port selection dag isel to new pass manager.
Only `AMDGPU` and `X86` support new pass version. `-verify-machineinstrs` in new p

[NewPM][CodeGen] Port selection dag isel to new pass manager (#83567)

Port selection dag isel to new pass manager.
Only `AMDGPU` and `X86` support new pass version. `-verify-machineinstrs` in new pass manager belongs to verify instrumentation, it is enabled by default.

show more ...

[RISCV] Merge RISCVCoalesceVSETVLI back into RISCVInsertVSETVLI (#92869)

We no longer need to separate the passes now that #70549 is landed and
this will unblock #89089.

It's not strictly NFC be

[RISCV] Merge RISCVCoalesceVSETVLI back into RISCVInsertVSETVLI (#92869)

We no longer need to separate the passes now that #70549 is landed and
this will unblock #89089.

It's not strictly NFC because it will move coalescing before register
allocation when -riscv-vsetvl-after-rvv-regalloc is disabled. But this
makes it closer to the original behaviour.

show more ...

[RISCV] Support postRA vsetvl insertion pass (#70549)

This patch try to get rid of vsetvl implict vl/vtype def-use chain and
improve the register allocation quality by moving the vsetvl insertion

[RISCV] Support postRA vsetvl insertion pass (#70549)

This patch try to get rid of vsetvl implict vl/vtype def-use chain and
improve the register allocation quality by moving the vsetvl insertion
pass after RVV register allocation

It will gain the benefit for the following optimization from

1. unblock scheduler's constraints by removing vl/vtype def-use chain
2. Support RVV re-materialization
3. Support partial spill

This patch add a new option `-riscv-vsetvl-after-rvv-regalloc=<1|0>` to
control this feature and default set as disable.

show more ...

[RISCV] Defer creating RISCVInsertVSETVLI to avoid leak with -stop-after (#92303)

As noted in
https://github.com/llvm/llvm-project/pull/91440#discussion_r1601976425,
if the pass pipeline stops ear

[RISCV] Defer creating RISCVInsertVSETVLI to avoid leak with -stop-after (#92303)

As noted in
https://github.com/llvm/llvm-project/pull/91440#discussion_r1601976425,
if the pass pipeline stops early because of -stop-after any allocated
passes added with insertPass will not be freed if they haven't already
been added.

This was showing up as a failure on the address sanitizer buildbots. We
can fix it by instead passing the pass ID instead so that allocation is
deferred.

show more ...

[RISCV] Move RISCVInsertVSETVLI to after phi elimination (#91440)

Split off from #70549, this patch moves RISCVInsertVSETVLI to after phi
elimination where we exit SSA and need to move to LiveVaria

[RISCV] Move RISCVInsertVSETVLI to after phi elimination (#91440)

Split off from #70549, this patch moves RISCVInsertVSETVLI to after phi
elimination where we exit SSA and need to move to LiveVariables.

The motivation for splitting this off is to avoid the large scheduling
diffs from moving completely to after regalloc, and instead focus on
converting the pass to work on LiveIntervals.

The two main changes required are updating VSETVLIInfo to store VNInfos
instead of MachineInstrs, which allows us to still check for PHI defs in
needVSETVLIPHI, and fixing up the live intervals of any AVL operands
after inserting new instructions.

On O3 the pass is inserted after the register coalescer, otherwise we
end up with a bunch of COPYs around eliminated PHIs that trip up
needVSETVLIPHI.

Co-authored-by: Piyou Chen <piyou.chen@sifive.com>

show more ...

[RISCV] Move RISCVInsertVSETVLI after CSR/VXRM passes (#91701)

This further splits off #91440 to inch RISCVInsertVSETVLI closer to post
vector regalloc.

As noted in #91440, most of the diffs are

[RISCV] Move RISCVInsertVSETVLI after CSR/VXRM passes (#91701)

This further splits off #91440 to inch RISCVInsertVSETVLI closer to post
vector regalloc.

As noted in #91440, most of the diffs are from moving vsetvli insertion
after the vxrm/csr insertion passes, but these are getting conflated
with the changes from moving to LiveIntervals.

One idea was that we could try and remove some of these diffs by
manually moving back the vsetvlis past the vxrm/csr instructions. But
this meant having to touch up the LiveIntervals again which seemed to
lead to even more diffs.

This instead just moves RISCVInsertVSETVLI after RISCVInsertReadWriteCSR
and RISCVInsertWriteVXRM so we can isolate those changes.

show more ...

[RISCV] Move RISCVDeadRegisterDefinitions to post vector regalloc (#90636)

Currently RISCVDeadRegisterDefinitions runs after vsetvli insertion, but
in #70549 vsetvli insertion runs after vector reg

[RISCV] Move RISCVDeadRegisterDefinitions to post vector regalloc (#90636)

Currently RISCVDeadRegisterDefinitions runs after vsetvli insertion, but
in #70549 vsetvli insertion runs after vector regalloc and as a result
we no longer convert some vsetvli a0, a0s to vsetvli x0, a0. This patch
moves it to after vector regalloc, but before scalar regalloc so we
still get the benefits of reducing register pressure.

show more ...

Reapply "[RISCV] Separate doLocalPostpass into new pass and move to post vector regalloc (#88295)"

The original commit was calling shrinkToUses on an interval for a virtual
register whose def was er

Reapply "[RISCV] Separate doLocalPostpass into new pass and move to post vector regalloc (#88295)"

The original commit was calling shrinkToUses on an interval for a virtual
register whose def was erased. This fixes it by calling shrinkToUses first
and removing the interval if we erase the old VL def.

show more ...

Revert "[RISCV] Separate doLocalPostpass into new pass and move to post vector regalloc (#88295)"

Seems to cause an address sanitizer failure on one of the buildbots related
to live intervals.

[RISCV] Separate doLocalPostpass into new pass and move to post vector regalloc (#88295)

This patch splits off part of the work to move vsetvli insertion to post
regalloc in #70549.

The doLocalP

[RISCV] Separate doLocalPostpass into new pass and move to post vector regalloc (#88295)

This patch splits off part of the work to move vsetvli insertion to post
regalloc in #70549.

The doLocalPostpass operates outside of RISCVInsertVSETVLI's dataflow,
so we can move it to its own pass. We can then move it to post vector
regalloc which should be a smaller change.

A couple of things that are different from #70549:

- This manually fixes up the LiveIntervals rather than recomputing it
via createAndComputeVirtRegInterval. I'm not sure if there's much of a
difference with either.
- For the postpass it's sufficient enough to just check isUndef() in
hasUndefinedMergeOp, i.e. we don't need to lookup the def in VNInfo.

Running on llvm-test-suite and SPEC CPU 2017 there aren't any changes in
the number of vsetvlis removed. There are some minor scheduling diffs as
well as extra spills and less spills in some cases (caused by transient
vsetvlis existing between RISCVInsertVSETVLI and RISCVCoalesceVSETVLI
when vec regalloc happens), but they are minor and should go away once
we finish moving the rest of RISCVInsertVSETVLI.

We could also potentially turn off this pass for unoptimised builds.

show more ...

[RISCV] Remove -riscv-split-regalloc flag (#89715)

Split vector and scalar regalloc has been enabled by default for 5
months now since d0a39e617ba301a76d28e2d82e1f657999c9dcfb, and shipped
with 18

[RISCV] Remove -riscv-split-regalloc flag (#89715)

Split vector and scalar regalloc has been enabled by default for 5
months now since d0a39e617ba301a76d28e2d82e1f657999c9dcfb, and shipped
with 18.1.0. I haven't heard of any issues with it so far, so this
proposes to remove the flag to reduce the number of configurations we
have to support.

show more ...

[RISCV][NFC] Add helpers for RVV register classes

There are two places in tree that use these helpers and there will
be more future usages.

Reviewers: asb, BeMg, lukel97

Reviewed By: BeMg, lukel97

[RISCV][NFC] Add helpers for RVV register classes

There are two places in tree that use these helpers and there will
be more future usages.

Reviewers: asb, BeMg, lukel97

Reviewed By: BeMg, lukel97

Pull Request: https://github.com/llvm/llvm-project/pull/84144

show more ...

[CodeGen] [ARM] Make RISC-V Init Undef Pass Target Independent and add support for the ARM Architecture. (#77770)

When using Greedy Register Allocation, there are times where
early-clobber values

[CodeGen] [ARM] Make RISC-V Init Undef Pass Target Independent and add support for the ARM Architecture. (#77770)

When using Greedy Register Allocation, there are times where
early-clobber values are ignored, and assigned the same register. This
is illeagal behaviour for these intructions. To get around this, using
Pseudo instructions for early-clobber registers gives them a definition
and allows Greedy to assign them to a different register. This then
meets the ARM Architecture Reference Manual and matches the defined
behaviour.

This patch takes the existing RISC-V patch and makes it target
independent, then adds support for the ARM Architecture. Doing this will
ensure early-clobber restraints are followed when using the ARM
Architecture. Making the pass target independent will also open up
possibility that support other architectures can be added in the future.

show more ...

[CodeGen] Port AtomicExpand to new Pass Manager (#71220)

Port the `atomicexpand` pass to the new Pass Manager.
Fixes #64559

[RISCV] Enable -riscv-enable-sink-fold by default. (#82026)

AArch64 has had it enabled since late November, so hopefully the main
issues have been resolved.

I see a small reduction in dynamic in

[RISCV] Enable -riscv-enable-sink-fold by default. (#82026)

AArch64 has had it enabled since late November, so hopefully the main
issues have been resolved.

I see a small reduction in dynamic instruction count on every benchmark
in specint2017. The best improvement was 0.3% so nothing amazing.

show more ...

[RISCV] Enable the TypePromotion pass from AArch64/ARM.

This pass looks for unsigned icmps that have illegal types and tries
to widen the use/def graph to improve the placement of the zero
extends t

[RISCV] Enable the TypePromotion pass from AArch64/ARM.

This pass looks for unsigned icmps that have illegal types and tries
to widen the use/def graph to improve the placement of the zero
extends that type legalization would need to insert.

I've explicitly disabled it for i32 by adding a check for
isSExtCheaperThanZExt to the pass.

The generated code isn't perfect, but my data shows a net
dynamic instruction count improvement on spec2017 for both base and
Zba+Zbb+Zbs.

show more ...

[RISCV] Remove unused variable 'ST' in RISCVTargetMachine.cpp (NFC)

llvm-project/llvm/lib/Target/RISCV/RISCVTargetMachine.cpp:358:27:
error: unused variable 'ST' [-Werror,-Wunused-variable]
cons

[RISCV] Remove unused variable 'ST' in RISCVTargetMachine.cpp (NFC)

llvm-project/llvm/lib/Target/RISCV/RISCVTargetMachine.cpp:358:27:
error: unused variable 'ST' [-Werror,-Wunused-variable]
const RISCVSubtarget &ST = C->MF->getSubtarget<RISCVSubtarget>();
^
1 error generated.

show more ...

[Sched] Add MacroFusion mutation if fusions are not empty (#72227)

We can get the fusions list by `getMacroFusions` and if it is not
empty, then we will add the MacroFusion mutation automatically.

[RISCV] Use TableGen-based macro fusion (#72224)

We convert existed macro fusions to TableGen.

Bacause `Fusion` depend on `Instruction` definitions which is defined
below `RISCVFeatures.td`, so

[RISCV] Use TableGen-based macro fusion (#72224)

We convert existed macro fusions to TableGen.

Bacause `Fusion` depend on `Instruction` definitions which is defined
below `RISCVFeatures.td`, so we recommend user to add fusion features
when defining new processor.

show more ...

[RISCV] CodeGen of RVE and ilp32e/lp64e ABIs (#76777)

This commit includes the necessary changes to clang and LLVM to support
codegen of `RVE` and the `ilp32e`/`lp64e` ABIs.

The differences between

[RISCV] CodeGen of RVE and ilp32e/lp64e ABIs (#76777)

This commit includes the necessary changes to clang and LLVM to support
codegen of `RVE` and the `ilp32e`/`lp64e` ABIs.

The differences between `RVE` and `RVI` are:
* `RVE` reduces the integer register count to 16(x0-x16).
* The ABI should be `ilp32e` for 32 bits and `lp64e` for 64 bits.

`RVE` can be combined with all current standard extensions.

The central changes in ilp32e/lp64e ABI, compared to ilp32/lp64 are:
* Only 6 integer argument registers (rather than 8).
* Only 2 callee-saved registers (rather than 12).
* A Stack Alignment of 32bits (rather than 128bits).
* ilp32e isn't compatible with D ISA extension.

If `ilp32e` or `lp64` is used with an ISA that has any of the registers
x16-x31 and f0-f31, then these registers are considered temporaries.

To be compatible with the implementation of ilp32e in GCC, we don't use
aligned registers to pass variadic arguments and set stack alignment\
to 4-bytes for types with length of 2*XLEN.

FastCC is also supported on RVE, while GHC isn't since there is only one
avaiable register.

Differential Revision: https://reviews.llvm.org/D70401

show more ...