[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNo

[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and similar feed into
instruction insertion positions. Call-sites where that's necessary were
updated a year ago; but to ensure some type safety however, we'd like to
have all calls to getFirstNonPHI use the iterator-returning version.

This patch changes a bunch of call-sites calling getFirstNonPHI to use
getFirstNonPHIIt, which returns an iterator. All these call sites are
where it's obviously safe to fetch the iterator then dereference it. A
follow-up patch will contain less-obviously-safe changes.

We'll eventually deprecate and remove the instruction-pointer
getFirstNonPHI, but not before adding concise documentation of what
considerations are needed (very few).

---------

Co-authored-by: Stephen Tozer <Melamoto@gmail.com>

show more ...

[IR] Replace of PointerType::getUnqual(Type) with opaque version (NFC) (#123909)

Follow up to https://github.com/llvm/llvm-project/issues/123569

[SDAG] Use BatchAAResults for querying alias analysis (AA) results (#123934)

Once we get to SelectionDAG the IR should not be changing anymore, so we
can use BatchAAResults rather than AAResults to

[SDAG] Use BatchAAResults for querying alias analysis (AA) results (#123934)

Once we get to SelectionDAG the IR should not be changing anymore, so we
can use BatchAAResults rather than AAResults to cache AA queries.

This should be a NFC change for targets that enable AA during codegen
(such as AArch64), but also give a nice compile-time improvement in some
cases. See:
https://github.com/llvm/llvm-project/pull/123787#issuecomment-2606797041

Note: This follows Nikita's suggestion on #123787.

show more ...

[IR] Replace of PointerType::get(Type) with opaque version (NFC) (#123617)

In accordance with https://github.com/llvm/llvm-project/issues/123569

In order to keep the patch at reasonable size, this

[IR] Replace of PointerType::get(Type) with opaque version (NFC) (#123617)

In accordance with https://github.com/llvm/llvm-project/issues/123569

In order to keep the patch at reasonable size, this PR only covers for
the llvm subproject, unittests excluded.

show more ...

[SDAG] Add an ISD node to help lower vector.extract.last.active (#118810)

Based on feedback from the clastb codegen PR, I'm refactoring basic codegen for the vector.extract.last.active intrinsic to

[SDAG] Add an ISD node to help lower vector.extract.last.active (#118810)

Based on feedback from the clastb codegen PR, I'm refactoring basic codegen for the vector.extract.last.active intrinsic to lower to an ISD node in SelectionDAGBuilder then expand in LegalizeVectorOps, instead of doing everything in the builder.

The new ISD node (vector_find_last_active) only covers finding the index of the last active element of the mask, and extracting the element + handling passthru is left to existing ISD nodes.

show more ...

[Mips] Fix compiler crash when returning fp128 after calling a functi… (#117525)

…on returning { i8, i128 }

Fixes https://github.com/llvm/llvm-project/issues/96432.

[CodeGen] Use Register/MCRegister::isPhysical. NFC

Reapply "DiagnosticInfo: Clean up usage of DiagnosticInfoInlineAsm" (#119575) (#119634)

This reverts commit 40986feda8b1437ed475b144d5b9a208b008782a.

Reapply with fix to prevent temporary Twine fro

Reapply "DiagnosticInfo: Clean up usage of DiagnosticInfoInlineAsm" (#119575) (#119634)

This reverts commit 40986feda8b1437ed475b144d5b9a208b008782a.

Reapply with fix to prevent temporary Twine from going out of scope.

show more ...

Revert "DiagnosticInfo: Clean up usage of DiagnosticInfoInlineAsm" (#119575)

Reverts llvm/llvm-project#119485

Breaks builders, details in llvm/llvm-project#119485

[SelectionDAG] Fix "unused variable" warnings after #119268 (NFC) (#119550)

[SelectionDAG] Don't call ComputeValueVTs for "demote register" (NFC) (#119268)

`ComputeValueVTs` only breaks down aggregate types. For pointer types it
is equivalent to calling `TargetLoweringBase:

[SelectionDAG] Don't call ComputeValueVTs for "demote register" (NFC) (#119268)

`ComputeValueVTs` only breaks down aggregate types. For pointer types it
is equivalent to calling `TargetLoweringBase::getPointerTy`.

show more ...

DiagnosticInfo: Clean up usage of DiagnosticInfoInlineAsm (#119485)

Currently LLVMContext::emitError emits any error as an "inline asm"
error which does not make any sense. InlineAsm appears to be s

DiagnosticInfo: Clean up usage of DiagnosticInfoInlineAsm (#119485)

Currently LLVMContext::emitError emits any error as an "inline asm"
error which does not make any sense. InlineAsm appears to be special,
in that it uses a "LocCookie" from srcloc metadata, which looks like
a parallel mechanism to ordinary source line locations. This meant
that other types of failures had degraded source information reported
when available.

Introduce some new generic error types, and only use inline asm
in the appropriate contexts. The DiagnosticInfo types are still
a bit of a mess, and I'm not sure why DiagnosticInfoWithLocationBase
exists instead of just having an optional DiagnosticLocation in the
base class.

DK_Generic is for any error that derives from an IR level instruction,
and thus can pull debug locations directly from it. DK_GenericWithLoc
is functionally the generic codegen error, since it does not depend
on the IR and instead can construct a DiagnosticLocation from the
MI debug location.

show more ...

[TargetLowering] Return Align from getByValTypeAlignment (NFC) (#119233)

[RISCV] Handle zeroinitializer of vector tuple Type (#113995)

It doesn't make sense to add a new generic ISD to handle riscv tuple
type. Instead we use `SPLAT_VECTOR` for ISD and further lower to

[RISCV] Handle zeroinitializer of vector tuple Type (#113995)

It doesn't make sense to add a new generic ISD to handle riscv tuple
type. Instead we use `SPLAT_VECTOR` for ISD and further lower to
`VMV_V_X`.

Note: If there's `visitSPLAT_VECTOR` in generic DAG combiner, it needs
to skip riscv vector tuple type.

Stack on https://github.com/llvm/llvm-project/pull/114329

show more ...

[SelectionDAG] Add generic implementation for @llvm.expect.with.probability when optimizations are disabled (#117459)

Handle \@llvm.expect.with.probability in SelectionDAGBuilder, FastISel,
and Int

[SelectionDAG] Add generic implementation for @llvm.expect.with.probability when optimizations are disabled (#117459)

Handle \@llvm.expect.with.probability in SelectionDAGBuilder, FastISel,
and IntrinsicLowering in the same way \@llvm.expect is handled, where
the value is passed through as-is. This can be reached if the intrinsic
is used without optimizations, where it would otherwise be properly
transformed out.

Fixes #115411 for SelectionDAG. A similar patch is likely needed for
GlobalISel.

show more ...

[IR] Vector extract last active element intrinsic (#113587)

As discussed in #112738, it may be better to have an intrinsic to represent vector element extracts based on mask bits. This intrinsic is

[IR] Vector extract last active element intrinsic (#113587)

As discussed in #112738, it may be better to have an intrinsic to represent vector element extracts based on mask bits. This intrinsic is for the case of extracting the last active element, if any, or a default value if the mask is all-false.

The target-agnostic SelectionDAG lowering is similar to the IR in #106560.

show more ...

[AArch64] Add @llvm.experimental.vector.match (#101974)

This patch introduces an experimental intrinsic for matching the
elements of one vector against the elements of another.

For AArch64 targe

[AArch64] Add @llvm.experimental.vector.match (#101974)

This patch introduces an experimental intrinsic for matching the
elements of one vector against the elements of another.

For AArch64 targets that support SVE2, the intrinsic lowers to a MATCH
instruction for supported fixed and scalar vector types.

show more ...

[llvm] Remove redundant control flow statements (NFC) (#115831)

Identified with readability-redundant-control-flow.

[DAGCombiner] Add basic support for `trunc nsw/nuw` (#113808)

This patch adds basic support for `trunc nsw/nuw` in SDAG. It will allow
DAGCombiner to further eliminate in-reg `zext/sext` instructio

[DAGCombiner] Add basic support for `trunc nsw/nuw` (#113808)

This patch adds basic support for `trunc nsw/nuw` in SDAG. It will allow
DAGCombiner to further eliminate in-reg `zext/sext` instructions.

show more ...

[WebAssembly] Fix rethrow's index calculation (#114693)

So far we have assumed that we only rethrow the exception caught in the
innermost EH pad. This is true in code we directly generate, but afte

[WebAssembly] Fix rethrow's index calculation (#114693)

So far we have assumed that we only rethrow the exception caught in the
innermost EH pad. This is true in code we directly generate, but after
inlining this may not be the case. For example, consider this code:
```ll
ehcleanup:
%0 = cleanuppad ...
call @destructor
cleanupret from %0 unwind label %catch.dispatch
```

If `destructor` gets inlined into this function, the code can be like
```ll
ehcleanup:
%0 = cleanuppad ...
invoke @throwing_func
to label %unreachale unwind label %catch.dispatch.i

catch.dispatch.i:
catchswitch ... [ label %catch.start.i ]

catch.start.i:
%1 = catchpad ...
invoke @some_function
to label %invoke.cont.i unwind label %terminate.i

invoke.cont.i:
catchret from %1 to label %destructor.exit

destructor.exit:
cleanupret from %0 unwind label %catch.dispatch
```

We lower a `cleanupret` into `rethrow`, which assumes it rethrows the
exception caught by the nearest dominating EH pad. But after the
inlining, the nearest dominating EH pad is not `ehcleanup` but
`catch.start.i`.

The problem exists in the same manner in the new (exnref) EH, because it
assumes the exception comes from the nearest EH pad and saves an exnref
from that EH pad and rethrows it (using `throw_ref`).

This problem can be fixed easily if `cleanupret` has the basic block
where its matching `cleanuppad` is. The bitcode instruction `cleanupret`
kind of has that info (it has a token from the `cleanuppad`), but that
info is lost when when we enter ISel, because `TargetSelectionDAG.td`'s
`cleanupret` node does not have any arguments:

https://github.com/llvm/llvm-project/blob/5091a359d9807db8f7d62375696f93fc34226969/llvm/include/llvm/Target/TargetSelectionDAG.td#L700
Note that `catchret` already has two basic block arguments, even though
neither of them means `catchpad`'s BB.

This PR adds the `cleanuppad`'s BB as an argument to `cleanupret` node
in ISel and uses it in the Wasm backend. Because this node is also used
in X86 backend we need to note its argument there too but nothing more
needs to change there as long as X86 doesn't need it.

---

- Details about changes in the Wasm backend:

After this PR, our pseudo `RETHROW` instruction takes a BB, which means
the EH pad whose exception it needs to rethrow. There are currently two
ways to generate a `RETHROW`: one is from `llvm.wasm.rethrow` intrinsic
and the other is from `CLEANUPRET` we discussed above. In case of
`llvm.wasm.rethrow`, we add a '0' as a placeholder argument when it is
lowered to a `RETHROW`, and change it to a BB in LateEHPrepare. As
written in the comments, this PR doesn't change how this BB is computed.
The BB argument will be converted to an immediate argument as with other
control flow instructions in CFGStackify.

In case of `CLEANUPRET`, it already has a BB argument pointing to an EH
pad, so it is just converted to a `RETHROW` with the same BB argument in
LateEHPrepare. This will also be lowered to an immediate in CFGStackify
with other control flow instructions.

---

Fixes #114600.

show more ...

[SelectionDAGBuilder][X86] Don't form FMAXNUM for f16 vectors if FMAXNUM needs to be promoted. (#114943)

In #70357, I changed a isLegalOrCustom to isLegalOrCustomOrPromote in
visitSelect to enable

[SelectionDAGBuilder][X86] Don't form FMAXNUM for f16 vectors if FMAXNUM needs to be promoted. (#114943)

In #70357, I changed a isLegalOrCustom to isLegalOrCustomOrPromote in
visitSelect to enable integer min/max to be formed when the operation
was promoted. Unfortunately, this also affected floating point. For
floating point, fmaxnum may require a libcall so we also need to check
if the operation on the promoted type is legal or custom.

Other changes to RISC-V have seen made the original change untested so
this patch restores the original isLegalOrCustom.

Fixes #114520.

show more ...

[SelectionDAG] Remove unused includes (NFC) (#114697)

Identified with misc-include-cleaner.

Reapply "[SelectionDAG] Add preliminary plumbing for `samesign` flag"

Original commit: 19c8475871faee5ebb06281034872a85a2552675

Multiple 2-stage sanitizer buildbots were reporting failures, the iss

Reapply "[SelectionDAG] Add preliminary plumbing for `samesign` flag"

Original commit: 19c8475871faee5ebb06281034872a85a2552675

Multiple 2-stage sanitizer buildbots were reporting failures, the issue
has been addressed separately in 29246a92aee87e86cbc2bb64ee520d7385644f34.

show more ...

Revert "[SelectionDAG] Add preliminary plumbing for `samesign` flag" (#114647)

Crashes on ARM builds
https://lab.llvm.org/buildbot/#/builders/85/builds/2548
```
DAGCombiner.cpp:16157: SDValue (an

Revert "[SelectionDAG] Add preliminary plumbing for `samesign` flag" (#114647)

Crashes on ARM builds
https://lab.llvm.org/buildbot/#/builders/85/builds/2548
```
DAGCombiner.cpp:16157: SDValue (anonymous
namespace)::DAGCombiner::visitFREEZE(SDNode *):
Assertion `DAG.isGuaranteedNotToBeUndefOrPoison(R,
false) && "Can't create node that may be
undef/poison!"' failed.
```

Issue #114648

This reverts commit 19c8475871faee5ebb06281034872a85a2552675.

show more ...

[DAG] Simplify control flow in SelectionDAGBuilder::visitShuffleVector [NFC]

If we've handled ==, and < above, the only case left can be >. We don't
need to branch on this, and can instead assert a

[DAG] Simplify control flow in SelectionDAGBuilder::visitShuffleVector [NFC]

If we've handled ==, and < above, the only case left can be >. We don't
need to branch on this, and can instead assert and reduce indentation,
and simplify reasoning about the fallthrough path.

show more ...