[SelectionDAG] make INLINEASM_BR use MachineBasicBlocks instead of BlockAddresses

As part of re-architecting callbr to no longer use blockaddresses
(https://reviews.llvm.org/D129288), we don't reall

[SelectionDAG] make INLINEASM_BR use MachineBasicBlocks instead of BlockAddresses

As part of re-architecting callbr to no longer use blockaddresses
(https://reviews.llvm.org/D129288), we don't really need them in MIR.
They make comparing MachineBasicBlocks of indirect targets during
MachineVerifier a PITA.

Suggested by @efriedma from the discussion:
https://reviews.llvm.org/D130290#3669531

Reviewed By: efriedma, void

Differential Revision: https://reviews.llvm.org/D130316

show more ...

Untangle the mess which is MachineBasicBlock::hasAddressTaken().

There are two different senses in which a block can be "address-taken".
There can be a BlockAddress involved, which means we need to

Untangle the mess which is MachineBasicBlock::hasAddressTaken().

There are two different senses in which a block can be "address-taken".
There can be a BlockAddress involved, which means we need to map the
IR-level value to some specific block of machine code. Or there can be
constructs inside a function which involve using the address of a basic
block to implement certain kinds of control flow.

Mixing these together causes a problem: if target-specific passes are
marking random blocks "address-taken", if we have a BlockAddress, we
can't actually tell which MachineBasicBlock corresponds to the
BlockAddress.

So split this into two separate bits: one for BlockAddress, and one for
the machine-specific bits.

Discovered while trying to sort out related stuff on D102817.

Differential Revision: https://reviews.llvm.org/D124697

show more ...

[AVR] Support unaligned store

This patch really just extends D39946 towards stores as well as loads.
While the patch is in SelectionDAGBuilder, it only applies to AVR (the
only target that supports

[AVR] Support unaligned store

This patch really just extends D39946 towards stores as well as loads.
While the patch is in SelectionDAGBuilder, it only applies to AVR (the
only target that supports unaligned atomic operations).

Differential Revision: https://reviews.llvm.org/D128483

show more ...

[llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC

With C++17 there is no Clang pedantic warning or MSVC C5051.

[NFC] Add SmallVector constructor to allow creation of SmallVector<T> from ArrayRef of items convertible to type T

Extracted from https://reviews.llvm.org/D129781 and address comment:
https://review

[NFC] Add SmallVector constructor to allow creation of SmallVector<T> from ArrayRef of items convertible to type T

Extracted from https://reviews.llvm.org/D129781 and address comment:
https://reviews.llvm.org/D129781#3655571

Differential Revision: https://reviews.llvm.org/D130268

show more ...

[SelectionDAG] Handle IntToPtr constants in dbg.value

The function `handleDebugValue` has custom logic to handle certain kinds
constants, namely integers, floats and null pointers. However, it does

[SelectionDAG] Handle IntToPtr constants in dbg.value

The function `handleDebugValue` has custom logic to handle certain kinds
constants, namely integers, floats and null pointers. However, it does
not handle constant pointers created from IntToPtr ConstantExpressions.
This patch addresses the issue by replacing the Constant with its
integer operand.

A similar bug was addressed for GlobalISel in D130642.

Reviewed By: aprantl, #debug-info

Differential Revision: https://reviews.llvm.org/D130908

show more ...

Introduce @llvm.threadlocal.address intrinsic to access TLS variable

This belongs to a series of patches which try to solve the thread
identification problem in coroutines. See
https://discourse.llv

Introduce @llvm.threadlocal.address intrinsic to access TLS variable

This belongs to a series of patches which try to solve the thread
identification problem in coroutines. See
https://discourse.llvm.org/t/address-thread-identification-problems-with-coroutine/62015
for a full background.

The problem consists of two concrete problems: TLS variable and readnone
functions. This patch tries to convert the TLS problem to readnone
problem by converting the access of TLS variable to an intrinsic which
is marked as readnone.

The readnone problem would be addressed in following patches.

Reviewed By: nikic, jyknight, nhaehnle, ychen

Differential Revision: https://reviews.llvm.org/D125291

show more ...

CodeGen: Remove AliasAnalysis from regalloc

This was stored in LiveIntervals, but not actually used for anything
related to LiveIntervals. It was only used in one check for if a load
instruction is

CodeGen: Remove AliasAnalysis from regalloc

This was stored in LiveIntervals, but not actually used for anything
related to LiveIntervals. It was only used in one check for if a load
instruction is rematerializable. I also don't think this was entirely
correct, since it was implicitly assuming constant loads are also
dereferenceable.

Remove this and rely only on the invariant+dereferenceable flags in
the memory operand. Set the flag based on the AA query upfront. This
should have the same net benefit, but has the possible disadvantage of
making this AA query nonlazy.

Preserve the behavior of assuming pointsToConstantMemory implying
dereferenceable for now, but maybe this should be changed.

show more ...

[CodeGen] Qualify auto variables in for loops (NFC)

[stackmaps] Legalise patchpoint arguments.

This is similar to D125680, but for llvm.experimental.patchpoint
(instead of llvm.experimental.stackmap).

Differential review: https://reviews.llvm.org/D1

[stackmaps] Legalise patchpoint arguments.

This is similar to D125680, but for llvm.experimental.patchpoint
(instead of llvm.experimental.stackmap).

Differential review: https://reviews.llvm.org/D129268

show more ...

[IR] Don't use blockaddresses as callbr arguments

Following some recent discussions, this changes the representation
of callbrs in IR. The current blockaddress arguments are replaced
with `!` label

[IR] Don't use blockaddresses as callbr arguments

Following some recent discussions, this changes the representation
of callbrs in IR. The current blockaddress arguments are replaced
with `!` label constraints that refer directly to callbr indirect
destinations:

; Before:
%res = callbr i8* asm "", "=r,r,i"(i8* %x, i8* blockaddress(@test8, %foo))
to label %asm.fallthrough [label %foo]
; After:
%res = callbr i8* asm "", "=r,r,!i"(i8* %x)
to label %asm.fallthrough [label %foo]

The benefit of this is that we can easily update the successors of
a callbr, without having to worry about also updating blockaddress
references. This should allow us to remove some limitations:

* Allow unrolling/peeling/rotation of callbr, or any other
clone-based optimizations
(https://github.com/llvm/llvm-project/issues/41834)
* Allow duplicate successors
(https://github.com/llvm/llvm-project/issues/45248)

This is just the IR representation change though, I will follow up
with patches to remove limtations in various transformation passes
that are no longer needed.

Differential Revision: https://reviews.llvm.org/D129288

show more ...

[llvm] Use value instead of getValue (NFC)

[SDAG] Don't duplicate ParseConstraints() implementation SDAGBuilder (NFCI)

visitInlineAsm() in SDAGBuilder was duplicating a lot of the code
in ParseConstraints(), in particular all the logic to de

[SDAG] Don't duplicate ParseConstraints() implementation SDAGBuilder (NFCI)

visitInlineAsm() in SDAGBuilder was duplicating a lot of the code
in ParseConstraints(), in particular all the logic to determine the
operand value and constraint VT.

Rely on the data computed by ParseConstraints() instead, and update
its ConstraintVT implementation to match getCallOperandValEVT()
more precisely.

show more ...

[NFC][SelectionDAG] Fix debug prints in salvageUnresolvedDbgValue

The prints are printing pointer values - fix by dereferencing the pointers.

[stackmaps] Start legalizing live variable operands

Prior to this change, live variable operands passed to
`llvm.experimental.stackmap` would be emitted directly to target nodes,
meaning that they d

[stackmaps] Start legalizing live variable operands

Prior to this change, live variable operands passed to
`llvm.experimental.stackmap` would be emitted directly to target nodes,
meaning that they don't get legalised. The upshot of this is that LLVM
may crash when encountering illegally typed target nodes.

e.g. https://github.com/llvm/llvm-project/issues/21657

This change introduces a platform independent stackmap DAG node whose
operands are legalised as per usual, thus avoiding aforementioned
crashes.

Note that some kinds of argument are still not handled properly, namely
vectors, structs, and large integers, like i128s. These will need to be
addressed in follow-up changes.

Note also that this does not change the behaviour of
`llvm.experimental.patchpoint`. A follow up change will do the same for
this intrinsic.

Differential review:
https://reviews.llvm.org/D125680

show more ...

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instr

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instruction will be expanded
to CAS loop. There are already a couple of targets supporting the feature. I'll
create another patch(es) to enable them accordingly.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D127041

show more ...

[IR] Remove support for insertvalue constant expression

This removes the insertvalue constant expression, as part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
This is

[IR] Remove support for insertvalue constant expression

This removes the insertvalue constant expression, as part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
This is very similar to the extractvalue removal from D125795.
insertvalue is also not supported in bitcode, so no auto-ugprade
is necessary.

ConstantExpr::getInsertValue() can be replaced with
IRBuilder::CreateInsertValue() or ConstantFoldInsertValueInstruction(),
depending on whether a constant result is required (with the latter
being fallible).

The ConstantExpr::hasIndices() and ConstantExpr::getIndices()
methods also go away here, because there are no longer any constant
expressions with indices.

Differential Revision: https://reviews.llvm.org/D128719

show more ...

[ConstExpr] Remove more leftovers of extractvalue expression (NFC)

Remove some leftover bits of extractvalue handling after the
removal in D125795.

[Alignment] Replace commonAlignment with std::min

`commonAlignment` is a shortcut to pick the smallest of two `Align`
objects. As-is it doesn't bring much value compared to `std::min`.

Differential

[Alignment] Replace commonAlignment with std::min

`commonAlignment` is a shortcut to pick the smallest of two `Align`
objects. As-is it doesn't bring much value compared to `std::min`.

Differential Revision: https://reviews.llvm.org/D128345

show more ...

[IR] Move vector.insert/vector.extract out of experimental namespace

These intrinsics are now fundemental for SVE code generation and have been
present for a year and a half, hence move them out of

[IR] Move vector.insert/vector.extract out of experimental namespace

These intrinsics are now fundemental for SVE code generation and have been
present for a year and a half, hence move them out of the experimental
namespace.

Differential Revision: https://reviews.llvm.org/D127976

show more ...

[llvm] Don't use Optional::hasValue (NFC)

This patch replaces Optional::hasValue with the implicit cast to bool
in conditionals only.

Revert "Don't use Optional::hasValue (NFC)"

This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.

Don't use Optional::hasValue (NFC)

[SLP] Add cost model for `llvm.powi.*` intrinsics (REAPPLIED)

Patch was reverted in 4c5f10a due to buildbot failures, now being
reapplied with updated AArch64 and RISCV tests.

This patch adds handl

[SLP] Add cost model for `llvm.powi.*` intrinsics (REAPPLIED)

Patch was reverted in 4c5f10a due to buildbot failures, now being
reapplied with updated AArch64 and RISCV tests.

This patch adds handling for the llvm.powi.* intrinsics in
BasicTTIImplBase::getIntrinsicInstrCost() and improves vectorization.
Closes #53887.

Differential Revision: https://reviews.llvm.org/D128172

show more ...

Revert "[NFC] Remove dead code"

This reverts commit 8ba2cbff70f2c49a8926451c59cc260d67b706cf.