[DebugInfo] Remove `dbg.addr` from CodeGen

As part of this work, removing `SDDbgValue::clearIsEmitted` originally added for
`dbg.addr` in 045c67769d7fe577fc38cccb6fb40fd814437447 was attempted, but

[DebugInfo] Remove `dbg.addr` from CodeGen

As part of this work, removing `SDDbgValue::clearIsEmitted` originally added for
`dbg.addr` in 045c67769d7fe577fc38cccb6fb40fd814437447 was attempted, but it
appears some tests for `DBG_INSTR_REF` now depend on that behaviour as well, so
it was kept and comments were updated instead.

Part of `dbg.addr` removal
Discussed in https://discourse.llvm.org/t/what-is-the-status-of-dbg-addr/62898

Differential Revision: https://reviews.llvm.org/D144800

show more ...

[NFC] Make FPClassTest a bitmask enumeration

This is recommit of 2e416cdd52, fixed to be accepatble by GCC.
The original commit message is below.

With this change bitwise operations are allowed for

[NFC] Make FPClassTest a bitmask enumeration

This is recommit of 2e416cdd52, fixed to be accepatble by GCC.
The original commit message is below.

With this change bitwise operations are allowed for FPClassTest
enumeration, it must simplify using this type. Also some functions
changed to get argument of type FPClassTest instead of unsigned.

Differential Revision: https://reviews.llvm.org/D144241

show more ...

[PatternMatch] Don't require DataLayout for m_VScale()

The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I beli

[PatternMatch] Don't require DataLayout for m_VScale()

The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I believe it is sufficient to check for the canonical
<vscale x 1 x i8> form here -- I don't think there's a need to
recognize exotic variations like <vscale x 1 x i4> as a vscale
constant representation as well.

Differential Revision: https://reviews.llvm.org/D144566

show more ...

[VP] Reorder is_int_min_poison/is_zero_poison operand before mask for vp.abs/ctlz/cttz.

The patch ensures last two operands of vp.abs/ctlz/cttz are mask and evl.

Reviewed By: craig.topper

Differen

[VP] Reorder is_int_min_poison/is_zero_poison operand before mask for vp.abs/ctlz/cttz.

The patch ensures last two operands of vp.abs/ctlz/cttz are mask and evl.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D144536

show more ...

Revert "[NFC] Make FPClassTest a bitmask enumeration"

This reverts commit e7613c1d9b259bdf2b0b06b4169d9a10dd553406.

GCC issues an error:

In file included from /home/buildbot/as-builder-4/lld-x86_6

Revert "[NFC] Make FPClassTest a bitmask enumeration"

This reverts commit e7613c1d9b259bdf2b0b06b4169d9a10dd553406.

GCC issues an error:

In file included from /home/buildbot/as-builder-4/lld-x86_64-ubuntu-fast/llvm-project/llvm/unittests/ADT/BitmaskEnumTest.cpp:9:
/home/buildbot/as-builder-4/lld-x86_64-ubuntu-fast/llvm-project/llvm/include/llvm/ADT/BitmaskEnum.h:66:22: error: explicit specialization of template<class E, class Enable> struct llvm::is_bitmask_enum outside its namespace must use a nested-name-specifier [-fpermissive]
66 | template <> struct is_bitmask_enum<Enum> : std::true_type {}; \
| ^~~~~~~~~~~~~~~~~~~~~
/home/buildbot/as-builder-4/lld-x86_64-ubuntu-fast/llvm-project/llvm/unittests/ADT/BitmaskEnumTest.cpp:30:1: note: in expansion of macro LLVM_DECLARE_ENUM_AS_BITMASK
30 | LLVM_DECLARE_ENUM_AS_BITMASK(Flags2, V4);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

show more ...

[NFC] Make FPClassTest a bitmask enumeration

This is recommit of 2e416cdd52, reverted in 8555ab2fcd, because GCC
complains on extra qualification. The macro LLVM_DECLARE_ENUM_AS_BITMASK
does not spe

[NFC] Make FPClassTest a bitmask enumeration

This is recommit of 2e416cdd52, reverted in 8555ab2fcd, because GCC
complains on extra qualification. The macro LLVM_DECLARE_ENUM_AS_BITMASK
does not specify llvm:: anymore, so the macro must occur in the namespace
llvm. Documentation updated accordingly. The original commit message is below.

With this change bitwise operations are allowed for FPClassTest
enumeration, it must simplify using this type. Also some functions
changed to get argument of type FPClassTest instead of unsigned.

Differential Revision: https://reviews.llvm.org/D144241

show more ...

Revert "[NFC] Make FPClassTest a bitmask enumeration"

This reverts commit 2e416cdd52c1079b8c7cb1f7d7e557c889a4fb56.

Breaks the GCC build:

In file included from /home/npopov/repos/llvm-project/llvm

Revert "[NFC] Make FPClassTest a bitmask enumeration"

This reverts commit 2e416cdd52c1079b8c7cb1f7d7e557c889a4fb56.

Breaks the GCC build:

In file included from /home/npopov/repos/llvm-project/llvm/include/llvm/ADT/FloatingPointMode.h:18,
from /home/npopov/repos/llvm-project/llvm/include/llvm/ADT/APFloat.h:20,
from /home/npopov/repos/llvm-project/llvm/lib/Support/APFloat.cpp:14:
/home/npopov/repos/llvm-project/llvm/include/llvm/ADT/BitmaskEnum.h:66:22: error: extra qualification not allowed [-fpermissive]
66 | template <> struct llvm::is_bitmask_enum<Enum> : std::true_type {}; \
| ^~~~
/home/npopov/repos/llvm-project/llvm/include/llvm/ADT/FloatingPointMode.h:223:1: note: in expansion of macro ‘LLVM_DECLARE_ENUM_AS_BITMASK’
223 | LLVM_DECLARE_ENUM_AS_BITMASK(FPClassTest, /* LargestValue */ fcPosInf);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/npopov/repos/llvm-project/llvm/include/llvm/ADT/BitmaskEnum.h:67:22: error: extra qualification not allowed [-fpermissive]
67 | template <> struct llvm::largest_bitmask_enum_bit<Enum> { \
| ^~~~
/home/npopov/repos/llvm-project/llvm/include/llvm/ADT/FloatingPointMode.h:223:1: note: in expansion of macro ‘LLVM_DECLARE_ENUM_AS_BITMASK’
223 | LLVM_DECLARE_ENUM_AS_BITMASK(FPClassTest, /* LargestValue */ fcPosInf);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
[43/4396] Building CXX object lib/Supp...iles/LLVMSupport.dir/CommandLine.cpp.o

show more ...

[NFC] Make FPClassTest a bitmask enumeration

With this change bitwise operations are allowed for FPClassTest
enumeration, it must simplify using this type. Also some functions
changed to get argumen

[NFC] Make FPClassTest a bitmask enumeration

With this change bitwise operations are allowed for FPClassTest
enumeration, it must simplify using this type. Also some functions
changed to get argument of type FPClassTest instead of unsigned.

Differential Revision: https://reviews.llvm.org/D144241

show more ...

[IR] Add new intrinsics interleave and deinterleave vectors

This patch adds 2 new intrinsics:

; Interleave two vectors into a wider vector
<vscale x 4 x i64> @llvm.vector.interleave2.nxv2i64(<v

[IR] Add new intrinsics interleave and deinterleave vectors

This patch adds 2 new intrinsics:

; Interleave two vectors into a wider vector
<vscale x 4 x i64> @llvm.vector.interleave2.nxv2i64(<vscale x 2 x i64> %even, <vscale x 2 x i64> %odd)

; Deinterleave the odd and even lanes from a wider vector
{<vscale x 2 x i64>, <vscale x 2 x i64>} @llvm.vector.deinterleave2.nxv2i64(<vscale x 4 x i64> %vec)

The main motivator for adding these intrinsics is to support vectorization of
complex types using scalable vectors.

The intrinsics are kept simple by only supporting a stride of 2, which makes
them easy to lower and type-legalize. A stride of 2 is sufficient to handle
complex types which only have a real/imaginary component.

The format of the intrinsics matches how `shufflevector` is used in
LoopVectorize. For example:

using cf = std::complex<float>;

void foo(cf * dst, int N) {
for (int i=0; i<N; ++i)
dst[i] += cf(1.f, 2.f);
}

For this loop, LoopVectorize:
(1) Loads a wide vector (e.g. <8 x float>)
(2) Extracts odd lanes using shufflevector (leading to <4 x float>)
(3) Extracts even lanes using shufflevector (leading to <4 x float>)
(4) Performs the addition
(5) Interleaves the two <4 x float> vectors into a single <8 x float> using
shufflevector
(6) Stores the wide vector.

In this example, we can 1-1 replace shufflevector in (2) and (3) with the
deinterleave intrinsic, and replace the shufflevector in (5) with the
interleave intrinsic.

The SelectionDAG nodes might be extended to support higher strides (3, 4, etc)
as well in the future.

Similar to what was done for vector.splice and vector.reverse, the intrinsic
is lowered to a shufflevector when the type is fixed width, so to benefit from
existing code that was written to recognize/optimize shufflevector patterns.

Note that this approach does not prevent us from adding new intrinsics for other
strides, or adding a more generic shuffle intrinsic in the future. It just solves
the immediate problem of being able to vectorize loops with complex math.

Reviewed By: paulwalker-arm

Differential Revision: https://reviews.llvm.org/D141924

show more ...

[llvm][SelectionDAGBuilder] codegen callbr.landingpad intrinsic

Given a CallBrInst, retain its first virtual register in SelectionDagBuilder's
FunctionLoweringInfo if there's corresponding landingpa

[llvm][SelectionDAGBuilder] codegen callbr.landingpad intrinsic

Given a CallBrInst, retain its first virtual register in SelectionDagBuilder's
FunctionLoweringInfo if there's corresponding landingpad. Walk the list
of COPY MachineInstr to find the original virtual and physical registers
defined by the INLINEASM_BR MachineInst.

Test cases from https://reviews.llvm.org/D139565.
Link: https://github.com/llvm/llvm-project/issues/59538

Part 3 from
https://discourse.llvm.org/t/rfc-syncing-asm-goto-with-outputs-with-gcc/65453/8

Follow up patches still need to wire up CallBrPrepare into the pass
pipelines.

Reviewed By: efriedma, void

Differential Revision: https://reviews.llvm.org/D140160

show more ...

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

Revert "[SelectionDAG] Add pcsections recursively on SDNode values"
Revert "[SelectionDAG] Add missing setValue calls in visitIntrinsicCall"

This reverts commit 0c64e1b68f36640ffe82fc90e6279c50617ad

Revert "[SelectionDAG] Add pcsections recursively on SDNode values"
Revert "[SelectionDAG] Add missing setValue calls in visitIntrinsicCall"

This reverts commit 0c64e1b68f36640ffe82fc90e6279c50617ad1cc.
This reverts commit 1142e6c7c795de7f80774325a07ed49bc95a48c9.

It spuriously added !pcsections where they shouldn't be. See added test
case in test/CodeGen/X86/pcsections.ll as an example. The reason is that
the SelectionDAG chains operations in a basic block as "operands"
pointing to preceding instructions. This resulted in setting the
metadata on _all_ instructions preceding the one that should have the
metadata.

Reverting for now because the semantics of !pcsections was completely
buggy now.

show more ...

[SDAG] fix miscompiles caused by using ValueTracking matchSelectPattern to create FMINIMUM/FMAXIMUM

ValueTracking attempts to match compare+select patterns to FP min/max
operations, but it was creat

[SDAG] fix miscompiles caused by using ValueTracking matchSelectPattern to create FMINIMUM/FMAXIMUM

ValueTracking attempts to match compare+select patterns to FP min/max
operations, but it was created before the newer IEEE-754-2019
minimum/maximum ops were defined. Ie, matchSelectPattern() does not
account for the -0.0/+0.0 behavior that is specified in the newer
standard.

FMINIMUM/FMAXIMUM nodes were created to map to the newer standard:

/// FMINIMUM/FMAXIMUM - NaN-propagating minimum/maximum that also treat -0.0
/// as less than 0.0. While FMINNUM_IEEE/FMAXNUM_IEEE follow IEEE 754-2008
/// semantics, FMINIMUM/FMAXIMUM follow IEEE 754-2018 draft semantics.

We could adjust ValueTracking to deal with signed zero, but it seems like
a moot point given the divergent NaN behavior discussed in D143056, so just
delete this possibility to avoid bugs when converting IR to SDAG.

Differential Revision: https://reviews.llvm.org/D143106

show more ...

DAG: Remove redundant check for return alignment

This is already what the CallBase getRetAlign does

Use llvm::count{lr}_{zero,one} (NFC)

Lift EHPersonalities from Analysis to IR (NFC)

Computing EH-related information was only relevant for analysis passes so far. Lifting it to IR will allow the IR Verifier to calculate EH funclet colo

Lift EHPersonalities from Analysis to IR (NFC)

Computing EH-related information was only relevant for analysis passes so far. Lifting it to IR will allow the IR Verifier to calculate EH funclet coloring and validate funclet operand bundles in a follow-up step.

Reviewed By: rnk, compnerd

Differential Revision: https://reviews.llvm.org/D138122

show more ...

[SelectionDAG] Add missing setValue calls in visitIntrinsicCall

Add missing setValue calls in SelectionDAGBuilder for mem-transfer
intrinsic calls. These setValue calls are required in order to prop

[SelectionDAG] Add missing setValue calls in visitIntrinsicCall

Add missing setValue calls in SelectionDAGBuilder for mem-transfer
intrinsic calls. These setValue calls are required in order to propagate
pcsections metadata from IR to MIR.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D141048

show more ...

[llvm][DiagnosticInfo] handle function pointer casts

As pointed out by @arsenm in https://reviews.llvm.org/D141451#4045099,
we don't handle ConstantExpressions for dontcall-{warn|error} IR Fn
Attrs.

[llvm][DiagnosticInfo] handle function pointer casts

As pointed out by @arsenm in https://reviews.llvm.org/D141451#4045099,
we don't handle ConstantExpressions for dontcall-{warn|error} IR Fn
Attrs.

Use CallBase::getCalledOperand() and Value::stripPointerCasts() should
the call to CallBase::getCalledFunction return nullptr.

I don't know how to express the IR test case in C, otherwise I'd add a
clang test, too.

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D142058

show more ...

IR: Add atomicrmw uinc_wrap and udec_wrap

These are essentially add/sub 1 with a clamping value.

AMDGPU has instructions for these. CUDA/HIP expose these as
atomicInc/atomicDec. Currently we use ta

IR: Add atomicrmw uinc_wrap and udec_wrap

These are essentially add/sub 1 with a clamping value.

AMDGPU has instructions for these. CUDA/HIP expose these as
atomicInc/atomicDec. Currently we use target intrinsics for these,
but those do no carry the ordering and syncscope. Add these to
atomicrmw so we can carry these and benefit from the regular
legalization processes.

show more ...

[llvm] Use llvm::bit_floor (NFC)

In all these cases, the arguments to Log2_32 are known to be nonzero,
so we don't have to worry about "1 << -1".

Use llvm::popcount instead of llvm::countPopulation(NFC)

[Assignment Tracking][NFC] Replace LLVM command line option with a module flag

Remove LLVM flag -experimental-assignment-tracking. Assignment tracking is
still enabled from Clang with the command li

[Assignment Tracking][NFC] Replace LLVM command line option with a module flag

Remove LLVM flag -experimental-assignment-tracking. Assignment tracking is
still enabled from Clang with the command line -Xclang
-fexperimental-assignment-tracking which tells Clang to ask LLVM to run the
pass declare-to-assign. That pass converts conventional debug intrinsics to
assignment tracking metadata. With this patch it now also sets a module flag
debug-info-assignment-tracking with the value `i1 true` (using the flag conflict
rule `Max` since enabling assignment tracking on IR that contains only
conventional debug intrinsics should cause no issues).

Update the docs and tests too.

Reviewed By: CarlosAlbertoEnciso

Differential Revision: https://reviews.llvm.org/D142027

show more ...

DAG/GlobalISel: Fix broken/redundant setting of MODereferenceable

This was incorrectly setting dereferenceable on unaligned
operands. getLoadMemOperandFlags does the alignment dereferenceabilty
chec

DAG/GlobalISel: Fix broken/redundant setting of MODereferenceable

This was incorrectly setting dereferenceable on unaligned
operands. getLoadMemOperandFlags does the alignment dereferenceabilty
check without alignment, and then both paths went on to check
isDereferenceableAndAlignedPointer. Make getLoadMemOperandFlags check
isDereferenceableAndAlignedPointer, and remove the second call.

show more ...

[NFC] Use TypeSize::getKnownMinValue() instead of TypeSize::getKnownMinSize()

This change is one of a series to implement the discussion from
https://reviews.llvm.org/D141134.

[DebugInfo] Produce variadic DBG_INSTR_REFs from ISel

This patch modifies SelectionDAG and FastISel to produce DBG_INSTR_REFs with
variadic expressions, and produce DBG_INSTR_REFs for debug values w

[DebugInfo] Produce variadic DBG_INSTR_REFs from ISel

This patch modifies SelectionDAG and FastISel to produce DBG_INSTR_REFs with
variadic expressions, and produce DBG_INSTR_REFs for debug values with variadic
location expressions. The former essentially means just prepending
DW_OP_LLVM_arg, 0 to the existing expression. The latter is achieved in
MachineFunction::finalizeDebugInstrRefs and InstrEmitter::EmitDbgInstrRef.

Reviewed By: jmorse, Orlando

Differential Revision: https://reviews.llvm.org/D133929

show more ...