[FEnv] File with properties of constrained intrinsics

Summary
In several places we need to enumerate all constrained intrinsics or IR
nodes that should be represented by them. It is easy to miss som

[FEnv] File with properties of constrained intrinsics

Summary
In several places we need to enumerate all constrained intrinsics or IR
nodes that should be represented by them. It is easy to miss some of
the cases. To make working with these intrinsics more convenient and
robust, this change introduces file containing definitions of all
constrained intrinsics and some of their properties. This file can be
included to generate constrained intrinsics processing code.

Reviewers: kpn, andrew.w.kaylor, cameron.mcinally, uweigand

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D69887

show more ...

[SVE][CodeGen] Scalable vector MVT size queries

* Implements scalable size queries for MVTs, split out from D53137.

* Contains a fix for FindMemType to avoid using scalable vector type
to contain

[SVE][CodeGen] Scalable vector MVT size queries

* Implements scalable size queries for MVTs, split out from D53137.

* Contains a fix for FindMemType to avoid using scalable vector type
to contain non-scalable types.

* Explicit casts for several places where implicit integer sign
changes or promotion from 32 to 64 bits caused problems.

* CodeGenDAGPatterns will treat scalable and non-scalable vector types
as different.

Reviewers: greened, cameron.mcinally, sdesmalen, rovka

Reviewed By: rovka

Differential Revision: https://reviews.llvm.org/D66871

show more ...

[FPEnv] Add constrained intrinsics for lrint and lround

Earlier in the year intrinsics for lrint, llrint, lround and llround were
added to llvm. The constrained versions are now implemented here.

R

[FPEnv] Add constrained intrinsics for lrint and lround

Earlier in the year intrinsics for lrint, llrint, lround and llround were
added to llvm. The constrained versions are now implemented here.

Reviewed by: andrew.w.kaylor, craig.topper, cameron.mcinally
Approved by: craig.topper
Differential Revision: https://reviews.llvm.org/D64746

llvm-svn: 373900

show more ...

[SDAG] Add a isSimple cover functon to MemSDNode, just as we have in IR/MI [NFC]

Uses are in reviews D66322 and D66318. Submitted separately to control rebuild times.

llvm-svn: 371445

Introduce infrastructure for an incremental port of SelectionDAG atomic load/store handling

This is the first patch in a large sequence. The eventual goal is to have unordered atomic loads and store

Introduce infrastructure for an incremental port of SelectionDAG atomic load/store handling

This is the first patch in a large sequence. The eventual goal is to have unordered atomic loads and stores - and possibly ordered atomics as well - handled through the normal ISEL codepaths for loads and stores. Today, there handled w/instances of AtomicSDNodes. The result of which is that all transforms need to be duplicated to work for unordered atomics. The benefit of the current design is that it's harder to introduce a silent miscompile by adding an transform which forgets about atomicity. See the thread on llvm-dev titled "FYI: proposed changes to atomic load/store in SelectionDAG" for further context.

Note that this patch is NFC unless the experimental flag is set.

The basic strategy I plan on taking is:

introduce infrastructure and a flag for testing (this patch)
Audit uses of isVolatile, and apply isAtomic conservatively*
piecemeal conservative* update generic code and x86 backedge code in individual reviews w/tests for cases which didn't check volatile, but can be found with inspection
flip the flag at the end (with minimal diffs)
Work through todo list identified in (2) and (3) exposing performance ops

(*) The "conservative" bit here is aimed at minimizing the number of diffs involved in (4). Ideally, there'd be none. In practice, getting it down to something reviewable by a human is the actual goal. Note that there are (currently) no paths which produce LoadSDNode or StoreSDNode with atomic MMOs, so we don't need to worry about preserving any behaviour there.

We've taken a very similar strategy twice before with success - once at IR level, and once at the MI level (post ISEL).

Differential Revision: https://reviews.llvm.org/D66309

llvm-svn: 371441

show more ...

[FPEnv] Add fptosi and fptoui constrained intrinsics.

This implements constrained floating point intrinsics for FP to signed and
unsigned integers.

Quoting from D32319:
The purpose of the constrain

[FPEnv] Add fptosi and fptoui constrained intrinsics.

This implements constrained floating point intrinsics for FP to signed and
unsigned integers.

Quoting from D32319:
The purpose of the constrained intrinsics is to force the optimizer to
respect the restrictions that will be necessary to support things like the
STDC FENV_ACCESS ON pragma without interfering with optimizations when
these restrictions are not needed.

Reviewed by: Andrew Kaylor, Craig Topper, Hal Finkel, Cameron McInally, Roman Lebedev, Kit Barton
Approved by: Craig Topper
Differential Revision: http://reviews.llvm.org/D63782

llvm-svn: 370228

show more ...

[SDAG] Minor code cleanup/standardization of atomic accessors [NFC]

llvm-svn: 369057

[SelectionDAG] Extend base addressing modes supported by MGATHER/MSCATTER

Summary:
Before this patch MGATHER/MSCATTER is capable of representing all
common addressing modes, but only when illegal ty

[SelectionDAG] Extend base addressing modes supported by MGATHER/MSCATTER

Summary:
Before this patch MGATHER/MSCATTER is capable of representing all
common addressing modes, but only when illegal types are used.
This patch adds an IndexType property so more representations
are available when using legal types only.

Original modes:
vector of bases
base + vector of signed scaled offsets

New modes:
base + vector of signed unscaled offsets
base + vector of unsigned scaled offsets
base + vector of unsigned unscaled offsets

The current behaviour of addressing modes for gather/scatter remains
unchanged.

Patch by Paul Walker.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D65636

llvm-svn: 368008

show more ...

SelectionDAG, MI, AArch64: Widen target flags fields/arguments from unsigned char to unsigned.

This makes the field wider than MachineOperand::SubReg_TargetFlags so that
we don't end up silently tru

SelectionDAG, MI, AArch64: Widen target flags fields/arguments from unsigned char to unsigned.

This makes the field wider than MachineOperand::SubReg_TargetFlags so that
we don't end up silently truncating any higher bits. We should still catch
any bits truncated from the MachineOperand field as a consequence of the
assertion in MachineOperand::setTargetFlags().

Differential Revision: https://reviews.llvm.org/D65465

llvm-svn: 367474

show more ...

[DAGCombiner] Support (shl (ext (shl x, c1)), c2) -> 0 non-uniform folds.

Use matchBinaryPredicate instead of isConstOrConstSplat to let us handle non-uniform shift cases.

This requires us to twea

[DAGCombiner] Support (shl (ext (shl x, c1)), c2) -> 0 non-uniform folds.

Use matchBinaryPredicate instead of isConstOrConstSplat to let us handle non-uniform shift cases.

This requires us to tweak matchBinaryPredicate to allow it to (optionally) handle constants with different type widths.

llvm-svn: 363792

show more ...

Allow target to handle STRICT floating-point nodes

The ISD::STRICT_ nodes used to implement the constrained floating-point
intrinsics are currently never passed to the target back-end, which makes
i

Allow target to handle STRICT floating-point nodes

The ISD::STRICT_ nodes used to implement the constrained floating-point
intrinsics are currently never passed to the target back-end, which makes
it impossible to handle them correctly (e.g. mark instructions are depending
on a floating-point status and control register, or mark instructions as
possibly trapping).

This patch allows the target to use setOperationAction to switch the action
on ISD::STRICT_ nodes to Legal. If this is done, the SelectionDAG common code
will stop converting the STRICT nodes to regular floating-point nodes, but
instead pass the STRICT nodes to the target using normal SelectionDAG
matching rules.

To avoid having the back-end duplicate all the floating-point instruction
patterns to handle both strict and non-strict variants, we make the MI
codegen explicitly aware of the floating-point exceptions by introducing
two new concepts:

- A new MCID flag "mayRaiseFPException" that the target should set on any
instruction that possibly can raise FP exception according to the
architecture definition.
- A new MI flag FPExcept that CodeGen/SelectionDAG will set on any MI
instruction resulting from expansion of any constrained FP intrinsic.

Any MI instruction that is *both* marked as mayRaiseFPException *and*
FPExcept then needs to be considered as raising exceptions by MI-level
codegen (e.g. scheduling).

Setting those two new flags is straightforward. The mayRaiseFPException
flag is simply set via TableGen by marking all relevant instruction
patterns in the .td files.

The FPExcept flag is set in SDNodeFlags when creating the STRICT_ nodes
in the SelectionDAG, and gets inherited in the MachineSDNode nodes created
from it during instruction selection. The flag is then transfered to an
MIFlag when creating the MI from the MachineSDNode. This is handled just
like fast-math flags like no-nans are handled today.

This patch includes both common code changes required to implement the
new features, and the SystemZ implementation.

Reviewed By: andrew.w.kaylor

Differential Revision: https://reviews.llvm.org/D55506

llvm-svn: 362663

show more ...

[DAG] isBitwiseNot / isConstOrConstSplat - add support for build vector undefs + truncation (PR41020)

Add (opt-in) support for implicit truncation to isConstOrConstSplat, which allows us to match tr

[DAG] isBitwiseNot / isConstOrConstSplat - add support for build vector undefs + truncation (PR41020)

Add (opt-in) support for implicit truncation to isConstOrConstSplat, which allows us to match truncated 'all ones' cases in isBitwiseNot.

PR41020 compares against using ISD::isBuildVectorAllOnes() instead, but that predicate silently accepts any UNDEF elements in the build vector which might not be what we want in isBitwiseNot - so I've added an opt-in 'AllowUndefs' flag that is set to false by default but will allow us to enable it on individual cases where its safe.

Differential Revision: https://reviews.llvm.org/D62783

llvm-svn: 362323

show more ...

[codeview] Fix SDNode representation of annotation labels

Before this change, they were erroneously constructed with the EH_LABEL
SDNode opcode, which caused other passes to interact with them in
in

[codeview] Fix SDNode representation of annotation labels

Before this change, they were erroneously constructed with the EH_LABEL
SDNode opcode, which caused other passes to interact with them in
incorrect ways. See the FIXME about fastisel that this addresses in the
existing test case.

Fixes PR41890

llvm-svn: 360818

show more ...

[SDAG, x86] allow targets to override test for binop opcodes

This follows the pattern of the existing isCommutativeBinOp().

x86 shows improvements from vector narrowing for the min/max opcodes.

ll

[SDAG, x86] allow targets to override test for binop opcodes

This follows the pattern of the existing isCommutativeBinOp().

x86 shows improvements from vector narrowing for the min/max opcodes.

llvm-svn: 360639

show more ...

Add constrained fptrunc and fpext intrinsics.

The new fptrunc and fpext intrinsics are constrained versions of the
regular fptrunc and fpext instructions.

Reviewed by: Andrew Kaylor, Craig Topper,

Add constrained fptrunc and fpext intrinsics.

The new fptrunc and fpext intrinsics are constrained versions of the
regular fptrunc and fpext instructions.

Reviewed by: Andrew Kaylor, Craig Topper, Cameron McInally, Conner Abbot
Approved by: Craig Topper
Differential Revision: https://reviews.llvm.org/D55897

llvm-svn: 360581

show more ...

[SelectionDAG] include FP min/max variants as binary operators

The x86 test diffs don't look great because of extra move ops,
but FP min/max should clearly be included in the list.

llvm-svn: 359399

[SelectionDAG] move splat util functions up from x86 lowering

This was supposed to be NFC, but the change in SDLoc
definitions causes instruction scheduling changes.

There's nothing x86-specific in

[SelectionDAG] move splat util functions up from x86 lowering

This was supposed to be NFC, but the change in SDLoc
definitions causes instruction scheduling changes.

There's nothing x86-specific in this code, and it can
likely be used from DAGCombiner's simplifyVBinOp().

llvm-svn: 358930

show more ...

[SelectionDAG] soften splat mask assert/unreachable (PR41535)

These are general queries, so they should not die when given
a degenerate input like an all undef mask. Callers should be
able to deal w

[SelectionDAG] soften splat mask assert/unreachable (PR41535)

These are general queries, so they should not die when given
a degenerate input like an all undef mask. Callers should be
able to deal with an op that will eventually be simplified away.

llvm-svn: 358761

show more ...

[AIX] SelectionDAGNodes.h: Pack bit-fields that are meant to be packed

Summary:
Certain classes in the subject file are expected to provide different
views of a two-byte field as a collection of var

[AIX] SelectionDAGNodes.h: Pack bit-fields that are meant to be packed

Summary:
Certain classes in the subject file are expected to provide different
views of a two-byte field as a collection of various bit-fields. On AIX,
the canonical layout of bit-fields would cause these classes to span
four bytes. Applying the `pack` pragma for compilers that employ the AIX
canonical layout allows these classes to fit within the expected two
bytes. In the future, the pragma would also likely need to be applied
when building with Clang on AIX.

Reviewers: xingxue, sfertile, jasonliu

Reviewed By: xingxue

Subscribers: jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60164

llvm-svn: 357661

show more ...

[DAG] Fix Lifetime Node ID hashing.

llvm-svn: 357179

[SelectionDAG] Strengthen assertions about usage of AtomicSDNodes

In D57601, I described the expectations around usage of AtomicSDNode and LoadSDNode/StoreSDNode w.r.t. atomic and volatiles. This pa

[SelectionDAG] Strengthen assertions about usage of AtomicSDNodes

In D57601, I described the expectations around usage of AtomicSDNode and LoadSDNode/StoreSDNode w.r.t. atomic and volatiles. This patch simply embeds those expectations in assertions so that they can't be accidentally weakened.

Note: The reason only AtomicSDNodes of ATOMIC_LOAD and ATOMIC_STORE opcode are currently checked is that AMDGPU has an intrinsic which gets lowered to an ATOMIC_LOAD_FADD w/o a corresponding atomic MMO. This is suspicious, and I've brought it to the attention of the relevant developers. Once resolved, I'll strengthen that assertion.

Note 2: If this breaks your out-of-tree backend, go read the update instructions associated w/the previously mentioned patch. This will assert on (a subset of) things you need to update per those instructions.

Differential Revision: https://reviews.llvm.org/D58738

llvm-svn: 355069

show more ...

[SelectionDAG] Add demanded elts variants to isConstOrConstSplat helpers. NFCI.

These helpers extend the existing isConstOrConstSplat helper checks to support DemandedElts masks as well.

We already

[SelectionDAG] Add demanded elts variants to isConstOrConstSplat helpers. NFCI.

These helpers extend the existing isConstOrConstSplat helper checks to support DemandedElts masks as well.

We already had a local version of this in SelectionDAG that computeKnownBits/ComputeNumSignBits made use of, but this adds the functionality directly to the BuildVectorSDNode node and extends isConstOrConstSplat etc. to use that.

This will allow us to reuse the functionality in SimplifyDemandedVectorElts/SimplifyDemandedBits.

Differential Revision: https://reviews.llvm.org/D58503

llvm-svn: 354797

show more ...

Re-land part of r354244 "[DAGCombiner] Eliminate dead stores to stack."

This part introduces the lifetime node.

llvm-svn: 354578

Revert r354244 "[DAGCombiner] Eliminate dead stores to stack."

Breaks some bots.

llvm-svn: 354245

[DAGCombiner] Eliminate dead stores to stack.

Summary:
A store to an object whose lifetime is about to end can be removed.

See PR40550 for motivation.

Reviewers: niravd

Subscribers: llvm-commits

[DAGCombiner] Eliminate dead stores to stack.

Summary:
A store to an object whose lifetime is about to end can be removed.

See PR40550 for motivation.

Reviewers: niravd

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D57541

llvm-svn: 354244

show more ...