[SelectionDAG] Add `STRICT_BF16_TO_FP` and `STRICT_FP_TO_BF16` (#80056)

This patch adds the support for `STRICT_BF16_TO_FP` and
`STRICT_FP_TO_BF16`.

Revert "[SelectionDAG] Add `STRICT_BF16_TO_FP` and `STRICT_FP_TO_BF16` (#80056)"

This reverts commit b0c158bd947c360a4652eb0de3a4794f46deb88b.

The changes in `compiler-rt` broke tests.

[SelectionDAG] Add `STRICT_BF16_TO_FP` and `STRICT_FP_TO_BF16` (#80056)

This patch adds the support for `STRICT_BF16_TO_FP` and
`STRICT_FP_TO_BF16`.

[llvm] Move CodeGenTypes library to its own directory (#79444)

Finally addresses https://reviews.llvm.org/D148769#4311232 :)

No behavior change.

[SelectionDAG] Add and use SDNode::getAsAPIntVal() helper (#77455)

This is the logical equivalent for #76710 for APInt and uses the same
naming scheme.

Converted existing users through:
`git gr

[SelectionDAG] Add and use SDNode::getAsAPIntVal() helper (#77455)

This is the logical equivalent for #76710 for APInt and uses the same
naming scheme.

Converted existing users through:
`git grep -l "cast<ConstantSDNode>\(.*\).*getAPIntValueValue" | xargs
sed -E -i
's/cast<ConstantSDNode>\((.*)\)->getAPIntValue/\1->getAsAPIntVal/'`

show more ...

[RFC][SelectionDAG] Add and use SDNode::getAsZExtVal() helper (#76710)

This follows on from #76708, allowing
`cast<ConstantSDNode>(N)->getZExtValue()` to be replaced with just
`N->getAsZextVal();`

[RFC][SelectionDAG] Add and use SDNode::getAsZExtVal() helper (#76710)

This follows on from #76708, allowing
`cast<ConstantSDNode>(N)->getZExtValue()` to be replaced with just
`N->getAsZextVal();`

Introduced via `git grep -l "cast<ConstantSDNode>\(.*\).*getZExtValue" |
xargs sed -E -i
's/cast<ConstantSDNode>\((.*)\)->getZExtValue/\1->getAsZExtVal/'` and
then using `git clang-format` on the result.

show more ...

[SelectionDAG] Add initial plumbing for the disjoint flag. (#76751)

This copies the flag from IR to the SDNode in SelectionDAGBuilder, clears
the flag in SimplifyDemandedBits, and adds it to canCre

[SelectionDAG] Add initial plumbing for the disjoint flag. (#76751)

This copies the flag from IR to the SDNode in SelectionDAGBuilder, clears
the flag in SimplifyDemandedBits, and adds it to canCreateUndefOrPoison.

Uses of the flag will come in later patches.

show more ...

[SelectionDAG] Fix copy/paste mistake in SDNodeFlags::intersectWith

The NonNeg flag was being Anded with the Exact flag.

[DAG] ISD::is*Load/is*Store - merge isa<>/cast<> calls into single dyn_cast<>. NFCI.

cast<> repeats most of the work that isa<> will have already done (and even calls assert(isa<>) in debug builds)

[DAG] ISD::is*Load/is*Store - merge isa<>/cast<> calls into single dyn_cast<>. NFCI.

cast<> repeats most of the work that isa<> will have already done (and even calls assert(isa<>) in debug builds) - just use dyn_cast and a pointer check to avoid all this duplicated work.

show more ...

[SelectionDAG] Add initial support for nneg flag on ISD::ZERO_EXTEND. (#70872)

This adds the nneg flag to SDNodeFlags and the node printing code.
SelectionDAGBuilder will add this flag to the node

[SelectionDAG] Add initial support for nneg flag on ISD::ZERO_EXTEND. (#70872)

This adds the nneg flag to SDNodeFlags and the node printing code.
SelectionDAGBuilder will add this flag to the node if the target doesn't
prefer sign extend.

A future RISC-V patch can remove the sign extend preference from
SelectionDAGBuilder.

I've also added the flag to the DAG combine that converts
ISD::SIGN_EXTEND to ISD::ZERO_EXTEND.

show more ...

[DAGCombiner] Extend `combineFMulOrFDivWithIntPow2` to work for non-splat float vecs

Do so by extending `matchUnaryPredicate` to also work for
`ConstantFPSDNode` types then encapsulate the constant

[DAGCombiner] Extend `combineFMulOrFDivWithIntPow2` to work for non-splat float vecs

Do so by extending `matchUnaryPredicate` to also work for
`ConstantFPSDNode` types then encapsulate the constant checks in a
lambda and pass it to `matchUnaryPredicate`.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D154868

show more ...

SelectionDAG: Swap operands of atomic_store

Irritatingly, atomic_store had operands in the opposite order from
regular store. This made it difficult to share patterns between
regular and atomic stor

SelectionDAG: Swap operands of atomic_store

Irritatingly, atomic_store had operands in the opposite order from
regular store. This made it difficult to share patterns between
regular and atomic stores.

There was a previous incomplete attempt to move atomic_store into the
regular StoreSDNode which would be better.

I think it was a mistake for all atomicrmw to swap the operand order,
so maybe it's better to take this one step further.

https://reviews.llvm.org/D123143

show more ...

[FPEnv] Update comment about nofpexcept default. NFC

It no longer defaults to false as of 63336795f0d5

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D154973

[SelectionDAG] Remove isNullValue and isAllOnesValue

These functions have been deprecated since:

commit f271e5d9d44e833ef2264e5b0e9aa5f8383c173d
Author: Kazu Hirata <kazu@google.com>
Date:

[SelectionDAG] Remove isNullValue and isAllOnesValue

These functions have been deprecated since:

commit f271e5d9d44e833ef2264e5b0e9aa5f8383c173d
Author: Kazu Hirata <kazu@google.com>
Date: Sun Mar 12 18:25:07 2023 -0700

Differential Revision: https://reviews.llvm.org/D153317

show more ...

[FPEnv] Intrinsics for access to FP environment

The change implements intrinsics 'get_fpenv', 'set_fpenv' and 'reset_fpenv'.
They are used to read floating-point environment, set it or reset to
some

[FPEnv] Intrinsics for access to FP environment

The change implements intrinsics 'get_fpenv', 'set_fpenv' and 'reset_fpenv'.
They are used to read floating-point environment, set it or reset to
some default state. They do the same actions as C library functions
'fegetenv' and 'fesetenv'. By default these intrinsics are lowered to calls
to these functions.

The new intrinsics specify FP environment as a value of integer type, it
is convenient of most targets where the FP state is a content of some
register. Some targets however use long representations. On X86 the size
of FP environment is 256 bits, and even half of this size is not a legal
ibteger type. To facilitate legalization in such cases, two sets of DAG
nodes is used. Nodes GET_FPENV and SET_FPENV are used when FP
environment may be represented by a legal integer type. Nodes
GET_FPENV_MEM and SET_FPENV_MEM consider FP environment as a region in
memory, much like `fesetenv` and `fegetenv` do. They are used when
target has long representation for floationg-point state.

Differential Revision: https://reviews.llvm.org/D71742

show more ...

[SDAG] Preserve unpredictable metadata, teach X86CmovConversion to respect this metadata

Sometimes an developer would like to have more control over cmov vs branch. We have unpredictable metadata in

[SDAG] Preserve unpredictable metadata, teach X86CmovConversion to respect this metadata

Sometimes an developer would like to have more control over cmov vs branch. We have unpredictable metadata in LLVM IR, but currently it is ignored by X86 backend. Propagate this metadata and avoid cmov->branch conversion in X86CmovConversion for cmov with this metadata.

Example:

```
int MaxIndex(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is converted to branch by X86CmovConversion
if (a[i] > a[t]) t = i;
}
return t;
}

int MaxIndex2(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is preserved
if (__builtin_unpredictable(a[i] > a[t])) t = i;
}
return t;
}
```

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D118118

show more ...

Restore CodeGen/MachineValueType.h from `Support`

This is rework of;

- rG13e77db2df94 (r328395; MVT)

Since `LowLevelType.h` has been restored to `CodeGen`, `MachinveValueType.h`
can be restored

Restore CodeGen/MachineValueType.h from `Support`

This is rework of;

- rG13e77db2df94 (r328395; MVT)

Since `LowLevelType.h` has been restored to `CodeGen`, `MachinveValueType.h`
can be restored as well.

Depends on D148767

Differential Revision: https://reviews.llvm.org/D149024

show more ...

Revert "[Coverity] Fix explicit null dereferences"

This reverts commit 22b23a5213b57ce1834f5b50fbbf8a50297efc8a.

This commit caused the following two build bots to start failing:
https://lab.llvm.o

Revert "[Coverity] Fix explicit null dereferences"

This reverts commit 22b23a5213b57ce1834f5b50fbbf8a50297efc8a.

This commit caused the following two build bots to start failing:
https://lab.llvm.org/buildbot/#/builders/216/builds/20322
https://lab.llvm.org/buildbot/#/builders/123/builds/18511

show more ...

[Coverity] Fix explicit null dereferences

This change fixes static code analysis errors

Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D148912

Fix uninitialized scalar members in CodeGen

This change fixes some static code analysis warnings.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D148811

[nfc][llvm] Replace pointer cast functions in PointerUnion by llvm casting functions.

This patch replaces the uses of PointerUnion.is function by llvm::isa,
PointerUnion.get function by llvm::cast,

[nfc][llvm] Replace pointer cast functions in PointerUnion by llvm casting functions.

This patch replaces the uses of PointerUnion.is function by llvm::isa,
PointerUnion.get function by llvm::cast, and PointerUnion.dyn_cast by
llvm::dyn_cast_if_present. This is according to the FIXME in
the definition of the class PointerUnion.

This patch does not remove them as they are being used in other
subprojects.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D148449

show more ...

[DAG] mergeStore - peek through truncates when finding dead store(trunc(load())) patterns

Extend the existing store(load()) removal code to account for intermediate truncates that some targets won't

[DAG] mergeStore - peek through truncates when finding dead store(trunc(load())) patterns

Extend the existing store(load()) removal code to account for intermediate truncates that some targets won't remove with canCombineTruncStore - we only care about the load/store MemoryVT.

Fixes regression from D146121

show more ...

[SelectionDAG] Deprecate isNullValue and isAllOnesValue

This patch deprecates them as there are no known uses of these
functions in the project.

Differential Revision: https://reviews.llvm.org/D145

[SelectionDAG] Deprecate isNullValue and isAllOnesValue

This patch deprecates them as there are no known uses of these
functions in the project.

Differential Revision: https://reviews.llvm.org/D145357

show more ...

IR: Add atomicrmw uinc_wrap and udec_wrap

These are essentially add/sub 1 with a clamping value.

AMDGPU has instructions for these. CUDA/HIP expose these as
atomicInc/atomicDec. Currently we use ta

IR: Add atomicrmw uinc_wrap and udec_wrap

These are essentially add/sub 1 with a clamping value.

AMDGPU has instructions for these. CUDA/HIP expose these as
atomicInc/atomicDec. Currently we use target intrinsics for these,
but those do no carry the ordering and syncscope. Add these to
atomicrmw so we can carry these and benefit from the regular
legalization processes.

show more ...

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955

show more ...