[CodeGen] llvm::Optional => std::optional

[CodeGen] Add missing copy assignment operator

When building on Windows with clang-cl ToT, before this patch I was seeing:
```
[2690/5505] Building CXX object lib/Target/ARM/CMakeFiles/LLVMARMCodeGe

[CodeGen] Add missing copy assignment operator

When building on Windows with clang-cl ToT, before this patch I was seeing:
```
[2690/5505] Building CXX object lib/Target/ARM/CMakeFiles/LLVMARMCodeGen.dir/ARMISelLowering.cpp.obj
In file included from D:/git/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:14:
In file included from D:/git/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.h:23:
D:/git/llvm-project/llvm/include\llvm/CodeGen/SelectionDAGNodes.h(760,5): warning: definition of implicit copy assignment operator for 'use_iterator' is deprecated because it has a user-declared copy constructor [-Wdeprecated-copy]
use_iterator(const use_iterator &I) = default;
^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\include\xutility(1015,13): note: in implicit copy assignment operator for 'llvm::SDNode::use_iterator' first required here
_It = _STD forward<_UIter>(_UIt);
^
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\include\xutility(5604,5): note: in instantiation of function template specialization 'std::_Seek_wrapped<llvm::SDNode::use_iterator, llvm::SDNode::use_iterator &>' requested here
_Seek_wrapped(_First, _UFirst);
^
D:/git/llvm-project/llvm/include\llvm/ADT/STLExtras.h(1737,15): note: in instantiation of function template specialization 'std::find_if<llvm::SDNode::use_iterator, (lambda at D:/git/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:15310:39)>' requested here
return std::find_if(adl_begin(Range), adl_end(Range), P);
^
1 warning generated.
```

Differential Revision: https://reviews.llvm.org/D138715

show more ...

[SelectionDAG] Remove deprecated MemSDNode->getAlignment()

I noticed a an assertion error when building MIPS code that loaded from
NULL. Loading from NULL ends up being a load with maximum alignment

[SelectionDAG] Remove deprecated MemSDNode->getAlignment()

I noticed a an assertion error when building MIPS code that loaded from
NULL. Loading from NULL ends up being a load with maximum alignment, and
due to integer truncation the value maximum was interpreted as 0 and the
assertion in MipsDAGToDAGISel::Select() failed. This previously happened
to work, but the maximum alignment was increased in
df84c1fe78130a86445d57563dea742e1b85156a, so it no longer fits into a 32
bit integer.
Instead of just fixing the one MIPS case, this patch removes all uses of
the deprecated getAlignment() call and replaces them with getAlign().

Differential Revision: https://reviews.llvm.org/D138420

show more ...

Add a const version of SDUse::getUser [nfc]

[SelectionDAG] Update stale comment on isOneOrOneSplat. NFC

[SelectionDAG] Use consistent type sizes for opcode

[SelectionDAG] Add helper function to check whether a SDValue is neutral element. NFC.

Using this helper makes work about neutral elements more easier. Although I only
find one case now, I think it

[SelectionDAG] Add helper function to check whether a SDValue is neutral element. NFC.

Using this helper makes work about neutral elements more easier. Although I only
find one case now, I think it will have more chance to be used since so many
combine works are related to neutral elements.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D133866

show more ...

KCFI sanitizer

The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type ide

KCFI sanitizer

The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type identifier for each
function and injects verification code before indirect calls.

Unlike the current CFI schemes implemented in LLVM, KCFI does not
require LTO, does not alter function references to point to a jump
table, and never breaks function address equality. KCFI is intended
to be used in low-level code, such as operating system kernels,
where the existing schemes can cause undue complications because
of the aforementioned properties. However, unlike the existing
schemes, KCFI is limited to validating only function pointers and is
not compatible with executable-only memory.

KCFI does not provide runtime support, but always traps when a
type mismatch is encountered. Users of the scheme are expected
to handle the trap. With `-fsanitize=kcfi`, Clang emits a `kcfi`
operand bundle to indirect calls, and LLVM lowers this to a
known architecture-specific sequence of instructions for each
callsite to make runtime patching easier for users who require this
functionality.

A KCFI type identifier is a 32-bit constant produced by taking the
lower half of xxHash64 from a C++ mangled typename. If a program
contains indirect calls to assembly functions, they must be
manually annotated with the expected type identifiers to prevent
errors. To make this easier, Clang generates a weak SHN_ABS
`__kcfi_typeid_<function>` symbol for each address-taken function
declaration, which can be used to annotate functions in assembly
as long as at least one C translation unit linked into the program
takes the function address. For example on AArch64, we might have
the following code:

```
.c:
int f(void);
int (*p)(void) = f;
p();

.s:
.4byte __kcfi_typeid_f
.global f
f:
...
```

Note that X86 uses a different preamble format for compatibility
with Linux kernel tooling. See the comments in
`X86AsmPrinter::emitKCFITypeId` for details.

As users of KCFI may need to locate trap locations for binary
validation and error handling, LLVM can additionally emit the
locations of traps to a `.kcfi_traps` section.

Similarly to other sanitizers, KCFI checking can be disabled for a
function with a `no_sanitize("kcfi")` function attribute.

Relands 67504c95494ff05be2a613129110c9bcf17f6c13 with a fix for
32-bit builds.

Reviewed By: nickdesaulniers, kees, joaomoreira, MaskRay

Differential Revision: https://reviews.llvm.org/D119296

show more ...

Revert "KCFI sanitizer"

This reverts commit 67504c95494ff05be2a613129110c9bcf17f6c13 as using
PointerEmbeddedInt to store 32 bits breaks 32-bit arm builds.

KCFI sanitizer

The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type ide

KCFI sanitizer

The KCFI sanitizer, enabled with `-fsanitize=kcfi`, implements a
forward-edge control flow integrity scheme for indirect calls. It
uses a !kcfi_type metadata node to attach a type identifier for each
function and injects verification code before indirect calls.

Unlike the current CFI schemes implemented in LLVM, KCFI does not
require LTO, does not alter function references to point to a jump
table, and never breaks function address equality. KCFI is intended
to be used in low-level code, such as operating system kernels,
where the existing schemes can cause undue complications because
of the aforementioned properties. However, unlike the existing
schemes, KCFI is limited to validating only function pointers and is
not compatible with executable-only memory.

KCFI does not provide runtime support, but always traps when a
type mismatch is encountered. Users of the scheme are expected
to handle the trap. With `-fsanitize=kcfi`, Clang emits a `kcfi`
operand bundle to indirect calls, and LLVM lowers this to a
known architecture-specific sequence of instructions for each
callsite to make runtime patching easier for users who require this
functionality.

A KCFI type identifier is a 32-bit constant produced by taking the
lower half of xxHash64 from a C++ mangled typename. If a program
contains indirect calls to assembly functions, they must be
manually annotated with the expected type identifiers to prevent
errors. To make this easier, Clang generates a weak SHN_ABS
`__kcfi_typeid_<function>` symbol for each address-taken function
declaration, which can be used to annotate functions in assembly
as long as at least one C translation unit linked into the program
takes the function address. For example on AArch64, we might have
the following code:

```
.c:
int f(void);
int (*p)(void) = f;
p();

.s:
.4byte __kcfi_typeid_f
.global f
f:
...
```

Note that X86 uses a different preamble format for compatibility
with Linux kernel tooling. See the comments in
`X86AsmPrinter::emitKCFITypeId` for details.

As users of KCFI may need to locate trap locations for binary
validation and error handling, LLVM can additionally emit the
locations of traps to a `.kcfi_traps` section.

Similarly to other sanitizers, KCFI checking can be disabled for a
function with a `no_sanitize("kcfi")` function attribute.

Reviewed By: nickdesaulniers, kees, joaomoreira, MaskRay

Differential Revision: https://reviews.llvm.org/D119296

show more ...

[SVE] Extend findMoreOptimalIndexType so BUILD_VECTORs do not force 64bit indices.

Extends findMoreOptimalIndexType to allow ISD::BUILD_VECTOR based
indices to be truncated when such truncation is l

[SVE] Extend findMoreOptimalIndexType so BUILD_VECTORs do not force 64bit indices.

Extends findMoreOptimalIndexType to allow ISD::BUILD_VECTOR based
indices to be truncated when such truncation is lossless. This can
enable the use of 32bit gather/scatter indices thus making it less
likely to have to split a gather/scatter in two.

Depends on D125194

Differential Revision: https://reviews.llvm.org/D130533

show more ...

[CodeGen] Generate efficient assembly for freeze(poison) version of `mm*_cast*` intel intrinsics

This patch makes the variants of `mm*_cast*` intel intrinsics that use `shufflevector(freeze(poison),

[CodeGen] Generate efficient assembly for freeze(poison) version of `mm*_cast*` intel intrinsics

This patch makes the variants of `mm*_cast*` intel intrinsics that use `shufflevector(freeze(poison), ..)` emit efficient assembly.
(These intrinsics are planned to use `shufflevector(freeze(poison), ..)` after shufflevector's semantics update; relevant thread: D103874)

To do so, this patch

1. Updates `LowerAVXCONCAT_VECTORS` in X86ISelLowering.cpp to recognize `FREEZE(UNDEF)` operand of `CONCAT_VECTOR` in addition to `UNDEF`
2. Updates X86InstrVecCompiler.td to recognize `insert_subvector` of `FREEZE(UNDEF)` vector as its first operand.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D130339

show more ...

SelectionDAGNodes.h - fix Wdocumentation warnings. NFC.

SelectionDAGNodes.h - fix Wdocumentation warnings. NFC.

[SVE][SelectionDAG] Use INDEX to generate matching instances of BUILD_VECTOR.

This patch starts small, only detecting sequences of the form
<a, a+n, a+2n, a+3n, ...> where a and n are ConstantSDNode

[SVE][SelectionDAG] Use INDEX to generate matching instances of BUILD_VECTOR.

This patch starts small, only detecting sequences of the form
<a, a+n, a+2n, a+3n, ...> where a and n are ConstantSDNodes.

Differential Revision: https://reviews.llvm.org/D125194

show more ...

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instr

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instruction will be expanded
to CAS loop. There are already a couple of targets supporting the feature. I'll
create another patch(es) to enable them accordingly.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D127041

show more ...

[SelectionDAG] Remove duplicate "is scaled" information from gather/scatter SDNodes.

During early gather/scatter enablement two different approaches
were taken to represent scaled indices:

* A Scal

[SelectionDAG] Remove duplicate "is scaled" information from gather/scatter SDNodes.

During early gather/scatter enablement two different approaches
were taken to represent scaled indices:

* A Scale operand whereby byte_offsets = Index * Scale
* An IndexType whereby byte_offsets = Index * sizeof(MemVT.ElementType)

Having multiple representations is bad as shown by this patch which
fixes instances where the two are out of sync. The dedicated scale
operand is more flexible and pervasive so this patch removes the
UNSCALED values from IndexType. This means all indices are scaled
but the scale can be one, hence unscaled. SDNodes now use the scale
operand to answer the "isScaledIndex" question.

I toyed with the idea of keeping the UNSCALED enums and helper
functions but because they will have no uses and force SDNodes to
validate the set of supported values I figured it's best to remove
them. We can re-add them if there's a real need. For similar
reasons I've kept the IndexType enum when a bool could be used as I
think being explicitly looks better.

Depends On D123347

Differential Revision: https://reviews.llvm.org/D123381

show more ...

[ISD::IndexType] Helper functions for common queries.

Add helper functions to query the signed and scaled properties
of ISD::IndexType along with functions to change them.

Remove setIndexType from

[ISD::IndexType] Helper functions for common queries.

Add helper functions to query the signed and scaled properties
of ISD::IndexType along with functions to change them.

Remove setIndexType from MaskedGatherSDNode because it only has
one usage and typically should only be changed alongside its
index operand.

Minimise the direct use of the enum values to lay the groundwork
for more refactoring.

Differential Revision: https://reviews.llvm.org/D123347

show more ...

[DAGCombine] Fold (X & ~Y) | Y with truncated not

This extends the (X & ~Y) | Y to X | Y fold to also work if ~Y is
a truncated not (when taking into account the mask X). This is
done by exporting t

[DAGCombine] Fold (X & ~Y) | Y with truncated not

This extends the (X & ~Y) | Y to X | Y fold to also work if ~Y is
a truncated not (when taking into account the mask X). This is
done by exporting the infrastructure added in D124856 and reusing
it here.

I've retained the old value of AllowUndefs=false, though probably
this can be switched to true with extra test coverage.

Differential Revision: https://reviews.llvm.org/D124930

show more ...

[NFC][CodeGen] Add comments for SDNode debug ID

Normally, we place fields serving for debug purpose declarations
under `#if LLVM_ENABLE_ABI_BREAKING_CHECKS`. For `SDNode::PersistentId` and
`Selectio

[NFC][CodeGen] Add comments for SDNode debug ID

Normally, we place fields serving for debug purpose declarations
under `#if LLVM_ENABLE_ABI_BREAKING_CHECKS`. For `SDNode::PersistentId` and
`SelectionDAG::NextPersistentId`, we do not want to do so because it adds
unneeded complexity without noticeable benefits (see discussion with @thakis
in D120714). This patch adds comments describing why we don't place those
fields under `#if` not to confuse anyone more.

Differential Revision: https://reviews.llvm.org/D123238

show more ...

Revert "[CodeGen] Place SDNode debug ID declaration under appropriate #if"

This reverts commit 83a798d4b0e17ac41d5430f1290d3661343eee1e.

As discussed in D120714 with @thakis, the patch added unneed

Revert "[CodeGen] Place SDNode debug ID declaration under appropriate #if"

This reverts commit 83a798d4b0e17ac41d5430f1290d3661343eee1e.

As discussed in D120714 with @thakis, the patch added unneeded complexity
without noticeable benefits.

show more ...

[CodeGen] Place SDNode debug ID declaration under appropriate #if

Place PersistentId declaration under #if LLVM_ENABLE_ABI_BREAKING_CHECKS to
reduce memory usage when it is not needed.

Differential

[CodeGen] Place SDNode debug ID declaration under appropriate #if

Place PersistentId declaration under #if LLVM_ENABLE_ABI_BREAKING_CHECKS to
reduce memory usage when it is not needed.

Differential Revision: https://reviews.llvm.org/D120714

show more ...

[DAG] Add llvm::isMinSignedConstant helper. NFC

Pulled out of D122754

[VP] Strided loads/stores

This patch introduces two new experimental IR intrinsics and SDAG nodes
to represent vector strided loads and stores.

Reviewed By: simoll

Differential Revision: https://r

[VP] Strided loads/stores

This patch introduces two new experimental IR intrinsics and SDAG nodes
to represent vector strided loads and stores.

Reviewed By: simoll

Differential Revision: https://reviews.llvm.org/D114884

show more ...

[ISEL] remove unnecessary getNode(); NFC

Reviewed By: RKSimon, craig.topper

Differential Revision: https://reviews.llvm.org/D120049