[AArch64] Improve bcvtn2 and remove aarch64_neon_bfcvt intrinsics (#120363)

This started out as trying to combine bf16 fpround to BFCVT2
instructions, but ended up removing the aarch64.neon.nfcvt i

[AArch64] Improve bcvtn2 and remove aarch64_neon_bfcvt intrinsics (#120363)

This started out as trying to combine bf16 fpround to BFCVT2
instructions, but ended up removing the aarch64.neon.nfcvt intrinsics in
favour of generating fpround instructions directly. This simplifies the
patterns and can lead to other optimizations. The BFCVT2 instruction is
adjusted to makes sure the types are valid, and a bfcvt2 is now
generated in more place. The old intrinsics are auto-upgraded to fptrunc
instructions too.

show more ...

[AutoUpgrade] Avoid unnecessary pointer bitcasts (NFCI)

Not needed with opaque pointers.

[WebAssembly] Add `-i128:128` to the `datalayout` string. (#119204)

Clang [defaults to aligning `__int128_t` to 16 bytes], while LLVM
`datalayout` strings [default to aligning `i128` to 8 bytes]. W

[WebAssembly] Add `-i128:128` to the `datalayout` string. (#119204)

Clang [defaults to aligning `__int128_t` to 16 bytes], while LLVM
`datalayout` strings [default to aligning `i128` to 8 bytes]. Wasm is
currently using the defaults for both, so it's inconsistent. Fix this by
adding `-i128:128` to Wasm's `datalayout` string so that it aligns
`i128` to 16 bytes too.

This is similar to
[llvm/llvm-project@dbad963](https://github.com/llvm/llvm-project/commit/dbad963a69fd7b16c6838f81b61167fbf00a413c)
for SPARC.

This fixes rust-lang/rust#133991; see that issue for further discussion.

[defaults to aligning `__int128_t` to 16 bytes]:
https://github.com/llvm/llvm-project/blob/f8b4182f076f8fe55f9d5f617b5a25008a77b22f/clang/lib/Basic/TargetInfo.cpp#L77
[default to aligning `i128` to 8 bytes]:
https://llvm.org/docs/LangRef.html#langref-datalayout

show more ...

[PowerPC] Update data layout aligment of i128 to 16 (#118004)

Fix 64-bit PowerPC part of
https://github.com/llvm/llvm-project/issues/102783.

[IR] Vector extract last active element intrinsic (#113587)

As discussed in #112738, it may be better to have an intrinsic to represent vector element extracts based on mask bits. This intrinsic is

[IR] Vector extract last active element intrinsic (#113587)

As discussed in #112738, it may be better to have an intrinsic to represent vector element extracts based on mask bits. This intrinsic is for the case of extracting the last active element, if any, or a default value if the mask is all-false.

The target-agnostic SelectionDAG lowering is similar to the IR in #106560.

show more ...

[MIPS] LLVM data layout give i128 an alignment of 16 for mips64 (#112084)

Fix parts of #102783.

[NVPTX] Remove nvvm.ldg.global.* intrinsics (#112834)

Remove these intrinsics which can be better represented by load
instructions with `!invariant.load` metadata:

- llvm.nvvm.ldg.global.i
- ll

[NVPTX] Remove nvvm.ldg.global.* intrinsics (#112834)

Remove these intrinsics which can be better represented by load
instructions with `!invariant.load` metadata:

- llvm.nvvm.ldg.global.i
- llvm.nvvm.ldg.global.f
- llvm.nvvm.ldg.global.p

show more ...

[SimplifyLibCall][Attribute] Fix bug where we may keep `range` attr with incompatible type (#112649)

In a variety of places we change the bitwidth of a parameter but don't
update the attributes.

[SimplifyLibCall][Attribute] Fix bug where we may keep `range` attr with incompatible type (#112649)

In a variety of places we change the bitwidth of a parameter but don't
update the attributes.

The issue in this case is from the `range` attribute when inlining
`__memset_chk`. `optimizeMemSetChk` will replace an `i32` with an
`i8`, and if the `i32` had a `range` attr assosiated it will cause an
error.

Fixes #112633

show more ...

[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112706)

Convert many instances of:
Fn = Intrinsic::getOrInsertDeclaration(...);
CreateCall(Fn, ...)
to the equivalent CreateIntrinsi

[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112706)

Convert many instances of:
Fn = Intrinsic::getOrInsertDeclaration(...);
CreateCall(Fn, ...)
to the equivalent CreateIntrinsic call.

show more ...

[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112546)

Convert almost every instance of:
CreateCall(Intrinsic::getOrInsertDeclaration(...), ...)
to the equivalent CreateIntrinsic cal

[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112546)

Convert almost every instance of:
CreateCall(Intrinsic::getOrInsertDeclaration(...), ...)
to the equivalent CreateIntrinsic call.

show more ...

[clang][aarch64] Add support for the MSVC qualifiers __ptr32, __ptr64, __sptr, __uptr for AArch64 (#111879)

MSVC has a set of qualifiers to allow using 32-bit signed/unsigned
pointers when building

[clang][aarch64] Add support for the MSVC qualifiers __ptr32, __ptr64, __sptr, __uptr for AArch64 (#111879)

MSVC has a set of qualifiers to allow using 32-bit signed/unsigned
pointers when building 64-bit targets. This is useful for WoW code
(i.e., the part of Windows that handles running 32-bit application on a
64-bit OS). Currently this is supported on x64 using the 270, 271 and
272 address spaces, but does not work for AArch64 at all.

This change adds the same 270, 271 and 272 address spaces to AArch64 and
adjusts the data layout string accordingly. Clang will generate the
correct address space casts, but these will currently be ignored until
the AArch64 backend is updated to handle them.

Partially fixes #62536

This is a resurrected version of <https://reviews.llvm.org/D158857>
(originally created by @a_vorobev) - I've cleaned it up a little, fixed
the rest of the tests and added to auto-upgrade for the data layout.

show more ...

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is a

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).

show more ...

AMDGPU: Remove flat/global fmin/fmax intrinsics (#105642)

These have been replaced with atomicrmw

AMDGPU: Add noalias.addrspace metadata when autoupgrading atomic intrinsics (#102599)

This will be needed to continue generating the raw instruction in the flat case.

[AArch64][SVE] Fix definition of bfloat fcvt intrinsics. (#110281)

Affected intrinsics:
llvm.aarch64.sve.fcvt.bf16f32
llvm.aarch64.sve.fcvtnt.bf16f32

The named intrinsics took a predica

[AArch64][SVE] Fix definition of bfloat fcvt intrinsics. (#110281)

Affected intrinsics:
llvm.aarch64.sve.fcvt.bf16f32
llvm.aarch64.sve.fcvtnt.bf16f32

The named intrinsics took a predicate based on the smallest element type
when it should be based on the largest. The intrinsics have been replace
by v2 equivalents and affected code ported to use them.

Patch includes changes to getSVEPredicateBitCast() that ensure the
generated code for the auto-upgraded old intrinsics is unchanged.

show more ...

[SPARC] Fix regression from UpgradeDataLayoutString change (#110608)

It turns out that we cannot rely on the presence of `-i64:64` as a
position reference when adding the `-i128:128` datalayout str

[SPARC] Fix regression from UpgradeDataLayoutString change (#110608)

It turns out that we cannot rely on the presence of `-i64:64` as a
position reference when adding the `-i128:128` datalayout string due to
some custom datalayout strings lacking it (e.g ones used by bugpoint,
among other things).
Do not add the `-i128:128` string in that case.

This fixes the regression introduced in
https://github.com/llvm/llvm-project/pull/106951.

show more ...

[SPARC] Align i128 to 16 bytes in SPARC datalayouts (#106951)

Align i128s to 16 bytes, following the example at
https://reviews.llvm.org/D86310.

clang already does this implicitly, but do it in

[SPARC] Align i128 to 16 bytes in SPARC datalayouts (#106951)

Align i128s to 16 bytes, following the example at
https://reviews.llvm.org/D86310.

clang already does this implicitly, but do it in backend code too for
the benefit of other frontends (see e.g
https://github.com/llvm/llvm-project/issues/102783 &
https://github.com/rust-lang/rust/issues/128950).

show more ...

Reland "[NVVM] Upgrade nvvm.ptr.* intrinics to addrspace cast" (#110262)

Remove the following intrinsics which can be trivially replaced with an
`addrspacecast`

* llvm.nvvm.ptr.gen.to.global

Reland "[NVVM] Upgrade nvvm.ptr.* intrinics to addrspace cast" (#110262)

Remove the following intrinsics which can be trivially replaced with an
`addrspacecast`

* llvm.nvvm.ptr.gen.to.global
* llvm.nvvm.ptr.gen.to.shared
* llvm.nvvm.ptr.gen.to.constant
* llvm.nvvm.ptr.gen.to.local
* llvm.nvvm.ptr.global.to.gen
* llvm.nvvm.ptr.shared.to.gen
* llvm.nvvm.ptr.constant.to.gen
* llvm.nvvm.ptr.local.to.gen

Also, cleanup the NVPTX lowering of `addrspacecast` making it more
concise.

This was reverted to avoid conflicts while reverting #107655. Re-landing
unchanged.

show more ...

Reland "[NVPTX] deprecate nvvm.rotate.* intrinsics, cleanup funnel-shift handling" (#110025)

This change deprecates the following intrinsics which can be trivially
converted to llvm funnel-shift in

Reland "[NVPTX] deprecate nvvm.rotate.* intrinsics, cleanup funnel-shift handling" (#110025)

This change deprecates the following intrinsics which can be trivially
converted to llvm funnel-shift intrinsics:

- @llvm.nvvm.rotate.b32
- @llvm.nvvm.rotate.right.b64
- @llvm.nvvm.rotate.b64

This fixes a bug in the previous version (#107655) which flipped the
order of the operands to the PTX funnel shift instruction. In LLVM IR
the high bits are the first arg and the low bits are the second arg,
while in PTX this is reversed.

show more ...

Revert "[NVPTX] deprecate nvvm.rotate.* intrinsics, cleanup funnel-shift handling (#107655)"

This reverts commit 9ac00b85e05d21be658d6aa0c91cbe05bb5dbde2.

Revert "[NVVM] Upgrade nvvm.ptr.* intrinics to addrspace cast (#109710)"

This reverts commit 36757613b73908f055674a8df0b51cc00aa04373.

[NVVM] Upgrade nvvm.ptr.* intrinics to addrspace cast (#109710)

Remove the following intrinsics which can be trivially replaced with an
`addrspacecast`

* llvm.nvvm.ptr.gen.to.global
* llvm.

[NVVM] Upgrade nvvm.ptr.* intrinics to addrspace cast (#109710)

Remove the following intrinsics which can be trivially replaced with an
`addrspacecast`

* llvm.nvvm.ptr.gen.to.global
* llvm.nvvm.ptr.gen.to.shared
* llvm.nvvm.ptr.gen.to.constant
* llvm.nvvm.ptr.gen.to.local
* llvm.nvvm.ptr.global.to.gen
* llvm.nvvm.ptr.shared.to.gen
* llvm.nvvm.ptr.constant.to.gen
* llvm.nvvm.ptr.local.to.gen

Also, cleanup the NVPTX lowering of `addrspacecast` making it more
concise.

show more ...

[NVPTX] deprecate nvvm.rotate.* intrinsics, cleanup funnel-shift handling (#107655)

This change deprecates the following intrinsics which can be trivially
converted to llvm funnel-shift intrinsics:

[NVPTX] deprecate nvvm.rotate.* intrinsics, cleanup funnel-shift handling (#107655)

This change deprecates the following intrinsics which can be trivially
converted to llvm funnel-shift intrinsics:

- @llvm.nvvm.rotate.b32
- @llvm.nvvm.rotate.right.b64
- @llvm.nvvm.rotate.b64

show more ...

[NVPTX] Remove nvvm.bitcast.* intrinsics (#107936)

Remove the following intrinsics which correspond directly to a bitcast:

- llvm.nvvm.bitcast.f2i
- llvm.nvvm.bitcast.i2f
- llvm.nvvm.bitcast.d2

[NVPTX] Remove nvvm.bitcast.* intrinsics (#107936)

Remove the following intrinsics which correspond directly to a bitcast:

- llvm.nvvm.bitcast.f2i
- llvm.nvvm.bitcast.i2f
- llvm.nvvm.bitcast.d2ll
- llvm.nvvm.bitcast.ll2d

show more ...

[AutoUpgrade] Preserve attributes when upgrading named struct return

For example, if the argument has an alignment attribute, preserve it.