[TableGen] Use MemoryEffects to represent intrinsic memory effects (NFCI)

The TableGen implementation was using a homegrown implementation of
FunctionModRefInfo. This switches it to use MemoryEffect

[TableGen] Use MemoryEffects to represent intrinsic memory effects (NFCI)

The TableGen implementation was using a homegrown implementation of
FunctionModRefInfo. This switches it to use MemoryEffects instead.
This makes the code simpler, and will allow exposing the full
representational power of MemoryEffects in the future. Among other
things, this will allow us to map IntrHasSideEffects to an
inaccessiblemem readwrite, rather than just ignoring it entirely
in most cases.

To avoid layering issues, this moves the ModRef.h header from IR
to Support, so that it can be included in the TableGen layer.

Differential Revision: https://reviews.llvm.org/D137641

show more ...

[IR] Switch everything to use memory attribute

This switches everything to use the memory attribute proposed in
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
The old argmemo

[IR] Switch everything to use memory attribute

This switches everything to use the memory attribute proposed in
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
The old argmemonly, inaccessiblememonly and inaccessiblemem_or_argmemonly
attributes are dropped. The readnone, readonly and writeonly attributes
are restricted to parameters only.

The old attributes are auto-upgraded both in bitcode and IR.
The bitcode upgrade is a policy requirement that has to be retained
indefinitely. The IR upgrade is mainly there so it's not necessary
to update all tests using memory attributes in this patch, which
is already large enough. We could drop that part after migrating
tests, or retain it longer term, to make it easier to import IR
from older LLVM versions.

High-level Function/CallBase APIs like doesNotAccessMemory() or
setDoesNotAccessMemory() are mapped transparently to the memory
attribute. Code that directly manipulates attributes (e.g. via
AttributeList) on the other hand needs to switch to working with
the memory attribute instead.

Differential Revision: https://reviews.llvm.org/D135780

show more ...

[llvm] Qualify auto in range-based for loops (NFC)

[llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC

With C++17 there is no Clang pedantic warning or MSVC C5051.

[IR] Simplify Intrinsic::getDeclaration. NFC.

[clang][CodeGen] add fn_ret_thunk_extern to synthetic fns

Follow up fix to
commit 2240d72f15f3 ("[X86] initial -mfunction-return=thunk-extern
support")
https://reviews.llvm.org/D129572

@nathanchanc

[clang][CodeGen] add fn_ret_thunk_extern to synthetic fns

Follow up fix to
commit 2240d72f15f3 ("[X86] initial -mfunction-return=thunk-extern
support")
https://reviews.llvm.org/D129572

@nathanchance reported that -mfunction-return=thunk-extern was failing
to annotate the asan and tsan contructors.
https://lore.kernel.org/llvm/Ys7pLq+tQk5xEa%2FB@dev-arch.thelio-3990X/

I then noticed the same occurring for gcov synthetic functions.

Similar to
commit 2786e67 ("[IR][sanitizer] Add module flag "frame-pointer" and set
it for cc1 -mframe-pointer={non-leaf,all}")
define a new module level MetaData, "fn_ret_thunk_extern", then when set
adds the fn_ret_thunk_extern IR Fn Attr to synthetically created
Functions.

Fixes https://github.com/llvm/llvm-project/issues/56514

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D129709

show more ...

[DirectX] add thread/group id DXIL operations.

Add DXIL operation for thread/group id operations.

ID Name Description
93 ThreadId reads the thread ID
94 Gro

[DirectX] add thread/group id DXIL operations.

Add DXIL operation for thread/group id operations.

ID Name Description
93 ThreadId reads the thread ID
94 GroupId reads the group ID (SV_GroupID)
95 ThreadIdInGroup reads the thread ID within the group (SV_GroupThreadID)
96 FlattenedThreadIdInGroup provides a flattened index for a given thread within a given group (SV_GroupIndex)

Also add llvm intrinsic which map to these intrinsics to DXIL operation.

Reviewed By: beanz

Differential Revision: https://reviews.llvm.org/D127990

show more ...

Rename GCCBuiltin into ClangBuiltin

This patch is needed because developers expect "GCCBuiltin" items to be the GCC intrinsics equivalent and not the Clang internals.

Reviewed By: #libc_abi, RKSimo

Rename GCCBuiltin into ClangBuiltin

This patch is needed because developers expect "GCCBuiltin" items to be the GCC intrinsics equivalent and not the Clang internals.

Reviewed By: #libc_abi, RKSimon, xbolva00

Differential Revision: https://reviews.llvm.org/D127460

show more ...

Don't use Optional::hasValue (NFC)

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as expand, which should be
remove

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as expand, which should be
removed once targets set this explicitly.

Adjusts 11 lit tests to reflect slightly different behavior during
DAG combine.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

Revert "[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4."

This reverts commit 430ac5c3029c52e391e584c6d4447e6e361fae99.

Due to failures in Clang tests.

Differential Revision: https:

Revert "[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4."

This reverts commit 430ac5c3029c52e391e584c6d4447e6e361fae99.

Due to failures in Clang tests.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as `expand`, which should be
remo

[ValueTypes] Define MVTs for v128i2/v64i4 as well as i2 and i4.

Adds MVT::v128i2, MVT::v64i4, and implied MVT::i2, MVT::i4.

Keeps MVT::i2, MVT::i4 lowering actions as `expand`, which should be
removed once targets set this explicitly.

Adjusts 11 lit tests to reflect slightly different behavior during
DAG combine.

Differential Revision: https://reviews.llvm.org/D125247

show more ...

Assert on polymorphic pointer intrinsic param

Opaque pointers cannot be polymorphic on the pointed type given their
lack thereof. However they are currently accepted by tablegen but the
intrinsic si

Assert on polymorphic pointer intrinsic param

Opaque pointers cannot be polymorphic on the pointed type given their
lack thereof. However they are currently accepted by tablegen but the
intrinsic signature verifier trips when verifying any further
polymorphic type because the opaque pointer codepath for pointers will
not push the pointed type in ArgTys.

This commit adds an assert to easily catch such cases instead of having
the generic signature match failure.

Reviewed By: #opaque-pointers, nikic

Differential Revision: https://reviews.llvm.org/D125764

show more ...

[IR] Require intrinsic struct return type to be anonymous

This is an alternative to D122376. Rather than working around the
problem, this patch requires that struct return types in intrinsics
are an

[IR] Require intrinsic struct return type to be anonymous

This is an alternative to D122376. Rather than working around the
problem, this patch requires that struct return types in intrinsics
are anonymous/literal and adds auto-upgrade code to convert
existing uses of intrinsics with named struct types.

This ensures that the mapping between intrinsic name and
intrinsic function type is actually bijective, as it is supposed
to be.

This also fixes https://github.com/llvm/llvm-project/issues/37891.

Differential Revision: https://reviews.llvm.org/D122471

show more ...

[VP] Preserve address space of pointer for strided load/store intrinsics.

This adds LLVMAnyPointerToElt to use instead of LLVMPointerToElt.
This allows us to preserve the address space as part of th

[VP] Preserve address space of pointer for strided load/store intrinsics.

This adds LLVMAnyPointerToElt to use instead of LLVMPointerToElt.
This allows us to preserve the address space as part of the type
overload for the intrinsic, but still require the vector element
type to match the pointer type.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D122042

show more ...

[IR] Allow matching pointer to vector with opaque pointers.

Allows for skipping the pointer to vector type if opaque pointers
are enabled and the matching pointer is a vector pointer when
matching a

[IR] Allow matching pointer to vector with opaque pointers.

Allows for skipping the pointer to vector type if opaque pointers
are enabled and the matching pointer is a vector pointer when
matching an intrinsic signature in the verifier.

No test added since lacking a target using intrinsic with pointer
to vector arguments.

Differential Revision: https://reviews.llvm.org/D122203

show more ...

[IR][IPSCCP] Treat different function type as address taken (PR54258)

Without opaque pointers, this code currently treats a call through
a bitcast as the function being address taken, and IPSCCP rel

[IR][IPSCCP] Treat different function type as address taken (PR54258)

Without opaque pointers, this code currently treats a call through
a bitcast as the function being address taken, and IPSCCP relies
on this for correctness. Match the same behavior under opaque
pointers by checking that the function types are the same.

Fixes https://github.com/llvm/llvm-project/issues/54258.

show more ...

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tabl

Extend the `uwtable` attribute with unwind table kind

We have the `clang -cc1` command-line option `-funwind-tables=1|2` and
the codegen option `VALUE_CODEGENOPT(UnwindTables, 2, 0) ///< Unwind
tables (1) or asynchronous unwind tables (2)`. However, this is
encoded in LLVM IR by the presence or the absence of the `uwtable`
attribute, i.e. we lose the information whether to generate want just
some unwind tables or asynchronous unwind tables.

Asynchronous unwind tables take more space in the runtime image, I'd
estimate something like 80-90% more, as the difference is adding
roughly the same number of CFI directives as for prologues, only a bit
simpler (e.g. `.cfi_offset reg, off` vs. `.cfi_restore reg`). Or even
more, if you consider tail duplication of epilogue blocks.
Asynchronous unwind tables could also restrict code generation to
having only a finite number of frame pointer adjustments (an example
of *not* having a finite number of `SP` adjustments is on AArch64 when
untagging the stack (MTE) in some cases the compiler can modify `SP`
in a loop).
Having the CFI precise up to an instruction generally also means one
cannot bundle together CFI instructions once the prologue is done,
they need to be interspersed with ordinary instructions, which means
extra `DW_CFA_advance_loc` commands, further increasing the unwind
tables size.

That is to say, async unwind tables impose a non-negligible overhead,
yet for the most common use cases (like C++ exceptions), they are not
even needed.

This patch extends the `uwtable` attribute with an optional
value:
- `uwtable` (default to `async`)
- `uwtable(sync)`, synchronous unwind tables
- `uwtable(async)`, asynchronous (instruction precise) unwind tables

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D114543

show more ...

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avo

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652

show more ...

[IR] Handle opaque pointers in PtrToArgument mangling

It appears that this mangling type is currently unused. Make it
compatible with opaque pointers in case it becomes used again...

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecatin

[NFC] Remove uses of PointerType::getElementType()

Instead use either Type::getPointerElementType() or
Type::getNonOpaquePointerElementType().

This is part of D117885, in preparation for deprecating the API.

show more ...

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor sligh

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor slightly heavier, and involves extra
allocation for each new string attribute. Storing the attribute key/value as
strings implies extra allocation/copy step.

Use a sorted vector instead. Given the low number of attributes generally
involved, this is cheaper, as showcased by

https://llvm-compile-time-tracker.com/compare.php?from=5de322295f4ade692dc4f1823ae4450ad3c48af2&to=05bc480bf641a9e3b466619af43a2d123ee3f71d&stat=instructions

Differential Revision: https://reviews.llvm.org/D116599

show more ...

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for bu

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for building and removing, which introduced odd situation like creation of
Attribute with dummy value because the only relevant part was the attribute
kind.

Differential Revision: https://reviews.llvm.org/D116110

show more ...

[WebAssembly] Implementation of intrinsic for ref.null and HeapType removal

This patch implements the intrinsic for ref.null.
In the process of implementing int_wasm_ref_null_func() and
int_wasm_ref

[WebAssembly] Implementation of intrinsic for ref.null and HeapType removal

This patch implements the intrinsic for ref.null.
In the process of implementing int_wasm_ref_null_func() and
int_wasm_ref_null_extern() intrinsics, it removes the redundant
HeapType.

This also causes the textual assembler syntax for ref.null to
change. Instead of receiving an argument: `func` or `extern`, the
instruction mnemonic is either ref.null_func or ref.null_extern,
without the need for a further operand.

Reviewed By: tlively

Differential Revision: https://reviews.llvm.org/D114979

show more ...

[IR,TableGen] Add support for vec3 intrinsic arguments

Add generic support for vec3 types, and in particular define
llvm_v3f32_ty which will be used by AMDGPU's
llvm.amdgcn.image.bvh.intersect.ray i

[IR,TableGen] Add support for vec3 intrinsic arguments

Add generic support for vec3 types, and in particular define
llvm_v3f32_ty which will be used by AMDGPU's
llvm.amdgcn.image.bvh.intersect.ray intrinsic.

Differential Revision: https://reviews.llvm.org/D114956

show more ...