[SPIR-V]: Add SPIR-V extension: SPV_KHR_cooperative_matrix (#96091)

This PR adds SPIR-V extension SPV_KHR_cooperative_matrix that "adds a
new set of types known as "cooperative matrix" types, where

[SPIR-V]: Add SPIR-V extension: SPV_KHR_cooperative_matrix (#96091)

This PR adds SPIR-V extension SPV_KHR_cooperative_matrix that "adds a
new set of types known as "cooperative matrix" types, where the storage
for and computations performed on the matrix are spread across a set of
invocations such as a subgroup" (see
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_KHR_cooperative_matrix.asciidoc).

This PR also fixes https://github.com/llvm/llvm-project/issues/96170, a
new test cases is attached
(llvm/test/CodeGen/SPIRV/transcoding/OpPtrCastToGeneric.ll).

show more ...

[SPIR-V] Improve type inference, addrspacecast and dependencies between SPIR-V entities and required capability/extensions (#94626)

This PR continues https://github.com/llvm/llvm-project/pull/94467

[SPIR-V] Improve type inference, addrspacecast and dependencies between SPIR-V entities and required capability/extensions (#94626)

This PR continues https://github.com/llvm/llvm-project/pull/94467 and
contains fixes in emission of type intrinsics, constant recording and
corresponding test cases:
* type-deduce-global-dup.ll -- fix of integer constant emission on
32-bit platforms and correct type deduction for globals
* type-deduce-simple-for.ll -- fix of GEP translation (there was an
issue previously that led to incorrect translation/broken logic of
for-range implementation)

This PR also:
* fixes a cast between identical storage classes and updates the test
case to include validation run by spirv-val,
* ensures that Bitcast for pointers satisfies the requirement that the
address spaces must match and adds the corresponding test case,
* improve encode in Tablegen and decode in code of dependencies between
SPIR-V entities and required capability/extensions,
* prevent emission of identical OpTypePointer instructions.

show more ...

[SPIR-V] Introduce support of llvm.ptr.annotation to SPIR-V Backend and implement extensions which make use of spirv.Decorations (#93561)

This PR introduces support of llvm.ptr.annotation to SPIR-V

[SPIR-V] Introduce support of llvm.ptr.annotation to SPIR-V Backend and implement extensions which make use of spirv.Decorations (#93561)

This PR introduces support of llvm.ptr.annotation to SPIR-V Backend, and
implement several extensions which make use of spirv.Decorations and
llvm.ptr.annotation to annotate global variables and pointers:

- SPV_INTEL_cache_controls
- SPV_INTEL_global_variable_host_access
- SPV_INTEL_global_variable_fpga_decorations

show more ...

[SPIR-V] Inline assembly support (#93164)

This PR introduces support for inline assembly calls for SPIR-V Backend
in general, and support for SPV_INTEL_inline_assembly [1] extension in
particular.

[SPIR-V] Inline assembly support (#93164)

This PR introduces support for inline assembly calls for SPIR-V Backend
in general, and support for SPV_INTEL_inline_assembly [1] extension in
particular. The former part of the PR is agnostic towards
vendor-specific requirements and resolves the task of supporting
successful transformation of inline assembly as long as it's possible
without specific SPIR-V instruction codes.

As a part of the PR there appears an opportunity to bring coherent
inline assembly information up to latest passes of the transformation
process (emitting final SPIR-V instructions), so that PR makes it easy
to add any another required flavor of inline assembly, other then
supported by the vendor specific SPV_INTEL_inline_assembly extension,
if/when needed.

At the moment, however, SPV_INTEL_inline_assembly is the only
implemented way to bring LLVM IR inline assembly calls up to valid
SPIR-V instructions and also the default one. This means that inline
assembly calls will generate an error message of such extension is not
used to prevent LLVM-generated error messages at the final stages of
translation. When the SPV_INTEL_inline_assembly extension is mentioned
among supported, translation of inline assembly is intercepted by this
extension implementation on a pre-legalizer step, and this is a place
where support for a new inline assembly extension may be added if
needed.

This PR also extends support for register classes, improves type
inference during pre-legalizer pass, and fixes a minor bug with
asm-printing of string literals.

[1]
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_inline_assembly.asciidoc

show more ...

[SPIR-V] Add cl_khr_kernel_clock / SPV_KHR_shader_clock extension (#92771)

Recognize `cl_khr_kernel_clock` builtins and translate them to
`OpReadClockKHR` instructions. The `Scope` operand is deduc

[SPIR-V] Add cl_khr_kernel_clock / SPV_KHR_shader_clock extension (#92771)

Recognize `cl_khr_kernel_clock` builtins and translate them to
`OpReadClockKHR` instructions. The `Scope` operand is deduced from the
builtin function name.

spirv-val does not pass yet due to OpReadClockKHR only supporting the
valid scopes for Vulkan (Device and Subgroup, but not Workgroup), so
leave validation disabled with a TODO.

show more ...

[SPIR-V] Emit SPIR-V generator magic number and version (#87951)

This patch:
- Adds SPIR-V backend's registered generator magic number to the emitted
binary. The magic number consists of the gener

[SPIR-V] Emit SPIR-V generator magic number and version (#87951)

This patch:
- Adds SPIR-V backend's registered generator magic number to the emitted
binary. The magic number consists of the generator ID (43) and LLVM
major version.
- Adds SPIR-V version to the binary.
- Allows reading the expected (maximum supported) SPIR-V version from
the target triple.
- Uses VersionTuple for representing versions throughout the backend's
codebase.
- Registers v1.6 for spirv32 and spirv64 triple.

See more: https://github.com/KhronosGroup/SPIRV-Headers/commit/7d500c

show more ...

[SPIR-V] SPIR-V Backend must generate a valid OCL version if working in OpenCL environment (#89199)

If there is no information about OpenCL version we are forced to
generate OpenCL 1.0 by default f

[SPIR-V] SPIR-V Backend must generate a valid OCL version if working in OpenCL environment (#89199)

If there is no information about OpenCL version we are forced to
generate OpenCL 1.0 by default for the OpenCL environment to avoid
puzzling run-times with Unknown/0.0 version output. For a reference,
LLVM-SPIRV Translator avoids potential issues with run-times in a
similar manner.

show more ...

[SPIR-V] Add WaveGetLaneIndex() intrinsic support (#85979)

Add support to generate valid SPIR-V for the WaveGetLaneIndex() HLSL
builtin.

To implement this, I had to fix a few small issues in the

[SPIR-V] Add WaveGetLaneIndex() intrinsic support (#85979)

Add support to generate valid SPIR-V for the WaveGetLaneIndex() HLSL
builtin.

To implement this, I had to fix a few small issues in the backend, like
the i8* pointer type being emitted, even if we have the type information
elsewhere.

Signed-off-by: Nathan Gauër <brioche@google.com>

show more ...

[SPIRV] Add Lifetime intrinsics/instructions (#85391)

This PR:
* adds Lifetime intrinsics/instructions
* fixes how the binary header is emitted (correct version and better
approximation of Bound)

[SPIRV] Add Lifetime intrinsics/instructions (#85391)

This PR:
* adds Lifetime intrinsics/instructions
* fixes how the binary header is emitted (correct version and better
approximation of Bound)
* add validation into more test cases

show more ...

[SPIR-V] Add support for the SPIR-V extension SPV_INTEL_bfloat16_conversion (#83443)

This PR is to add support for the SPIR-V extension
SPV_INTEL_bfloat16_conversion
(https://github.com/KhronosGro

[SPIR-V] Add support for the SPIR-V extension SPV_INTEL_bfloat16_conversion (#83443)

This PR is to add support for the SPIR-V extension
SPV_INTEL_bfloat16_conversion
(https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/INTEL/SPV_INTEL_bfloat16_conversion.asciidoc)
and OpenCL extension cl_intel_bfloat16_conversions
(https://registry.khronos.org/OpenCL/extensions/intel/cl_intel_bfloat16_conversions.html).

show more ...

[SPIR-V] Add support for SPV_KHR_float_controls (#83418)

This PR is to add explicit support for SPV_KHR_float_controls
(https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_K

[SPIR-V] Add support for SPV_KHR_float_controls (#83418)

This PR is to add explicit support for SPV_KHR_float_controls
(https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_KHR_float_controls.asciidoc).
This extension is included into SPIR-V after version 1.4, but in case of
lower versions it is to be included explicitly and OpExtension must be
present in the module with `OpExtension "SPV_KHR_float_controls"`.

This PR fixes this issue and fixes the test case
test/CodeGen/SPIRV/exec_mode_float_control_khr.ll to account for a
version lower than 1.4.

show more ...

[SPIR-V]: add SPIR-V extension: SPV_INTEL_variable_length_array (#83002)

This PR adds SPIR-V extension SPV_INTEL_variable_length_array that
allows to allocate local arrays whose number of elements

[SPIR-V]: add SPIR-V extension: SPV_INTEL_variable_length_array (#83002)

This PR adds SPIR-V extension SPV_INTEL_variable_length_array that
allows to allocate local arrays whose number of elements is unknown at
compile time:
* add a new SPIR-V internal intrinsic:int_spv_alloca_array
* legalize G_STACKSAVE and G_STACKRESTORE
* implement allocation of arrays (previously getArraySize() of
AllocaInst was not used)
* add tests

show more ...

Add support for the SPV_INTEL_usm_storage_classes extension (#82247)

Add support for the SPV_INTEL_usm_storage_classes extension:
*
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-ex

Add support for the SPV_INTEL_usm_storage_classes extension (#82247)

Add support for the SPV_INTEL_usm_storage_classes extension:
*
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_usm_storage_classes.asciidoc

show more ...

[SPIRV] Add support for the SPV_KHR_subgroup_rotate extension (#82374)

This PR adds support for the SPV_KHR_subgroup_rotate extension that
enables rotating values across invocations within a subgro

[SPIRV] Add support for the SPV_KHR_subgroup_rotate extension (#82374)

This PR adds support for the SPV_KHR_subgroup_rotate extension that
enables rotating values across invocations within a subgroup:
*
https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/KHR/SPV_KHR_subgroup_rotate.asciidoc

show more ...

Add support for the SPIR-V extension SPV_KHR_uniform_group_instructions (#82064)

This PR is to add support for the SPIR-V extension
SPV_KHR_uniform_group_instructions that adds new instructions to

Add support for the SPIR-V extension SPV_KHR_uniform_group_instructions (#82064)

This PR is to add support for the SPIR-V extension
SPV_KHR_uniform_group_instructions that adds new instructions to SPIR-V
to support additional group operations within uniform control flow.

show more ...

Add support for atomic instruction on floating-point numbers (#81683)

This PR adds support for atomic instruction on floating-point numbers:

* SPV_EXT_shader_atomic_float_add
* SPV_EXT_shader_at

Add support for atomic instruction on floating-point numbers (#81683)

This PR adds support for atomic instruction on floating-point numbers:

* SPV_EXT_shader_atomic_float_add
* SPV_EXT_shader_atomic_float_min_max
* SPV_EXT_shader_atomic_float16_add

and fixes asm printer output for half floating-type.

show more ...

add support for the SPV_KHR_linkonce_odr extension (#81512)

This PR adds support for the SPV_KHR_linkonce_odr extension and modifies
existing negative test with a positive check for the extension a

add support for the SPV_KHR_linkonce_odr extension (#81512)

This PR adds support for the SPV_KHR_linkonce_odr extension and modifies
existing negative test with a positive check for the extension and
proper linkage type in case when the extension is enabled.

SPV_KHR_linkonce_odr adds a "LinkOnceODR" linkage type, allowing proper
translation of, for example, C++ templates classes merging during
linking from different modules and supporting any other cases when a
global variable/function must be merged with equivalent global
variable(s)/function(s) from other modules during the linking process.

show more ...

let a user select preferred/unpreferred capabilities in a list of enabling capabilities (#81476)

By SPIR-V specification: "If an instruction, enumerant, or other feature
specifies multiple enabling

let a user select preferred/unpreferred capabilities in a list of enabling capabilities (#81476)

By SPIR-V specification: "If an instruction, enumerant, or other feature
specifies multiple enabling capabilities, only one such
capability needs to be declared to use the feature."

However, one capability may be preferred over another. One important
case is Shader capability that may not be supported by a backend, but
always is inserted if "OpDecorate SpecId" is found, because Enabling
Capabilities for the latter is the list of Shader and Kernel, where
Shader is coming first and thus always selected as the first available
option.

In this PR we address the problem by keeping current behaviour of
selecting the first option among enabling capabilities as is, but giving
a user a way to filter capabilities during the selection process via a
newly introduced "--avoid-spirv-capabilities" command line option. This
option is to avoid selection of certain capabilities if there are other
available enabling capabilities.

This PR is changing also existing pruneCapabilities() function. It
doesn't remove capability from module requirement anymore, but only adds
implicitly required capabilities recursively, so its name is changed
accordingly. This change fixes the present bug in collecting required by
a module capabilities. Before the change, introduced by this PR,
pruneCapabilities() function has been removing, for example, Kernel
capability from required by a module, because Kernel is initially
required and the second time it was needed pruneCapabilities() removed
it by mistake.

show more ...

Add support for SPIR-V extension: SPV_INTEL_function_pointers (#80759)

This PR adds initial support for "SPV_INTEL_function_pointers" SPIR-V
extension:
https://github.com/intel/llvm/blob/sycl/sycl

Add support for SPIR-V extension: SPV_INTEL_function_pointers (#80759)

This PR adds initial support for "SPV_INTEL_function_pointers" SPIR-V
extension:
https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_function_pointers.asciidoc

The goal of the extension is to support indirect function calls and
translation of function pointers into SPIR-V.

show more ...

Add support for SPIR-V extension: SPV_INTEL_subgroups (#81023)

The goal of this PR is to implement SPV_INTEL_subgroups extension in
SPIR-V Backend.

[SPIRV] Use llvm::find (NFC)

[SPIR-V] improve performance of Module Analysis stage in the part of processing "other instructions" (#76047)

The goal of this PR is to fix an issue when Module Analysis stage is not
able to comple

[SPIR-V] improve performance of Module Analysis stage in the part of processing "other instructions" (#76047)

The goal of this PR is to fix an issue when Module Analysis stage is not
able to complete processing of a really big LLVM source:
https://github.com/llvm/llvm-project/issues/76048.

There is an example of a bulky LLVM source:
https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/main/test/SpecConstants/long-spec-const-composite.ll

Processing of this file with
`llc -mtriple=spirv64-unknown-unknown -O0 long-spec-const-composite.ll
-o long-spec-const-composite.spvt`
to produce SPIR-V output using LLVM SPIR-V backend takes too long, and
I've never been able to see it actually completes. After the patch from
this PR applied elapsed time for me is ~30 sec.

The fix changes underlying data structure to be `std::set` to trace
instructions with identical operands instead of the existing approach of
the `findSameInstrInMS()` function.

show more ...

[SPIR-V] Add Float16 support when targeting Vulkan (#77115)

Add Float16 to Vulkan's available capabilities, and guard Float16Buffer
(Kernel-only capability) against being added outside OpenCL
envi

[SPIR-V] Add Float16 support when targeting Vulkan (#77115)

Add Float16 to Vulkan's available capabilities, and guard Float16Buffer
(Kernel-only capability) against being added outside OpenCL
environments.

Add tests to verify half and half vector types, and validate with
spirv-val.

Fixes #66398

show more ...

[llvm] Use DenseMap::contains (NFC)

[SPIRV] Do not remove capability from all caps (#68194)

We were removing bit_instructions cap from All caps but this was a
mistake.

Test SPV_KHR_bit_instructions was failing. Remove function
re

[SPIRV] Do not remove capability from all caps (#68194)

We were removing bit_instructions cap from All caps but this was a
mistake.

Test SPV_KHR_bit_instructions was failing. Remove function
removeCapabilityIf. It was not being done correctly and is now
unnecessary.

show more ...