[SPIR-V] Change a way SPIR-V Backend API works with user facing options (#124745)

This PR fixes https://github.com/llvm/llvm-project/issues/124703:
* added a new API call `SPIRVTranslate` that is t

[SPIR-V] Change a way SPIR-V Backend API works with user facing options (#124745)

This PR fixes https://github.com/llvm/llvm-project/issues/124703:
* added a new API call `SPIRVTranslate` that is to replace entirely old
`SPIRVTranslateModule` after existing clients switch into the new
function;
* the new `SPIRVTranslate` doesn't require option parsing, replacing the
`Opts` argument with explicit `CodeGenOptLevel` and `Triple` arguments;
* the old `SPIRVTranslateModule` call is a wrapper for `SPIRVTranslate`,
it doesn't require option parsing either and doesn't hold any logic
inside except for converting string options into `CodeGenOptLevel` and
`Triple` arguments;
* usage of the extensions list is reworked to avoid writes to the global
cl::opt variable `lib/Target/SPIRV/SPIRVSubtarget.cpp::Extensions` --
instead a new class member in SPIRVSubtarget.cpp is implemented that
allows to replace supported extensions after SPIRVSubtarget.cpp is
created;
* both API calls don't require option parsing and don't write to global
cl::opt variables.

Other related/required changes:
* SPIRV::Capability::Shader is marked as an capability of lesser
priority for OpenCL environment (to remediate absence of the
"avoid-spirv-capabilities" command line option in API calls);
* unit tests are updated and extended to cover testing of a newer API
call;
* old API call is marked with TODO to remove it after existing clients
switch into the new function.

show more ...

[SPIRV] Expand RWBuffer load and store from HLSL (#122355)

The code pattern that clang will generate for HLSL has changed from the
original plan. This allows the SPIR-V backend to generate code for

[SPIRV] Expand RWBuffer load and store from HLSL (#122355)

The code pattern that clang will generate for HLSL has changed from the
original plan. This allows the SPIR-V backend to generate code for the
current code generation.

It looks for patterns of the form:

```
%1 = @llvm.spv.resource.handlefrombinding
%2 = @llvm.spv.resource.getpointer(%1, index)
load/store %2
```

These three llvm-ir instruction are treated as a single unit that will

1. Generate or find the global variable identified by the call to
`resource.handlefrombinding`.
2. Generate an OpLoad of the variable to get the handle to the image.
3. Generate an OpImageRead or OpImageWrite using that handle with the
given index.

This will generate the OpLoad in the same BB as the read/write.

Note: Now that `resource.handlefrombinding` is not processed on its own,
many existing tests had to be removed. We do not have intrinsics that
are able to use handles to sampled images, input attachments, etc., so
we cannot generate the load of the handle. These tests are removed for
now, and will be added when those resource types are fully implemented.

show more ...

[HLSL][SPIRV][DXIL] Implement `WaveActiveSum` intrinsic (#118580)

``` - add clang builtin to Builtins.td
- link builtin in hlsl_intrinsics
- add codegen for spirv intrinsic and two

[HLSL][SPIRV][DXIL] Implement `WaveActiveSum` intrinsic (#118580)

``` - add clang builtin to Builtins.td
- link builtin in hlsl_intrinsics
- add codegen for spirv intrinsic and two directx intrinsics to retain
signedness information of the operands in CGBuiltin.cpp
- add semantic analysis in SemaHLSL.cpp
- add lowering of spirv intrinsic to spirv backend in
SPIRVInstructionSelector.cpp
- add lowering of directx intrinsics to WaveActiveOp dxil op in
DXIL.td

- add test cases to illustrate passespendent pr merges.
```
Resolves #70106

---------

Co-authored-by: Finn Plummer <canadienfinn@gmail.com>

show more ...

[SPIR-V] Prefer SPV_INTEL_optnone over SPV_EXT_optnone when both extensions are available (#122082)

This PR fixes https://github.com/llvm/llvm-project/issues/122075. We
prefer SPV_INTEL_optnone ove

[SPIR-V] Prefer SPV_INTEL_optnone over SPV_EXT_optnone when both extensions are available (#122082)

This PR fixes https://github.com/llvm/llvm-project/issues/122075. We
prefer SPV_INTEL_optnone over SPV_EXT_optnone when both extensions are
available, otherwise, when a target specifies a required extension
explicitly rather than allowing any of those (e.g., by providing
--spirv-ext=all command line argument), the Backend's behavior remains
unchanged. An existing test case is updated to check the case of 2
alternative extensions available at the same time.

show more ...

[SPIR-V] Overhaul module analysis to improve translation speed and simplify the underlying logics (#120415)

This PR is to address legacy issues with module analysis that currently
uses a complicate

[SPIR-V] Overhaul module analysis to improve translation speed and simplify the underlying logics (#120415)

This PR is to address legacy issues with module analysis that currently
uses a complicated and not so efficient approach to trace dependencies
between SPIR-V id's via a duplicate tracker data structures and an
explicitly built dependency graph. Even a quick performance check
without any specialized benchmarks points to this part of the
implementation as a biggest bottleneck.

This PR specifically:
* eliminates a need to build a dependency graph as a data structure,
* updates the test suite (mainly, by fixing incorrect CHECK's referring
to a hardcoded order of definitions, contradicting the spec requirement
to allow certain definitions to go "in any order", see
https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_logical_layout_of_a_module),
* improves function pointers implementation so that it now passes
EXPENSIVE_CHECKS (thus removing 3 XFAIL's in the test suite).

As a quick sanity check of whether goals of the PR are achieved, we can
measure time of translation for any big LLVM IR. While testing the PR in
the local development environment, improvements of the x5 order have
been observed.

For example, the SYCL test case "group barrier" that is a ~1Mb binary IR
input shows the following values of the naive performance metric that we
can nevertheless apply here to roughly estimate effects of the PR.

before the PR:
```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj

real 3m33.241s
user 3m14.688s
sys 0m18.530s
```

after the PR

```
$ time llc -O0 -mtriple=spirv64v1.6-unknown-unknown _group_barrier_phi.bc -o 1 --filetype=obj

real 0m42.031s
user 0m38.834s
sys 0m3.193s
```

Next work should probably address Duplicate Tracker further, as it needs
analysis now from the perspective of what parts of it are not necessary
now, after changing the approach to implementation of the module
analysis step.

show more ...

[SPIR-V] Add saturation and float rounding mode decorations, a subset of arithmetic constrained floating-point intrinsics, and SPV_INTEL_float_controls2 extension (#119862)

This PR adds the followin

[SPIR-V] Add saturation and float rounding mode decorations, a subset of arithmetic constrained floating-point intrinsics, and SPV_INTEL_float_controls2 extension (#119862)

This PR adds the following features:
* saturation and float rounding mode decorations,
* arithmetic constrained floating-point intrinsics (strict_fadd,
strict_fsub, strict_fmul, strict_fdiv, strict_frem, strict_fma and
strict_fldexp),
* and SPV_INTEL_float_controls2 extension,
* using recent improvements of emit-intrinsics step, this PR also
simplifies pre- and post-legalizer steps and improves instruction
selection.

show more ...

[SPIR-V] Improve general validity of emitted code between passes (#119202)

This PR improves general validity of emitted code between passes due to
generation of `TargetOpcode::PHI` instead of `SPIR

[SPIR-V] Improve general validity of emitted code between passes (#119202)

This PR improves general validity of emitted code between passes due to
generation of `TargetOpcode::PHI` instead of `SPIRV::OpPhi` after
Instruction Selection, fixing generation of OpTypePointer instructions
and using of proper virtual register classes.

Using `TargetOpcode::PHI` instead of `SPIRV::OpPhi` after Instruction
Selection has a benefit to support existing optimization passes
immediately, as an alternative path to disable those passes that use
`MI.isPHI()`. This PR makes it possible thus to revert
https://github.com/llvm/llvm-project/pull/116060 actions and get back to
use the `MachineSink` pass.

This PR is a solution of the problem discussed in details in
https://github.com/llvm/llvm-project/pull/110507. It accepts an advice
from code reviewers of the PR #110507 to postpone generation of OpPhi
rather than to patch CodeGen. This solution allows to unblock
improvements wrt. expensive checks and makes it unrelated to the general
points of the discussion about OpPhi vs. G_PHI/PHI.

This PR contains numerous small patches of emitted code validity that
allows to substantially pass rate with expensive checks. Namely, the
test suite with expensive checks set ON now has only 12 fails out of 569
total test cases.

FYI @bogner

show more ...

[SPIR-V] Add SPV_INTEL_joint_matrix extension (#118578)

The spec is available here:
https://github.com/intel/llvm/pull/12497

The PR doesn't add OpCooperativeMatrixApplyFunctionINTEL instruction

[SPIR-V] Add SPV_INTEL_joint_matrix extension (#118578)

The spec is available here:
https://github.com/intel/llvm/pull/12497

The PR doesn't add OpCooperativeMatrixApplyFunctionINTEL instruction as
it's still experimental and not properly tested E2E.

The PR also fixes few bugs in the related code:
1. CooperativeMatrixMulAddKHR optional operand must be literal, not a
constant;
2. Fixed available capabilities table creation for a case, when a single
extension adds few capabilities, that occupy not contiguous op codes.

---------

Signed-off-by: Sidorov, Dmitry <dmitry.sidorov@intel.com>

show more ...

[SPIR-V] Fix emission of debug and annotation instructions and add SPV_EXT_optnone SPIR-V extension (#118402)

This PR fixes:
* emission of OpNames (added newly inserted internal intrinsics and
bas

[SPIR-V] Fix emission of debug and annotation instructions and add SPV_EXT_optnone SPIR-V extension (#118402)

This PR fixes:
* emission of OpNames (added newly inserted internal intrinsics and
basic blocks)
* emission of function attributes (SRet is added)
* implementation of SPV_INTEL_optnone so that it emits OptNoneINTEL
Function Control flag, and add implementation of the SPV_EXT_optnone
SPIR-V extension.

show more ...

Add support for SPIR-V extension: SPV_INTEL_media_block_io (#118024)

This changes implements SPV_INTEL_media_block_io extension in SPIR-V
backend.

[SPIRV] Use `Op[S|U]Dot` when possible for integer dot product (#115095)

```
- use the new OpSDot/OpUDot instructions when capabilites allow in SPIRVInstructionSelector.cpp
- correct functionality

[SPIRV] Use `Op[S|U]Dot` when possible for integer dot product (#115095)

```
- use the new OpSDot/OpUDot instructions when capabilites allow in SPIRVInstructionSelector.cpp
- correct functionality of capability check onto input operand and not return operand type in SPIRVModuleAnalysis.cpp

- add test cases to demonstrate use case in idot.ll
```

Resolves #114632

show more ...

[HLSL] Implement WaveActiveAnyTrue intrinsic (#115902)

Resolves https://github.com/llvm/llvm-project/issues/99160

- [x] Implement `WaveActiveAnyTrue` clang builtin,
- [x] Link `WaveActiveAnyTr

[HLSL] Implement WaveActiveAnyTrue intrinsic (#115902)

Resolves https://github.com/llvm/llvm-project/issues/99160

- [x] Implement `WaveActiveAnyTrue` clang builtin,
- [x] Link `WaveActiveAnyTrue` clang builtin with `hlsl_intrinsics.h`
- [x] Add sema checks for `WaveActiveAnyTrue` to
`CheckHLSLBuiltinFunctionCall` in `SemaChecking.cpp`
- [x] Add codegen for `WaveActiveAnyTrue` to `EmitHLSLBuiltinExpr` in
`CGBuiltin.cpp`
- [x] Add codegen tests to
`clang/test/CodeGenHLSL/builtins/WaveActiveAnyTrue.hlsl`
- [x] Add sema tests to
`clang/test/SemaHLSL/BuiltIns/WaveActiveAnyTrue-errors.hlsl`
- [x] Create the `int_dx_WaveActiveAnyTrue` intrinsic in
`IntrinsicsDirectX.td`
- [x] Create the `DXILOpMapping` of `int_dx_WaveActiveAnyTrue` to `113`
in `DXIL.td`
- [x] Create the `WaveActiveAnyTrue.ll` and
`WaveActiveAnyTrue_errors.ll` tests in `llvm/test/CodeGen/DirectX/`
- [x] Create the `int_spv_WaveActiveAnyTrue` intrinsic in
`IntrinsicsSPIRV.td`
- [x] In SPIRVInstructionSelector.cpp create the `WaveActiveAnyTrue`
lowering and map it to `int_spv_WaveActiveAnyTrue` in
`SPIRVInstructionSelector::selectIntrinsic`.
- [x] Create SPIR-V backend test case in
`llvm/test/CodeGen/SPIRV/hlsl-intrinsics/WaveActiveAnyTrue.ll`

---------

Co-authored-by: Finn Plummer <50529406+inbelic@users.noreply.github.com>
Co-authored-by: Greg Roth <grroth@microsoft.com>

show more ...

[SPIRV] Add write to image buffer for shaders. (#115927)

This commit adds an intrinsic that will write to an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

[SPIRV] Add write to image buffer for shaders. (#115927)

This commit adds an intrinsic that will write to an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

We cannot reuse the existing openCL write_image function because that is
not a reserved name in HLSL. There is not much common code to factor
out.

show more ...

[HLSL] Adding HLSL `clip` function. (#114588)

Adding HLSL `clip` function.
- adding llvm intrinsic
- adding sema checks
- adding dxil lowering
- ading spirv lowering
- adding sema tests
- addi

[HLSL] Adding HLSL `clip` function. (#114588)

Adding HLSL `clip` function.
- adding llvm intrinsic
- adding sema checks
- adding dxil lowering
- ading spirv lowering
- adding sema tests
- adding codegen tests
- adding lowering tests

Closes #99093

---------

Co-authored-by: Joao Saffran <jderezende@microsoft.com>

show more ...

[SPIRV] Add reads from image buffer for shaders. (#115178)

This commit adds an intrinsic that will read from an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clan

[SPIRV] Add reads from image buffer for shaders. (#115178)

This commit adds an intrinsic that will read from an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

We cannot reuse the existing openCL readimage function because that is
not a reserved name in HLSL.

I considered trying to refactor generateReadImageInst, so that we could
share code between the two implementations. However, most of the code in
generateReadImageInst is concerned with trying to figure out which type
of image read is being done. Once we factor out the code that will be
common, then we end up with just a single call to the MIRBuilder being
common.

show more ...

[HLSL][SPIRV][DXIL] Implement `dot4add_i8packed` intrinsic (#113623)

- create a clang built-in in Builtins.td
- link dot4add_i8packed in hlsl_intrinsics.h
- add lowering to spirv backend through e

[HLSL][SPIRV][DXIL] Implement `dot4add_i8packed` intrinsic (#113623)

- create a clang built-in in Builtins.td
- link dot4add_i8packed in hlsl_intrinsics.h
- add lowering to spirv backend through expansion of operation as OPSDot
is missing up to SPIRV 1.6 in SPIRVInstructionSelector.cpp
- add lowering to spirv backend using OpSDot in applicable SPIRV version
or if SPV_KHR_integer_dot_product is enabled
- add dot4add_i8packed intrinsic to IntrinsicsDirectX.td and mapping to
DXIL.td op Dot4AddI8Packed

- add tests for HLSL intrinsic lowering to dx/spv intrinsic in
dot4add_i8packed.hlsl
- add tests for sema checks in dot4add_i8packed-errors.hlsl
- add test of spir-v lowering in SPIRV/dot4add_i8packed.ll
- add test to dxil lowering in DirectX/dot4add_i8packed.ll

Resolves #99220

show more ...

[SPIRV][HLSL] Handle arrays of resources (#111564)

This commit adds the ability to get a particular resource from an array
of resources using the handle_fromBinding intrinsic.

The main changes a

[SPIRV][HLSL] Handle arrays of resources (#111564)

This commit adds the ability to get a particular resource from an array
of resources using the handle_fromBinding intrinsic.

The main changes are:

1. Create an array when generating the type.
2. Add capabilities from

[SPV_EXT_descriptor_indexing](https://htmlpreview.github.io/?https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/EXT/SPV_EXT_descriptor_indexing.html).

We are still missing the ability to declare a runtime array. That will
be done in a follow up PR.

show more ...

[SPIR-V] Implement support of the SPV_INTEL_split_barrier SPIRV extension (#112359)

This PR implements support of the SPV_EXT_arithmetic_fence SPIRV
extension
(https://github.com/KhronosGroup/SPIR

[SPIR-V] Implement support of the SPV_INTEL_split_barrier SPIRV extension (#112359)

This PR implements support of the SPV_EXT_arithmetic_fence SPIRV
extension
(https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/INTEL/SPV_INTEL_split_barrier.asciidoc)
and adds builtins from
https://registry.khronos.org/OpenCL/extensions/intel/cl_intel_split_work_group_barrier.html

show more ...

[SPIR-V] Emit DebugTypePointer from NonSemantic DI (#109287)

Implementation of DebugTypePointer from
NonSemantic.Shader.DebugInfo.100.

[SPIR-V] Duplicates Tracker accounts for possible changes in Constant usage after optimization (#110835)

This PR introduces changes into processing of internal/service data in
SPIRV Backend so that

[SPIR-V] Duplicates Tracker accounts for possible changes in Constant usage after optimization (#110835)

This PR introduces changes into processing of internal/service data in
SPIRV Backend so that Duplicates Tracker accounts for possible changes
in Constant usage after optimization, namely this PR fixes the case when
a Constant register stored in Duplicates Tracker after all passes is
represented by a non-constant expression. In this case we may be sure
that it neither is able to create a duplicate nor is in need of a
special placement as a Constant instruction.

This PR doesn't introduce a new feature, and in this case we rely on
existing set of test cases in the SPIRV Backend test suite to ensure
that this PR doesn't break existing assumptions without introducing new
test cases. There is a reproducer of the issue available as part of SYCL
CTS test suite, however it's a binary of several MB's size. Given the
subtlety of the issue, reduction of the reproducer to a reasonable site
for inclusion into the SPIRV Backend test suite doesn't seem realistic.

show more ...

[SPIRV] Make access qualifier optional for spirv.Image type (#110852)

The SPIRV backend has a special type named `spirv.Image`. This type is
meant to correspond to the OpTypeImage instruction in SP

[SPIRV] Make access qualifier optional for spirv.Image type (#110852)

The SPIRV backend has a special type named `spirv.Image`. This type is
meant to correspond to the OpTypeImage instruction in SPIR-V, but there
is one difference. The access qualifier operand in OpTypeImage is
optional. On top of that, the access qualifiers are only valid for
kernels, and not for shaders.

We want to reuse this type when generating shader from HLSL, but we
can't use the access qualifier. This commit make the access qualifer
optional in the target extension type.

The same is done for `spirv.SampledImage`.

Contributes to #81036

show more ...

[SPIR-V] Implement support of the SPV_EXT_arithmetic_fence SPIRV extension (#110500)

This PR implements support of the SPV_EXT_arithmetic_fence SPIRV
extension:
https://htmlpreview.github.io/?http

[SPIR-V] Implement support of the SPV_EXT_arithmetic_fence SPIRV extension (#110500)

This PR implements support of the SPV_EXT_arithmetic_fence SPIRV
extension:
https://htmlpreview.github.io/?https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/EXT/SPV_EXT_arithmetic_fence.html.

show more ...

[SPIR-V] Emit DebugTypeBasic for NonSemantic DI (#106980)

The commit introduces support for fundamental DI instruction. Metadata
handlers required for this instruction is stored inside debug record

[SPIR-V] Emit DebugTypeBasic for NonSemantic DI (#106980)

The commit introduces support for fundamental DI instruction. Metadata
handlers required for this instruction is stored inside debug records
(https://llvm.org/docs/SourceLevelDebugging.html) parts of the module
which rises the necessity of it's traversal.

show more ...

[SPIR-V] Ensure that OpExtInst instructions generated by NonSemantic_Shader_DebugInfo_100 are not mixed up with other OpExtInst instructions (#107007)

This PR is to ensure that OpExtInst instruction

[SPIR-V] Ensure that OpExtInst instructions generated by NonSemantic_Shader_DebugInfo_100 are not mixed up with other OpExtInst instructions (#107007)

This PR is to ensure that OpExtInst instructions generated by
NonSemantic_Shader_DebugInfo_100 are not mixed up with other OpExtInst
instructions.

Original implementation
(https://github.com/llvm/llvm-project/pull/97558) has introduced an
issue by moving OpExtInst instruction with the 3rd operand equal to
DebugSource (value 35) or DebugCompilationUnit (value 1) even if
OpExtInst is not generated by NonSemantic_Shader_DebugInfo_100
implementation code.

The reproducer is attached as a new test case. The code of the test case
reproduces the issue, because "lgamma" has the same code (35) inside
OpenCL_std as DebugSource inside NonSemantic_Shader_DebugInfo_100.

show more ...

[SPIRV] Emitting DebugSource, DebugCompileUnit (#97558)

This commit introduces emission of DebugSource, DebugCompileUnit from
NonSemantic.Shader.DebugInfo.100 and required OpString with filename.

[SPIRV] Emitting DebugSource, DebugCompileUnit (#97558)

This commit introduces emission of DebugSource, DebugCompileUnit from
NonSemantic.Shader.DebugInfo.100 and required OpString with filename.
NonSemantic.Shader.DebugInfo.100 is divided, following DWARF into two
main concepts – emitting DIE and Line.
In DWARF .debug_abbriev and .debug_info sections are responsible for
emitting tree with information (DEIs) about e.g. types, compilation
unit. Corresponding to that in NonSemantic.Shader.DebugInfo.100 have
instructions like DebugSource, DebugCompileUnit etc. which preforms same
role in SPIR-V file. The difference is in fact that in SPIR-V there are
no sections but logical layout which forces order of the instruction
emission.
The NonSemantic.Shader.DebugInfo.100 requires for this type of global
information to be emitted after OpTypeXXX and OpConstantXXX
instructions.
One of the goals was to minimize changes and interaction with
SPIRVModuleAnalysis as possible which current commit achieves by
emitting it’s instructions directly into MachineFunction.
The possibility of duplicates are mitigated by guard inside pass which
emits the global information only once in one function.
By that method duplicates don’t have chance to be emitted.
From that point, adding new debug global instructions should be
straightforward.

show more ...