[SPIR-V] Fixup storage class for global private (#116636)

Adds a new address spaces: `hlsl_private`. Variables with such address
space will be emitted with a `Private` storage class.
This is usefu

[SPIR-V] Fixup storage class for global private (#116636)

Adds a new address spaces: `hlsl_private`. Variables with such address
space will be emitted with a `Private` storage class.
This is useful for variables global to a SPIR-V module, since up to now,
they were still emitted with a `Function` storage class, which is wrong.

---------

Signed-off-by: Nathan Gauër <brioche@google.com>

show more ...

[SPIR-V] Implement intrinsics llvm.scmp.* and llvm.ucmp.* (#117341)

This PR add translation of intrinsics `llvm.scmp.*` and `llvm.ucmp.*`.

[SPIR-V] Fix SPIR-V extension SPV_INTEL_function_pointers: introduce CodeSectionINTEL (#117250)

This PR fixes generation of OpConstantFunctionPointerINTEL instruction
for the SPIR-V extension SPV_I

[SPIR-V] Fix SPIR-V extension SPV_INTEL_function_pointers: introduce CodeSectionINTEL (#117250)

This PR fixes generation of OpConstantFunctionPointerINTEL instruction
for the SPIR-V extension SPV_INTEL_function_pointers. Result type of
OpConstantFunctionPointerINTEL must be OpTypePointer with Storage Class
operand equal to CodeSectionINTEL.

See also https://github.com/llvm/llvm-project/pull/116636

CC: @MrSidims

show more ...

[SPIRV] Use `Op[S|U]Dot` when possible for integer dot product (#115095)

```
- use the new OpSDot/OpUDot instructions when capabilites allow in SPIRVInstructionSelector.cpp
- correct functionality

[SPIRV] Use `Op[S|U]Dot` when possible for integer dot product (#115095)

```
- use the new OpSDot/OpUDot instructions when capabilites allow in SPIRVInstructionSelector.cpp
- correct functionality of capability check onto input operand and not return operand type in SPIRVModuleAnalysis.cpp

- add test cases to demonstrate use case in idot.ll
```

Resolves #114632

show more ...

[HLSL] Implement WaveActiveAnyTrue intrinsic (#115902)

Resolves https://github.com/llvm/llvm-project/issues/99160

- [x] Implement `WaveActiveAnyTrue` clang builtin,
- [x] Link `WaveActiveAnyTr

[HLSL] Implement WaveActiveAnyTrue intrinsic (#115902)

Resolves https://github.com/llvm/llvm-project/issues/99160

- [x] Implement `WaveActiveAnyTrue` clang builtin,
- [x] Link `WaveActiveAnyTrue` clang builtin with `hlsl_intrinsics.h`
- [x] Add sema checks for `WaveActiveAnyTrue` to
`CheckHLSLBuiltinFunctionCall` in `SemaChecking.cpp`
- [x] Add codegen for `WaveActiveAnyTrue` to `EmitHLSLBuiltinExpr` in
`CGBuiltin.cpp`
- [x] Add codegen tests to
`clang/test/CodeGenHLSL/builtins/WaveActiveAnyTrue.hlsl`
- [x] Add sema tests to
`clang/test/SemaHLSL/BuiltIns/WaveActiveAnyTrue-errors.hlsl`
- [x] Create the `int_dx_WaveActiveAnyTrue` intrinsic in
`IntrinsicsDirectX.td`
- [x] Create the `DXILOpMapping` of `int_dx_WaveActiveAnyTrue` to `113`
in `DXIL.td`
- [x] Create the `WaveActiveAnyTrue.ll` and
`WaveActiveAnyTrue_errors.ll` tests in `llvm/test/CodeGen/DirectX/`
- [x] Create the `int_spv_WaveActiveAnyTrue` intrinsic in
`IntrinsicsSPIRV.td`
- [x] In SPIRVInstructionSelector.cpp create the `WaveActiveAnyTrue`
lowering and map it to `int_spv_WaveActiveAnyTrue` in
`SPIRVInstructionSelector::selectIntrinsic`.
- [x] Create SPIR-V backend test case in
`llvm/test/CodeGen/SPIRV/hlsl-intrinsics/WaveActiveAnyTrue.ll`

---------

Co-authored-by: Finn Plummer <50529406+inbelic@users.noreply.github.com>
Co-authored-by: Greg Roth <grroth@microsoft.com>

show more ...

[NFC][SPIRV] Cleanup selectOpWithSrc functions (#117077)

As a follow up request from
https://github.com/llvm/llvm-project/pull/111082#discussion_r1811132876
the following non functional changes ha

[NFC][SPIRV] Cleanup selectOpWithSrc functions (#117077)

As a follow up request from
https://github.com/llvm/llvm-project/pull/111082#discussion_r1811132876
the following non functional changes have been make
- `selectNAryOpWithSrcs` has been renamed to `selectOpWithSrcs`
- Calls to `selectUnOpWithSrc` have been replaced with
`selectOpWithSrcs`
- `selectUnOpWithSrc` has been deleted

show more ...

[SPIRV] Add write to image buffer for shaders. (#115927)

This commit adds an intrinsic that will write to an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

[SPIRV] Add write to image buffer for shaders. (#115927)

This commit adds an intrinsic that will write to an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

We cannot reuse the existing openCL write_image function because that is
not a reserved name in HLSL. There is not much common code to factor
out.

show more ...

[HLSL] Adding HLSL `clip` function. (#114588)

Adding HLSL `clip` function.
- adding llvm intrinsic
- adding sema checks
- adding dxil lowering
- ading spirv lowering
- adding sema tests
- addi

[HLSL] Adding HLSL `clip` function. (#114588)

Adding HLSL `clip` function.
- adding llvm intrinsic
- adding sema checks
- adding dxil lowering
- ading spirv lowering
- adding sema tests
- adding codegen tests
- adding lowering tests

Closes #99093

---------

Co-authored-by: Joao Saffran <jderezende@microsoft.com>

show more ...

[SPIR-V] Implement builtins for OpIAddCarry/OpISubBorrow and improve/fix type inference (#115192)

This PR is to solve several intertwined issues with type inference while
adding support for builtin

[SPIR-V] Implement builtins for OpIAddCarry/OpISubBorrow and improve/fix type inference (#115192)

This PR is to solve several intertwined issues with type inference while
adding support for builtins for OpIAddCarry and OpISubBorrow:
* OpIAddCarry and OpISubBorrow generation in a way of supporting SPIR-V
friendly builtins `__spirv_...` -- introduces a new element to account
for, namely, `ptr sret (%struct) %0` argument that is a place to put a
result of the instruction;
* fix early definition of SPIR-V types during call lowering -- namely,
the goal of the PR is to ensure that correct types are applied to
virtual registers which were used as arguments in call lowering and so
caused early definition of SPIR-V types; reproducers are attached as a
new test cases;
* improve parsing of builtin names (e.g., understand a name of a kind
`"anon<int, int> __spirv_IAddCarry<int, int>(int, int)"` that was
incorrectly parsed as `anon` before the PR);
* improve type inference and fix access to erased from parent after
visit instructions -- before the PR visiting of instructions in
emitintrinsics pass replaced old alloca's, bitcast's, etc. instructions
with a newly generated internal SPIR-V intrinsics and after erasing old
instructions there were still references to them in a postprocessing
working list, while records for newly deduced pointee types were lost;
this PR fixes the issue by adding as consistent wrt. internal data
structures action `SPIRVEmitIntrinsics::replaceAllUsesWith()` that fixes
above mentioned problems;
* LLVM IR add/sub instructions result in logical SPIR-V instructions
when applied to bool type;
* fix validation of pointer types for frexp and lgamma_r,
* fix hardcoded reference to AS0 as a Function storage class in
lib/Target/SPIRV/SPIRVBuiltins.cpp -- now it's
`storageClassToAddressSpace(SPIRV::StorageClass::Function)`,
* re-use the same OpTypeStruct for two identical references to struct's
in arithmetic with overflow instructions.

show more ...

[SPIRV] Mark maybe unused extractSubvector variable (#116117)

Change #115178 introduced a variable that is only used in an assert,
which could result in an unused variable warning in builds without

[SPIRV] Mark maybe unused extractSubvector variable (#116117)

Change #115178 introduced a variable that is only used in an assert,
which could result in an unused variable warning in builds without
asserts enabled. This just addes the maybe_unused attribute to silence
the warning.

show more ...

[SPIRV] Add reads from image buffer for shaders. (#115178)

This commit adds an intrinsic that will read from an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clan

[SPIRV] Add reads from image buffer for shaders. (#115178)

This commit adds an intrinsic that will read from an image buffer. We
chose to match the name of the DXIL intrinsic for simplicity in clang.

We cannot reuse the existing openCL readimage function because that is
not a reserved name in HLSL.

I considered trying to refactor generateReadImageInst, so that we could
share code between the two implementations. However, most of the code in
generateReadImageInst is concerned with trying to figure out which type
of image read is being done. Once we factor out the code that will be
common, then we end up with just a single call to the MIRBuilder being
common.

show more ...

[SPIRV] Audit `select` Result in SPIRVInstructionSelector (#115193)

- as per the definition of `select` in GlobalISel/InstructionSelector.h
the return value is a boolean denoting if the select was

[SPIRV] Audit `select` Result in SPIRVInstructionSelector (#115193)

- as per the definition of `select` in GlobalISel/InstructionSelector.h
the return value is a boolean denoting if the select was successful
- doing `Result |=` is incorrect as all inserted instructions should be
succesful, hence we change to using `Result &=`
- ensure that the return value of all BuildMI instructions are
propagated correctly

show more ...

[DXIL][SPIRV] Lower `WaveActiveCountBits` intrinsic (#113382)

```
- add codegen for llvm builtin to spirv/directx intrinsic in CGBuiltin.cpp
- add lowering of spirv intrinsic to spirv backend

[DXIL][SPIRV] Lower `WaveActiveCountBits` intrinsic (#113382)

```
- add codegen for llvm builtin to spirv/directx intrinsic in CGBuiltin.cpp
- add lowering of spirv intrinsic to spirv backend in SPIRVInstructionSelector.cpp
- add lowering of directx intrinsic to dxil op in DXIL.td

- add test cases to illustrate passes
- add test case for semantic analysis
```

Resolves #80176

show more ...

[HLSL][SPIRV] Added clamp intrinsic (#113394)

Fixes #88052

- Added the following intrinsics:
- `int_spv_uclamp`
- `int_spv_sclamp`
- `int_spv_fclamp`
- Updated DirectX counterparts to h

[HLSL][SPIRV] Added clamp intrinsic (#113394)

Fixes #88052

- Added the following intrinsics:
- `int_spv_uclamp`
- `int_spv_sclamp`
- `int_spv_fclamp`
- Updated DirectX counterparts to have the same three clamp intrinsics.
- Update the clamp.hlsl unit tests to include SPIRV
- Added the SPIRV specific tests

show more ...

[HLSL][SPIRV][DXIL] Implement `dot4add_u8packed` intrinsic (#115068)

```- create a clang built-in in Builtins.td
- link dot4add_u8packed in hlsl_intrinsics.h
- add lowering to spirv backend throug

[HLSL][SPIRV][DXIL] Implement `dot4add_u8packed` intrinsic (#115068)

```- create a clang built-in in Builtins.td
- link dot4add_u8packed in hlsl_intrinsics.h
- add lowering to spirv backend through expansion of operation as OpUDot is missing up to SPIRV 1.6 in SPIRVInstructionSelector.cpp
- add lowering to spirv backend using OpUDot if applicable SPIRV version or SPV_KHR_integer_dot_product is enabled
- add dot4add_u8packed intrinsic to IntrinsicsDirectX.td and mapping to DXIL.td op Dot4AddU8Packed

- add tests for HLSL intrinsic lowering to dx/spv intrinsic in dot4add_u8packed.hlsl
- add tests for sema checks in dot4add_u8packed-errors.hlsl
- add test of spir-v lowering in SPIRV/dot4add_u8packed.ll
- add test to dxil lowering in DirectX/dot4add_u8packed.ll
```

Resolves #99219

show more ...

[HLSL] implement elementwise firstbithigh hlsl builtin (#111082)

Implements elementwise firstbithigh hlsl builtin.
Implements firstbituhigh intrinsic for spirv and directx, which handles
unsigned

[HLSL] implement elementwise firstbithigh hlsl builtin (#111082)

Implements elementwise firstbithigh hlsl builtin.
Implements firstbituhigh intrinsic for spirv and directx, which handles
unsigned integers
Implements firstbitshigh intrinsic for spirv and directx, which handles
signed integers.
Fixes #113486
Closes #99115

show more ...

[HLSL][SPIRV][DXIL] Implement `dot4add_i8packed` intrinsic (#113623)

- create a clang built-in in Builtins.td
- link dot4add_i8packed in hlsl_intrinsics.h
- add lowering to spirv backend through e

[HLSL][SPIRV][DXIL] Implement `dot4add_i8packed` intrinsic (#113623)

- create a clang built-in in Builtins.td
- link dot4add_i8packed in hlsl_intrinsics.h
- add lowering to spirv backend through expansion of operation as OPSDot
is missing up to SPIRV 1.6 in SPIRVInstructionSelector.cpp
- add lowering to spirv backend using OpSDot in applicable SPIRV version
or if SPV_KHR_integer_dot_product is enabled
- add dot4add_i8packed intrinsic to IntrinsicsDirectX.td and mapping to
DXIL.td op Dot4AddI8Packed

- add tests for HLSL intrinsic lowering to dx/spv intrinsic in
dot4add_i8packed.hlsl
- add tests for sema checks in dot4add_i8packed-errors.hlsl
- add test of spir-v lowering in SPIRV/dot4add_i8packed.ll
- add test to dxil lowering in DirectX/dot4add_i8packed.ll

Resolves #99220

show more ...

[SPIR-V] Fix OpDecorate emission after vreg def. (#114426)

In SPIR-V, OpDecorate instructions are allowed to forward-declare a
virtual register. But while we are at the MIR level, we must comply wi

[SPIR-V] Fix OpDecorate emission after vreg def. (#114426)

In SPIR-V, OpDecorate instructions are allowed to forward-declare a
virtual register. But while we are at the MIR level, we must comply with
stricter rules, meaning OpDecorate should be emited after, not before
the reg definition.
(In some cases, we defined those just before, switching to just after).

Related to #110652

---------

Signed-off-by: Nathan Gauër <brioche@google.com>

show more ...

[SPIRV][HLSL] Handle arrays of resources (#111564)

This commit adds the ability to get a particular resource from an array
of resources using the handle_fromBinding intrinsic.

The main changes a

[SPIRV][HLSL] Handle arrays of resources (#111564)

This commit adds the ability to get a particular resource from an array
of resources using the handle_fromBinding intrinsic.

The main changes are:

1. Create an array when generating the type.
2. Add capabilities from

[SPV_EXT_descriptor_indexing](https://htmlpreview.github.io/?https://github.com/KhronosGroup/SPIRV-Registry/blob/main/extensions/EXT/SPV_EXT_descriptor_indexing.html).

We are still missing the ability to declare a runtime array. That will
be done in a follow up PR.

show more ...

[SPIRV] Add GroupMemoryBarrierWithGroupSync intrinsic (#111888)

partially fixes #70103

### Changes
* Added int_spv_group_memory_barrier_with_group_sync intrinsic in
IntrinsicsSPIRV.td
* Added

[SPIRV] Add GroupMemoryBarrierWithGroupSync intrinsic (#111888)

partially fixes #70103

### Changes
* Added int_spv_group_memory_barrier_with_group_sync intrinsic in
IntrinsicsSPIRV.td
* Added lowering for int_spv_group_memory_barrier_with_group_sync in
SPIRVInstructionSelector.cpp
* Added SPIRV backend test case

### Related PRs
* [[clang][HLSL] Add GroupMemoryBarrierWithGroupSync intrinsic
#111883](https://github.com/llvm/llvm-project/pull/111883)
* [[DXIL] Add GroupMemoryBarrierWithGroupSync intrinsic
#111884](https://github.com/llvm/llvm-project/pull/111884)

show more ...

[HLSL] Implement `WaveReadLaneAt` intrinsic (#111010)

- create a clang built-in in Builtins.td
- add semantic checking in SemaHLSL.cpp
- link the WaveReadLaneAt api in hlsl_intrinsics.h

[HLSL] Implement `WaveReadLaneAt` intrinsic (#111010)

- create a clang built-in in Builtins.td
- add semantic checking in SemaHLSL.cpp
- link the WaveReadLaneAt api in hlsl_intrinsics.h
- add lowering to spirv backend op GroupNonUniformShuffle
with Scope = 2 (Group) in SPIRVInstructionSelector.cpp
- add WaveReadLaneAt intrinsic to IntrinsicsDirectX.td and mapping
to DXIL.td

- add tests for HLSL intrinsic lowering to spirv intrinsic in
WaveReadLaneAt.hlsl
- add tests for sema checks in WaveReadLaneAt-errors.hlsl
- add spir-v backend tests in WaveReadLaneAt.ll
- add test to show scalar dxil lowering functionality

- note that this doesn't include support for the scalarizer to
handle WaveReadLaneAt will be added in a future pr

This is the first part #70104

show more ...

[HLSL] Implement the `degrees` intrinsic (#111209)

- add degrees builtin
- link degrees api in hlsl_intrinsics.h
- add degrees intrinsic to IntrinsicsDirectX.td
- add degrees intrinsi

[HLSL] Implement the `degrees` intrinsic (#111209)

- add degrees builtin
- link degrees api in hlsl_intrinsics.h
- add degrees intrinsic to IntrinsicsDirectX.td
- add degrees intrinsic to IntrinsicsSPIRV.td
- add lowering from clang builtin to dx/spv intrinsics in CGBuiltin.cpp
- add semantic checks to SemaHLSL.cpp
- add expansion of directx intrinsic to llvm fmul for DirectX in
DXILIntrinsicExpansion.cpp
- add mapping to spir-v intrinsic in SPIRVInstructionSelector.cpp

- add test coverage:
- degrees.hlsl -> check hlsl lowering to dx/spv degrees intrinsics
- degrees-errors.hlsl/half-float-only-errors -> check semantic warnings
- hlsl-intrinsics/degrees.ll -> check lowering of spir-v degrees
intrinsic to SPIR-V backend
- DirectX/degrees.ll -> check expansion and scalarization of directx
degrees intrinsic to fmul

Resolves #99104

show more ...

[SPIRV] Implement handle_fromBinding intrinsic. (#111052)

Implement the intrinsic `llvm.spv.handle.fromBinding`, which returns the
handle for a global resource. This involves creating a global vari

[SPIRV] Implement handle_fromBinding intrinsic. (#111052)

Implement the intrinsic `llvm.spv.handle.fromBinding`, which returns the
handle for a global resource. This involves creating a global variable
that matches the return-type, set, and binding in the call, and
returning the handle to that resource.

This commit implements the scalar version. It does not handle arrays of
resources yet. It also does not handle storage buffers yet. We do not
have the type for the storage buffers designed yet.

Part of #81036

show more ...

Reapply "[SPIRV] Add radians intrinsic"

I had too many tabs open and reverted #110800 by mistake, it was
supposed to be #110616.

This reverts commit dec6fe3de0f6475ea83391e5b0b4036cf56db35b.

Revert "[SPIRV] Add radians intrinsic" (#111398)

Reverts llvm/llvm-project#110800

`llvm\test\CodeGen\DirectX\radians.ll` is failing after this change.
@adam-yang please send a new PR with the is

Revert "[SPIRV] Add radians intrinsic" (#111398)

Reverts llvm/llvm-project#110800

`llvm\test\CodeGen\DirectX\radians.ll` is failing after this change.
@adam-yang please send a new PR with the issue resolved once you've had
time to investigate.

show more ...