[OpenMP] Replace nvvm.annotation usage with kernel calling conventions (#122320)

Specifying a kernel with the `ptx_kernel` or `amdgpu_kernel` calling
convention is a more idiomatic and compile-time

[OpenMP] Replace nvvm.annotation usage with kernel calling conventions (#122320)

Specifying a kernel with the `ptx_kernel` or `amdgpu_kernel` calling
convention is a more idiomatic and compile-time performant than using
the `nvvm.annoation !"kernel"` metadata.

Transition OMPIRBuilder to use calling conventions for PTX kernels and
no longer emit `nvvm.annoation`. Update OpenMPOpt to work with kernels
specified via calling convention as well as metadata. Update OpenMP
tests to use the calling conventions.

show more ...

[Attributor] Enable `AAAddressSpace` in `OpenMPOpt` (#104363)

This reverts commit e592c2dcf5b7d2da6c2564f5d9990aa34079bad4.

We can finally reland the PR since the issue that caused the PR to be

[Attributor] Enable `AAAddressSpace` in `OpenMPOpt` (#104363)

This reverts commit e592c2dcf5b7d2da6c2564f5d9990aa34079bad4.

We can finally reland the PR since the issue that caused the PR to be
reverted has been resolved in
https://github.com/llvm/llvm-project/pull/104051.

show more ...

[Attributor][NFC] update tests (#91011)

[OpenMP][NFC] Split the reduction buffer size into two components

Before we tracked the size of the teams reduction buffer in order to
allocate it at runtime per kernel launch. This patch splits the

[OpenMP][NFC] Split the reduction buffer size into two components

Before we tracked the size of the teams reduction buffer in order to
allocate it at runtime per kernel launch. This patch splits the number
into two parts, the size of the reduction data (=all reduction
variables) and the (maximal) length of the buffer. This will allow us to
allocate less if we need less, e.g., if we have less teams than the
maximal length. It also allows us to move code from clangs codegen into
the runtime as we now know how large the reduction data is.

show more ...

[OpenMP] Introduce the KernelLaunchEnvironment as implicit argument (#70401)

The KernelEnvironment is for compile time information about a kernel. It
allows the compiler to feed information to the

[OpenMP] Introduce the KernelLaunchEnvironment as implicit argument (#70401)

The KernelEnvironment is for compile time information about a kernel. It
allows the compiler to feed information to the runtime. The
KernelLaunchEnvironment is for dynamic information *per* kernel launch.
It allows the rutime to feed information to the kernel that is not
shared with other invocations of the kernel. The first use case is to
replace the globals that synchronize teams reductions with per-launch
versions. This allows concurrent teams reductions. More uses cases will
follow, e.g., per launch memory pools.

Fixes: https://github.com/llvm/llvm-project/issues/70249

show more ...

Revert "Revert "[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)""

This reverts commit ddbaa11e9f43a38d50d62a9b9b07c3653b6bf8ab.

Reapply the original commit, the br

Revert "Revert "[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)""

This reverts commit ddbaa11e9f43a38d50d62a9b9b07c3653b6bf8ab.

Reapply the original commit, the broken test was repaired in 5e51363f38d083ab326736c0d4d1b5f9fe0de080 in the meantime.

show more ...

Revert "[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)"

This reverts commit c2a1249a8257ed033a98e32e425539c6da6700ec.

The MLIR bots are broken with an omp test fa

Revert "[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)"

This reverts commit c2a1249a8257ed033a98e32e425539c6da6700ec.

The MLIR bots are broken with an omp test failure.

show more ...

[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)

The runtime needs to know about the acceptable launch bounds, especially
if the compiler (middle- or backend) assum

[OpenMP][NFC] Add min/max threads/teams count into the KernelEnvironment (#70257)

The runtime needs to know about the acceptable launch bounds, especially
if the compiler (middle- or backend) assumed those bounds. While this
patch does not yet inform the runtime, it stores the bounds in a place
that can/will be accessed and is associated with the kernel.

show more ...

Revert "Reapply "[Attributor] Enable AAAddressSpace for OpenMPOpt (#65544)"""

This reverts commit c5525a6e8fb7f7c2ce7126ac5b17aaff01ac407f.
AMD BB is not happy again.

Reapply "[Attributor] Enable AAAddressSpace for OpenMPOpt (#65544)""

This reverts commit e592c2dcf5b7d2da6c2564f5d9990aa34079bad4 that
reverts e91e3cf.

Revert "[Attributor] Enable AAAddressSpace for OpenMPOpt (#65544)"

This reverts commit e91e3cf0748a80e1d7219c13fa6a7622321f4936 because
AMD BB is not happy with it.

[Attributor] Enable AAAddressSpace for OpenMPOpt (#65544)

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`. They are accessible on the host by reading the corresponding global variable, but not from the device. Besides, some assumptions, such as no nested parallelism, are not per kernel basis, preventing us applying per kernel optimization in the device runtime.

This is a combination and refinement of patch series D116908, D116909, and D116910.

Depend on D155886.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D142569

show more ...

[OpenMP] Make the nested parallelism global hidden

Summary:
These will probably be removed with the kernel environment, but they
should have hidden visibliity so they can be optimized out.

Revert commits for kernel environment

This reverts commits for kernel environments as they causes issues in AMD BB.

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`. They are accessible on the host by reading the corresponding global variable, but not from the device. Besides, some assumptions, such as no nested parallelism, are not per kernel basis, preventing us applying per kernel optimization in the device runtime.

This is a combination and refinement of patch series D116908, D116909, and D116910.

Depend on D155886.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D142569

show more ...

[OpenMP] Use "kernel" attribute consistently

[Attributor][OpenMP-Opt][NFC] Run the update test checks script

Revert "[OpenMP] Introduce kernel environment"

This reverts commit 35cfadfbe2decd9633560b3046fa6c17523b2fa9.

It makes a couple of buildbots unhappy because of the following test failures:
- `Transf

Revert "[OpenMP] Introduce kernel environment"

This reverts commit 35cfadfbe2decd9633560b3046fa6c17523b2fa9.

It makes a couple of buildbots unhappy because of the following test failures:
- `Transforms/OpenMP/add_attributes.ll'`
- `mapping/declare_mapper_target_data.cpp` on AMDGPU

show more ...

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`

[OpenMP] Introduce kernel environment

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like `__kernel_name_exec_mode`. They are accessible on the host by reading the corresponding global variable, but not from the device. Besides, some assumptions, such as no nested parallelism, are not per kernel basis, preventing us applying per kernel optimization in the device runtime.

This is a combination and refinement of patch series D116908, D116909, and D116910.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D142569

show more ...

OpenMPOpt: Check nested parallelism in target region

Analysis that determines if a parallel region can reach another parallel region in any target region of the TU.
A new global var is emitted with

OpenMPOpt: Check nested parallelism in target region

Analysis that determines if a parallel region can reach another parallel region in any target region of the TU.
A new global var is emitted with the name of the kernel + "_nested_parallelism", which is either 0 or 1 depending on the result.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D141010

show more ...