[Driver][SYCL] Add initial SYCL offload compilation support (#117268)

Introduces the SYCL based toolchain and initial toolchain construction
when using the '-fsycl' option. This option will enable

[Driver][SYCL] Add initial SYCL offload compilation support (#117268)

Introduces the SYCL based toolchain and initial toolchain construction
when using the '-fsycl' option. This option will enable SYCL based
offloading, creating a SPIR-V based IR file packaged into the compiled
host object.

This includes early support for creating the host/device object using
the new offloading model. The device object is created using the
spir64-unknown-unknown target triple.

New/Updated Options:
-fsycl Enables SYCL offloading for host and device
-fsycl-device-only
Enables device only compilation for SYCL
-fsycl-host-only
Enables host only compilation for SYCL

RFC Reference:
https://discourse.llvm.org/t/rfc-sycl-driver-enhancements/74092

This is a reland of: https://github.com/llvm/llvm-project/pull/107493

show more ...

Revert "[Driver][SYCL] Add initial SYCL offload compilation support" (#116381)

Reverts llvm/llvm-project#107493

Failing bots include:
https://lab.llvm.org/buildbot/#/builders/190/builds/9546
ht

Revert "[Driver][SYCL] Add initial SYCL offload compilation support" (#116381)

Reverts llvm/llvm-project#107493

Failing bots include:
https://lab.llvm.org/buildbot/#/builders/190/builds/9546
https://lab.llvm.org/buildbot/#/builders/46/builds/7938

show more ...

[Driver][SYCL] Add initial SYCL offload compilation support (#107493)

Introduces the SYCL based toolchain and initial toolchain construction
when using the '-fsycl' option. This option will enable

[Driver][SYCL] Add initial SYCL offload compilation support (#107493)

Introduces the SYCL based toolchain and initial toolchain construction
when using the '-fsycl' option. This option will enable SYCL based
offloading, creating a SPIR-V based IR file packaged into the compiled
host object.

This includes early support for creating the host/device object using
the new offloading model. The device object is created using the
spir64-unknown-unknown target triple.

New/Updated Options:
-fsycl Enables SYCL offloading for host and device
-fsycl-device-only
Enables device only compilation for SYCL
-fsycl-host-only
Enables host only compilation for SYCL

RFC Reference:
https://discourse.llvm.org/t/rfc-sycl-driver-enhancements/74092

show more ...

[InstallAPI] Set InstallAPI as a standalone tool instead of CC1 action (#82293)

Installapi has important distinctions when compared to the clang driver,
so much that, it doesn't make much sense to

[InstallAPI] Set InstallAPI as a standalone tool instead of CC1 action (#82293)

Installapi has important distinctions when compared to the clang driver,
so much that, it doesn't make much sense to try to integrate into it.

This patch partially reverts the CC1 action & driver support to replace
with its own driver as a clang tool.

For distribution, we could use `LLVM_TOOL_LLVM_DRIVER_BUILD` mechanism
for integrating the functionality into clang such that the toolchain
size is less impacted.

show more ...

[clang][InstallAPI] Introduce basic driver to write out tbd files (#81571)

This introduces a basic outline of installapi as a clang driver option.
It captures relevant information as cc1 args, whic

[clang][InstallAPI] Introduce basic driver to write out tbd files (#81571)

This introduces a basic outline of installapi as a clang driver option.
It captures relevant information as cc1 args, which are common arguments
already passed to the linker to encode into TBD file outputs. This is
effectively an upstream for what already exists as `tapi installapi` in
Xcode toolchains, but directly in Clang. This patch does not handle any
AST traversing on input yet.

InstallAPI is broadly an operation that takes a series of header files
that represent a single dynamic library and generates a TBD file out of
it which represents all the linkable symbols and necessary attributes
for statically linking in clients. It is the linkable object in all
Apple SDKs and when building dylibs in Xcode. `clang -installapi` also
will support verification where it compares all the information recorded
for the TBD files against the already built binary, to catch possible
mismatches like when a declaration is missing a definition for an
exported symbol.

show more ...

[HLSL] [Dirver] add dxv as a VerifyDebug Job

New option --dxv-path is added for dxc mode to set the installation path for dxv.
If cannot find dxv, a warning will be report.

dxv will be executed wi

[HLSL] [Dirver] add dxv as a VerifyDebug Job

New option --dxv-path is added for dxc mode to set the installation path for dxv.
If cannot find dxv, a warning will be report.

dxv will be executed with command line dxv file_name -o file_name.
It will validate and sign the file and overwrite it.

Differential Revision: https://reviews.llvm.org/D141705

show more ...

[OpenMP] Adjust phases for AMDGPU offloading for OpenMP in save-temps mode

Currently, the behaviour of `-save-temps` changes the generated output
when offloading to AMDGPU. This is because we only h

[OpenMP] Adjust phases for AMDGPU offloading for OpenMP in save-temps mode

Currently, the behaviour of `-save-temps` changes the generated output
when offloading to AMDGPU. This is because we only have a single phase
and it contains the `-disable-llvm-passes` flags which results in
unoptimized bitcode. We need to make sure we generate another phase that
produces both the optimized and unoptimized bitcode. There used to be a
check that turned these phases into a no-op. But I believe it is more
correct to not generate them this way in the first place. Doing this
requires a bit of a hack, replacing an already generated phase action,
but it should be fine.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D141440

show more ...

[C++20] [Modules] Remove unmaintained Header Module

Currently there is a -emit-header-module mode, which can combine several
headers together as a module interface. However, this breaks our
assumpti

[C++20] [Modules] Remove unmaintained Header Module

Currently there is a -emit-header-module mode, which can combine several
headers together as a module interface. However, this breaks our
assumption (for standard c++ modules) about module interface. The module
interface should come from a module interface unit. And if it is a
header, it should be a header unit. And currently we have no ideas to
combine several headers together.

So I think this mode is an experimental one and it is not maintained and
it is not used. So it will be better to remove them.

Reviewed By: Bigcheese, dblaikie, bruno

Differential Revision: https://reviews.llvm.org/D137609

show more ...

[CUDA][HIP] Fix new driver crashing when using -save-temps in RDC-mode

Previously when using the `clang-offload-packager` we did not pass the
active offloading kinds. Then in Clang when we attempted

[CUDA][HIP] Fix new driver crashing when using -save-temps in RDC-mode

Previously when using the `clang-offload-packager` we did not pass the
active offloading kinds. Then in Clang when we attempted to detect when
there was host-offloading action that needed to be embedded in the host
we did not find it. This patch adds the active offloading kinds so we
know when there is input to be embedded.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D134189

show more ...

Use llvm::all_equal (NFC)

[OpenMP] Deprecate the old driver for OpenMP offloading

Recently OpenMP has transitioned to using the "new" driver which
primarily merges the device and host linking phases into a single
wrapper tha

[OpenMP] Deprecate the old driver for OpenMP offloading

Recently OpenMP has transitioned to using the "new" driver which
primarily merges the device and host linking phases into a single
wrapper that handles both at the same time. This replaced a few tools
that were only used for OpenMP offloading, such as the
`clang-offload-wrapper` and `clang-nvlink-wrapper`. The new driver
carries some marked benefits compared to the old driver that is now
being deprecated. Things like device-side LTO, static library
support, and more compatible tooling. As such, we should be able to
completely deprecate the old driver, at least for OpenMP. The old driver
support will still exist for CUDA and HIP, although both of these can
currently be compiled on Linux with `--offload-new-driver` to use the new
method.

Note that this does not deprecate the `clang-offload-bundler`, although
it is unused by OpenMP now, it is still used by the HIP toolchain both
as their device binary format and object format.

When I proposed deprecating this code I heard some vendors voice
concernes about needing to update their code in their fork. They should
be able to just revert this commit if it lands.

Reviewed By: jdoerfert, MaskRay, ye-luo

Differential Revision: https://reviews.llvm.org/D130020

show more ...

[Clang] Introduce clang-offload-packager tool to bundle device files

In order to do offloading compilation we need to embed files into the
host and create fatbainaries. Clang uses a special binary f

[Clang] Introduce clang-offload-packager tool to bundle device files

In order to do offloading compilation we need to embed files into the
host and create fatbainaries. Clang uses a special binary format to
bundle several files along with their metadata into a single binary
image. This is currently performed using the `-fembed-offload-binary`
option. However this is not very extensibile since it requires changing
the command flag every time we want to add something and makes optional
arguments difficult. This patch introduces a new tool called
`clang-offload-packager` that behaves similarly to CUDA's `fatbinary`.
This tool takes several input files with metadata and embeds it into a
single image that can then be embedded in the host.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D125165

show more ...

[OpenMP] Make clang argument handling for the new driver more generic

In preparation for accepting other offloading kinds with the new driver,
this patch makes the way we handle offloading actions m

[OpenMP] Make clang argument handling for the new driver more generic

In preparation for accepting other offloading kinds with the new driver,
this patch makes the way we handle offloading actions more generic. A
new field to get the associated device action's toolchain is used rather
than manually iterating a list. This makes building the arguments easier
and makes sure that we doin't rely on any implicit ordering.

Reviewed By: yaxunl

Differential Revision: https://reviews.llvm.org/D123313

show more ...

Ensure -extract-api handles multiple headers correctly

clang -extract-api should accept multiple headers and forward them to a
single CC1 instance. This change introduces a new ExtractAPIJobAction.

Ensure -extract-api handles multiple headers correctly

clang -extract-api should accept multiple headers and forward them to a
single CC1 instance. This change introduces a new ExtractAPIJobAction.
Currently API Extraction is done during the Precompile phase as this is
the current phase that matches the requirements the most. Adding a new
phase would need to change some logic in how phases are scheduled. If
the headers scheduled for API extraction are of different types the
driver emits a diagnostic.

Differential Revision: https://reviews.llvm.org/D121936

show more ...

[Clang] Introduce Clang Linker Wrapper Tool

This patch introduces a linker wrapper tool that allows us to preprocess
files before they are sent to the linker. This adds a dummy action and
job to the

[Clang] Introduce Clang Linker Wrapper Tool

This patch introduces a linker wrapper tool that allows us to preprocess
files before they are sent to the linker. This adds a dummy action and
job to the driver stage that builds the linker command as usual and then
replaces the command line with the wrapper tool.

Depends on D116543

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D116544

show more ...

[CUDA][HIP] Add -fuse-cuid

This patch added a distinct CUID for each input file, which is represented by InputAction.
clang initially creates an InputAction for each input file for the host compilat

[CUDA][HIP] Add -fuse-cuid

This patch added a distinct CUID for each input file, which is represented by InputAction.
clang initially creates an InputAction for each input file for the host compilation. In CUDA/HIP action
builder, each InputAction is given a CUID and cloned for each GPU arch, and the CUID is also cloned. In this way,
we guarantee the corresponding device and host compilation for the same file shared the
same CUID. On the other hand, different compilation units have different CUID.

-fuse-cuid=random|hash|none is added to control the method to generate CUID. The default
is hash. -cuid=X is also added to specify CUID explicitly, which overrides -fuse-cuid.

Reviewed by: Artem Belevich

Differential Revision: https://reviews.llvm.org/D95007

show more ...

Add Statically Linked Libraries

Add GNU Static Lib Tool, which supports the --emit-static-lib
flag. For HIP, a static library archive will be created and
consist of HIP Fat Binary host object with t

Add Statically Linked Libraries

Add GNU Static Lib Tool, which supports the --emit-static-lib
flag. For HIP, a static library archive will be created and
consist of HIP Fat Binary host object with the device images embedded.
Using llvm-ar to create the static archive. Also, delete existing
output file to ensure a new archive is created each time.

Reviewers: yaxunl, tra, rjmccall, echristo

Subscribers: echristo, JonChesterfield, scchan, msearles

Differential Revision: https://reviews.llvm.org/D78759

show more ...

[Clang][OpenMP Offload] Add new tool for wrapping offload device binaries

This patch removes the remaining part of the OpenMP offload linker scripts which was used for inserting device binaries into

[Clang][OpenMP Offload] Add new tool for wrapping offload device binaries

This patch removes the remaining part of the OpenMP offload linker scripts which was used for inserting device binaries into the output linked binary. Device binaries are now inserted into the host binary with a help of the wrapper bit-code file which contains device binaries as data. Wrapper bit-code file is dynamically created by the clang driver with a help of new tool clang-offload-wrapper which takes device binaries as input and produces bit-code file with required contents. Wrapper bit-code is then compiled to an object and resulting object is appended to the host linking by the clang driver.

This is the second part of the patch for eliminating OpenMP linker script (please see https://reviews.llvm.org/D64943).

Differential Revision: https://reviews.llvm.org/D68166

llvm-svn: 374219

show more ...

[clang][ifs] Clang Interface Stubs ToolChain plumbing.

Second Landing Attempt:

This patch enables end to end support for generating ELF interface stubs
directly from clang. Now the following:

clan

[clang][ifs] Clang Interface Stubs ToolChain plumbing.

Second Landing Attempt:

This patch enables end to end support for generating ELF interface stubs
directly from clang. Now the following:

clang -emit-interface-stubs -o libfoo.so a.cpp b.cpp c.cpp

will product an ELF binary with visible symbols populated. Visibility attributes
and -fvisibility can be used to control what gets populated.

* Adding ToolChain support for clang Driver IFS Merge Phase
* Implementing a default InterfaceStubs Merge clang Tool, used by ToolChain
* Adds support for the clang Driver to involve llvm-ifs on ifs files.
* Adds -emit-merged-ifs flag, to tell llvm-ifs to emit a merged ifs text file
instead of the final object format (normally ELF)


Differential Revision: https://reviews.llvm.org/D63978

llvm-svn: 374061

show more ...

Revert 373538 and follow-ups 373549 and 373552.

They break tests on (at least) macOS.

llvm-svn: 373556

[clang][ifs] Clang Interface Stubs ToolChain plumbing.

This patch enables end to end support for generating ELF interface stubs
directly from clang. Now the following:

clang -emit-interface-stubs -

[clang][ifs] Clang Interface Stubs ToolChain plumbing.

This patch enables end to end support for generating ELF interface stubs
directly from clang. Now the following:

clang -emit-interface-stubs -o libfoo.so a.cpp b.cpp c.cpp

will product an ELF binary with visible symbols populated. Visibility attributes
and -fvisibility can be used to control what gets populated.

* Adding ToolChain support for clang Driver IFS Merge Phase
* Implementing a default InterfaceStubs Merge clang Tool, used by ToolChain
* Adds support for the clang Driver to involve llvm-ifs on ifs files.
* Adds -emit-merged-ifs flag, to tell llvm-ifs to emit a merged ifs text file
instead of the final object format (normally ELF)


Differential Revision: https://reviews.llvm.org/D63978

llvm-svn: 373538

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[modules] Driver support for precompiling a collection of files as a single
action.

llvm-svn: 342305

[HIP] Fix output file extension

OffloadBundlingJobAction constructor accepts a list of JobAction as inputs.
The host JobAction is the last one. The file type of OffloadBundlingJobAction
should be de

[HIP] Fix output file extension

OffloadBundlingJobAction constructor accepts a list of JobAction as inputs.
The host JobAction is the last one. The file type of OffloadBundlingJobAction
should be determined by the host JobAction (the last one) instead of the first
one.

Since HIP emits LLVM bitcode for device compilation, device JobAction has
different file type as host Job Action. This bug causes incorrect output file
extension for HIP.

This patch fixes it by using the last input JobAction (host JobAction) to determine
file type of OffloadBundlingJobAction.

Differential Revision: https://reviews.llvm.org/D51336

llvm-svn: 340873

show more ...

[HIP] Add hip offload kind

There are quite differences in HIP action builder and action job creation,
which justifies to define a separate offload kind.

Differential Revision: https://reviews.llvm.

[HIP] Add hip offload kind

There are quite differences in HIP action builder and action job creation,
which justifies to define a separate offload kind.

Differential Revision: https://reviews.llvm.org/D46471

llvm-svn: 331811

show more ...