[Object] Remove unused includes (NFC) (#116750)

Identified with misc-include-cleaner.

[llvm] Use StringRef::operator== instead of StringRef::equals (NFC) (#91441)

I'm planning to remove StringRef::equals in favor of
StringRef::operator==.

- StringRef::operator==/!= outnumber Stri

[llvm] Use StringRef::operator== instead of StringRef::equals (NFC) (#91441)

I'm planning to remove StringRef::equals in favor of
StringRef::operator==.

- StringRef::operator==/!= outnumber StringRef::equals by a factor of
70 under llvm/ in terms of their usage.

- The elimination of StringRef::equals brings StringRef closer to
std::string_view, which has operator== but not equals.

- S == "foo" is more readable than S.equals("foo"), especially for
!Long.Expression.equals("str") vs Long.Expression != "str".

show more ...

[Object] Ensure header size not to underflow in `OffloadBinary::create`

Prevent potential integer underflows when header size is not valid.

Fixes: https://github.com/llvm/llvm-project/issues/86280.

[LinkerWrapper][NFC] Rename 'all' to 'generic' for architecture agnostic IR

Summary:
A previous patch introduced `all` as a special architecture. I have
decided I do not like this name and have chan

[LinkerWrapper][NFC] Rename 'all' to 'generic' for architecture agnostic IR

Summary:
A previous patch introduced `all` as a special architecture. I have
decided I do not like this name and have changed it to `generic`.

show more ...

[LinkerWrapper] Allow 'all' as a generic bundled architecture (#81193)

Summary:
Currently, the linker wrapper sorts input files into different link
jobs according to their architectures. Here we ass

[LinkerWrapper] Allow 'all' as a generic bundled architecture (#81193)

Summary:
Currently, the linker wrapper sorts input files into different link
jobs according to their architectures. Here we assume each architecture
is a unique and incompatible link job unless they are specifically
marked compatible. This patch simply adds an `all` target to represent
an architecture that should be linked against every single other
architecture.

This will be useful for modelling generic IR such as the ROCm device
libraries or the NVPTX libdevice.

show more ...

[LinkerWrapper] Support relocatable linking for offloading (#80066)

Summary:
The standard GPU compilation process embeds each intermediate object
file into the host file at the `.llvm.offloading` se

[LinkerWrapper] Support relocatable linking for offloading (#80066)

Summary:
The standard GPU compilation process embeds each intermediate object
file into the host file at the `.llvm.offloading` section so it can be
linked later. We also use a special section called something like
`omp_offloading_entries` to store all the globals that need to be
registered by the runtime. The linker-wrapper's job is to link the
embedded device code stored at this section and then emit code to
register the linked image and the kernels and globals in the offloading
entry section.

One downside to RDC linking is that it can become quite big for very
large projects that wish to make use of static linking. This patch
changes the support for relocatable linking via `-r` to support a kind
of "partial" RDC compilation for offloading languages.

This primarily requires manually editing the embedded data in the
output object file for the relocatable link. We need to rename the
output section to make it distinct from the input sections that will be
merged. We then delete the old embedded object code so it won't be
linked further. We then need to rename the old offloading section so
that it is private to the module. A runtime solution could also be done
to defer entries that don't belong to the given GPU executable, but this
is easier. Note that this does not work with COFF linking, only the ELF
method for handling offloading entries, that could be made to work
similarly.

Given this support, the following compilation path should produce two
distinct images for OpenMP offloading.
```
$ clang foo.c -fopenmp --offload-arch=native -c
$ clang foo.c -lomptarget.devicertl --offload-link -r -o merged.o
$ clang main.c merged.o -fopenmp --offload-arch=native
$ ./a.out
```

Or similarly for HIP to effectively perform non-RDC mode compilation for
a subset of files.

```
$ clang -x hip foo.c --offload-arch=native --offload-new-driver -fgpu-rdc -c
$ clang -x hip foo.c -lomptarget.devicertl --offload-link -r -o merged.o
$ clang -x hip main.c merged.o --offload-arch=native --offload-new-driver -fgpu-rdc
$ ./a.out
```

One question is whether or not this should be the default behavior of
`-r` when run through the linker-wrapper or a special option. Standard
`-r` behavior is still possible if used without invoking the
linker-wrapper and it guaranteed to be correct.

show more ...

[LinkerWrapper] Handle AMDGPU Target-IDs correctly when linking (#78359)

Summary:
The linker wrapper's job is to sort various embedded inputs into a list
of files that participate in a single link j

[LinkerWrapper] Handle AMDGPU Target-IDs correctly when linking (#78359)

Summary:
The linker wrapper's job is to sort various embedded inputs into a list
of files that participate in a single link job. So far, this has been
completely 1-to-1, that is, each input file participates in exactly one
link job. However, support for AMD's target-id requires that one input
file may participate in multiple link jobs. For example, if given a
`gfx90a` static library and a `gfx90a:xnack+` object file input, we
should link the gfx90a` target into the `gfx90a:xnack+` job. These are
considered separate CPUs that can be mutually linked more or less.

This patch adds the necessary logic to make this happen. It primarily
reworks the logic to copy relevant input files into a separate list. So,
it moves construction of the final list of link jobs into the extraction
phase. We also need to copy the files in the case that it is needed more
than once, as the entire workflow expects ownership of said file.

show more ...

[Object] Change OffloadBinary::write to return SmallString<0>

SmallString<0> is more flexible and avoids an unneeded copy in
ObjectYAML/OffloadEmitter.cpp.

Reviewed By: jhuber6

Differential Revisi

[Object] Change OffloadBinary::write to return SmallString<0>

SmallString<0> is more flexible and avoids an unneeded copy in
ObjectYAML/OffloadEmitter.cpp.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D159335

show more ...

OffloadBinary: Switch to MapVector<StringRef, StringRef> to stabilize iteration order

D122069 incorrectly uses StringMap iteration order
(https://llvm.org/docs/ProgrammersManual.html#llvm-adt-string

OffloadBinary: Switch to MapVector<StringRef, StringRef> to stabilize iteration order

D122069 incorrectly uses StringMap iteration order
(https://llvm.org/docs/ProgrammersManual.html#llvm-adt-stringmap-h).
Switch to MapVector.

show more ...

[Binary] Support extracting offloading files from COFF

This patch adds initial support for extracting offloading binaries from
`COFF` objects. This is a first step to allow building offloading files

[Binary] Support extracting offloading files from COFF

This patch adds initial support for extracting offloading binaries from
`COFF` objects. This is a first step to allow building offloading files
on Windows targets with the new driver.

Depends on D136796

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D136855

show more ...

[llvm-objdump][Offload] Use common offload extraction method

A previous patch introduced a common function used to extract offloading
binaries from an image. Therefore we no longer need to duplicate

[llvm-objdump][Offload] Use common offload extraction method

A previous patch introduced a common function used to extract offloading
binaries from an image. Therefore we no longer need to duplicate the
functionality in the `llvm-objdump` implementation. Functionally, this
removes the old warning behaviour when given malformed input. This has
been changed to a hard error, which is effectively the same.

This required a slight tweak in the linker wrapper to filter out the
user passing shared objects directly.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D136796

show more ...

[Object] Refactor code for extracting offload binaries

We currently extract offload binaries inside of the linker wrapper.
Other tools may wish to do the same extraction operation. This patch
simply

[Object] Refactor code for extracting offload binaries

We currently extract offload binaries inside of the linker wrapper.
Other tools may wish to do the same extraction operation. This patch
simply factors out this handling into the `OffloadBinary.h` interface.

Reviewed By: yaxunl

Differential Revision: https://reviews.llvm.org/D132689

show more ...

[llvm-objdump] Ensure offloading sections have proper alignment

Summary:
A previous patch added support for dumping offloading sections. The
tests for this feature added dummy input to the required

[llvm-objdump] Ensure offloading sections have proper alignment

Summary:
A previous patch added support for dumping offloading sections. The
tests for this feature added dummy input to the required section using
`llvm-objcopy`. This binary format has a required alignment of `8` which
was not being respected by the file copied with llvm-objcopy and would
cause failures on architectures sensitive to alignment problems or with
sanitizers. This patch adds the proper alignemnt and adds an error check
at least for the binary format so it's not completely opaque. This
should be improvbed so users actually get a helpful message.

show more ...

[Binary] Further improve malformed input handling for the OffloadBinary

Summary:
This patch adds some new sanity checks to make sure that the sizes of
the offsets are within the bounds of the file o

[Binary] Further improve malformed input handling for the OffloadBinary

Summary:
This patch adds some new sanity checks to make sure that the sizes of
the offsets are within the bounds of the file or what is expected by the
binary. This also improves the error handling of the version structure
to be built into the binary itself so we can change it easier.

show more ...

[Binary] Fix leftoever line

[Binary] Reserve the correct size for the OffloadBinary

Summary:
When writing the offload binary, we use a SmallVector. We already know
the size that we expect the buffer to take up so we should res

[Binary] Reserve the correct size for the OffloadBinary

Summary:
When writing the offload binary, we use a SmallVector. We already know
the size that we expect the buffer to take up so we should reserve all
that memory up-front to improve performance. Also this patch adds some
extra sanity checks for the binary format for safety.

show more ...

[Binary] Make the OffloadingImage type own the memory

Summary:
The OffloadingBinary uses a convenience struct to help manage the memory
that will be serialized using the binary format. This currentl

[Binary] Make the OffloadingImage type own the memory

Summary:
The OffloadingBinary uses a convenience struct to help manage the memory
that will be serialized using the binary format. This currently uses a
reference to an existing buffer, but this should own the memory instead
so it is easier to work with seeing as its only current use requires
saving the buffer anyway.

show more ...

[Binary] Align the image offset in OffloadBinary

Summary:
The OffloadBinary wraps around an embedded device image, commonly an ELF
or LLVM BC file. These file formats have alignment requirements for

[Binary] Align the image offset in OffloadBinary

Summary:
The OffloadBinary wraps around an embedded device image, commonly an ELF
or LLVM BC file. These file formats have alignment requirements for
parsing, so if the image is stored at an un-aligned offset from the
beginning of the file we will be unable to parse the embeded image
without copying the image buffer. This patch adds alignment padding
before the binary image is appended to ensure we can parse the symbolic
file it contains in-place without copying memory.

show more ...

[Object] Fix namespace style issues in D122069

https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

[Binary] Promote OffloadBinary to inherit from Binary

We use the `OffloadBinary` to create binary images of offloading files
and their corresonding metadata. This patch changes this to inherit from

[Binary] Promote OffloadBinary to inherit from Binary

We use the `OffloadBinary` to create binary images of offloading files
and their corresonding metadata. This patch changes this to inherit from
the base `Binary` class. This allows us to create and insepect these
more generically. This patch includes all the necessary glue to
implement this as a new binary format, along with added the magic bytes
we use to distinguish the offloading binary to the `file_magic`
implementation.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D126812

show more ...

[Object] Add binary format for bundling offloading metadata

We need to embed certain metadata along with a binary image when we wish
to perform a device-linking job on it. Currently this metadata wa

[Object] Add binary format for bundling offloading metadata

We need to embed certain metadata along with a binary image when we wish
to perform a device-linking job on it. Currently this metadata was
embedded in the section name of the data itself. This worked, but made
adding new metadata very difficult and didn't work if the user did any
sort of section linking.

This patch introduces a custom binary format for bundling offloading
metadata with a device object file. This binary format is fundamentally
a simple string map table with some additional data and an embedded
image. I decided to use a custom format rather than using an existing
format (ELF, JSON, etc) because of the specialty use-case of this. We
need a simple binary format that can be concatenated without requiring
other external dependencies.

This extension will make it easier to extend the linker wrapper's
capabilties with whatever data is necessary. Eventually this will allow
us to remove all the external arguments passed to the linker wrapper and
embed it directly in the host's linker so device linking behaves exactly
like host linking.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D122069

show more ...