"Reapply "[llvm-jitlink] Use concurrent linking by default." with more fixes.

This reapplies edca1d9bad2 which was reverted in 7ec139ad4bc due to bot
failures.

LocalDependencyPropagation.s is updat

"Reapply "[llvm-jitlink] Use concurrent linking by default." with more fixes.

This reapplies edca1d9bad2 which was reverted in 7ec139ad4bc due to bot
failures.

LocalDependencyPropagation.s is updated to use -num-threads=0 in order to
avoid interleaving debugging output.

ELFNixPlatform.h is updated to protect the deferred runtime function calls
map during bootstrap.

show more ...

Revert "Reapply "[llvm-jitlink] Use concurrent linking by default." with fixes. (#120958)"

Caused random failures.

This reverts commit 93d4b1f7a72f366c1ea91b2d65991266053be8d9.
(llvmorg-20-init-162

Revert "Reapply "[llvm-jitlink] Use concurrent linking by default." with fixes. (#120958)"

Caused random failures.

This reverts commit 93d4b1f7a72f366c1ea91b2d65991266053be8d9.
(llvmorg-20-init-16299-g93d4b1f7a72f)

show more ...

Reapply "[llvm-jitlink] Use concurrent linking by default." with fixes. (#120958)

Reapplies commit edca1d9bad2 which was reverted in 34531cff638 while I
investigated bot failures, (e.g.
https://la

Reapply "[llvm-jitlink] Use concurrent linking by default." with fixes. (#120958)

Reapplies commit edca1d9bad2 which was reverted in 34531cff638 while I
investigated bot failures, (e.g.
https://lab.llvm.org/buildbot/#/builders/137/builds/10791).

Commit 158a60051d2 should address the -check failures on the bots, which
were caused by checks running earlier under the concurrent linking
scheme before all files referenced by the checks had been fully linked.
This patch also fixes the -threads option failure by renaming the option
to -num-threads to avoid clashing with the ThreadCount cl::opt variable
defined in ThinLTOCodeGenerator.cpp.

show more ...

Revert "[llvm-jitlink] Use concurrent linking by default."

This reverts commit edca1d9bad2 while I investigate bot failures,
e.g. https://lab.llvm.org/buildbot/#/builders/137/builds/10791.

[llvm-jitlink] Use concurrent linking by default.

Adds a -threads option to llvm-jitlink. By default llvm-jitlink will now use a
DynamicThreadPoolTaskDispatcher with the number of materialization th

[llvm-jitlink] Use concurrent linking by default.

Adds a -threads option to llvm-jitlink. By default llvm-jitlink will now use a
DynamicThreadPoolTaskDispatcher with the number of materialization threads set
to whatever is returned by std::hardware_concurrency(). This brings the default
in-place linking behavior in line with the concurrent linking that is used for
-oop-executor and -oop-executor-connect mode.

In-place linking on the main thread can be forced by passing -threads=0.

show more ...

[llvm-jitlink] Allow optional stub-kind filter in stub_addr() expressions (#78369)

We use `jitlink-check` lines in LIT tests as the primary tool for
testing JITLink backends. Parsing and evaluation

[llvm-jitlink] Allow optional stub-kind filter in stub_addr() expressions (#78369)

We use `jitlink-check` lines in LIT tests as the primary tool for
testing JITLink backends. Parsing and evaluation of the expressions is
implemented in `RuntimeDyldChecker`. The `stub_addr(obj, name)`
expression allows to obtain the linker-generated stub for the external
symbol `name` in object file `obj`.

This patch adds support for a filter parameter to select one out of many
stubs. This is necessary for the AArch32 JITLink backend, which must be
able to emit two different kinds of stubs depending on the instruction
set state (Arm/Thumb) of the relocation site. Since the new parameter is
optional, we don't have to update existing tests.

Filters are regular expressions without brackets that match exactly one
existing stub. Given object file `armv7.o` with two stubs for external
function `ext` of kinds `armv7_abs_le` and `thumbv7_abs_le`, we get the
following filter results e.g.:
```
stub_addr(armv7.o, ext, thumb) thumbv7_abs_le
stub_addr(armv7.o, ext, thumbv7) thumbv7_abs_le
stub_addr(armv7.o, ext, armv7_abs_le) armv7_abs_le
stub_addr(armv7.o, ext, v7_.*_le) Error: "ext" has 2 candidate stubs in file "armv7.o". Please refine stub-kind filter "v7_.*_le" for disambiguation (encountered kinds are "thumbv7_abs_le", "armv7_abs_le").
stub_addr(armv7.o, ext, v8) Error: "ext" has 2 stubs in file "armv7.o", but none of them matches the stub-kind filter "v8" (all encountered kinds are "thumbv7_abs_le", "armv7_abs_le").
```

show more ...

[llvm-jitlink] Fix MSVC "not all control paths return a value" warning. NFC.

[llvm-jitlink] Refactor GOT and stubs registration (NFC) (#78365)

Add methods `registerGOTEntry()` and `registerStubEntry()` in
`Session::FileInfo` to factor out generic code from the individual
o

[llvm-jitlink] Refactor GOT and stubs registration (NFC) (#78365)

Add methods `registerGOTEntry()` and `registerStubEntry()` in
`Session::FileInfo` to factor out generic code from the individual
object type implementations.

show more ...

[llvm-jitlink] Support plain AArch32 stubs in jitlink-check's stub_addr() expressions (#73268)

We want to use regular `stub_addr()` expressions in `jitlink-check` lines to test the generation of stu

[llvm-jitlink] Support plain AArch32 stubs in jitlink-check's stub_addr() expressions (#73268)

We want to use regular `stub_addr()` expressions in `jitlink-check` lines to test the generation of stubs in AArch32, but we don't want this to require a standardized GOT-based PLT implementation. In terms of performance and binary size it doesn't seem beneficial. And in terms of patching branch targets, we should be able to handle range-extension- and interworking-stubs without a lot of extra logic.

In order to allow such AArch32 stubs we add a separate path for `stub_addr()` expressions in `llvm-jitlink-elf`. The relocations in our stubs are not pointing to the GOT, but to the external symbol directly. Thus, we have to avoid access to the block of the edge target. Instead we only return the symbol name, which is enough to use `stub_addr()` expressions in tests.

The name of the AArch32 stubs section differs from the conventional `$__STUBS` on purpose. It allows to add a regular PLT/GOT implementation as an orthogonal feature in the future. In order to also allow decoding of stub target addresses in the future, we mention the stub flavor in the section name as well.

show more ...

[llvm-jitlink] Avoid assertion failure in make_error parameter

If GOTSym is not defined, we cannot call `GOTSym.getBlock()`. It failed with:
```
Assertion failed: (Base->isDefined() && "Not a define

[llvm-jitlink] Avoid assertion failure in make_error parameter

If GOTSym is not defined, we cannot call `GOTSym.getBlock()`. It failed with:
```
Assertion failed: (Base->isDefined() && "Not a defined symbol"), function getBlock, file JITLink.h, line 554.
```

show more ...

[jitlink][rtdyld][checker] Re-apply 4b17c81d5a5 with fixes.

This re-applies 4b17c81d5a5, "[jitlink/rtdydl][checker] Add TargetFlag
dependent disassembler switching support", which was reverted in
48

[jitlink][rtdyld][checker] Re-apply 4b17c81d5a5 with fixes.

This re-applies 4b17c81d5a5, "[jitlink/rtdydl][checker] Add TargetFlag
dependent disassembler switching support", which was reverted in
4871a9ca546 due to bot failures.

The patch has been updated to add missing plumbing for Subtarget Features and
a CPU string, which should fix the failing tests.

https://reviews.llvm.org/D158280

show more ...

Revert "[jitlink/rtdydl][checker] Add TargetFlag dependent disassembler switching support"

This reverts commit 4b17c81d5a5d3e0f514026c2b7f9b623d901cc04.

Caused buildbot failures:
https://lab.llvm.o

Revert "[jitlink/rtdydl][checker] Add TargetFlag dependent disassembler switching support"

This reverts commit 4b17c81d5a5d3e0f514026c2b7f9b623d901cc04.

Caused buildbot failures:
https://lab.llvm.org/buildbot/#/builders/230/builds/18341
https://lab.llvm.org/buildbot/#/builders/109/builds/73169
https://lab.llvm.org/buildbot/#/builders/67/builds/12597

show more ...

[jitlink/rtdydl][checker] Add TargetFlag dependent disassembler switching support

Some targets such as AArch32 make use of TargetFlags to indicate ISA mode. Depending
on the TargetFlag, MCDisassembl

[jitlink/rtdydl][checker] Add TargetFlag dependent disassembler switching support

Some targets such as AArch32 make use of TargetFlags to indicate ISA mode. Depending
on the TargetFlag, MCDisassembler and similar target specific objects should be
reinitialized with the correct Target Triple. Backends with similar needs can
easily extend this implementation for their usecase.

The drivers llvm-rtdyld and llvm-jitlink have their SymbolInfo's extended to take
TargetFlag into account. RuntimeDyldChecker can now create necessary TargetInfo
to reinitialize MCDisassembler and MCInstPrinter. The required triple is obtained
from the new getTripleFromTargetFlag function by checking the TargetFlag.

In addition, breaking changes for RuntimeDyld COFF Thumb tests are fixed by making
the backend emit a TargetFlag.

Reviewed By: lhames, sgraenitz

Differential Revision: https://reviews.llvm.org/D158280

show more ...

[JITLink][ELF][x86-64] Add support for R_X86_64_GOTPC32 relocation.

Adds support for the R_X86_64_GOTPC32 relocation, which is a 32-bit delta to
the global offset table.

Since the delta to the GOT

[JITLink][ELF][x86-64] Add support for R_X86_64_GOTPC32 relocation.

Adds support for the R_X86_64_GOTPC32 relocation, which is a 32-bit delta to
the global offset table.

Since the delta to the GOT doesn't actually require any GOT entries to exist
this commit adds an extra fallback path to the getOrCreateGOTSymbol function:
If the symbol is in the extenal symbols list but no entry exists then the
symbol is turned into an absolute symbol pointing to an arbitrary address in
the current graph's allocation (accessing this address via the symbol would be
illegal, but any access should have triggered creation of a GOT entry which
would prevent this fallback path from being taken in the first place).

This commit also updates the llvm-jitlink tool to scrape the addresses of the
absolute symbols in the graph so that the testcase can see the now-absolute
_GLOBAL_OFFSET_TABLE_ symbol.

show more ...

[llvm] Use x.empty() instead of llvm::empty(x) (NFC)

I'm planning to deprecate and eventually remove llvm::empty.

I thought about replacing llvm::empty(x) with std::empty(x), but it
turns out that

[llvm] Use x.empty() instead of llvm::empty(x) (NFC)

I'm planning to deprecate and eventually remove llvm::empty.

I thought about replacing llvm::empty(x) with std::empty(x), but it
turns out that all uses can be converted to x.empty(). That is, no
use requires the ability of std::empty to accept C arrays and
std::initializer_list.

Differential Revision: https://reviews.llvm.org/D133677

show more ...

[llvm] Use llvm::find_if (NFC)

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.

show more ...

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbo

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.

show more ...

[JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.

ExecutorAddr is the preferred representation for executor process addresses now.

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builde

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builder, but may also be used by pass writers who
want to create pointer stubs for indirection.

This patch also switches the underlying type for LinkGraph content from
StringRef to ArrayRef<char>. This avoids any confusion when working with buffers
that contain null bytes in the middle like, for example, a newly added null
pointer content array. ;)

show more ...

[JITLink][ELF/x86-64] Add support for GOTOFF64 relocation.

[JITLink][ELF/x86-64] Add support for R_X86_64_GOTPC64 and R_X86_64_GOT64.

Start adding support for ELF x86-64 large code model, PIC relocations.

[llvm-jitlink] Add suppport for testing GOT entries and stubs for ELF.

This enables regression testing of GOT and stub handling with
llvm-jitlink.

[JITLink] Improve llvm-jitlink regression testing support for ELF.

This patch adds a jitlink pass, 'registerELFGraphInfo', that records section
and symbol information about each LinkGraph in the llv

[JITLink] Improve llvm-jitlink regression testing support for ELF.

This patch adds a jitlink pass, 'registerELFGraphInfo', that records section
and symbol information about each LinkGraph in the llvm-jitlink session object.
This allows symbols and sections to be referred to by name in llvm-jitlink
regression tests. This will enable a testcase to be written for
https://reviews.llvm.org/D80613.

show more ...