[JITLink][MachO] Add support for non-subsections-via-symbols objects.

This patch updates MachOLinkGraphBuilder to honor the MH_SUBSECTIONS_VIA_SYMBOLS
flag. Prior to this patch we assumed MH_SUBSECT

[JITLink][MachO] Add support for non-subsections-via-symbols objects.

This patch updates MachOLinkGraphBuilder to honor the MH_SUBSECTIONS_VIA_SYMBOLS
flag. Prior to this patch we assumed MH_SUBSECTIONS_VIA_SYMBOLS, but never
checked the flag.

If MH_SUBSECTIONS_VIA_SYMBOLS is set (the default for MachO output on modern
compilers) then MachOLinkGraphBuilder will break MachO section content into
jitlink::Blocks on symbol boundaries. (This is how JITLink has always handled
MachO sections previously).

If MH_SUBSECTIONS_VIA_SYMBOLS is not set then MachOLinkGraphBuilder will create
a single jitlink::Block for each MachO section.

Existing hand-written testcases that were _not_ using the
.subsections_via_symbols directive are updated to use it. A new testcase for
non-subsections-via-symbols behavior is included.

show more ...

[JITLink][MachO] Fix alignment bug in the c-string literal section graphifier.

This function had been assuming a 1-byte alignment, which isn't always correct.
This commit updates it to take the alig

[JITLink][MachO] Fix alignment bug in the c-string literal section graphifier.

This function had been assuming a 1-byte alignment, which isn't always correct.
This commit updates it to take the alignment from the __cstring section.

The key change is to the createContentBlock call, but the surrounding code is
updated with clearer debugging output to support the testcase (and any future
debugging work).

show more ...

[ORC][JITLink] Fix MachO absolute symbol handling, add test case.

This patch removes the unintended resolution of locally scoped absolute symbols
(which was causing unexpected definition errors).

I

[ORC][JITLink] Fix MachO absolute symbol handling, add test case.

This patch removes the unintended resolution of locally scoped absolute symbols
(which was causing unexpected definition errors).

It stops using the JITSymbolFlags::Absolute flag (it isn't set or used elsewhere,
and causes mismatch-flags asserts), and adds JITSymbolFlags::Exported to default
scoped absolute symbols.

Finally, we now set the scope of absolute symbols correctly in
MachOLinkGraphBuilder.

show more ...

[llvm] Use = default (NFC)

[llvm] Use true/false instead of 1/0 (NFC)

Identified with modernize-use-bool-literals.

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416

Re-apply "[JITLink] Update JITLink to use ExecutorAddr rather... " with fixes.

This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.

The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.

show more ...

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbo

Revert "[JITLink] Update JITLink to use ExecutorAddr rather than..."

This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.

show more ...

[JITLink] Update JITLink to use ExecutorAddr rather than JITTargetAddress.

ExecutorAddr is the preferred representation for executor process addresses now.

[NFC][llvm] Inclusive language: remove uses of sanity in llvm/lib/ExecutionEngine/

Reworded and removed code comments to avoid using `sanity check` and `sanity
test`.

Reapply "[ORC] Initial MachO debugging support (via GDB JIT debug.." with fixes.

This reapplies e1933a0488a50eb939210808fc895d374570d891 (which was reverted in
f55ba3525eb19baed7d3f23638cbbd880246a3

Reapply "[ORC] Initial MachO debugging support (via GDB JIT debug.." with fixes.

This reapplies e1933a0488a50eb939210808fc895d374570d891 (which was reverted in
f55ba3525eb19baed7d3f23638cbbd880246a370 due to bot failures, e.g.
https://lab.llvm.org/buildbot/#/builders/117/builds/2768).

The bot failures were due to a missing symbol error: We use the input object's
mangling to decide how to mangle the debug-info registration function name. This
caused lookup of the registration function to fail when the input object
mangling didn't match the host mangling.

Disbaling the test on non-Darwin platforms is the easiest short-term solution.
I have filed https://llvm.org/PR52503 with a proposed longer term solution.

show more ...

Revert "[ORC] Initial MachO debugging support (via GDB JIT debug..."

This reverts commit e1933a0488a50eb939210808fc895d374570d891 until I can look
into bot failures.

[ORC] Initial MachO debugging support (via GDB JIT debug registration interface)

This commit adds a new plugin, GDBJITDebugInfoRegistrationPlugin, that checks
for objects containing debug info and r

[ORC] Initial MachO debugging support (via GDB JIT debug registration interface)

This commit adds a new plugin, GDBJITDebugInfoRegistrationPlugin, that checks
for objects containing debug info and registers any debug info found via the
GDB JIT registration API.

To enable this registration without redundantly representing non-debug sections
this plugin synthesizes a new embedded object within a section of the LinkGraph.
An allocation action is used to make the registration call.

Currently MachO only. ELF users can still use the DebugObjectManagerPlugin. The
two are likely to be merged in the near future.

show more ...

[JITLink][MachO] Fix "find-symbol-by-address" logic.

Only search within the requested section, and allow one-past-then-end addresses.

This is needed to support section-end-address references to sec

[JITLink][MachO] Fix "find-symbol-by-address" logic.

Only search within the requested section, and allow one-past-then-end addresses.

This is needed to support section-end-address references to sections with no
symbols in them.

show more ...

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accide

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.

show more ...

Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."

This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager impl

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.

show more ...

[JITLink][MachO][arm64] Add support for splitting compact-unwind sections.

CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions b

[JITLink][MachO][arm64] Add support for splitting compact-unwind sections.

CompactUnwindSplitter splits compact-unwind sections on record boundaries and
adds keep-alive edges from target functions back to their respective records.

In MachO_arm64.cpp, a CompactUnwindSplitter pass is added to the pre-prune pass
list when setting up the standard pipeline.

This patch does not provide runtime support for compact-unwind, but is a first
step towards enabling it.

show more ...

[JITLink][MachO] Add more detail to error message.

[JITLink][MachO] Detect MachO::S_THREAD_LOCAL_ZEROFILL sections as zero-fill.

This will be used in upcoming MachO native TLV support patches to LLVM and
the ORC runtime.

[JITLink][MachO] Handle muliple symbols at same offset when splitting C-strings.

The C-string section splitting support added in f9649d123db triggered an assert
("Duplicate canonical symbol at addre

[JITLink][MachO] Handle muliple symbols at same offset when splitting C-strings.

The C-string section splitting support added in f9649d123db triggered an assert
("Duplicate canonical symbol at address") when multiple symbols were defined at
the the same offset within a C-string block (this triggered on arm64, where we
always add a block start symbol). The bug was caused by a failure to update the
record of the last canonical symbol address. The fix was to maintain this record
correctly, and move the auto-generation of the block-start symbol above the
handling for symbols defined in the object itself so that all symbols
(auto-generated and defined) are processed in address order.

show more ...

[JITLink][MachO] Split C-string literal sections on null-terminators.

MachO C-string literal sections should be split on null-terminator boundaries,
rather than the usual symbol boundaries. This pat

[JITLink][MachO] Split C-string literal sections on null-terminators.

MachO C-string literal sections should be split on null-terminator boundaries,
rather than the usual symbol boundaries. This patch updates
MachOLinkGraphBuilder to do that.

show more ...

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builde

[JITLink] Switch from StringRef to ArrayRef<char>, add some generic x86-64 utils

Adds utilities for creating anonymous pointers and jump stubs to x86_64.h. These
are used by the GOT and Stubs builder, but may also be used by pass writers who
want to create pointer stubs for indirection.

This patch also switches the underlying type for LinkGraph content from
StringRef to ArrayRef<char>. This avoids any confusion when working with buffers
that contain null bytes in the middle like, for example, a newly added null
pointer content array. ;)

show more ...

[JITLink][MachO] Use full <segment>,<section> names for MachO jitlink::Sections.

JITLink now requires section names to be unique. In MachO section names are only
guaranteed to be unique within their

[JITLink][MachO] Use full <segment>,<section> names for MachO jitlink::Sections.

JITLink now requires section names to be unique. In MachO section names are only
guaranteed to be unique within their containing segment (e.g. a '__const' section
in the '__DATA' segment does not clash with a '__const' section in the '__TEXT'
segment), so we need to use the fully qualified <segment>,<section> section
names (e.g. '__DATA,__const' or '__TEXT,__const') when constructing
jitlink::Sections for MachO objects.

show more ...

[JITLink][MachO][x86-64] Introduce generic x86-64 support.

This patch introduces generic x86-64 edge kinds, and refactors the MachO/x86-64
backend to use these edge kinds. This simplifies the implem

[JITLink][MachO][x86-64] Introduce generic x86-64 support.

This patch introduces generic x86-64 edge kinds, and refactors the MachO/x86-64
backend to use these edge kinds. This simplifies the implementation of the
MachO/x86-64 backend and makes it possible to write generic x86-64 passes and
utilities.

The new edge kinds are different from the original set used in the MachO/x86-64
backend. Several edge kinds that were not meaningfully distinguished in that
backend (e.g. the PCRelMinusN edges) have been merged into single edge kinds in
the new scheme (these edge kinds can be reintroduced later if we find a use for
them). At the same time, new edge kinds have been introduced to convey extra
information about the state of the graph. E.g. The Request*AndTransformTo**
edges represent GOT/TLVP relocations prior to synthesis of the GOT/TLVP
entries, and the 'Relaxable' suffix distinguishes edges that are candidates for
optimization from edges which should be left as-is (e.g. to enable runtime
redirection).

ELF/x86-64 will be refactored to use these generic edges at some point in the
future, and I anticipate a similar refactor to create a generic arm64 support
header too.

Differential Revision: https://reviews.llvm.org/D98305

show more ...

[JITLink][ORC] Enable creation / linking of raw jitlink::LinkGraphs.

Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is

[JITLink][ORC] Enable creation / linking of raw jitlink::LinkGraphs.

Separates link graph creation from linking. This allows raw LinkGraphs to be
created and passed to a link. ObjectLinkingLayer is updated to support emission
of raw LinkGraphs in addition to object buffers.

Raw LinkGraphs can be created by in-memory compilers to bypass object encoding /
decoding (though this prevents caching, as LinkGraphs have do not have an
on-disk representation), and by utility code to add programatically generated
data structures to the JIT target process.

show more ...