[JITLink] Use target triple for LinkGraph pointer size and endianness.

Removes LinkGraph's PointerSize and Endianness members and uses the triple to
find these values instead.

Also removes some red

[JITLink] Use target triple for LinkGraph pointer size and endianness.

Removes LinkGraph's PointerSize and Endianness members and uses the triple to
find these values instead.

Also removes some redundant Triple copies.

show more ...

[JITLink] Switch to SymbolStringPtr for Symbol names (#115796)

Use SymbolStringPtr for Symbol names in LinkGraph. This reduces string interning
on the boundary between JITLink and ORC, and allows p

[JITLink] Switch to SymbolStringPtr for Symbol names (#115796)

Use SymbolStringPtr for Symbol names in LinkGraph. This reduces string interning
on the boundary between JITLink and ORC, and allows pointer comparisons (rather
than string comparisons) between Symbol names. This should improve the
performance and readability of code that bridges between JITLink and ORC (e.g.
ObjectLinkingLayer and ObjectLinkingLayer::Plugins).

To enable use of SymbolStringPtr a std::shared_ptr<SymbolStringPool> is added to
LinkGraph and threaded through to its construction sites in LLVM and Bolt. All
LinkGraphs that are to have symbol names compared by pointer equality must point
to the same SymbolStringPool instance, which in ORC sessions should be the pool
attached to the ExecutionSession.
---------

Co-authored-by: Lang Hames <lhames@gmail.com>

show more ...

Re-apply "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetime..." with fixes.

This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b

Re-apply "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetime..." with fixes.

This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b4 due to bot failures.

The new patch contains fixes to ELFLinkGraphBuilder.h to better handle
non-SHT_ALLOC sections (these were being accidentally skipped in the previous
patch), and to skip SHT_NULL sections.

show more ...

Revert "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add ..."

This reverts commit 2cc64df0bd6a802eab592dbc282463c3e4a4281c while I investigate
bot failures (e.g. https://lab.llvm.org

Revert "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add ..."

This reverts commit 2cc64df0bd6a802eab592dbc282463c3e4a4281c while I investigate
bot failures (e.g. https://lab.llvm.org/buildbot/#/builders/3/builds/23081).

show more ...

[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add NoAlloc option.

The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* F

[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add NoAlloc option.

The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* Finalize: allocated memory lives until all finalize actions have been run,
then is destroyed.

This patch introduces a new 'NoAlloc' option. NoAlloc indicates that the
section should be ignored by the JITLinkMemoryManager -- the memory manager
should allocate neither working memory nor executor address space to blocks in
NoAlloc sections. The NoAlloc option is intended to support metadata sections
(e.g. debug info) that we want to keep in the graph and have fixed up if
necessary, but don't want allocated or transmitted to the executor (or we want
that allocation and transmission to be managed manually by plugins).

Since NoAlloc blocks are ignored by the JITLinkMemoryManager they will not have
working memory allocated to them by default post-allocation. Clients wishing to
modify the content of a block in a NoAlloc section should call
`Block::getMutableMemory(LinkGraph&)` to get writable memory allocated on the
LinkGraph's allocator (this memory will exist for the lifetime of the graph).
If no client requests mutable memory prior to the fixup phase then the generic
link algorithm will do so when it encounters the first edge in any given block.

Addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and should not be modified by the JITLinkMemoryManager. Plugins are responsible
for updating addresses if they add/remove content from these sections. The
meaning of addresses in NoAlloc-sections is backend/plugin defined, but for
fixup purposes they will be treated the same as addresses in Standard/Finalize
sections. References from Standard/Finalize sections to NoAlloc sections are
expected to be common (these represent metadata tracking executor addresses).
References from NoAlloc sections to Standard/Finalize sections are expected to
be rare/non-existent (they would represent JIT'd code / data tracking metadata
in the controller, which would be surprising). LinkGraphBuilders and specific
backends may impose additional constraints on edges between Standard/Finalize
and NoAlloc sections where required for correctness.

Differential Revision: https://reviews.llvm.org/D146183

show more ...

[ORC][JITLink] Move MemoryFlags.h (MemProt, AllocGroup,...) from JITLink to ORC.

Moving these types to OrcShared eliminates the need for the separate
WireProtectionFlags type.

[ORC] Move CWrapperFunctionResult out of the detail:: namespace.

This type has been moved up into the llvm::orc::shared namespace.

This type was originally put in the detail:: namespace on the assu

[ORC] Move CWrapperFunctionResult out of the detail:: namespace.

This type has been moved up into the llvm::orc::shared namespace.

This type was originally put in the detail:: namespace on the assumption that
few (if any) LLVM source files would need to use it. In practice it has been
needed in many places, and will continue to be needed until/unless
OrcTargetProcess is fully merged into the ORC runtime.

show more ...

[ORC] Call ExecutorProcessControl::disconnect in unit tests that require it.

Another follow-up to 2815ed57e3c and 19b4e3cfc6a. For unit tests that don't use
an ExecutionSession we need to call Execu

[ORC] Call ExecutorProcessControl::disconnect in unit tests that require it.

Another follow-up to 2815ed57e3c and 19b4e3cfc6a. For unit tests that don't use
an ExecutionSession we need to call ExecutorProcessControl::disconnect directly
to wait for the dispatcher to shut down.

https://llvm.org/PR52153

show more ...

[Support][ORC] Add an explicit release operation to OwningMemoryBlock.

This gives OwningMemoryBlock clients a way to check for errors on release.

rdar://84127175

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accide

Re-apply e50aea58d59, "Major JITLinkMemoryManager refactor". with fixes.

Adds explicit narrowing casts to JITLinkMemoryManager.cpp.

Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.

This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.

show more ...

Revert "[JITLink][ORC] Major JITLinkMemoryManager refactor."

This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager impl

[JITLink][ORC] Major JITLinkMemoryManager refactor.

This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.

Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.

To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).

In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).

Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.

Major features:

(1) More asynchrony:

The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.

(2) Improved control over graph address layout:

Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.

(3) Efficient tracking of allocated memory.

The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.

Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.

(4) Support for "allocation actions" and finalize-lifetime memory.

Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as

((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);

At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).

The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.

These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.

Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.

show more ...

[ORC] Rename ExecutorAddress to ExecutorAddr.

Removing the 'ess' suffix improves the ergonomics without sacrificing clarity.
Since this class is likely to be used more frequently in the future it's

[ORC] Rename ExecutorAddress to ExecutorAddr.

Removing the 'ess' suffix improves the ergonomics without sacrificing clarity.
Since this class is likely to be used more frequently in the future it's worth
some short term pain to fix this now.

show more ...

[ORC] Add finalization & deallocation actions, SimpleExecutorMemoryManager class

Finalization and deallocation actions are a key part of the upcoming
JITLinkMemoryManager redesign: They generalize t

[ORC] Add finalization & deallocation actions, SimpleExecutorMemoryManager class

Finalization and deallocation actions are a key part of the upcoming
JITLinkMemoryManager redesign: They generalize the existing finalization and
deallocate concepts (basically "copy-and-mprotect", and "munmap") to include
support for arbitrary registration and deregistration of parts of JIT linked
code. This allows us to register and deregister eh-frames, TLV sections,
language metadata, etc. using regular memory management calls with no additional
IPC/RPC overhead, which should both improve JIT performance and simplify
interactions between ORC and the ORC runtime.

The SimpleExecutorMemoryManager class provides executor-side support for memory
management operations, including finalization and deallocation actions.

This support is being added in advance of the rest of the memory manager
redesign as it will simplify the introduction of an EPC based
RuntimeDyld::MemoryManager (since eh-frame registration/deregistration will be
expressible as actions). The new RuntimeDyld::MemoryManager will in turn allow
us to remove older remote allocators that are blocking the rest of the memory
manager changes.

show more ...

[ORC] Add Shared/OrcRTBridge, and TargetProcess/OrcRTBootstrap.

This is a small first step towards reorganization of the ORC libraries:

Declarations for types and function names (as strings) to be

[ORC] Add Shared/OrcRTBridge, and TargetProcess/OrcRTBootstrap.

This is a small first step towards reorganization of the ORC libraries:

Declarations for types and function names (as strings) to be found in the
"ORC runtime bootstrap" set are moved into OrcRTBridge.h / OrcRTBridge.cpp.

The current implementation of the "ORC runtime bootstrap" functions is moved
into OrcRTBootstrap.h and OrcRTBootstrap.cpp. It is likely that this code will
eventually be moved into ORT-RT proper (in compiler RT).

The immediate goal of this change is to make these bootstrap functions usable
for clients other than SimpleRemoteEPC/SimpleRemoteEPCServer. The first planned
client is a new RuntimeDyld::MemoryManager that will run over EPC, which will
allow us to remove the old OrcRemoteTarget code.

show more ...

[ORC] Static cast more uint64_t to size_t

These instances don't have an obvious way to fail
nicely so I've just asserted they are within range.

Fixes the Arm 32 bit builds.

[ORC] Add EPCGenericJITLinkMemoryManager: memory management via EPC calls.

All ExecutorProcessControl subclasses must provide a JITLinkMemoryManager object
that can be used to allocate memory in the

[ORC] Add EPCGenericJITLinkMemoryManager: memory management via EPC calls.

All ExecutorProcessControl subclasses must provide a JITLinkMemoryManager object
that can be used to allocate memory in the executor process. The
EPCGenericJITLinkMemoryManager class provides an off-the-shelf
JITLinkMemoryManager implementation for JITs that do not need (or cannot
provide) a specialized JITLinkMemoryManager implementation. This simplifies the
process of creating new ExecutorProcessControl implementations.

show more ...