Revert "[ORC] Enable JIT support for the compact-unwind frame info format..."

This reverts 4f0325873faccfbe171bae4babceb65975ca892e and follow-up patches
(see below) while I investigate some ongoing

Revert "[ORC] Enable JIT support for the compact-unwind frame info format..."

This reverts 4f0325873faccfbe171bae4babceb65975ca892e and follow-up patches
(see below) while I investigate some ongoing failures on the buildbots.

---

Revert "[clang-repl] Try to XFAIL testcase on arm32 without affecting arm64
darwin."

This reverts commit fd174f0ff3e793fe96a6663b1488ed159cfe042f.

Revert "[clang-repl] The simple-exception test now passes on arm64-darwin."

This reverts commit c9bc242e387f4a4a3dfcd86561f3ec0ca8a72d62.

Revert "[ORC] Destroy defunct MaterializationUnits outside the session lock."

This reverts commit a001cc0e6cdcfa672b8aff9ce6d14782bb96356a.

Revert "[ORC] Add explicit narrowing casts to fix build errors."

This reverts commit 26fc07d5d88760ad659599184fd10181287d2d9e.

Revert "[ORC] Enable JIT support for the compact-unwind frame info format on
Darwin."

This reverts commit 4f0325873faccfbe171bae4babceb65975ca892e.

show more ...

[ORC] Enable JIT support for the compact-unwind frame info format on Darwin.

For Darwin/arm64 (including Apple Silicon Macs) this will enable exception
handling and stack unwinding in JIT'd code.

D

[ORC] Enable JIT support for the compact-unwind frame info format on Darwin.

For Darwin/arm64 (including Apple Silicon Macs) this will enable exception
handling and stack unwinding in JIT'd code.

Darwin supports two unwind-info formats: DWARF eh-frames and compact-unwind. On
Darwin/x86-64 compilers usually produce both by default, and ORC supported
exceptions and unwinding via eh-frames (same as on Linux), discarding the
redundant compact-unwind info. On Darwin/arm64 compilers typically default to
producing compact-unwind only, with DWARF eh-frames as a fallback for functions
that can't be described in compact-unwind. Since ORC did not previously support
the compact-unwind format and eh-frames were not present ORC was unable to
handle exceptions or unwinding by default in Darwin/arm64 JIT'd code.

This patch enables support for the compact-unwind-info format, and contains
three major moving parts:

(1) The JITLink CompactUnwindManager class is responsible for transforming the
__compact_unwind records produced by the linker into the __unwind_info
tables that libunwind parses during unwinding. To enable this the
CompactUnwindManager class provides three JITLink passes: The
prepareForPrune pass that splits the __compact_unwind section into
single-record blocks, allowing unused records to be dead-stripped; the
processAndReserveUnwindInfo pass that reserves space for the final
__unwind_info section, and the writeUnwindInfo pass that writes the
__unwind_info section.

(2) The OrcTargetProcess UnwindInfoManager class maintains a table of
registered JIT'd __unwind_info and __eh_frame sections, and handles
requests from libunwind for unwind info sections (by registering a callback
with libunwind's __unw_add_find_dynamic_unwind_sections function).

(3) The Orc UnwindInfoRegistrationPlugin, which scans LinkGraphs for
__unwind_info and __eh_frame sections to register with the
UnwindInfoManager.

This commit adds the CompactUnwindManager passes to the default JITLink
pipelines for Darwin/arm64 and Darwin/x86-64, and UnwindInfoManager intances to
the SelfExecutorProcessControl class (when built for apple platforms) and the
llvm-jitlink-executor tool.

The LLJIT class will now create an UnwindInfoRegistrationPlugin when targeting
a process running on Darwin if it detects that an UnwindInfoManager is
available to handle the registrations.

The ORC runtime macho_platform class already supported libunwind callbacks, so
out-of-process execution and unwinding support will work when loading the ORC
runtime.

The llvm-jitlink tool will only support compact-unwind when the orc-runtime is
loaded, as the UnwindInfoRegistrationPlugin requires access to an IR compiler
to load a helper module and llvm-jitlink does not provide an IR compiler.

show more ...

[ExecutionEngine] Remove unused includes (NFC) (#116749)

Identified with misc-include-cleaner.

[MC][re-land] Omit DWARF unwind info if compact unwind is present where eligible

This reverts commit d941d597837d9e1405086f008c9bd6a71e7263c9.

Differential Revision: https://reviews.llvm.org/D122258

Revert "[MC] Omit DWARF unwind info if compact unwind is present where eligible"

This reverts commit ef501bf85d8c869248e51371f0e74bcec0e7b229.

[MC] Omit DWARF unwind info if compact unwind is present where eligible

Previously, omitting unnecessary DWARF unwinds was only done in two
cases:
* For Darwin + aarch64, if no DWARF unwind info is

[MC] Omit DWARF unwind info if compact unwind is present where eligible

Previously, omitting unnecessary DWARF unwinds was only done in two
cases:
* For Darwin + aarch64, if no DWARF unwind info is needed for all the
functions in a TU, then the `__eh_frame` section would be omitted
entirely. If any one function needed DWARF unwind, then MC would emit
DWARF unwind entries for all the functions in the TU.
* For watchOS, MC would omit DWARF unwind on a per-function basis, as
long as compact unwind was available for that function.

This diff makes it so that we omit DWARF unwind on a per-function basis
for Darwin + aarch64 as well. In addition, we introduce the flag
`--emit-dwarf-unwind=` which can toggle between `always`,
`no-compact-unwind` (only emit DWARF when CU cannot be emitted for a
given function), and the target platform `default`. `no-compact-unwind`
is particularly useful for newer x86_64 platforms: we don't want to omit
DWARF unwind for x86_64 in general due to possible backwards compat
issues, but we should make it possible for people to opt into this
behavior if they are only targeting newer platforms.

**Motivation:** I'm working on adding support for `__eh_frame` to LLD,
but I'm concerned that we would suffer a perf hit. Processing compact
unwind is already expensive, and that's a simpler format than EH frames.
Given that MC currently produces one EH frame entry for every compact
unwind entry, I don't think processing them will be cheap. I tried to do
something clever on LLD's end to drop the unnecessary EH frames at parse
time, but this made the code significantly more complex. So I'm looking
at fixing this at the MC level instead.

**Addendum:** It turns out that there was a latent bug in the X86
backend when `OmitDwarfIfHaveCompactUnwind` is naively enabled, which is
not too surprising given that this combination has not been heretofore
used.

For functions that have unwind info that cannot be encoded with CU, MC
would end up dropping both the compact unwind entry (OK; existing
behavior) as well as the DWARF entries (not OK). This diff fixes things
so that we emit the DWARF entry, as well as a CU entry with encoding
`UNWIND_X86_MODE_DWARF` -- this basically tells the unwinder to look for
the DWARF entry. I'm not 100% sure the `UNWIND_X86_MODE_DWARF` CU entry
is necessary, this was the simplest fix. ld64 seems to be able to handle
both the absence and presence of this CU entry. Ultimately ld64 (and
LLD) will synthesize `UNWIND_X86_MODE_DWARF` if it is absent, so there
is no impact to the final binary size.

Reviewed By: davide, lhames

Differential Revision: https://reviews.llvm.org/D122258

show more ...

[llvm] Add null-termination capability to SmallVectorMemoryBuffer

Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (

[llvm] Add null-termination capability to SmallVectorMemoryBuffer

Most of `MemoryBuffer` interfaces expose a `RequiresNullTerminator` parameter that's being used to:
* determine how to open a file (`mmap` vs `open`),
* assert newly initialized buffer indeed has an implicit null terminator.

This patch adds the paramater to the `SmallVectorMemoryBuffer` constructors, meaning:
* null terminator can now be added to `SmallVector`s that didn't have one before,
* `SmallVectors` that had a null terminator before keep it even after the move.

In line with existing code, the new parameter is defaulted to `true`. This patch makes sure all calls to the `SmallVectorMemoryBuffer` constructor set it to `false` to preserve the current semantics.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D115331

show more ...

[ORC] Add generic initializer/deinitializer support.

Initializers and deinitializers are used to implement C++ static constructors
and destructors, runtime registration for some languages (e.g. with

[ORC] Add generic initializer/deinitializer support.

Initializers and deinitializers are used to implement C++ static constructors
and destructors, runtime registration for some languages (e.g. with the
Objective-C runtime for Objective-C/C++ code) and other tasks that would
typically be performed when a shared-object/dylib is loaded or unloaded by a
statically compiled program.

MCJIT and ORC have historically provided limited support for discovering and
running initializers/deinitializers by scanning the llvm.global_ctors and
llvm.global_dtors variables and recording the functions to be run. This approach
suffers from several drawbacks: (1) It only works for IR inputs, not for object
files (including cached JIT'd objects). (2) It only works for initializers
described by llvm.global_ctors and llvm.global_dtors, however not all
initializers are described in this way (Objective-C, for example, describes
initializers via specially named metadata sections). (3) To make the
initializer/deinitializer functions described by llvm.global_ctors and
llvm.global_dtors searchable they must be promoted to extern linkage, polluting
the JIT symbol table (extra care must be taken to ensure this promotion does
not result in symbol name clashes).

This patch introduces several interdependent changes to ORCv2 to support the
construction of new initialization schemes, and includes an implementation of a
backwards-compatible llvm.global_ctor/llvm.global_dtor scanning scheme, and a
MachO specific scheme that handles Objective-C runtime registration (if the
Objective-C runtime is available) enabling execution of LLVM IR compiled from
Objective-C and Swift.

The major changes included in this patch are:

(1) The MaterializationUnit and MaterializationResponsibility classes are
extended to describe an optional "initializer" symbol for the module (see the
getInitializerSymbol method on each class). The presence or absence of this
symbol indicates whether the module contains any initializers or
deinitializers. The initializer symbol otherwise behaves like any other:
searching for it triggers materialization.

(2) A new Platform interface is introduced in llvm/ExecutionEngine/Orc/Core.h
which provides the following callback interface:

- Error setupJITDylib(JITDylib &JD): Can be used to install standard symbols
in JITDylibs upon creation. E.g. __dso_handle.

- Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU): Generally
used to record initializer symbols.

- Error notifyRemoving(JITDylib &JD, VModuleKey K): Used to notify a platform
that a module is being removed.

Platform implementations can use these callbacks to track outstanding
initializers and implement a platform-specific approach for executing them. For
example, the MachOPlatform installs a plugin in the JIT linker to scan for both
__mod_inits sections (for C++ static constructors) and ObjC metadata sections.
If discovered, these are processed in the usual platform order: Objective-C
registration is carried out first, then static initializers are executed,
ensuring that calls to Objective-C from static initializers will be safe.

This patch updates LLJIT to use the new scheme for initialization. Two
LLJIT::PlatformSupport classes are implemented: A GenericIR platform and a MachO
platform. The GenericIR platform implements a modified version of the previous
llvm.global-ctor scraping scheme to provide support for Windows and
Linux. LLJIT's MachO platform uses the MachOPlatform class to provide MachO
specific initialization as described above.

Reviewers: sgraenitz, dblaikie

Subscribers: mgorny, hiraditya, mgrang, ributzka, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74300

show more ...

Revert "Remove redundant "std::move"s in return statements"

The build failed with

error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a7936bf85aa68aef94bd40c3b

Revert "Remove redundant "std::move"s in return statements"

The build failed with

error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a7936bf85aa68aef94bd40c3ba77d8ddf2.

show more ...

Remove redundant "std::move"s in return statements

[ORC] Fix a missing move in ce2207abaf9.

This should fix the build failure at
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-sde-avx512-linux/builds/32524
and others.

[ORC] Add support for emulated TLS to ORCv2.

This commit adds a ManglingOptions struct to IRMaterializationUnit, and replaces
IRCompileLayer::CompileFunction with a new IRCompileLayer::IRCompiler cl

[ORC] Add support for emulated TLS to ORCv2.

This commit adds a ManglingOptions struct to IRMaterializationUnit, and replaces
IRCompileLayer::CompileFunction with a new IRCompileLayer::IRCompiler class. The
ManglingOptions struct defines the emulated-TLS state (via a bool member,
EmulatedTLS, which is true if emulated-TLS is enabled and false otherwise). The
IRCompileLayer::IRCompiler class wraps an IRCompiler (the same way that the
CompileFunction typedef used to), but adds a method to return the
IRCompileLayer::ManglingOptions that the compiler will use.

These changes allow us to correctly determine the symbols that will be produced
when a thread local global variable defined at the IR level is compiled with or
without emulated TLS. This is required for ORCv2, where MaterializationUnits
must declare their interface up-front.

Most ORCv2 clients should not require any changes. Clients writing custom IR
compilers will need to wrap their compiler in an IRCompileLayer::IRCompiler,
rather than an IRCompileLayer::CompileFunction, however this should be a
straightforward change (see modifications to CompileUtils.* in this patch for an
example).

show more ...

[ORC] Add a utility to support dumping JIT'd objects to disk for debugging.

Adds a DumpObjects utility that can be used to dump JIT'd objects to disk.
Instances of DebugObjects may be used by Object

[ORC] Add a utility to support dumping JIT'd objects to disk for debugging.

Adds a DumpObjects utility that can be used to dump JIT'd objects to disk.
Instances of DebugObjects may be used by ObjectTransformLayer as no-op
transforms.

This patch also adds an ObjectTransformLayer to LLJIT and an example of how
to use this utility to dump JIT'd objects in LLJIT.

show more ...

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013

show more ...

[ORC] Move SimpleCompiler/ConcurrentIRCompiler definitions into a .cpp file.

SimpleCompiler is no longer templated, so there's no reason for this code to be
in a header any more.

llvm-svn: 359626