[llvm] Use std::make_unique (NFC) (#97165)

This patch is based on clang-tidy's modernize-make-unique but limited
to those cases where type names are mentioned twice like
std::unique_ptr<Type>(new

[llvm] Use std::make_unique (NFC) (#97165)

This patch is based on clang-tidy's modernize-make-unique but limited
to those cases where type names are mentioned twice like
std::unique_ptr<Type>(new Type()), which is a bit mouthful.

show more ...

[ORC] Drop Comdat when discarding IR symbol

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 and
87d7838202267a

[ORC] Drop Comdat when discarding IR symbol

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 and
87d7838202267a011639fcbf97263556ccf091dc with updates to the test:
* Force emission of the extra-module, to trigger the bug after D138264,
by providing a second symbol @g, and making the comdat nodeduplicate.
(Technically only one is needed, but two should be safer.)
* Name the comdat $f to avoid failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.
* Mark the test as UNSUPPORTED on macOS, MachO doesn't support COMDATs.

Differential Revision: https://reviews.llvm.org/D142443

show more ...

Revert "[ORC] Drop Comdat when discarding IR symbol"

A number of AArch64 bots report errors in clang-repl, for example
https://lab.llvm.org/buildbot/#/builders/197/builds/3920

This reverts commit 8

Revert "[ORC] Drop Comdat when discarding IR symbol"

A number of AArch64 bots report errors in clang-repl, for example
https://lab.llvm.org/buildbot/#/builders/197/builds/3920

This reverts commit 87d7838202267a011639fcbf97263556ccf091dc.

show more ...

[ORC] Drop Comdat when discarding IR symbol

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 with
updates to th

[ORC] Drop Comdat when discarding IR symbol

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 with
updates to the test:
* Force emission of the extra-module, to trigger the bug after D138264,
by providing a second symbol @g, and making the comdat nodeduplicate.
(Technically only one is needed, but two should be safer.)
* Name the comdat $f to avoid failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.

Differential Revision: https://reviews.llvm.org/D142443

show more ...

Revert "[ORC] Drop Comdat when discarding IR symbol"

Failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.

This reverts commit 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 while

Revert "[ORC] Drop Comdat when discarding IR symbol"

Failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.

This reverts commit 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 while
I investigate the problem and a solution that still triggers the
original problem.

show more ...

[ORC] Drop Comdat when discarding IR symbol

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

Differential Revision: https://reviews.llvm.org/D142443

Teach RuntimeDyld about COFF weak references and to consider comdat symbols weak.

Patch by Lang Hames and Sunho Kim!

Differential revision: https://reviews.llvm.org/D138264

[llvm] Use x.empty() instead of llvm::empty(x) (NFC)

I'm planning to deprecate and eventually remove llvm::empty.

I thought about replacing llvm::empty(x) with std::empty(x), but it
turns out that

[llvm] Use x.empty() instead of llvm::empty(x) (NFC)

I'm planning to deprecate and eventually remove llvm::empty.

I thought about replacing llvm::empty(x) with std::empty(x), but it
turns out that all uses can be converted to x.empty(). That is, no
use requires the ability of std::empty to accept C arrays and
std::initializer_list.

Differential Revision: https://reviews.llvm.org/D133677

show more ...

[llvm] Use = default (NFC)

[ORC] Add MaterializationUnit::Interface parameter to ObjectLayer::add.

Also moves object interface building functions out of Mangling.h and in to the
new ObjectFileInterfaces.h header, and updates

[ORC] Add MaterializationUnit::Interface parameter to ObjectLayer::add.

Also moves object interface building functions out of Mangling.h and in to the
new ObjectFileInterfaces.h header, and updates the llvm-jitlink tool to use
custom object interfaces rather than a custom link layer.

ObjectLayer::add overloads are added to match the old signatures (which
do not take a MaterializationUnit::Interface). These overloads use the
standard getObjectFileInterface function to build an interface.

Passing a MaterializationUnit::Interface explicitly makes it easier to alter
the effective interface of the object file being added, e.g. by changing symbol
visibility/linkage, or renaming symbols (in both cases the changes will need to
be mirrored by a JITLink pass at link time to update the LinkGraph to match the
explicit interface). Altering interfaces in this way can be useful when lazily
compiling (e.g. for renaming function bodies) or emulating linker options (e.g.
demoting all symbols to hidden visibility to emulate -load_hidden).

show more ...

[ORC] Add a MaterializationUnit::Interface struct.

MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and

[ORC] Add a MaterializationUnit::Interface struct.

MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.

Having a type for this will make functions that build materializer interfaces
more readable and maintainable.

show more ...

[Orc] Use extensible RTTI for the orc::ObjectLayer class hierarchy

So far we had no way to distinguish between JITLink and RuntimeDyld in lli. Instead, we used implicit knowledge that RuntimeDyld wo

[Orc] Use extensible RTTI for the orc::ObjectLayer class hierarchy

So far we had no way to distinguish between JITLink and RuntimeDyld in lli. Instead, we used implicit knowledge that RuntimeDyld would be used for linking ELF. In order to get D97337 to work with lli though, we have to move on and allow JITLink for ELF. This patch uses extensible RTTI to allow external clients to add their own layers without touching the LLVM sources.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D97338

show more ...

[ORC] Add support for resource tracking/removal (removable code).

This patch introduces new APIs to support resource tracking and removal in Orc.
It is intended as a thread-safe generalization of th

[ORC] Add support for resource tracking/removal (removable code).

This patch introduces new APIs to support resource tracking and removal in Orc.
It is intended as a thread-safe generalization of the removeModule concept from
OrcV1.

Clients can now create ResourceTracker objects (using
JITDylib::createResourceTracker) to track resources for each MaterializationUnit
(code, data, aliases, absolute symbols, etc.) added to the JIT. Every
MaterializationUnit will be associated with a ResourceTracker, and
ResourceTrackers can be re-used for multiple MaterializationUnits. Each JITDylib
has a default ResourceTracker that will be used for MaterializationUnits added
to that JITDylib if no ResourceTracker is explicitly specified.

Two operations can be performed on ResourceTrackers: transferTo and remove. The
transferTo operation transfers tracking of the resources to a different
ResourceTracker object, allowing ResourceTrackers to be merged to reduce
administrative overhead (the source tracker is invalidated in the process). The
remove operation removes all resources associated with a ResourceTracker,
including any symbols defined by MaterializationUnits associated with the
tracker, and also invalidates the tracker. These operations are thread safe, and
should work regardless of the the state of the MaterializationUnits. In the case
of resource transfer any existing resources associated with the source tracker
will be transferred to the destination tracker, and all future resources for
those units will be automatically associated with the destination tracker. In
the case of resource removal all already-allocated resources will be
deallocated, any if any program representations associated with the tracker have
not been compiled yet they will be destroyed. If any program representations are
currently being compiled then they will be prevented from completing: their
MaterializationResponsibility will return errors on any attempt to update the
JIT state.

Clients (usually Layer writers) wishing to track resources can implement the
ResourceManager API to receive notifications when ResourceTrackers are
transferred or removed. The MaterializationResponsibility::withResourceKeyDo
method can be used to create associations between the key for a ResourceTracker
and an allocated resource in a thread-safe way.

RTDyldObjectLinkingLayer and ObjectLinkingLayer are updated to use the
ResourceManager API to enable tracking and removal of memory allocated by the
JIT linker.

The new JITDylib::clear method can be used to trigger removal of every
ResourceTracker associated with the JITDylib (note that this will only
remove resources for the JITDylib, it does not run static destructors).

This patch includes unit tests showing basic usage. A follow-up patch will
update the Kaleidoscope and BuildingAJIT tutorial series to OrcV2 and will
use this API to release code associated with anonymous expressions.

show more ...

Re-apply "[ORC] Make MaterializationResponsibility immovable..." with fixes.

Re-applies c74900ca672 with fixes for the ThinLtoJIT example.

Revert "[ORC] Make MaterializationResponsibility immovable, pass by unique_ptr."

This reverts commit c74900ca67241bf963b7a4cfa1fae8eadf6bb8cd.

This appears to be breaking some builds on macOS and h

Revert "[ORC] Make MaterializationResponsibility immovable, pass by unique_ptr."

This reverts commit c74900ca67241bf963b7a4cfa1fae8eadf6bb8cd.

This appears to be breaking some builds on macOS and has been causing
build failures on Green Dragon (see below). I am reverting this for now,
to unblock testing on Green Dragon.

http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental/18144/console

[65/187] /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBUILD_EXAMPLES -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iexamples/ThinLtoJIT -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT -Iinclude -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -O3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.9 -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o -MF examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o.d -o examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o -c /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT/ThinLtoDiscoveryThread.cpp
FAILED: examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DBUILD_EXAMPLES -DGTEST_HAS_RTTI=0 -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iexamples/ThinLtoJIT -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT -Iinclude -I/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -O3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.9 -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o -MF examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o.d -o examples/ThinLtoJIT/CMakeFiles/ThinLtoJIT.dir/ThinLtoDiscoveryThread.cpp.o -c /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT/ThinLtoDiscoveryThread.cpp
In file included from /Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT/ThinLtoDiscoveryThread.cpp:7:
/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/examples/ThinLtoJIT/ThinLtoInstrumentationLayer.h:37:68: error: non-virtual member function marked 'override' hides virtual member function
void emit(MaterializationResponsibility R, ThreadSafeModule TSM) override;
^
/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm-project/llvm/include/llvm/ExecutionEngine/Orc/Layer.h:103:16: note: hidden overloaded virtual function 'llvm::orc::IRLayer::emit' declared here: type mismatch at 1st parameter ('std::unique_ptr<MaterializationResponsibility>' vs 'llvm::orc::MaterializationResponsibility')
virtual void emit(std::unique_ptr<MaterializationResponsibility> R,
^
1 error generated.

show more ...

[ORC] Make MaterializationResponsibility immovable, pass by unique_ptr.

Making MaterializationResponsibility instances immovable allows their
associated VModuleKeys to be updated by the ExecutionSes

[ORC] Make MaterializationResponsibility immovable, pass by unique_ptr.

Making MaterializationResponsibility instances immovable allows their
associated VModuleKeys to be updated by the ExecutionSession while the
responsibility is still in-flight. This will be used in the upcoming
removable code feature to enable safe merging of resource keys even if
there are active compiles using the keys being merged.

show more ...

[ORC][NFC] Refactor loop to determine name of init symbol in IRMaterializationUnit

This loop caused me a little headache once, because I didn't see the assigned variable is a member. The refactored

[ORC][NFC] Refactor loop to determine name of init symbol in IRMaterializationUnit

This loop caused me a little headache once, because I didn't see the assigned variable is a member. The refactored version appears more readable to me.

Differential Revision: https://reviews.llvm.org/D85922

show more ...

[ORC] Introduce JITSymbolFlags::HasMaterializeSideEffectsOnly flag.

This flag can be used to mark a symbol as existing only for the purpose of
enabling materialization. Such a symbol can be looked u

[ORC] Introduce JITSymbolFlags::HasMaterializeSideEffectsOnly flag.

This flag can be used to mark a symbol as existing only for the purpose of
enabling materialization. Such a symbol can be looked up to trigger
materialization with the lookup returning only once materialization is
complete. Symbols with this flag will never resolve however (to avoid
permanently polluting the symbol table), and should only be looked up using
the SymbolLookupFlags::WeaklyReferencedSymbol flag. The primary use case for
this flag is initialization symbols.

show more ...

[ORC] Move ostream operators for debugging output out of Core.h.

DebugUtils.h seems like a more appropriate home for these.

[ORC] Remove an out-of-date FIXME

[ORC] Add generic initializer/deinitializer support.

Initializers and deinitializers are used to implement C++ static constructors
and destructors, runtime registration for some languages (e.g. with

[ORC] Add generic initializer/deinitializer support.

Initializers and deinitializers are used to implement C++ static constructors
and destructors, runtime registration for some languages (e.g. with the
Objective-C runtime for Objective-C/C++ code) and other tasks that would
typically be performed when a shared-object/dylib is loaded or unloaded by a
statically compiled program.

MCJIT and ORC have historically provided limited support for discovering and
running initializers/deinitializers by scanning the llvm.global_ctors and
llvm.global_dtors variables and recording the functions to be run. This approach
suffers from several drawbacks: (1) It only works for IR inputs, not for object
files (including cached JIT'd objects). (2) It only works for initializers
described by llvm.global_ctors and llvm.global_dtors, however not all
initializers are described in this way (Objective-C, for example, describes
initializers via specially named metadata sections). (3) To make the
initializer/deinitializer functions described by llvm.global_ctors and
llvm.global_dtors searchable they must be promoted to extern linkage, polluting
the JIT symbol table (extra care must be taken to ensure this promotion does
not result in symbol name clashes).

This patch introduces several interdependent changes to ORCv2 to support the
construction of new initialization schemes, and includes an implementation of a
backwards-compatible llvm.global_ctor/llvm.global_dtor scanning scheme, and a
MachO specific scheme that handles Objective-C runtime registration (if the
Objective-C runtime is available) enabling execution of LLVM IR compiled from
Objective-C and Swift.

The major changes included in this patch are:

(1) The MaterializationUnit and MaterializationResponsibility classes are
extended to describe an optional "initializer" symbol for the module (see the
getInitializerSymbol method on each class). The presence or absence of this
symbol indicates whether the module contains any initializers or
deinitializers. The initializer symbol otherwise behaves like any other:
searching for it triggers materialization.

(2) A new Platform interface is introduced in llvm/ExecutionEngine/Orc/Core.h
which provides the following callback interface:

- Error setupJITDylib(JITDylib &JD): Can be used to install standard symbols
in JITDylibs upon creation. E.g. __dso_handle.

- Error notifyAdding(JITDylib &JD, const MaterializationUnit &MU): Generally
used to record initializer symbols.

- Error notifyRemoving(JITDylib &JD, VModuleKey K): Used to notify a platform
that a module is being removed.

Platform implementations can use these callbacks to track outstanding
initializers and implement a platform-specific approach for executing them. For
example, the MachOPlatform installs a plugin in the JIT linker to scan for both
__mod_inits sections (for C++ static constructors) and ObjC metadata sections.
If discovered, these are processed in the usual platform order: Objective-C
registration is carried out first, then static initializers are executed,
ensuring that calls to Objective-C from static initializers will be safe.

This patch updates LLJIT to use the new scheme for initialization. Two
LLJIT::PlatformSupport classes are implemented: A GenericIR platform and a MachO
platform. The GenericIR platform implements a modified version of the previous
llvm.global-ctor scraping scheme to provide support for Windows and
Linux. LLJIT's MachO platform uses the MachOPlatform class to provide MachO
specific initialization as described above.

Reviewers: sgraenitz, dblaikie

Subscribers: mgorny, hiraditya, mgrang, ributzka, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74300

show more ...

[ORC] Add support for emulated TLS to ORCv2.

This commit adds a ManglingOptions struct to IRMaterializationUnit, and replaces
IRCompileLayer::CompileFunction with a new IRCompileLayer::IRCompiler cl

[ORC] Add support for emulated TLS to ORCv2.

This commit adds a ManglingOptions struct to IRMaterializationUnit, and replaces
IRCompileLayer::CompileFunction with a new IRCompileLayer::IRCompiler class. The
ManglingOptions struct defines the emulated-TLS state (via a bool member,
EmulatedTLS, which is true if emulated-TLS is enabled and false otherwise). The
IRCompileLayer::IRCompiler class wraps an IRCompiler (the same way that the
CompileFunction typedef used to), but adds a method to return the
IRCompileLayer::ManglingOptions that the compiler will use.

These changes allow us to correctly determine the symbols that will be produced
when a thread local global variable defined at the IR level is compiled with or
without emulated TLS. This is required for ORCv2, where MaterializationUnits
must declare their interface up-front.

Most ORCv2 clients should not require any changes. Clients writing custom IR
compilers will need to wrap their compiler in an IRCompileLayer::IRCompiler,
rather than an IRCompileLayer::CompileFunction, however this should be a
straightforward change (see modifications to CompileUtils.* in this patch for an
example).

show more ...

[ORC] fix use-after-free detected by -Wreturn-stack-address

Summary:
llvm/lib/ExecutionEngine/Orc/Layer.cpp:53:12: warning: returning address of local temporary object [-Wreturn-stack-address]

In
`

[ORC] fix use-after-free detected by -Wreturn-stack-address

Summary:
llvm/lib/ExecutionEngine/Orc/Layer.cpp:53:12: warning: returning address of local temporary object [-Wreturn-stack-address]

In
```
StringRef IRMaterializationUnit::getName() const {
[...]
return TSM.withModuleDo(
[](const Module &M) { return M.getModuleIdentifier(); });
```
`getModuleIdentifier()` returns a `const std::string &`, but the implicit return type
of the lambda is `std::string` by value, and thus the returned `StringRef` refers
to a temporary `std::string`.

Detect by annotating `llvm::StringRef` with `[[gsl::Pointer]]`.

Reviewers: lhames, sgraenitz

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66440

llvm-svn: 369306

show more ...

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013

show more ...

[ORC] Change the locking scheme for ThreadSafeModule.

ThreadSafeModule/ThreadSafeContext are used to manage lifetimes and locking
for LLVMContexts in ORCv2. Prior to this patch contexts were locked

[ORC] Change the locking scheme for ThreadSafeModule.

ThreadSafeModule/ThreadSafeContext are used to manage lifetimes and locking
for LLVMContexts in ORCv2. Prior to this patch contexts were locked as soon
as an associated Module was emitted (to be compiled and linked), and were not
unlocked until the emit call returned. This could lead to deadlocks if
interdependent modules that shared contexts were compiled on different threads:
when, during emission of the first module, the dependence was discovered the
second module (which would provide the required symbol) could not be emitted as
the thread emitting the first module still held the lock.

This patch eliminates this possibility by moving to a finer-grained locking
scheme. Each client holds the module lock only while they are actively operating
on it. To make this finer grained locking simpler/safer to implement this patch
removes the explicit lock method, 'getContextLock', from ThreadSafeModule and
replaces it with a new method, 'withModuleDo', that implicitly locks the context,
calls a user-supplied function object to operate on the Module, then implicitly
unlocks the context before returning the result.

ThreadSafeModule TSM = getModule(...);
size_t NumFunctions = TSM.withModuleDo(
[](Module &M) { // <- context locked before entry to lambda.
return M.size();
});

Existing ORCv2 layers that operate on ThreadSafeModules are updated to use the
new method.

This method is used to introduce Module locking into each of the existing
layers.

llvm-svn: 367686

show more ...