[JITLink] Add convenience methods to LinkGraph to find symbols by name.

Adds new convenience methods findDefinedSymbolByName, findExternalSymbolByName
and findAbsoluteSymbolByName to the LinkGraph c

[JITLink] Add convenience methods to LinkGraph to find symbols by name.

Adds new convenience methods findDefinedSymbolByName, findExternalSymbolByName
and findAbsoluteSymbolByName to the LinkGraph class. These should be used to
find symbols of the given types by name.

COFFLinkGraphBuilder and MachOPlatform are updated to take advantage of the
new methods.

show more ...

[JITLink] Use target triple for LinkGraph pointer size and endianness.

Removes LinkGraph's PointerSize and Endianness members and uses the triple to
find these values instead.

Also removes some red

[JITLink] Use target triple for LinkGraph pointer size and endianness.

Removes LinkGraph's PointerSize and Endianness members and uses the triple to
find these values instead.

Also removes some redundant Triple copies.

show more ...

[JITLink][COFF] Use regular external symbol resolution for __ImageBase.

This fixes a concurrency bug in the COFF_x86_64 backend: lookupAsync was used
to find __ImageBase without blocking to wait for

[JITLink][COFF] Use regular external symbol resolution for __ImageBase.

This fixes a concurrency bug in the COFF_x86_64 backend: lookupAsync was used
to find __ImageBase without blocking to wait for the result.

Rather than adding promises / futures, this patch just adds an external
__ImageBase symbol if needed. This is the canonical JITLink solution for
resolving external symbols, and is simpler and more concurrency friendly than
using promises / futures with lookupAsync.

show more ...

[JITLink] Switch to SymbolStringPtr for Symbol names (#115796)

Use SymbolStringPtr for Symbol names in LinkGraph. This reduces string interning
on the boundary between JITLink and ORC, and allows p

[JITLink] Switch to SymbolStringPtr for Symbol names (#115796)

Use SymbolStringPtr for Symbol names in LinkGraph. This reduces string interning
on the boundary between JITLink and ORC, and allows pointer comparisons (rather
than string comparisons) between Symbol names. This should improve the
performance and readability of code that bridges between JITLink and ORC (e.g.
ObjectLinkingLayer and ObjectLinkingLayer::Plugins).

To enable use of SymbolStringPtr a std::shared_ptr<SymbolStringPool> is added to
LinkGraph and threaded through to its construction sites in LLVM and Bolt. All
LinkGraphs that are to have symbol names compared by pointer equality must point
to the same SymbolStringPool instance, which in ORC sessions should be the pool
attached to the ExecutionSession.
---------

Co-authored-by: Lang Hames <lhames@gmail.com>

show more ...

Re-apply "[ORC][JITLink] Treat common symbols as weak definitions." with fixes.

This reapplies 785d376d123, which was reverted in c49837f5f68 due to bot
failures. The fix was to relax some asserts t

Re-apply "[ORC][JITLink] Treat common symbols as weak definitions." with fixes.

This reapplies 785d376d123, which was reverted in c49837f5f68 due to bot
failures. The fix was to relax some asserts to allow common symbols to be
resolved with either common or weak flags, rather than requiring one or the
other.

show more ...

Revert "[ORC][JITLink] Treat common symbols as weak definitions."

This reverts commit 785d376d1231167688dd12f93c5c0a5d46cd4086 while I
investigate some bot failures (e.g.
https://lab.llvm.org/buildb

Revert "[ORC][JITLink] Treat common symbols as weak definitions."

This reverts commit 785d376d1231167688dd12f93c5c0a5d46cd4086 while I
investigate some bot failures (e.g.
https://lab.llvm.org/buildbot/#/builders/3/builds/1983).

show more ...

[ORC][JITLink] Treat common symbols as weak definitions.

Duplicate common definitions should be coaleseced, rather than being treated as
duplicate definitions. Strong definitions should override com

[ORC][JITLink] Treat common symbols as weak definitions.

Duplicate common definitions should be coaleseced, rather than being treated as
duplicate definitions. Strong definitions should override common definitions.

rdar://132314264

show more ...

Use llvm::endianness::{big,little,native} (NFC)

Note that llvm::support::endianness has been renamed to
llvm::endianness while becoming an enum class as opposed to an
enum. This patch replaces suppo

Use llvm::endianness::{big,little,native} (NFC)

Note that llvm::support::endianness has been renamed to
llvm::endianness while becoming an enum class as opposed to an
enum. This patch replaces support::{big,little,native} with
llvm::endianness::{big,little,native}.

show more ...

Use llvm::endianness (NFC)

Now that llvm::support::endianness has been renamed to
llvm::endianness, we can use the shorter form. This patch replaces
support::endianness with llvm::endianness.

[ORC] Rename MemLifetimePolicy to MemLifetime.

The *Policy suffix came from the earlier MemAllocPolicy type, where it was
included to distinguish the type from a memory-allocation operation.
MemLife

[ORC] Rename MemLifetimePolicy to MemLifetime.

The *Policy suffix came from the earlier MemAllocPolicy type, where it was
included to distinguish the type from a memory-allocation operation.
MemLifetime is a noun already, so the *Policy suffix is just dead weight now.

show more ...

Fix typos in comments of ExecutionEngine (NFC)

Reviewed By: sgraenitz

Differential Revision: https://reviews.llvm.org/D157226

[JITLink] Use SubtargetFeatures to store features in LinkGraph

D149522 introduced target features to LinkGraph. However, to avoid a
public dependency on MC, the features were stored in a std::vector

[JITLink] Use SubtargetFeatures to store features in LinkGraph

D149522 introduced target features to LinkGraph. However, to avoid a
public dependency on MC, the features were stored in a std::vector
instead of using SubtargetFeatures directly.

Since SubtargetFeatures was moved from MC to TargetParser (D150549), we
can now use it directly to store the features. This patch implements
that and removes the (private) dependency on MC.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D153749

show more ...

[Orc][Coff] Skip registration of voltbl sections

We're getting asserts for duplicate section registration during
linking which stems back to these sections. From previous
discussions, it seems like

[Orc][Coff] Skip registration of voltbl sections

We're getting asserts for duplicate section registration during
linking which stems back to these sections. From previous
discussions, it seems like these are metadata sections that can
be dropped. See the discussion in D116474 and
https://bugs.llvm.org/show_bug.cgi?id=45111.

Differential Revision: https://reviews.llvm.org/D152574

show more ...

[JITLink] Add target features to LinkGraph

This patch adds SubtargetFeatures to LinkGraph. Similar to Triple, some
targets might use this information while linking.

One example, and the reason this

[JITLink] Add target features to LinkGraph

This patch adds SubtargetFeatures to LinkGraph. Similar to Triple, some
targets might use this information while linking.

One example, and the reason this patch was written, is linker relaxation
on RISC-V: different relaxations are possible depending on if the C
extension is enabled.

Note that the features are stored as `std::vector<std::string>` to prevent a
public dependency on MC. There is still a private dependency to be able to
convert SubtargetFeatures to a vector.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D149522

show more ...

Re-apply "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetime..." with fixes.

This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b

Re-apply "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetime..." with fixes.

This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b4 due to bot failures.

The new patch contains fixes to ELFLinkGraphBuilder.h to better handle
non-SHT_ALLOC sections (these were being accidentally skipped in the previous
patch), and to skip SHT_NULL sections.

show more ...

Revert "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add ..."

This reverts commit 2cc64df0bd6a802eab592dbc282463c3e4a4281c while I investigate
bot failures (e.g. https://lab.llvm.org

Revert "[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add ..."

This reverts commit 2cc64df0bd6a802eab592dbc282463c3e4a4281c while I investigate
bot failures (e.g. https://lab.llvm.org/buildbot/#/builders/3/builds/23081).

show more ...

[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add NoAlloc option.

The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* F

[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add NoAlloc option.

The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* Finalize: allocated memory lives until all finalize actions have been run,
then is destroyed.

This patch introduces a new 'NoAlloc' option. NoAlloc indicates that the
section should be ignored by the JITLinkMemoryManager -- the memory manager
should allocate neither working memory nor executor address space to blocks in
NoAlloc sections. The NoAlloc option is intended to support metadata sections
(e.g. debug info) that we want to keep in the graph and have fixed up if
necessary, but don't want allocated or transmitted to the executor (or we want
that allocation and transmission to be managed manually by plugins).

Since NoAlloc blocks are ignored by the JITLinkMemoryManager they will not have
working memory allocated to them by default post-allocation. Clients wishing to
modify the content of a block in a NoAlloc section should call
`Block::getMutableMemory(LinkGraph&)` to get writable memory allocated on the
LinkGraph's allocator (this memory will exist for the lifetime of the graph).
If no client requests mutable memory prior to the fixup phase then the generic
link algorithm will do so when it encounters the first edge in any given block.

Addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and should not be modified by the JITLinkMemoryManager. Plugins are responsible
for updating addresses if they add/remove content from these sections. The
meaning of addresses in NoAlloc-sections is backend/plugin defined, but for
fixup purposes they will be treated the same as addresses in Standard/Finalize
sections. References from Standard/Finalize sections to NoAlloc sections are
expected to be common (these represent metadata tracking executor addresses).
References from NoAlloc sections to Standard/Finalize sections are expected to
be rare/non-existent (they would represent JIT'd code / data tracking metadata
in the controller, which would be surprising). LinkGraphBuilders and specific
backends may impose additional constraints on edges between Standard/Finalize
and NoAlloc sections where required for correctness.

Differential Revision: https://reviews.llvm.org/D146183

show more ...

[JITLink][ORC] Add LinkGraph::allocateCString method.

Renames the existing allocateString method to allocateContent and adds a pair of
allocateCString methods.

The previous allocateString method di

[JITLink][ORC] Add LinkGraph::allocateCString method.

Renames the existing allocateString method to allocateContent and adds a pair of
allocateCString methods.

The previous allocateString method did not include a null-terminator. It behaved
the same as allocateContent except with a Twine input, rather than an
ArrayRef<char>. Renaming allocateString to allocateBuffer (overloading the
existing method) makes this clearer.

The new allocateCString methods allocate the given content plus a
null-terminator character, and return a buffer covering both the string and
null-terminator. This makes them suitable for creating c-string content for
jitlink::Blocks.

Existing users of the old allocateString method have been updated to use the
new allocateContent overload.

show more ...

Revert D139098 "[Alignment] Use Align for ObjectFile::getSectionAlignment"

This breaks lld.

This reverts commit 10c47465e2505ddfee4e62a2ab2e535abea3ec56.

[Alignment] Use Align for ObjectFile::getSectionAlignment

Differential Revision: https://reviews.llvm.org/D139098

[llvm] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of ma

[llvm] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[ORC][JITLink] Move MemoryFlags.h (MemProt, AllocGroup,...) from JITLink to ORC.

Moving these types to OrcShared eliminates the need for the separate
WireProtectionFlags type.

[ORC][JITLink] Retain Weak flags in JITDylib interfaces, propagate to LinkGraph.

Previously we stripped Weak flags from JITDylib symbol table entries once they
were resolved (there was no particular

[ORC][JITLink] Retain Weak flags in JITDylib interfaces, propagate to LinkGraph.

Previously we stripped Weak flags from JITDylib symbol table entries once they
were resolved (there was no particularly good reason for this). Now we want to
retain them and query them when setting the Linkage on external symbols in
LinkGraphs during symbol resolution (this was the motivation for 75404e9ef88).
Making weak linkage of external definitions discoverable in the LinkGraph will
in turn allow future plugins to implement correct handling for them (by
recording locations that depend on exported weak definitions and pointing all
of these at one chosen definition at runtime).

show more ...

[JITLink] Introduce new weakly-referenced concept separate from linkage.

Introduces two new methods on Symbol: isWeaklyReferenced and
setWeaklyReferenced. These are now used to track/set whether an

[JITLink] Introduce new weakly-referenced concept separate from linkage.

Introduces two new methods on Symbol: isWeaklyReferenced and
setWeaklyReferenced. These are now used to track/set whether an external symbol
is weakly referenced, rather than having the Symbol's linkage set to weak.

This change is a first step towards proper handling of weak defs used across
JITDylib boundaries: It frees up the Linkage field on external symbols so that
it can be used to represent the linkage of the definition that the symbol resolves
to. It is expected that Platform plugins will use this information to track
locations that need to be updated if the selected weak definition changes (e.g.
because JITDylibs were dlclosed and then dlopened again in a different order).

show more ...

[JITLink][COFF] Use DLLImportDefinitionGenerator for creating PLT stubs.

Uses DLLImportDefinitionGenerator for creating PLT stubs. It removes previous approach for dllimport stub creation which can'

[JITLink][COFF] Use DLLImportDefinitionGenerator for creating PLT stubs.

Uses DLLImportDefinitionGenerator for creating PLT stubs. It removes previous approach for dllimport stub creation which can't deal with jump thunks.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D132524

show more ...