[BOLT] Never emit "large" functions (#115974)

"Large" functions are functions that are too big to fit into their
original slots after code modifications. CheckLargeFunctions pass is
designed to pr

[BOLT] Never emit "large" functions (#115974)

"Large" functions are functions that are too big to fit into their
original slots after code modifications. CheckLargeFunctions pass is
designed to prevent such functions from emission. Extend this pass to
work with functions with constant islands.

Now that CheckLargeFunctions covers all functions, it guarantees that we
will never see such functions after code emission on all platforms
(previously it was guaranteed on x86 only). Hence, we can get rid of
RewriteInstance extensions that were meant to support "large" functions.

show more ...

[BOLT] Use AsmInfo for address size. NFCI (#115932)

Use AsmInfo instead of DWARFObj interface for extracting address size
and format.

[BOLT][NFC] Track fragment relationships using EquivalenceClasses

Three-way splitting can create references between split fragments (warm
to cold or vice versa) that are not handled by
`isChildOf/is

[BOLT][NFC] Track fragment relationships using EquivalenceClasses

Three-way splitting can create references between split fragments (warm
to cold or vice versa) that are not handled by
`isChildOf/isParentOf/isChildOrParentOf`. Generalize fragment
relationships to allow checking if two functions belong to one group,
potentially in presence of ICF which can join multiple groups.

Test Plan: NFC for existing tests

Reviewers: maksfb, ayermolo, rafaelauler, dcci

Reviewed By: rafaelauler

Pull Request: https://github.com/llvm/llvm-project/pull/99979

show more ...

[BOLT] Refactor interface for instruction labels. NFCI (#83209)

To avoid accidentally setting the label twice for the same instruction,
which can lead to a "lost" label, introduce getOrSetInstLabel

[BOLT] Refactor interface for instruction labels. NFCI (#83209)

To avoid accidentally setting the label twice for the same instruction,
which can lead to a "lost" label, introduce getOrSetInstLabel()
function. Rename existing functions to getInstLabel()/setInstLabel() to
make it explicit that they operate on instruction labels. Add an
assertion in setInstLabel() that the instruction did not have a prior
label set.

show more ...

[BOLT][NFC] Log through JournalingStreams (#81524)

Make core BOLT functionality more friendly to being used as a
library instead of in our standalone driver llvm-bolt. To
accomplish this, we augme

[BOLT][NFC] Log through JournalingStreams (#81524)

Make core BOLT functionality more friendly to being used as a
library instead of in our standalone driver llvm-bolt. To
accomplish this, we augment BinaryContext with journaling streams
that are to be used by most BOLT code whenever something needs to
be logged to the screen. Users of the library can decide if logs
should be printed to a file, no file or to the screen, as
before. To illustrate this, this patch adds a new option
`--log-file` that allows the user to redirect BOLT logging to a
file on disk or completely hide it by using
`--log-file=/dev/null`. Future BOLT code should now use
`BinaryContext::outs()` for printing important messages instead of
`llvm::outs()`. A new test log.test enforces this by verifying that
no strings are print to screen once the `--log-file` option is
used.

In previous patches we also added a new BOLTError class to report
common and fatal errors, so code shouldn't call exit(1) now. To
easily handle problems as before (by quitting with exit(1)),
callers can now use
`BinaryContext::logBOLTErrorsAndQuitOnFatal(Error)` whenever code
needs to deal with BOLT errors. To test this, we have fatal.s
that checks we are correctly quitting and printing a fatal error
to the screen.

Because this is a significant change by itself, not all code was
yet ported. Code from Profiler libs (DataAggregator and friends)
still print errors directly to screen.

Co-authored-by: Rafael Auler <rafaelauler@fb.com>

Test Plan: NFC

show more ...

[BOLT][NFC] Propagate BOLTErrors from Core, RewriteInstance, and passes (2/2) (#81523)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch cont

[BOLT][NFC] Propagate BOLTErrors from Core, RewriteInstance, and passes (2/2) (#81523)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch continue the migration
on libCore, libRewrite and libPasses to use the new BOLTError
class whenever a failure occurs.

Test Plan: NFC

Co-authored-by: Rafael Auler <rafaelauler@fb.com>

show more ...

[BOLT] Determine address size from binary (#74870)

Query the executable for address size.

[BOLT] Fix C++ exceptions when LPStart is specified (#72737)

Whenever LPStartEncoding was different from DW_EH_PE_omit, we used to
miscalculate LPStart. As a result, landing pads were assigned wron

[BOLT] Fix C++ exceptions when LPStart is specified (#72737)

Whenever LPStartEncoding was different from DW_EH_PE_omit, we used to
miscalculate LPStart. As a result, landing pads were assigned wrong
addresses. Fix that.

show more ...

[BOLT] Fix build after 0df1546

[BOLT] Use Label annotation instead of EHLabel pseudo. NFCI. (#70179)

When we need to attach EH label to an instruction, we can now use Label
annotation instead of EHLabel pseudo instruction.

[BOLT] Add isParentOf and isParentOrChildOf BF checks

Add helper methods and simplify cases where we want to check if two functions
are parent-child of each other (function-fragment relationship).

[BOLT] Add isParentOf and isParentOrChildOf BF checks

Add helper methods and simplify cases where we want to check if two functions
are parent-child of each other (function-fragment relationship).

Reviewed By: #bolt, rafauler

Differential Revision: https://reviews.llvm.org/D142668

show more ...

[BOLT] Rename BF::isParentFragment -> isChildOf

`isChildOf` is a more concise name for the check. Also, there's no need to
test if the function is a fragment before doing `isChildOf` check.

Reviewe

[BOLT] Rename BF::isParentFragment -> isChildOf

`isChildOf` is a more concise name for the check. Also, there's no need to
test if the function is a fragment before doing `isChildOf` check.

Reviewed By: #bolt, rafauler, maksfb

Differential Revision: https://reviews.llvm.org/D142667

show more ...

[AArch64] fix bug #55005 handle DW_CFA_GNU_NegateRAState

GCC on AArch64 uses DW_CFA_GNU_NegateRAState for return address signing.

Differential Revision: https://reviews.llvm.org/D142572

[BOLT][NFC] Use std::optional in MCPlusBuilder

Reviewed By: maksfb, #bolt

Differential Revision: https://reviews.llvm.org/D139260

[DebugInfo] llvm::Optional => std::optional

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

[bolt] Use llvm::sys::RWMutex instead of std::shared_timed_mutex

This has the following advantages:
- std::shared_timed_mutex is macOS 10.12+ only. llvm::sys::RWMutex
automatically switches to a d

[bolt] Use llvm::sys::RWMutex instead of std::shared_timed_mutex

This has the following advantages:
- std::shared_timed_mutex is macOS 10.12+ only. llvm::sys::RWMutex
automatically switches to a different implementation internally
when targeting older macOS versions.
- bolt only needs std::shared_mutex, not std::shared_timed_mutex.
llvm::sys::RWMutex automatically uses std::shared_mutex internally
where available.

std::shared_mutex and RWMutex have the same API, so no code changes
other than types and includes are needed.

Differential Revision: https://reviews.llvm.org/D138423

show more ...

[BOLT] Preserve original LSDA type encoding

In non-pie binaries BOLT unconditionally converted type encoding
from indirect to absptr, which broke std exceptions since pointers
to their typeinfo were

[BOLT] Preserve original LSDA type encoding

In non-pie binaries BOLT unconditionally converted type encoding
from indirect to absptr, which broke std exceptions since pointers
to their typeinfo were only assigned at runtime in .data section.
In this patch we preserve original encoding so that indirect
remains indirect and can be resolved at runtime, and absolute remains absolute.

Reviewed By: rafauler, maksfb

Differential Revision: https://reviews.llvm.org/D132484

show more ...

[BOLT] Emit LSDA call sites for all fragments

For exception handling, LSDA call sites have to be emitted for each
fragment individually. With this patch, call sites and respective LSDA
symbols are g

[BOLT] Emit LSDA call sites for all fragments

For exception handling, LSDA call sites have to be emitted for each
fragment individually. With this patch, call sites and respective LSDA
symbols are generated and associated with each fragment of their
function, such that they can be used by the emitter.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D132052

show more ...

[BOLT] Allocate FunctionFragment on heap

This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-s

[BOLT] Allocate FunctionFragment on heap

This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132050

show more ...

Revert "[BOLT] Allocate FunctionFragment on heap"

This reverts commit 101344af1af82d1633c773b718788eaa813d7f79.

[BOLT] Allocate FunctionFragment on heap

This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-s

[BOLT] Allocate FunctionFragment on heap

This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132050

show more ...

[BOLT] Make exception handling fragment aware

This adds basic fragment awareness in the exception handling passes and
generates the necessary symbols for fragments.

Reviewed By: rafauler

Different

[BOLT] Make exception handling fragment aware

This adds basic fragment awareness in the exception handling passes and
generates the necessary symbols for fragments.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130520

show more ...

[bolt] silence unused variables warnings

[BOLT] Add function layout class

This patch adds a dedicated class to keep track of each function's
layout. It also lays the groundwork for splitting functions into
multiple fragments (as opposed to

[BOLT] Add function layout class

This patch adds a dedicated class to keep track of each function's
layout. It also lays the groundwork for splitting functions into
multiple fragments (as opposed to a strict hot/cold split).

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D129518

show more ...

[BOLT] Support split landing pad

We previously support split jump table, where some jump table entries
target different fragments of same function. In this fix, we provide
support for another type o

[BOLT] Support split landing pad

We previously support split jump table, where some jump table entries
target different fragments of same function. In this fix, we provide
support for another type of intra-indirect transfer: landing pad.

When C++ exception handling is used, compiler emits .gcc_except_table
that describes the location of catch block (landing pad) for specific
range that potentially invokes a throw(). Normally landing pads reside
in the function, but with -fsplit-machine-functions, landing pads can
be moved to another fragment. The intuition is, landing pads are rarely
executed, so compiler can move them to .cold section.

This update will mark all fragments that have landing pad to another
fragment as non-simple, and later propagate non-simple to all related
fragments.

This update also includes one manual test case: split-landing-pad.s

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D128561

show more ...