[BOLT] Avoid repeated set lookups (NFC) (#112157)

[NFC][BOLT] Rename createDummyReturnFunction to createReturnInstructi.. (#98448)

`createDummyReturnFunction` is not creating a function but instead only
a function body that is simply a return stat

[NFC][BOLT] Rename createDummyReturnFunction to createReturnInstructi.. (#98448)

`createDummyReturnFunction` is not creating a function but instead only
a function body that is simply a return statement.
This patch renames it to: `createReturnInstructionList`

show more ...

[BOLT][NFC] Separate isReversibleBranch's 2 semantics (#95572)

`isUnsupportedBranch` was renamed (and inverted) to `isReversibleBranch`, as that was how it was being used. But one use in `BinaryFu

[BOLT][NFC] Separate isReversibleBranch's 2 semantics (#95572)

`isUnsupportedBranch` was renamed (and inverted) to `isReversibleBranch`, as that was how it was being used. But one use in `BinaryFunction::disassemble` was using the original meaning to detect unsupported branches, and the `isUnsupportedBranch` had 2 separate semantic checks.

Move the unsupported branch check from `isReversibleBranch` to a new entry point: `isUnsupportedInstruction`. Call that from `BinaryFunction::disassemble`.

Move the dynamic branch check from X86's isReversibleBranch to the base class, as it is not an architecture-specific check.

Remove unnecessary `isReversibleBranch` calls from Instrumentation and X86 MCPlusBuilder.

show more ...

[BOLT][NFC] Rename isUnsupportedBranch to isReversibleBranch (#92447)

`isUnsupportedBranch` is not a very informative name, and doesn't match
its corresponding `reverseBranchCondition`, as I noted

[BOLT][NFC] Rename isUnsupportedBranch to isReversibleBranch (#92447)

`isUnsupportedBranch` is not a very informative name, and doesn't match
its corresponding `reverseBranchCondition`, as I noted in PR #92018.
Here's a renaming to a more mnemonic name.

show more ...

[BOLT][AArch64] Skip BBs only instead of functions (#81989)

After [this
](https://github.com/llvm/llvm-project/commit/846eb76761c858cbfc75700bf68445e0e3ade48e)
commit we noticed that the size of f

[BOLT][AArch64] Skip BBs only instead of functions (#81989)

After [this
](https://github.com/llvm/llvm-project/commit/846eb76761c858cbfc75700bf68445e0e3ade48e)
commit we noticed that the size of fdata file decreased a lot. That's
why the better and more precise way will be to skip basic blocks with
exclusive instructions only instead of the whole function

show more ...

[BOLT][NFC] Log through JournalingStreams (#81524)

Make core BOLT functionality more friendly to being used as a
library instead of in our standalone driver llvm-bolt. To
accomplish this, we augme

[BOLT][NFC] Log through JournalingStreams (#81524)

Make core BOLT functionality more friendly to being used as a
library instead of in our standalone driver llvm-bolt. To
accomplish this, we augment BinaryContext with journaling streams
that are to be used by most BOLT code whenever something needs to
be logged to the screen. Users of the library can decide if logs
should be printed to a file, no file or to the screen, as
before. To illustrate this, this patch adds a new option
`--log-file` that allows the user to redirect BOLT logging to a
file on disk or completely hide it by using
`--log-file=/dev/null`. Future BOLT code should now use
`BinaryContext::outs()` for printing important messages instead of
`llvm::outs()`. A new test log.test enforces this by verifying that
no strings are print to screen once the `--log-file` option is
used.

In previous patches we also added a new BOLTError class to report
common and fatal errors, so code shouldn't call exit(1) now. To
easily handle problems as before (by quitting with exit(1)),
callers can now use
`BinaryContext::logBOLTErrorsAndQuitOnFatal(Error)` whenever code
needs to deal with BOLT errors. To test this, we have fatal.s
that checks we are correctly quitting and printing a fatal error
to the screen.

Because this is a significant change by itself, not all code was
yet ported. Code from Profiler libs (DataAggregator and friends)
still print errors directly to screen.

Co-authored-by: Rafael Auler <rafaelauler@fb.com>

Test Plan: NFC

show more ...

[BOLT][NFC] Add BOLTError and return it from passes (1/2) (#81522)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch we add a new class
BOLT

[BOLT][NFC] Add BOLTError and return it from passes (1/2) (#81522)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch we add a new class
BOLTError and auxiliary functions `createFatalBOLTError()` and
`createNonFatalBOLTError()` that allow BOLT code to bubble up the
problem to the caller by using the Error class as a return
type (or Expected). Also changes passes to use these.

Co-authored-by: Rafael Auler <rafaelauler@fb.com>

Test Plan: NFC

show more ...

[BOLT][NFC] Return Error from BinaryFunctionPass::runOnFunctions (#81521)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch we change the
in

[BOLT][NFC] Return Error from BinaryFunctionPass::runOnFunctions (#81521)

As part of the effort to refactor old error handling code that
would directly call exit(1), in this patch we change the
interface to `BinaryFunctionPass` to return an Error on
`runOnFunctions()`. This gives passes the ability to report a
serious problem to the caller (RewriteInstance class), so the
caller may decide how to best handle the exceptional situation.

Co-authored-by: Rafael Auler <rafaelauler@fb.com>

Test Plan: NFC

show more ...

[BOLT] Fix typos (#68121)

Closes https://github.com/llvm/llvm-project/issues/63097

Before merging please make sure the change to
bolt/include/bolt/Passes/StokeInfo.h is correct.

bolt/include/

[BOLT] Fix typos (#68121)

Closes https://github.com/llvm/llvm-project/issues/63097

Before merging please make sure the change to
bolt/include/bolt/Passes/StokeInfo.h is correct.

bolt/include/bolt/Passes/StokeInfo.h

```diff
// This Pass solves the two major problems to use the Stoke program without
- // proting its code:
+ // probing its code:
```

I'm still not happy about the awkward wording in this comment.

bolt/include/bolt/Passes/FixRelaxationPass.h

```
$ ed -s bolt/include/bolt/Passes/FixRelaxationPass.h <<<'9,12p'
// This file declares the FixRelaxations class, which locates instructions with
// wrong targets and fixes them. Such problems usually occures when linker
// relaxes (changes) instructions, but doesn't fix relocations types properly
// for them.
$
```


bolt/docs/doxygen.cfg.in
bolt/include/bolt/Core/BinaryContext.h
bolt/include/bolt/Core/BinaryFunction.h
bolt/include/bolt/Core/BinarySection.h
bolt/include/bolt/Core/DebugData.h
bolt/include/bolt/Core/DynoStats.h
bolt/include/bolt/Core/Exceptions.h
bolt/include/bolt/Core/MCPlusBuilder.h
bolt/include/bolt/Core/Relocation.h
bolt/include/bolt/Passes/FixRelaxationPass.h
bolt/include/bolt/Passes/InstrumentationSummary.h
bolt/include/bolt/Passes/ReorderAlgorithm.h
bolt/include/bolt/Passes/StackReachingUses.h
bolt/include/bolt/Passes/StokeInfo.h
bolt/include/bolt/Passes/TailDuplication.h
bolt/include/bolt/Profile/DataAggregator.h
bolt/include/bolt/Profile/DataReader.h
bolt/lib/Core/BinaryContext.cpp
bolt/lib/Core/BinarySection.cpp
bolt/lib/Core/DebugData.cpp
bolt/lib/Core/DynoStats.cpp
bolt/lib/Core/Relocation.cpp
bolt/lib/Passes/Instrumentation.cpp
bolt/lib/Passes/JTFootprintReduction.cpp
bolt/lib/Passes/ReorderData.cpp
bolt/lib/Passes/RetpolineInsertion.cpp
bolt/lib/Passes/ShrinkWrapping.cpp
bolt/lib/Passes/TailDuplication.cpp
bolt/lib/Rewrite/BoltDiff.cpp
bolt/lib/Rewrite/DWARFRewriter.cpp
bolt/lib/Rewrite/RewriteInstance.cpp
bolt/lib/Utils/CommandLineOpts.cpp
bolt/runtime/instr.cpp
bolt/test/AArch64/got-ld64-relaxation.test
bolt/test/AArch64/unmarked-data.test
bolt/test/X86/Inputs/dwarf5-cu-no-debug-addr-helper.s
bolt/test/X86/Inputs/linenumber.cpp
bolt/test/X86/double-jump.test
bolt/test/X86/dwarf5-call-pc-function-null-check.test
bolt/test/X86/dwarf5-split-dwarf4-monolithic.test
bolt/test/X86/dynrelocs.s
bolt/test/X86/fallthrough-to-noop.test
bolt/test/X86/tail-duplication-cache.s
bolt/test/runtime/X86/instrumentation-ind-calls.s

show more ...

[BOLT][AArch64] Fix instrumentation deadloop

According to ARMv8-a architecture reference manual B2.10.5 software
must avoid having any explicit memory accesses between exclusive load
and associated

[BOLT][AArch64] Fix instrumentation deadloop

According to ARMv8-a architecture reference manual B2.10.5 software
must avoid having any explicit memory accesses between exclusive load
and associated store instruction. Otherwise exclusive monitor might
clear the exclusivity without application-related cause which may
result in the deadloop. Disable instrumentation for such functions,
since between exclusive load and store there might be branches and we
would insert instrumentation snippet which contains loads and stores.

The better solution would be to analyze with BFS finding the exact BBs
between load and store and not instrumenting them. Or even better to
recognize such sequences and replace them with more complex one, e.g.
loading value non exclusively, and for the brach where exclusive store
is made make exclusive load and store sequentially, but for now just
disable instrumentation for such functions completely.

Differential Revision: https://reviews.llvm.org/D159520

show more ...

[BOLT][Instrumentation] Initial instrumentation support for AArch64

This commit adds code generation for AArch64 instrumentation,
including direct and indirect calls support.

Reviewed By: rafauler,

[BOLT][Instrumentation] Initial instrumentation support for AArch64

This commit adds code generation for AArch64 instrumentation,
including direct and indirect calls support.

Reviewed By: rafauler, yota9

Differential Revision: https://reviews.llvm.org/D151899

show more ...

[BOLT][NFC] Change signature of MCPlusBuilder::isUnsupportedBranch()

Make MCPlusBuilder::isUnsupportedBranch() take MCInst, not opcode.

Reviewed By: Amir

Differential Revision: https://reviews.llv

[BOLT][NFC] Change signature of MCPlusBuilder::isUnsupportedBranch()

Make MCPlusBuilder::isUnsupportedBranch() take MCInst, not opcode.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D152765

show more ...

[BOLT][NFC] Use llvm::is_contained

Apply the replacement throughout BOLT.

Reviewed By: #bolt, rafauler

Differential Revision: https://reviews.llvm.org/D145464

[BOLT][NFC] Return instruction list from createInstrIncMemory

Leverage move semantics for `std::vector`.

This also makes it consistent with `createInstrumentationSnippet`.

Reviewed By: Elvina

Dif

[BOLT][NFC] Return instruction list from createInstrIncMemory

Leverage move semantics for `std::vector`.

This also makes it consistent with `createInstrumentationSnippet`.

Reviewed By: Elvina

Differential Revision: https://reviews.llvm.org/D145465

show more ...

[BOLT][Instrumentation] Preserve red zone for functions with tail calls only

Allow a function with tail calls only to clobber its red zone.

Fixes https://github.com/llvm/llvm-project/issues/61114.

[BOLT][Instrumentation] Preserve red zone for functions with tail calls only

Allow a function with tail calls only to clobber its red zone.

Fixes https://github.com/llvm/llvm-project/issues/61114.

Reviewed By: #bolt, yota9

Differential Revision: https://reviews.llvm.org/D145202

show more ...

[BOLT] Fix intermittent crash with instrumentation

When createInstrumentedIndirectCall() was invoked for tail calls, we
attached annotation instruction twice to the new call instruction.
First in cr

[BOLT] Fix intermittent crash with instrumentation

When createInstrumentedIndirectCall() was invoked for tail calls, we
attached annotation instruction twice to the new call instruction.
First in createDirectCall(), and then again while copying over the
metadata operands.

As a result, the annotations were not properly stripped for such calls
before the call to freeAnnotations() in LowerAnnotations pass. That lead
to use-after-free while restoring the offsets with setOffset() call.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D144806

show more ...

[BOLT][NFC] Replace anonymous namespace functions with static

Follow LLVM Coding Standards guideline on using anonymous namespaces
(https://llvm.org/docs/CodingStandards.html#anonymous-namespaces)
a

[BOLT][NFC] Replace anonymous namespace functions with static

Follow LLVM Coding Standards guideline on using anonymous namespaces
(https://llvm.org/docs/CodingStandards.html#anonymous-namespaces)
and use `static` modifier for function definitions.

Reviewed By: #bolt, maksfb

Differential Revision: https://reviews.llvm.org/D143124

show more ...

[bolt] Use llvm::sys::RWMutex instead of std::shared_timed_mutex

This has the following advantages:
- std::shared_timed_mutex is macOS 10.12+ only. llvm::sys::RWMutex
automatically switches to a d

[bolt] Use llvm::sys::RWMutex instead of std::shared_timed_mutex

This has the following advantages:
- std::shared_timed_mutex is macOS 10.12+ only. llvm::sys::RWMutex
automatically switches to a different implementation internally
when targeting older macOS versions.
- bolt only needs std::shared_mutex, not std::shared_timed_mutex.
llvm::sys::RWMutex automatically uses std::shared_mutex internally
where available.

std::shared_mutex and RWMutex have the same API, so no code changes
other than types and includes are needed.

Differential Revision: https://reviews.llvm.org/D138423

show more ...

[BOLT] Add function layout class

This patch adds a dedicated class to keep track of each function's
layout. It also lays the groundwork for splitting functions into
multiple fragments (as opposed to

[BOLT] Add function layout class

This patch adds a dedicated class to keep track of each function's
layout. It also lays the groundwork for splitting functions into
multiple fragments (as opposed to a strict hot/cold split).

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D129518

show more ...

[BOLT][NFC] Use range-based STL wrappers

Replace `std::` algorithms taking begin/end iterators with `llvm::` counterparts
accepting ranges.

Reviewed By: rafauler

Differential Revision: https://rev

[BOLT][NFC] Use range-based STL wrappers

Replace `std::` algorithms taking begin/end iterators with `llvm::` counterparts
accepting ranges.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D128154

show more ...

[BOLT][NFCI] Refactor interface for adding basic blocks

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D127935

[BOLT][NFC] Move Offset annotation to Group 1

Summary:
Move the annotation to avoid dynamic memory allocations.
Improves the CPU time of instrumenting a large binary by 1% (+-0.8%, p-value 0.01)

Te

[BOLT][NFC] Move Offset annotation to Group 1

Summary:
Move the annotation to avoid dynamic memory allocations.
Improves the CPU time of instrumenting a large binary by 1% (+-0.8%, p-value 0.01)

Test Plan: NFC

Reviewers: maksfb

FBD30091656

show more ...

[BOLT][NFC] Fix braces usage in Passes

Summary:
Refactor bolt/*/Passes to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).

(cherry p

[BOLT][NFC] Fix braces usage in Passes

Summary:
Refactor bolt/*/Passes to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).

(cherry picked from FBD33344642)

show more ...

[BOLT][NFC] Fix file-description comments

Summary: Fix comments at the start of source files.

(cherry picked from FBD33274597)

[BOLT][NFC] Reformat with clang-format

Summary: Selectively apply clang-format to BOLT code base.

(cherry picked from FBD33119052)