Reapply "[MemProf] Use radix tree for alloc contexts in bitcode summaries" (#117395) (#117404)

This reverts commit fdb050a5024320ec29d2edf3f2bc686c3a84abaa, and
restores ccb4702038900d82d1041ff6107

Reapply "[MemProf] Use radix tree for alloc contexts in bitcode summaries" (#117395) (#117404)

This reverts commit fdb050a5024320ec29d2edf3f2bc686c3a84abaa, and
restores ccb4702038900d82d1041ff610788740f5cef723, with a fix for build
bot failures.

Specifically, add ProfileData to the dependences of the BitWriter
library, which was causing shared library builds of LLVM to fail.
Reproduced the failure with a shared library build and confirmed this
change fixes that build failure.

show more ...

Revert "[MemProf] Use radix tree for alloc contexts in bitcode summaries" (#117395)

Reverts llvm/llvm-project#117066

This is causing some build bot failures that need investigation.

[MemProf] Use radix tree for alloc contexts in bitcode summaries (#117066)

Leverage the support added to represent allocation contexts in a more
compact way via a radix tree in the indexed profile

[MemProf] Use radix tree for alloc contexts in bitcode summaries (#117066)

Leverage the support added to represent allocation contexts in a more
compact way via a radix tree in the indexed profile to similarly reduce
sizes of the bitcode summaries.

For a large target, this reduced the size of the per-module summaries by
about 18% and in the distributed combined index files by 28%.

show more ...

[MemProf] Print full context hash when reporting hinted bytes (#114465)

Improve the information printed when -memprof-report-hinted-sizes is
enabled. Now print the full context hash computed from t

[MemProf] Print full context hash when reporting hinted bytes (#114465)

Improve the information printed when -memprof-report-hinted-sizes is
enabled. Now print the full context hash computed from the original
profile, similar to what we do when reporting matching statistics. This
will make it easier to correlate with the profile.

Note that the full context hash must be computed at profile match time
and saved in the metadata and summary, because we may trim the context
during matching when it isn't needed for distinguishing hotness.
Similarly, due to the context trimming, we may have more than one full
context id and total size pair per MIB in the metadata and summary,
which now get a list of these pairs.

Remove the old aggregate size from the metadata and summary support.
One other change from the prior support is that we no longer write the
size information into the combined index for the LTO backends, which
don't use this information, which reduces unnecessary bloat in
distributed index files.

show more ...

[IR][AArch64][PAC] Add "ptrauth(...)" Constant to represent signed pointers. (#85738)

This defines a new kind of IR Constant that represents a ptrauth signed
pointer, as used in AArch64 PAuth.

I

[IR][AArch64][PAC] Add "ptrauth(...)" Constant to represent signed pointers. (#85738)

This defines a new kind of IR Constant that represents a ptrauth signed
pointer, as used in AArch64 PAuth.

It allows representing most kinds of signed pointer constants used thus
far in the llvm ptrauth implementations, notably those used in the
Darwin and ELF ABIs being implemented for c/c++. These signed pointer
constants are then lowered to ELF/MachO relocations.

These can be simply thought of as a constant `llvm.ptrauth.sign`, with
the interesting addition of discriminator computation: the `ptrauth`
constant can also represent a combined blend, when both address and
integer discriminator operands are used. Both operands are otherwise
optional, with default values 0/null.

show more ...

Reapply [RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

Reaplying after revert in #85382 (861ebe6446296c96578807363aa292c69d827773).
Fixed intermittent test failure by avoiding p

Reapply [RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

Reaplying after revert in #85382 (861ebe6446296c96578807363aa292c69d827773).
Fixed intermittent test failure by avoiding piping output in some RUN lines.

If --write-experimental-debuginfo-iterators-to-bitcode is true (default false)
and --expermental-debuginfo-iterators is also true then the new debug info
format (non-instruction records) is written to bitcode directly.

Added the following records:

FUNC_CODE_DEBUG_RECORD_LABEL
FUNC_CODE_DEBUG_RECORD_VALUE
FUNC_CODE_DEBUG_RECORD_DECLARE
FUNC_CODE_DEBUG_RECORD_ASSIGN
FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE

The last one has an abbrev in FUNCTION_BLOCK BLOCK_INFO. Incidentally, this uses
the last value available without widening the code-length for FUNCTION_BLOCK
from 4 to 5 bits.

Records are formatted as follows:

All DbgRecord start with:
1. DILocation

FUNC_CODE_DEBUG_RECORD_LABEL
2. DILabel

DPValues then share common fields:
2. DILocalVariable
3. DIExpression

FUNC_CODE_DEBUG_RECORD_VALUE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_DECLARE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE
4. Location Value (single)

FUNC_CODE_DEBUG_RECORD_ASSIGN
4. Location Metadata
5. DIAssignID
6. DIExpression (address)
7. Location Metadata (address)

Encoding the DILocation metadata reference directly appeared to yield smaller
bitcode files than encoding the operands seperately (as is done with instruction
DILocations).

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE is by far the most common DbgRecord record
in optimized code (order of 5x-10x over other kinds). Unoptimized code should
only contain FUNC_CODE_DEBUG_RECORD_DECLARE.

show more ...

Revert "[RemoveDIs] Read/write DbgRecords directly from/to bitcode" (#85382)

Reverts llvm/llvm-project#83251

Buildbot: https://lab.llvm.org/buildbot/#/builders/139/builds/61485

[RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

If --write-experimental-debuginfo-iterators-to-bitcode is true (default false)
and --expermental-debuginfo-iterators is also true

[RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

If --write-experimental-debuginfo-iterators-to-bitcode is true (default false)
and --expermental-debuginfo-iterators is also true then the new debug info
format (non-instruction records) is written to bitcode directly.

Added the following records:

FUNC_CODE_DEBUG_RECORD_LABEL
FUNC_CODE_DEBUG_RECORD_VALUE
FUNC_CODE_DEBUG_RECORD_DECLARE
FUNC_CODE_DEBUG_RECORD_ASSIGN
FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE

The last one has an abbrev in FUNCTION_BLOCK BLOCK_INFO. Incidentally, this uses
the last value available without widening the code-length for FUNCTION_BLOCK
from 4 to 5 bits.

Records are formatted as follows:

All DbgRecord start with:
1. DILocation

FUNC_CODE_DEBUG_RECORD_LABEL
2. DILabel

DPValues then share common fields:
2. DILocalVariable
3. DIExpression

FUNC_CODE_DEBUG_RECORD_VALUE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_DECLARE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE
4. Location Value (single)

FUNC_CODE_DEBUG_RECORD_ASSIGN
4. Location Metadata
5. DIAssignID
6. DIExpression (address)
7. Location Metadata (address)

Encoding the DILocation metadata reference directly appeared to yield smaller
bitcode files than encoding the operands seperately (as is done with instruction
DILocations).

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE is by far the most common DbgRecord record
in optimized code (order of 5x-10x over other kinds). Unoptimized code should
only contain FUNC_CODE_DEBUG_RECORD_DECLARE.

show more ...

[Bitcode(Reader|Writer)] Convert Optional to std::optional

[Bitcode] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of

[Bitcode] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Restore "[MemProf] ThinLTO summary support" with more fixes

This restores commit 98ed423361de2f9dc0113a31be2aa04524489ca9 and
follow on fix 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b, which were
rever

Restore "[MemProf] ThinLTO summary support" with more fixes

This restores commit 98ed423361de2f9dc0113a31be2aa04524489ca9 and
follow on fix 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b, which were
reverted in 5d938eb6f79b16f55266dd23d5df831f552ea082 due to an
MSVC bot failure. I've included a fix for that failure.

Differential Revision: https://reviews.llvm.org/D135714

show more ...

Revert "Restore "[MemProf] ThinLTO summary support" with fixes"

This reverts commit 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b.
This reverts commit 98ed423361de2f9dc0113a31be2aa04524489ca9.

Seemingly

Revert "Restore "[MemProf] ThinLTO summary support" with fixes"

This reverts commit 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b.
This reverts commit 98ed423361de2f9dc0113a31be2aa04524489ca9.

Seemingly MSVC has some kind of issue with this patch, in terms of linking:

https://lab.llvm.org/buildbot/#/builders/123/builds/14137

I'll post more detail on D135714 momentarily.

show more ...

Restore "[MemProf] ThinLTO summary support" with fixes

This restores 47459455009db4790ffc3765a2ec0f8b4934c2a4, which was
reverted in commit 452a14efc84edf808d1e2953dad2c694972b312f, along with
fixes

Restore "[MemProf] ThinLTO summary support" with fixes

This restores 47459455009db4790ffc3765a2ec0f8b4934c2a4, which was
reverted in commit 452a14efc84edf808d1e2953dad2c694972b312f, along with
fixes for a couple of bot failures.

show more ...

Revert "[MemProf] ThinLTO summary support"

This reverts commit 47459455009db4790ffc3765a2ec0f8b4934c2a4.

Revert while I try to fix a couple of non-Linux build failures.

[MemProf] ThinLTO summary support

Implements the ThinLTO summary support for memprof related metadata.

This includes support for the assembly format, and for building the
summary from IR during Mod

[MemProf] ThinLTO summary support

Implements the ThinLTO summary support for memprof related metadata.

This includes support for the assembly format, and for building the
summary from IR during ModuleSummaryAnalysis.

To reduce space in both the bitcode format and the in memory index,
we do 2 things:
1. We keep a single vector of all uniq stack id hashes, and record the
index into this vector in the callsite and allocation memprof
summaries.
2. When building the combined index during the LTO link, the callsite
and allocation memprof summaries are only kept on the FunctionSummary
of the prevailing copy.

Differential Revision: https://reviews.llvm.org/D135714

show more ...

Remove redundant initialization of Optional (NFC)

Don't use Optional::hasValue (NFC)

[Bitcode] materialize Functions early when BlockAddress taken

IRLinker builds a work list of functions to materialize, then moves them
from a source module to a destination module one at a time.

Th

[Bitcode] materialize Functions early when BlockAddress taken

IRLinker builds a work list of functions to materialize, then moves them
from a source module to a destination module one at a time.

This is a problem for blockaddress Constants, since they need not refer
to the function they are used in; IPSCCP is quite good at sinking these
constants deep into other functions when passed as arguments.

This would lead to curious errors during LTO:
ld.lld: error: Never resolved function from blockaddress ...
based on the ordering of function definitions in IR.

The problem was that IRLinker would basically do:

for function f in worklist:
materialize f
splice f from source module to destination module

in one pass, with Functions being lazily added to the running worklist.
This confuses BitcodeReader, which cannot disambiguate whether a
blockaddress is referring to a function which has not yet been parsed
("materialized") or is simply empty because its body was spliced out.
This causes BitcodeReader to insert Functions into its BasicBlockFwdRefs
list incorrectly, as it will never re-materialize an already
materialized (but spliced out) function.

Because of the possibility that blockaddress Constants may appear in
Functions other than the ones they reference, this patch adds a new
bitcode function code FUNC_CODE_BLOCKADDR_USERS that is a simple list of
Functions that contain BlockAddress Constants that refer back to this
Function, rather then the Function they are scoped in. We then
materialize those functions when materializing `f` from the example loop
above. This might over-materialize Functions should the user of
BitcodeReader ultimately decide not to link those Functions, but we can
at least now we can avoid this ordering related issue with blockaddresses.

Fixes: https://github.com/llvm/llvm-project/issues/52787
Fixes: https://github.com/ClangBuiltLinux/linux/issues/1215

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D120781

show more ...

[Support/Hash functions] Change the `final()` and `result()` of the hashing functions to return an array of bytes

Returning `std::array<uint8_t, N>` is better ergonomics for the hashing functions us

[Support/Hash functions] Change the `final()` and `result()` of the hashing functions to return an array of bytes

Returning `std::array<uint8_t, N>` is better ergonomics for the hashing functions usage, instead of a `StringRef`:

* When returning `StringRef`, client code is "jumping through hoops" to do string manipulations instead of dealing with fixed array of bytes directly, which is more natural
* Returning `std::array<uint8_t, N>` avoids the need for the hasher classes to keep a field just for the purpose of wrapping it and returning it as a `StringRef`

As part of this patch also:

* Introduce `TruncatedBLAKE3` which is useful for using BLAKE3 as the hasher type for `HashBuilder` with non-default hash sizes.
* Make `MD5Result` inherit from `std::array<uint8_t, 16>` which improves & simplifies its API.

Differential Revision: https://reviews.llvm.org/D123100

show more ...

[Bitcode] Handle invalid abbrev number error more gracefully

Avoid report_fatal_error(), propagate the error upwards instead.

[llvm] Use true/false instead of 1/0 (NFC)

Identified with modernize-use-bool-literals.

[llvm][IR] Add no_cfi constant

With Control-Flow Integrity (CFI), the LowerTypeTests pass replaces
function references with CFI jump table references, which is a problem
for low-level code that need

[llvm][IR] Add no_cfi constant

With Control-Flow Integrity (CFI), the LowerTypeTests pass replaces
function references with CFI jump table references, which is a problem
for low-level code that needs the address of the actual function body.

For example, in the Linux kernel, the code that sets up interrupt
handlers needs to take the address of the interrupt handler function
instead of the CFI jump table, as the jump table may not even be mapped
into memory when an interrupt is triggered.

This change adds the no_cfi constant type, which wraps function
references in a value that LowerTypeTestsModule::replaceCfiUses does not
replace.

Link: https://github.com/ClangBuiltLinux/linux/issues/1353

Reviewed By: nickdesaulniers, pcc

Differential Revision: https://reviews.llvm.org/D108478

show more ...

[llvm] Use range-based for loops (NFC)

[llvm] Use range-based for loops (NFC)

Bitcode: Use Expected<T>::takeError() and moveInto() more, NFC

Avoid naming some Expected<T> values in the Bitcode reader by using
takeError() and moveInto() more often. This follows the smaller set

Bitcode: Use Expected<T>::takeError() and moveInto() more, NFC

Avoid naming some Expected<T> values in the Bitcode reader by using
takeError() and moveInto() more often. This follows the smaller set of
changes included in 2410fb4616b2c08bbaddd44e6c11da8285fbd1d3.

show more ...