[MemProf] Prune unneeded non-cold contexts (#124823)

We can take advantage of the fact that we subsequently only clone cold
allocation contexts, since not cold behavior is the default, and
signifi

[MemProf] Prune unneeded non-cold contexts (#124823)

We can take advantage of the fact that we subsequently only clone cold
allocation contexts, since not cold behavior is the default, and
significantly reduce the amount of metadata (and later ThinLTO summary
and MemProfContextDisambiguation graph nodes) by pruning unnecessary not
cold contexts when building metadata from the trie.

Specifically, we only need to keep notcold contexts that overlap the
longest with cold allocations, to know how deeply to clone those
contexts to expose the cold allocation behavior.

For a large target this reduced ThinLTO bitcode object sizes by about
35%. It reduced the ThinLTO indexing time by about half and the peak
ThinLTO indexing memory by about 20%.

show more ...

[MemProf] Convert Hot contexts to NotCold early (#124219)

While we convert hot contexts to notcold contexts during the cloning
step, their existence was greatly limiting the context trimming
perfo

[MemProf] Convert Hot contexts to NotCold early (#124219)

While we convert hot contexts to notcold contexts during the cloning
step, their existence was greatly limiting the context trimming
performed when we add the MemProf profile to the IR. To address this,
any hot contexts are converted to notcold contexts immediately after
first checking for unambiguous allocation types, and before checking it
again and before adding metadata while performing context trimming.

Note that hot hints are now disabled by default, however, this avoids
adding unnecessary overhead if they are re-enabled.

show more ...

[MemProf] Disable hot hints by default (#124338)

By default we were marking some contexts as hot, and adding hot hints to
unambiguously hot allocations. However, there is not yet support for
cloni

[MemProf] Disable hot hints by default (#124338)

By default we were marking some contexts as hot, and adding hot hints to
unambiguously hot allocations. However, there is not yet support for
cloning to expose hot allocation contexts, and none is planned for the
forseeable future.

While we convert hot contexts to notcold contexts during the cloning
step, their existence was greatly limiting the context trimming
performed when we add the MemProf profile to the IR. This change simply
disables the generation of hot contexts / hints by default, as few
allocations were unambiguously hot.

A subsequent change will address the issue when hot hints are optionally
enabled. See PR124219 for details.

This change resulted in significant overhead reductions for a large
target:
~48% reduction in the per-module ThinLTO bitcode summary sizes
~72% reduction in the distributed ThinLTO bitcode combined summary sizes
~68% reduction in thin link time
~34% reduction in thin link peak memory

show more ...

[NFC] Cleanup in ADT and Analysis headers. (#104484)

Remove unused directly includes and forward declarations in ADT and
Analysis headers.

Fix for windows bot failure (switch uint to unsigned)

Fix windows bot failure from b8d2f7177c39af7be371ba7f46cb00b9c63ef8f5:

https://lab.llvm.org/buildbot/#/builders/216/builds/20923

by switching

Fix for windows bot failure (switch uint to unsigned)

Fix windows bot failure from b8d2f7177c39af7be371ba7f46cb00b9c63ef8f5:

https://lab.llvm.org/buildbot/#/builders/216/builds/20923

by switching uint to unsigned, as the former is not recognized there.

show more ...

[MemProf] Add hot allocation type

Add "Hot" AllocationType (in addition to existing cold, notcold).

Use lifetime access density as metric to identify hot allocations.
Treat hot as notcold for MemPr

[MemProf] Add hot allocation type

Add "Hot" AllocationType (in addition to existing cold, notcold).

Use lifetime access density as metric to identify hot allocations.
Treat hot as notcold for MemProfContextDisambiguation for now
before the disambiguation for "hot" is done.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D149932

show more ...

[MemProf] Use profiled lifetime access density directly

Now that the runtime tracks the lifetime access density directly, we can
use that directly in the threshold checks instead of less accurately

[MemProf] Use profiled lifetime access density directly

Now that the runtime tracks the lifetime access density directly, we can
use that directly in the threshold checks instead of less accurately
computing from other statistics.

Differential Revision: https://reviews.llvm.org/D149684

show more ...

[MemProf] Make hasSingleAllocType helper non-static

As suggested in D140908, make the hasSingleAllocType helper non-static
so that it can be used in other files. Add unit testing.

Differential Revi

[MemProf] Make hasSingleAllocType helper non-static

As suggested in D140908, make the hasSingleAllocType helper non-static
so that it can be used in other files. Add unit testing.

Differential Revision: https://reviews.llvm.org/D144318

show more ...

[MemProf] Add helper to access the back (last) call stack id

This is split out of D140908 as suggested.

Differential Revision: https://reviews.llvm.org/D143184

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0

Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part

Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955

show more ...

Restore "[MemProf] ThinLTO summary support" with more fixes

This restores commit 98ed423361de2f9dc0113a31be2aa04524489ca9 and
follow on fix 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b, which were
rever

Restore "[MemProf] ThinLTO summary support" with more fixes

This restores commit 98ed423361de2f9dc0113a31be2aa04524489ca9 and
follow on fix 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b, which were
reverted in 5d938eb6f79b16f55266dd23d5df831f552ea082 due to an
MSVC bot failure. I've included a fix for that failure.

Differential Revision: https://reviews.llvm.org/D135714

show more ...

Revert "Restore "[MemProf] ThinLTO summary support" with fixes"

This reverts commit 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b.
This reverts commit 98ed423361de2f9dc0113a31be2aa04524489ca9.

Seemingly

Revert "Restore "[MemProf] ThinLTO summary support" with fixes"

This reverts commit 00c22351ba697dbddb4b5bf0ad94e4bcea4b316b.
This reverts commit 98ed423361de2f9dc0113a31be2aa04524489ca9.

Seemingly MSVC has some kind of issue with this patch, in terms of linking:

https://lab.llvm.org/buildbot/#/builders/123/builds/14137

I'll post more detail on D135714 momentarily.

show more ...

Restore "[MemProf] ThinLTO summary support" with fixes

This restores 47459455009db4790ffc3765a2ec0f8b4934c2a4, which was
reverted in commit 452a14efc84edf808d1e2953dad2c694972b312f, along with
fixes

Restore "[MemProf] ThinLTO summary support" with fixes

This restores 47459455009db4790ffc3765a2ec0f8b4934c2a4, which was
reverted in commit 452a14efc84edf808d1e2953dad2c694972b312f, along with
fixes for a couple of bot failures.

show more ...

Revert "[MemProf] ThinLTO summary support"

This reverts commit 47459455009db4790ffc3765a2ec0f8b4934c2a4.

Revert while I try to fix a couple of non-Linux build failures.

[MemProf] ThinLTO summary support

Implements the ThinLTO summary support for memprof related metadata.

This includes support for the assembly format, and for building the
summary from IR during Mod

[MemProf] ThinLTO summary support

Implements the ThinLTO summary support for memprof related metadata.

This includes support for the assembly format, and for building the
summary from IR during ModuleSummaryAnalysis.

To reduce space in both the bitcode format and the in memory index,
we do 2 things:
1. We keep a single vector of all uniq stack id hashes, and record the
index into this vector in the callsite and allocation memprof
summaries.
2. When building the combined index during the LTO link, the callsite
and allocation memprof summaries are only kept on the FunctionSummary
of the prevailing copy.

Differential Revision: https://reviews.llvm.org/D135714

show more ...

[MemProf] Add memprof metadata related analysis utilities

Adds a number of utilities that are used to help create and update
memprof related metadata. These will be used during profile matching
and

[MemProf] Add memprof metadata related analysis utilities

Adds a number of utilities that are used to help create and update
memprof related metadata. These will be used during profile matching
and annotation, as well as by the inliner when updating the metadata.
Also adds unit tests for the utilities.

See also related RFCs:
RFC: Sanitizer-based Heap Profiler [1]
RFC: A binary serialization format for MemProf [2]
RFC: IR metadata format for MemProf [3]
(Note that the IR metadata format has changed from the RFC during
implementation, as described in the preceeding patch adding the basic
metadata and verification support.)

Depends on D128141.

Differential Revision: https://reviews.llvm.org/D128854

show more ...