Don't use Optional::hasValue (NFC)

[llvm] Remove unneeded cl::ZeroOrMore for cl::opt options. NFC

Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the err

[llvm] Remove unneeded cl::ZeroOrMore for cl::opt options. NFC

Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the error has been removed,
cl::ZeroOrMore is unneeded.

Also remove cl::init(false) while touching the lines.

show more ...

[memprof] Keep and display symbol names in the RawMemProfReader.

Extend the Frame struct to hold the symbol name if requested
when a RawMemProfReader object is constructed. This change updates the
t

[memprof] Keep and display symbol names in the RawMemProfReader.

Extend the Frame struct to hold the symbol name if requested
when a RawMemProfReader object is constructed. This change updates the
tests and removes the need to pass --debug to obtain the mapping from
GUID to symbol names.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D126344

show more ...

Fix "the the" typo in documentation and user facing strings

There are many more instances of this pattern, but I chose to limit this change to .rst files (docs), anything in libcxx/include, and stri

Fix "the the" typo in documentation and user facing strings

There are many more instances of this pattern, but I chose to limit this change to .rst files (docs), anything in libcxx/include, and string literals. These have the highest chance of being seen by end users.

Reviewed By: #libc, Mordante, martong, ldionne

Differential Revision: https://reviews.llvm.org/D124708

show more ...

[ThreadPool] add ability to group tasks into separate groups

This is needed for parallelizing of loading modules symbols in LLDB
(D122975). Currently LLDB can parallelize indexing symbols
when loadi

[ThreadPool] add ability to group tasks into separate groups

This is needed for parallelizing of loading modules symbols in LLDB
(D122975). Currently LLDB can parallelize indexing symbols
when loading a module, but modules are loaded sequentially. If LLDB
index cache is enabled, this means that the cache loading is not
parallelized, even though it could. However doing that creates
a threadpool-within-threadpool situation, so the number of threads
would not be properly limited.

This change adds ThreadPoolTaskGroup as a simple type that can be
used with ThreadPool calls to put tasks into groups that can be
independently waited for (even recursively from within a task)
but still run in the same thread pool.

Differential Revision: https://reviews.llvm.org/D123225

show more ...

[CSSPGO] Rename ProfileIsCSNested and ProfileIsCSFlat

To be more clear and definitive, I'm renaming `ProfileIsCSFlat` back to `ProfileIsCS` which stands for full context-sensitive flat profiles. `P

[CSSPGO] Rename ProfileIsCSNested and ProfileIsCSFlat

To be more clear and definitive, I'm renaming `ProfileIsCSFlat` back to `ProfileIsCS` which stands for full context-sensitive flat profiles. `ProfileIsCSNested` is now renamed to `ProfileIsPreInlined` and is extended to be applicable for CS flat profiles too. More specifically, `ProfileIsPreInlined` is for any kind of profiles (flat or nested) that contain 'ShouldBeInlined' contexts. The flag is encoded in the profile summary section for extbinary profiles and is computed on-the-fly for text profiles.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D122602

show more ...

[memprof] Deduplicate and outline frame storage in the memprof profile.

The current implementation of memprof information in the indexed profile
format stores the representation of each calling cont

[memprof] Deduplicate and outline frame storage in the memprof profile.

The current implementation of memprof information in the indexed profile
format stores the representation of each calling context fram inline.
This patch uses an interned representation where the frame contents are
stored in a separate on-disk hash table. The table is indexed via a hash
of the contents of the frame. With this patch, the compressed size of a
large memprof profile reduces by ~22%.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D123094

show more ...

[llvm-profdata] Unify default cutoffs for detailed summary printing

Use `ProfileSummaryBuilder::DefaultCutoffs` for llvm-profdata detailed summary printing for Instr profile.

Differential Revision:

[llvm-profdata] Unify default cutoffs for detailed summary printing

Use `ProfileSummaryBuilder::DefaultCutoffs` for llvm-profdata detailed summary printing for Instr profile.

Differential Revision: https://reviews.llvm.org/D122210

show more ...

Reland "[memprof] Store callsite metadata with memprof records."

This reverts commit f4b794427e8037a4e952cacdfe7201e961f31a6f.

Reland with underlying msan issue fixed in D122260.

Revert "[memprof] Store callsite metadata with memprof records."

This reverts commit 0d362c90d335509c57c0fbd01ae1829e2b9c3765.

Reason: Causes the MSan buildbot to fail (see comments on
https://revi

Revert "[memprof] Store callsite metadata with memprof records."

This reverts commit 0d362c90d335509c57c0fbd01ae1829e2b9c3765.

Reason: Causes the MSan buildbot to fail (see comments on
https://reviews.llvm.org/D121179 for more information

show more ...

[memprof] Store callsite metadata with memprof records.

To ease profile annotation, each of the callsites in a function can be
annotated with profile data - "IR metadata format for MemProf" [1]. Thi

[memprof] Store callsite metadata with memprof records.

To ease profile annotation, each of the callsites in a function can be
annotated with profile data - "IR metadata format for MemProf" [1]. This
patch extends the on-disk serialized record format to store the debug
information for allocation callsites incl inline frames. This change is
incompatible with the existing format i.e. indexed profiles must be
regenerated, raw profiles are unaffected.

[1] https://groups.google.com/g/llvm-dev/c/aWHsdMxKAfE/m/WtEmRqyhAgAJ

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D121179

show more ...

Reland "[memprof] Extend the index prof format to include memory profiles."

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profi

Reland "[memprof] Extend the index prof format to include memory profiles."

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

This commit also includes the changes reviewed separately in D120093.

Differential Revision: https://reviews.llvm.org/D120103

show more ...

Revert "Reland "[memprof] Extend the index prof format to include memory profiles.""

This reverts commit 807ba7aace188ada83ddb4477265728e97346af1.

Reland "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 85355a560a33897453df2ef959e255ee725eebce.

This patch adds support for optional memory profile informa

Reland "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 85355a560a33897453df2ef959e255ee725eebce.

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

Differential Revision: https://reviews.llvm.org/D118653

show more ...

Revert "Reland "[memprof] Extend the index prof format to include memory profiles.""

This reverts commit de54e4ab78ef09b60f870e8df6f8a87e56d6bd94 [1/4]

Reland "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 0f73fb18ca333e38cdb9ffa701a8db026c56041d.

Use llvm/Profile/MIBEntryDef.inc instead of relative path.

Reland "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 0f73fb18ca333e38cdb9ffa701a8db026c56041d.

Use llvm/Profile/MIBEntryDef.inc instead of relative path.

Generated the raw profile data with `-mllvm
-enable-name-compression=false` so that builbots where the reader is
built without zlib do not fail.

Also updated the test build instructions.

show more ...

[llvm-profdata] Fix use-after-move

Revert "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 43c2348c5b926df6bdbc5b70efaa35ecdefe12d5.

Buildbots are failing with an error on reading memprof test

Revert "[memprof] Extend the index prof format to include memory profiles."

This reverts commit 43c2348c5b926df6bdbc5b70efaa35ecdefe12d5.

Buildbots are failing with an error on reading memprof testdata.
"Inputs/basic.profraw: profile uses zlib
compression but the profile reader was built without zlib support"

https://lab.llvm.org/buildbot/#/builders/16/builds/24490

show more ...

[memprof] Extend the index prof format to include memory profiles.

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header

[memprof] Extend the index prof format to include memory profiles.

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

Differential Revision: https://reviews.llvm.org/D118653

show more ...

Revert "Revert "[ProfileData] Read and symbolize raw memprof profiles.""

This reverts commit dbf47d227d080e4eb7239b589660f51d7b08afa9.

Reapply https://reviews.llvm.org/D116784 now that
https://revi

Revert "Revert "[ProfileData] Read and symbolize raw memprof profiles.""

This reverts commit dbf47d227d080e4eb7239b589660f51d7b08afa9.

Reapply https://reviews.llvm.org/D116784 now that
https://reviews.llvm.org/D118413 has landed with a couple of fixes:
* fix raw profile reader unaligned access identified by ubsan
* fix windows build by using MOCK_CONST_METHOD3 instead of MOCK_METHOD.

show more ...

Revert "[ProfileData] Read and symbolize raw memprof profiles."

This reverts commit 26f978d4c5ad0d2217940ef7625b0c3c0d576988.

This patch added a transitive dependency on libcurl via symbolize.
See

Revert "[ProfileData] Read and symbolize raw memprof profiles."

This reverts commit 26f978d4c5ad0d2217940ef7625b0c3c0d576988.

This patch added a transitive dependency on libcurl via symbolize.
See discussion
https://reviews.llvm.org/D116784#inline-1137928
https://reviews.llvm.org/D113717#3295350

show more ...

[ProfileData] Read and symbolize raw memprof profiles.

This change extends the RawMemProfReader to read all the sections of the
raw profile and symbolize the virtual addresses recorded as part of th

[ProfileData] Read and symbolize raw memprof profiles.

This change extends the RawMemProfReader to read all the sections of the
raw profile and symbolize the virtual addresses recorded as part of the
callstack for each allocation. For now the symbolization is used to
display the contents of the profile with llvm-profdata.

Differential Revision: https://reviews.llvm.org/D116784

show more ...

[memprof] Print out the summary in YAML format.

Print out the profile summary in YAML format to make it easier to for
tools and tests to read in the contents of the raw profile.

Differential Revisi

[memprof] Print out the summary in YAML format.

Print out the profile summary in YAML format to make it easier to for
tools and tests to read in the contents of the raw profile.

Differential Revision: https://reviews.llvm.org/D116783

show more ...

[InstrProf] Add single byte coverage mode

Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhea

[InstrProf] Add single byte coverage mode

Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block

The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.

When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D116180

show more ...

[InstrProf][NFC] Refactor Profile kind into a bitset enum.

This change refactors the ProfileKind enum into a bitset enum to
represent the different attributes a profile can have. This change
simplif

[InstrProf][NFC] Refactor Profile kind into a bitset enum.

This change refactors the ProfileKind enum into a bitset enum to
represent the different attributes a profile can have. This change
simplifies the logic in the instrprof writer when multiple profiles are
merged together. In the future we plan on introducing a new memory
profile section which will extend the enum by one additional entry.
Without this change when accounting for memory profiles will have to be
maintained separately and will make the logic more complex.

Differential Revision: https://reviews.llvm.org/D115393

show more ...