[memprof] Add Version2 of the indexed MemProf format (#89100)

This patch adds Version2 of the indexed MemProf format. The new
format comes with a hash table from CallStackId to actual call stacks

[memprof] Add Version2 of the indexed MemProf format (#89100)

This patch adds Version2 of the indexed MemProf format. The new
format comes with a hash table from CallStackId to actual call stacks
llvm::SmallVector<FrameId>. The rest of the format refers to call
stacks with CallStackId. This "values + references" model effectively
deduplicates call stacks. Without this patch, a large indexed memprof
file of mine shrinks from 4.4GB to 1.6GB, a 64% reduction.

This patch does not make Version2 generally available yet as I am
planning to make a few more changes to the format.

show more ...

[memprof] Use structured binding (NFC) (#88363)

[memprof] Rename RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h} (NFC) (#88200)

This patch renames RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h},
respectively. Also, it re-creates RawMemProf

[memprof] Rename RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h} (NFC) (#88200)

This patch renames RawMemProfReader.{cpp,h} to MemProfReader.{cpp,h},
respectively. Also, it re-creates RawMemProfReader.h just to include
MemProfReader.h for compatibility with out-of-tree users.

show more ...

[InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (#66825)

(The profile format change is split into a standalone change into https://github.com/llvm/llvm-project/pull/8169

[InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (#66825)

(The profile format change is split into a standalone change into https://github.com/llvm/llvm-project/pull/81691)

* For InstrFDO value profiling, implement instrumentation and lowering for virtual table address.
* This is controlled by `-enable-vtable-value-profiling` and off by default.
* When the option is on, raw profiles will carry serialized `VTableProfData` structs and compressed vtables as payloads.

* Implement profile reader and writer support
* Raw profile reader is used by `llvm-profdata` but not compiler. Raw profile reader will construct InstrProfSymtab with symbol names, and map profiled runtime address to vtable symbols.
* Indexed profile reader is used by `llvm-profdata` and compiler. When initialized, the reader stores a pointer to the beginning of in-memory compressed vtable names and the length of string. When used in `llvm-profdata`, reader decompress the string to show symbols of a profiled site. When used in compiler, string decompression doesn't
happen since IR is used to construct InstrProfSymtab.
* Indexed profile writer collects the list of vtable names, and stores that to index profiles.
* Text profile reader and writer support are added but mostly follow the implementation for indirect-call value type.
* `llvm-profdata show -show-vtables <args> <profile>` is implemented.

rfc in
https://discourse.llvm.org/t/rfc-dynamic-type-profiling-and-optimizations-in-llvm/74600#pick-instrumentation-points-and-instrument-runtime-types-7

show more ...

[memprof] Add MemProf version (#86414)

This patch adds a version field to the MemProf section of the indexed
profile format, calling the new version "version 1". The existing
version is called "v

[memprof] Add MemProf version (#86414)

This patch adds a version field to the MemProf section of the indexed
profile format, calling the new version "version 1". The existing
version is called "version 0".

The writer supports both versions via a command-line option:

llvm-profdata merge --memprof-version=1 ...

The reader supports both versions by automatically detecting the
version from the header.

show more ...

[PGO] Add support for writing previous indexed format (#84505)

Enable temporary support to ease use of new llvm-profdata with slightly
older indexed profiles after 16e74fd48988ac95551d0f64e1b36f78a

[PGO] Add support for writing previous indexed format (#84505)

Enable temporary support to ease use of new llvm-profdata with slightly
older indexed profiles after 16e74fd48988ac95551d0f64e1b36f78a82a89a2,
which bumped the indexed format for type profiling.

show more ...

Rename llvm::ThreadPool -> llvm::DefaultThreadPool (NFC) (#83702)

The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.

This is a breaking change: cli

Rename llvm::ThreadPool -> llvm::DefaultThreadPool (NFC) (#83702)

The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.

This is a breaking change: clients who use to create a ThreadPool must
now create a DefaultThreadPool instead.

show more ...

[NFC][InstrProf]Generalize getParsedIRPGOFuncName to getParsedIRPGOName (#81054)

- Function getParsedIRPGOFuncName splits name by delimiter. The `[filename;]mangled-name` format could be generalized

[NFC][InstrProf]Generalize getParsedIRPGOFuncName to getParsedIRPGOName (#81054)

- Function getParsedIRPGOFuncName splits name by delimiter. The `[filename;]mangled-name` format could be generalized for non-function global values (e.g., vtables for type profiling). So rename the
function.
- Use kGlobalIdentifierDelimiter rather than semicolon directly for defragmentation.

show more ...

[llvm-profdata] Simplify a string equality comparison (NFC)

Added feature in llvm-profdata merge to filter functions from the profile (#78378)

`--function=<regex>` Include functions matching regex in the output
`--no-function=<regex>` Exclude functions matc

Added feature in llvm-profdata merge to filter functions from the profile (#78378)

`--function=<regex>` Include functions matching regex in the output
`--no-function=<regex>` Exclude functions matching regex from the output

If both are specified, `--no-function` has a higher precedence if a
function name matches both filters

show more ...

[llvm-driver] Fix usage of `InitLLVM` on Windows (#76306)

Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld

[llvm-driver] Fix usage of `InitLLVM` on Windows (#76306)

Previously, some tools such as `clang` or `lld` which require strict
order for certain command-line options, such as `clang -cc1` or `lld
-flavor`, would not longer work on Windows, when these tools were linked
as part of `llvm-driver`. This was caused by `InitLLVM` which was part
of the `*_main()` function of these tools, which in turn calls
`windows::GetCommandLineArguments`. That function completly replaces
argc/argv by new UTF-8 contents, so any ajustements to argc/argv made by
`llvm-driver` prior to calling these tools was reset.

`InitLLVM` is now called by the `llvm-driver`. Any tool that
participates in (or is part of) the `llvm-driver` doesn't call
`InitLLVM` anymore.

show more ...

[InstrProf] No linkage prefixes in IRPGO names (#76994)

Change the format of IRPGO counter names to
`[<filepath>;]<mangled-name>` which is computed by
`GlobalValue::getGlobalIdentifier()` to fix #

[InstrProf] No linkage prefixes in IRPGO names (#76994)

Change the format of IRPGO counter names to
`[<filepath>;]<mangled-name>` which is computed by
`GlobalValue::getGlobalIdentifier()` to fix #74565.

In fe051934cbb0aaf25d960d7d45305135635d650b
(https://reviews.llvm.org/D156569) the format of IRPGO counter names was
changed to be `[<filepath>;]<linkage-name>` where `<linkage-name>` is
basically `F.getName()` with some prefix, e.g., `_` or `l_` on Mach-O
(yes, it is confusing that `<linkage-name>` is computed with
`Mangler().getNameWithPrefix()` while `<mangled-name>` is just
`F.getName()`). We discovered in #74565 that this causes some missed
import issues on some targets and #74008 is a partial fix.

Since `<mangled-name>` may not match the `<linkage-name>` on some
targets like Mach-O, we will need to post-process the output of
`llvm-profdata order` before passing to the linker via `-order_file`.

Profiles generated after fe051934cbb0aaf25d960d7d45305135635d650b will
become stale after this diff, but I think this is acceptable since that
patch landed after the LLVM 18 cut which hasn't been released yet.

show more ...

[llvm-profdata] Use semicolon as the delimiter for supplementary profiles. (#75080)

When merging instrFDO profiles with afdo profile as supplementary, instrFDO counters for static functions are stor

[llvm-profdata] Use semicolon as the delimiter for supplementary profiles. (#75080)

When merging instrFDO profiles with afdo profile as supplementary, instrFDO counters for static functions are stored with function's PGO name (with filename.cpp; prefix).
- This pull request fixes the delimiter used when a PGO function name is 'normalized' for AFDO look-up.

show more ...

[llvm-profdata] Modernize FuncSampleStats, ValueSitesStats, and HotFuncInfo (NFC)

[Profile] Add binary profile correlation for code coverage. (#69493)

## Motivation
Since we don't need the metadata sections at runtime, we can somehow
offload them from memory at runtime. Initial

[Profile] Add binary profile correlation for code coverage. (#69493)

## Motivation
Since we don't need the metadata sections at runtime, we can somehow
offload them from memory at runtime. Initially, I explored [debug info
correlation](https://discourse.llvm.org/t/instrprofiling-lightweight-instrumentation/59113),
which is used for PGO with value profiling disabled. However, it
currently only works with DWARF and it's be hard to add such artificial
debug info for every function in to CodeView which is used on Windows.
So, offloading profile metadata sections at runtime seems to be a
platform independent option.

## Design
The idea is to use new section names for profile name and data sections
and mark them as metadata sections. Under this mode, the new sections
are non-SHF_ALLOC in ELF. So, they are not loaded into memory at runtime
and can be stripped away as a post-linking step. After the process
exits, the generated raw profiles will contains only headers + counters.
llvm-profdata can be used correlate raw profiles with the unstripped
binary to generate indexed profile.

## Data
For chromium base_unittests with code coverage on linux, the binary size
overhead due to instrumentation reduced from 64M to 38.8M (39.4%) and
the raw profile files size reduce from 128M to 68M (46.9%)
```
$ bloaty out/cov/base_unittests.stripped -- out/no-cov/base_unittests.stripped
FILE SIZE VM SIZE
-------------- --------------
+121% +30.4Mi +121% +30.4Mi .text
[NEW] +14.6Mi [NEW] +14.6Mi __llvm_prf_data
[NEW] +10.6Mi [NEW] +10.6Mi __llvm_prf_names
[NEW] +5.86Mi [NEW] +5.86Mi __llvm_prf_cnts
+95% +1.75Mi +95% +1.75Mi .eh_frame
+108% +400Ki +108% +400Ki .eh_frame_hdr
+9.5% +211Ki +9.5% +211Ki .rela.dyn
+9.2% +95.0Ki +9.2% +95.0Ki .data.rel.ro
+5.0% +87.3Ki +5.0% +87.3Ki .rodata
[ = ] 0 +13% +47.0Ki .bss
+40% +1.78Ki +40% +1.78Ki .got
+12% +1.49Ki +12% +1.49Ki .gcc_except_table
[ = ] 0 +65% +1.23Ki .relro_padding
+62% +1.20Ki [ = ] 0 [Unmapped]
+13% +448 +19% +448 .init_array
+8.8% +192 [ = ] 0 [ELF Section Headers]
+0.0% +136 +0.0% +80 [7 Others]
+0.1% +96 +0.1% +96 .dynsym
+1.2% +96 +1.2% +96 .rela.plt
+1.5% +80 +1.2% +64 .plt
[ = ] 0 -99.2% -3.68Ki [LOAD #5 [RW]]
+195% +64.0Mi +194% +64.0Mi TOTAL
$ bloaty out/cov-cor/base_unittests.stripped -- out/no-cov/base_unittests.stripped
FILE SIZE VM SIZE
-------------- --------------
+121% +30.4Mi +121% +30.4Mi .text
[NEW] +5.86Mi [NEW] +5.86Mi __llvm_prf_cnts
+95% +1.75Mi +95% +1.75Mi .eh_frame
+108% +400Ki +108% +400Ki .eh_frame_hdr
+9.5% +211Ki +9.5% +211Ki .rela.dyn
+9.2% +95.0Ki +9.2% +95.0Ki .data.rel.ro
+5.0% +87.3Ki +5.0% +87.3Ki .rodata
[ = ] 0 +13% +47.0Ki .bss
+40% +1.78Ki +40% +1.78Ki .got
+12% +1.49Ki +12% +1.49Ki .gcc_except_table
+13% +448 +19% +448 .init_array
+0.1% +96 +0.1% +96 .dynsym
+1.2% +96 +1.2% +96 .rela.plt
+1.2% +64 +1.2% +64 .plt
+2.9% +64 [ = ] 0 [ELF Section Headers]
+0.0% +40 +0.0% +40 .data
+1.2% +32 +1.2% +32 .got.plt
+0.0% +24 +0.0% +8 [5 Others]
[ = ] 0 -22.9% -872 [LOAD #5 [RW]]
-74.5% -1.44Ki [ = ] 0 [Unmapped]
[ = ] 0 -76.5% -1.45Ki .relro_padding
+118% +38.8Mi +117% +38.8Mi TOTAL
```

A few things to note:
1. llvm-profdata doesn't support filter raw profiles by binary id yet,
so when a raw profile doesn't belongs to the binary being digested by
llvm-profdata, merging will fail. Once this is implemented,
llvm-profdata should be able to only merge raw profiles with the same
binary id as the binary and discard the rest (with mismatched/missing
binary id). The workflow I have in mind is to have scripts invoke
llvm-profdata to get all binary ids for all raw profiles, and
selectively choose the raw pnrofiles with matching binary id and the
binary to llvm-profdata for merging.
2. Note: In COFF, currently they are still loaded into memory but not
used. I didn't do it in this patch because I noticed that `.lcovmap` and
`.lcovfunc` are loaded into memory. A separate patch will address it.
3. This should works with PGO when value profiling is disabled as debug
info correlation currently doing, though I haven't tested this yet.

show more ...

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[NFC] Typo fix (#74033)

Fix spelling error from `linakge` to `linkage`.

Co-authored-by: lifengxiang <lifengxiang.1025@bytedance.com>

[nfc][llvm-profdata] Use cl::Subcommand to organize subcommand and options in llvm-profdata (#71328)

- The motivation is to reduce the number of arguments passed around
(e.g., from `show_main` to `

[nfc][llvm-profdata] Use cl::Subcommand to organize subcommand and options in llvm-profdata (#71328)

- The motivation is to reduce the number of arguments passed around
(e.g., from `show_main` to `show*Profile`). In order to do this, move
function-defined options to global variables, and create
`cl::SubCommand` for {show, merge, overlap, order} to organize options.
- The side-effect by extracting function local options to a C++
namespace is that the extracted options are no longer (lazily)
initialized when the enclosing function runs for the first time.
- `cl::Subcommand` support (introduced in
https://lists.llvm.org/pipermail/llvm-dev/2016-June/101804.html) could
put options in a per-subcommand namespace.
- One option could belong to multiple subcommand. This patch defines
most of the options once and associates them with multiple subcommands
except
1. `overlap` and `show` both has `value-cutoff` with different default
values
([former](https://github.com/llvm/llvm-project/blob/64f62de96609dc3ea9a8a914a9e9445b7f4d625d/llvm/tools/llvm-profdata/llvm-profdata.cpp#L2352)
vs
[latter](https://github.com/llvm/llvm-project/blob/64f62de96609dc3ea9a8a914a9e9445b7f4d625d/llvm/tools/llvm-profdata/llvm-profdata.cpp#L3009)).
Define 'OverlapValueCutoff' and 'ShowValueCutoff' respectively.
2. `show` supports three profile formats in `ProfileKind` while
{`merge`, `overlap`} supports two. Define separate options.
- Clean up obsolete code as a result, including `-h` and `--version`
customizations. These two options are supported for all commands.
Results pasted.
- [-h and
--help](https://gist.github.com/minglotus-6/387490e5eeda2dd2f9c440a424d6f360)
output.
-
[--version](https://gist.github.com/minglotus-6/f905abcc3a346957bd797f2f84c18c1b)
- [llvm-profdata show
--help](https://gist.github.com/minglotus-6/f143079f02af243a94758138c0af471a)

This PR should be `llvm-profdata` only. It depends on
https://github.com/llvm/llvm-project/pull/71981

show more ...

[Profile] Refactor profile correlation. (#70856)

Refactor some code from https://github.com/llvm/llvm-project/pull/69493.

#70712 was reverted due to linking failures. So, `-debug-info-correlate`

[Profile] Refactor profile correlation. (#70856)

Refactor some code from https://github.com/llvm/llvm-project/pull/69493.

#70712 was reverted due to linking failures. So, `-debug-info-correlate` remains unchanged and no new flag added.

show more ...

[llvm-profdata] Emit warning when counter value is greater than 2^56. (#69513)

Fixes #65416

Revert "[Profile] Refactor profile correlation. (#70712)"

This reverts commit 4b383d0af93136b80841fc140da0823dfc441dd4.

[Profile] Refactor profile correlation. (#70712)

Refactor some code from https://github.com/llvm/llvm-project/pull/69493.

Rebase of https://github.com/llvm/llvm-project/pull/69656 on top of main

[Profile] Refactor profile correlation. (#70712)

Refactor some code from https://github.com/llvm/llvm-project/pull/69493.

Rebase of https://github.com/llvm/llvm-project/pull/69656 on top of main
as it was messed up.

show more ...

[llvm-profdata] Do not create numerical strings for MD5 function names read from a Sample Profile. (#66164)

This is phase 2 of the MD5 refactoring on Sample Profile following
https://reviews.llvm.o

[llvm-profdata] Do not create numerical strings for MD5 function names read from a Sample Profile. (#66164)

This is phase 2 of the MD5 refactoring on Sample Profile following
https://reviews.llvm.org/D147740

In previous implementation, when a MD5 Sample Profile is read, the
reader first converts the MD5 values to strings, and then create a
StringRef as if the numerical strings are regular function names, and
later on IPO transformation passes perform string comparison over these
numerical strings for profile matching. This is inefficient since it
causes many small heap allocations.
In this patch I created a class `ProfileFuncRef` that is similar to
`StringRef` but it can represent a hash value directly without any
conversion, and it will be more efficient (I will attach some benchmark
results later) when being used in associative containers.

ProfileFuncRef guarantees the same function name in string form or in
MD5 form has the same hash value, which also fix a few issue in IPO
passes where function matching/lookup only check for function name
string, while returns a no-match if the profile is MD5.

When testing on an internal large profile (> 1 GB, with more than 10
million functions), the full profile load time is reduced from 28 sec to
25 sec in average, and reading function offset table from 0.78s to 0.7s

show more ...

[NFC]Rename InstrProf::getFuncName{,orExternalSymbol} to getFuncOrValName{,IfDefined} (#68240)

- This function looks up MD5ToNameMap to return a name for a given MD5.
https://github.com/llvm/llvm-p

[NFC]Rename InstrProf::getFuncName{,orExternalSymbol} to getFuncOrValName{,IfDefined} (#68240)

- This function looks up MD5ToNameMap to return a name for a given MD5.
https://github.com/llvm/llvm-project/pull/66825 adds MD5 of global
variable names into this map. So rename methods and update comments

show more ...

[llvm-profdata] Modernize SampleOverlapStats (NFC)