[llvm-profdata] Add block percent to detailed summary (#105915)

[memprof] Improve the way we express Frames in YAML (#119629)

This patch does two things:

- During deserialization, we accept a function name for Frame as an
alternative to the usual GUID expr

[memprof] Improve the way we express Frames in YAML (#119629)

This patch does two things:

- During deserialization, we accept a function name for Frame as an
alternative to the usual GUID expressed as a hexadecimal number.

- During serialization, we print a GUID of Frame as a 16-digit
hexadecimal number prefixed with 0x in the usual way. (Without this
patch, we print a decimal number, which is not customary.)

The patch uses a machinery called "normalization" in YAML I/O, which
lets us serialize and deserialize into an alternative data structure.
For our use case, we have an alternative Frame data structure, which
is identical to "struct Frame" except that Function is of type
GUIDHex64 instead of GlobalValue::GUID. This alternative type
supports the two bullet points above without modifying "struct Frame"
at all.

show more ...

[memprof] Move YAML support to MemProfYAML.h (NFC) (#119515)

The YAML support is increasing in size, so this patch moves it to a
separate file.

[memprof] Add YAML read/write support to llvm-profdata (#118915)

This patch adds YAML read/write support to llvm-profdata. The primary
intent is to accommodate MemProf profiles in test cases, ther

[memprof] Add YAML read/write support to llvm-profdata (#118915)

This patch adds YAML read/write support to llvm-profdata. The primary
intent is to accommodate MemProf profiles in test cases, thereby
avoiding the binary format.

The read support is via llvm-profdata merge. This is useful when we
want to verify that the compiler does the right thing on a given .ll
file and a MemProf profile in a test case. In the test case, we would
convert the MemProf profile in YAML to an indexed profile and invoke
the compiler on the .ll file along with the indexed profile.

The write support is via llvm-profdata show --memory. This is useful
when we wish to convert an indexed MemProf profile to YAML while
writing tests. We would compile a test case in C++, run it for an
indexed MemProf profile, and then convert it to the text format.

show more ...

[PGO] Add option to always instrumenting loop entries (#116789)

This patch extends the PGO infrastructure with an option to prefer the
instrumentation of loop entry blocks.
This option is a genera

[PGO] Add option to always instrumenting loop entries (#116789)

This patch extends the PGO infrastructure with an option to prefer the
instrumentation of loop entry blocks.
This option is a generalization of
https://github.com/llvm/llvm-project/commit/19fb5b467bb97f95eace1f3637d2d1041cebd3ce,
and helps to cover cases where the loop exit is never executed.
An example where this can occur are event handling loops.

Note that change does NOT change the default behavior.

show more ...

[memprof] Remove MemProf format Version 1 (#117357)

This patch removes MemProf format Version 1 now that Version 2 and 3
are working well.

[memprof] Add MemProfReader::takeMemProfData (#116769)

This patch adds MemProfReader::takeMemProfData, a function to return
the complete MemProf profile from the reader. We can directly pass
its

[memprof] Add MemProfReader::takeMemProfData (#116769)

This patch adds MemProfReader::takeMemProfData, a function to return
the complete MemProf profile from the reader. We can directly pass
its return value to InstrProfWriter::addMemProfData without having to
deal with the indivual components of the MemProf profile. The new
function is named "take", but it doesn't do std::move yet because of
type differences (DenseMap v.s. MapVector).

The end state I'm trying to get to is roughly as follows:

- MemProfReader accepts IndexedMemProfData as a parameter as opposed
to the three individual components (frames, call stacks, and
records).

- MemProfReader keeps IndexedMemProfData as a class member without
decomposing it into its individual components.

- MemProfReader returns IndexedMemProfData like:

IndexedMemProfData takeMemProfData() {
return std::move(MemProfData);
}

show more ...

[llvm-profdata] fix typo (#116754)

[memprof] Remove MemProf format Version 0 (#116442)

This patch removes MemProf format Version 0 now that version 2 and 3
seem to be working well.

I'm not touching version 1 for now because some

[memprof] Remove MemProf format Version 0 (#116442)

This patch removes MemProf format Version 0 now that version 2 and 3
seem to be working well.

I'm not touching version 1 for now because some tests still rely on
version 1.

Note that Version 0 is identical to Version 1 except that the MemProf
section of the indexed format has a MemProf version field.

show more ...

[MemProf] Support for random hotness when writing profile (#113998)

Add support for generating random hotness in the memprof profile writer,
to be used for testing. The random seed is printed to st

[MemProf] Support for random hotness when writing profile (#113998)

Add support for generating random hotness in the memprof profile writer,
to be used for testing. The random seed is printed to stderr, and an
additional option enables providing a specific seed in order to
reproduce a particular random profile.

show more ...

[tools] Don't call StringRef::str() when calling StringMap::find (NFC) (#113119)

StringMap::find takes StringRef. We don't need to create an instance
of std::string from StringRef only to convert

[tools] Don't call StringRef::str() when calling StringMap::find (NFC) (#113119)

StringMap::find takes StringRef. We don't need to create an instance
of std::string from StringRef only to convert it right back to
StringRef.

show more ...

[llvm-profdata] Default to MemProf version 3 (#108863)

It's very confusing to have support for Verion 3 but not default to
it. This patch teaches llvm-profdata to use MemProf version 3 by
default.

[llvm-profdata] Avoid repeated hash lookups (NFC) (#111629)

[llvm-profdata] Fix typo in usage (#110434)

From `profata` to `profdata`

[InstrProf] Add debuginfod correlation support (#106606)

This patch adds debuginfod support into llvm-profdata to
find the assosicated executable by a build id in a raw
profile to correlate a prof

[InstrProf] Add debuginfod correlation support (#106606)

This patch adds debuginfod support into llvm-profdata to
find the assosicated executable by a build id in a raw
profile to correlate a profile with a provided correlation
kind (debug-info or binary).

show more ...

[llvm-profdata] Enabled functionality to write split-layout profile (#101795)

Using the flag `-split_layout` in llvm-profdata merge, the output
profile can write profiles with and without inlined f

[llvm-profdata] Enabled functionality to write split-layout profile (#101795)

Using the flag `-split_layout` in llvm-profdata merge, the output
profile can write profiles with and without inlined function into two
different extbinary sections (and their FuncOffsetTable too). The
section without inlined functions are marked with `SecFlagFlat` and is
skipped by ThinLTO because it provides no useful info.

The split layout feature was already implemented in SampleProfWriter but
previously there is no way to use it from llvm-profdata.

show more ...

[NFC] Coding style: drop `k` in `kGlobalIdentifierDelimiter` (#98230)

[NFC] Coding style fixes: SampleProf (#98208)

Also some control flow simplifications.

Notably, this doesn't address `sampleprof_error`. I *think* the style
there tries to match `std::error_categ

[NFC] Coding style fixes: SampleProf (#98208)

Also some control flow simplifications.

Notably, this doesn't address `sampleprof_error`. I *think* the style
there tries to match `std::error_category`.

Also left `hash_value` as-is, because it matches what we do in Hashing.h

show more ...

[TypeProf][InstrFDO]Omit vtable symbols in indexed profiles by default (#96520)

- The indexed iFDO profiles contains compressed vtable names for `llvm-profdata show --show-vtables` debugging
us

[TypeProf][InstrFDO]Omit vtable symbols in indexed profiles by default (#96520)

- The indexed iFDO profiles contains compressed vtable names for `llvm-profdata show --show-vtables` debugging
usage. An optimized build doesn't need it and doesn't decompress the blob now [1], since optimized binary has the
source code and IR to find vtable symbols.
- The motivation is to avoid increasing profile size when it's not necessary.
- This doesn't change the indexed profile format and thereby doesn't need a version change.

[1] https://github.com/llvm/llvm-project/blob/eac925fb81f26342811ad1765e8f9919628e2254/llvm/include/llvm/ProfileData/InstrProfReader.h#L696-L699

show more ...

[llvm-profdata] Clean up traverseAllValueSites (NFC) (#95467)

If NV == 0, nothing interesting happens after the "if" statement. We
should just "continue" to the next value site.

While I am at i

[llvm-profdata] Clean up traverseAllValueSites (NFC) (#95467)

If NV == 0, nothing interesting happens after the "if" statement. We
should just "continue" to the next value site.

While I am at it, this patch migrates a use of getValueForSite to
getValueArrayForSite.

show more ...

[memprof] Add MemProf format Version 3 (#93608)

This patch adds Version 3 for development purposes. For now, this
patch adds V3 as a copy of V2.

For the most part, this patch adds "case Version

[memprof] Add MemProf format Version 3 (#93608)

This patch adds Version 3 for development purposes. For now, this
patch adds V3 as a copy of V2.

For the most part, this patch adds "case Version3:" wherever "case
Version2:" appears. One exception is writeMemProfV3, which is copied
from writeMemProfV2 but updated to write out memprof::Version3 to the
MemProf header. We'll incrementally modify writeMemProfV3 in
subsequent patches.

show more ...

[InstrProf] Evaluate function order using test traces (#92451)

The `llvm-profdata order` command is used to compute a function order
using traces from the input profile. Add the `--num-test-traces`

[InstrProf] Evaluate function order using test traces (#92451)

The `llvm-profdata order` command is used to compute a function order
using traces from the input profile. Add the `--num-test-traces` flag to
keep aside N traces to evalute this order. These test traces are assumed
to be the actual function execution order in some experiment. The output
is a number that represents how many page faults we got. Lower is
better.

I tested on a large profile I already had.
```
llvm-profdata order default.profdata --num-test-traces=30
# Ordered 149103 functions
# Total area under the page fault curve: 2.271827e+09
...
```

I also improved `TemporalProfTraceTy::createBPFunctionNodes()` in a few
ways:
* Simplified how `UN`s are computed
* Change how the initial `Node` order is computed
* Filter out rare and common `UN`s
* Output vector is an aliased argument instead of a return

These changes slightly improved the evaluation in my test.
```
llvm-profdata order default.profdata --num-test-traces=30
# Ordered 149103 functions
# Total area under the page fault curve: 2.268586e+09
...
```

show more ...

[llvm-profdata] Fix some style and clang-tidy issues

Fix #92761
Fix #92762

[InstrProf] Remove unused argv in llvm-profdata.cpp (#92435)

https://github.com/llvm/llvm-project/pull/71328 refactored
`llvm-profdata.cpp` to use subcommands (which is super nice), but left
many

[InstrProf] Remove unused argv in llvm-profdata.cpp (#92435)

https://github.com/llvm/llvm-project/pull/71328 refactored
`llvm-profdata.cpp` to use subcommands (which is super nice), but left
many unused `argv` variables. This opts to use `ProgName` where
necessary, and removes `argv` otherwise.

show more ...

[memprof] Reduce schema for Version2 (#89876)

Curently, the compiler only uses several fields of MemoryInfoBlock.
Serializing all fields into the indexed MemProf file simply wastes
storage.

Thi

[memprof] Reduce schema for Version2 (#89876)

Curently, the compiler only uses several fields of MemoryInfoBlock.
Serializing all fields into the indexed MemProf file simply wastes
storage.

This patch limits the schema down to four fields for Version2 by
default. It retains the old behavior of serializing all fields via:

llvm-profdata merge --memprof-version=2 --memprof-full-schema

This patch reduces the size of the indexed MemProf profile I have by
40% (1.6GB down to 1.0GB).

show more ...