[SampleFDO] New hierarchical discriminator for FS SampleFDO (llvm-profdata part)

This patch was split from https://reviews.llvm.org/D102246
[SampleFDO] New hierarchical discriminator for Flow Sensit

[SampleFDO] New hierarchical discriminator for FS SampleFDO (llvm-profdata part)

This patch was split from https://reviews.llvm.org/D102246
[SampleFDO] New hierarchical discriminator for Flow Sensitive SampleFDO
This is for llvm-profdata part of change. It sets the bit masks for the
profile reader in llvm-profdata. Also add an internal option
"-fs-discriminator-pass" for show and merge command to process the profile
offline.

This patch also moved setDiscriminatorMaskedBitFrom() to
SampleProfileReader::create() to simplify the interface.

Differential Revision: https://reviews.llvm.org/D103550

show more ...

[CSSPGO][llvm-profdata] Support trimming cold context when merging profiles

The change adds support for triming and merging cold context when mergine CSSPGO profiles using llvm-profdata. This is sim

[CSSPGO][llvm-profdata] Support trimming cold context when merging profiles

The change adds support for triming and merging cold context when mergine CSSPGO profiles using llvm-profdata. This is similar to the context profile trimming in llvm-profgen, however the flexibility to trim cold context after profile is generated can be useful.

Differential Revision: https://reviews.llvm.org/D100528

show more ...

[CSSPGO] Fix dangling context strings and improve profile order consistency and error handling

This patch fixed the following issues along side with some refactoring:

1. Fix bugs where StringRef fo

[CSSPGO] Fix dangling context strings and improve profile order consistency and error handling

This patch fixed the following issues along side with some refactoring:

1. Fix bugs where StringRef for context string out live the underlying std::string. We now keep string table in profile generator to hold std::strings. We also do the same for bracketed context strings in profile writer.
2. Make sure profile output strictly follow (total sample, name) order. Previously, there's inconsistency between ProfileMap's key and FunctionSamples's name, leading to inconsistent ordering. This is now fixed by introducing context profile canonicalization. Assertions are also added to make sure ProfileMap's key and FunctionSamples's name are always consistent.
3. Enhanced error handling for profile writing to make sure we bubble up errors properly for both llvm-profgen and llvm-profdata when string table is not populated correctly for extended binary profile.
4. Keep all internal context representation bracket free. This avoids creating new strings for context trimming, merging and preinline. getNameWithContext API is now simplied accordingly.
5. Factor out the code for context trimming and merging into SampleContextTrimmer in SampleProf.cpp. This enables llvm-profdata to use the trimmer when merging profiles. Changes in llvm-profgen will be in separate patch.

Differential Revision: https://reviews.llvm.org/D100090

show more ...

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a C

[SystemZ][z/OS][Windows] Add new OF_TextWithCRLF flag and use this flag instead of OF_Text

Problem:
On SystemZ we need to open text files in text mode. On Windows, files opened in text mode adds a CRLF '\r\n' which may not be desirable.

Solution:
This patch adds two new flags

- OF_CRLF which indicates that CRLF translation is used.
- OF_TextWithCRLF = OF_Text | OF_CRLF indicates that the file is text and uses CRLF translation.

Developers should now use either the OF_Text or OF_TextWithCRLF for text files and OF_None for binary files. If the developer doesn't want carriage returns on Windows, they should use OF_Text, if they do want carriage returns on Windows, they should use OF_TextWithCRLF.

So this is the behaviour per platform with my patch:

z/OS:
OF_None: open in binary mode
OF_Text : open in text mode
OF_TextWithCRLF: open in text mode

Windows:
OF_None: open file with no carriage return
OF_Text: open file with no carriage return
OF_TextWithCRLF: open file with carriage return

The Major change is in llvm/lib/Support/Windows/Path.inc to only set text mode if the OF_CRLF is set.
```
if (Flags & OF_CRLF)
CrtOpenFlags |= _O_TEXT;
```

These following files are the ones that still use OF_Text which I left unchanged. I modified all these except raw_ostream.cpp in recent patches so I know these were previously in Binary mode on Windows.
./llvm/lib/Support/raw_ostream.cpp
./llvm/lib/TableGen/Main.cpp
./llvm/tools/dsymutil/DwarfLinkerForBinary.cpp
./llvm/unittests/Support/Path.cpp
./clang/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp
./clang/lib/Frontend/CompilerInstance.cpp
./clang/lib/Driver/Driver.cpp
./clang/lib/Driver/ToolChains/Clang.cpp

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D99426

show more ...

[llvm-profdata] Make sure to consume Error on the error path of setIsIRLevelProfile

Encountered a crash while running a debug build, where this code path would be taken due to a mismatch in profile

[llvm-profdata] Make sure to consume Error on the error path of setIsIRLevelProfile

Encountered a crash while running a debug build, where this code path would be taken due to a mismatch in profile coverage data versions. Without consuming the error, an assert would be triggered inside the destructor of Error.

Differential Revision: https://reviews.llvm.org/D99457

show more ...

[CSSPGO] llvm-profdata support for CS profile.

Context-sensitive AutoFDO profile has a different name scheme where full calling contexts are encoded as function names. When processing CS proifle, ll

[CSSPGO] llvm-profdata support for CS profile.

Context-sensitive AutoFDO profile has a different name scheme where full calling contexts are encoded as function names. When processing CS proifle, llvm-profdata should use full contexts instead of leaf function names.

Reviewed By: wmi, wenlei, wlei

Differential Revision: https://reviews.llvm.org/D97998

show more ...

[llvm-profdata] Emit Error when Invalid MemOpSize Section is Created by llvm-profdata

Under certain (currently unknown) conditions, llvm-profdata is outputting
profiles that have two consecutive ent

[llvm-profdata] Emit Error when Invalid MemOpSize Section is Created by llvm-profdata

Under certain (currently unknown) conditions, llvm-profdata is outputting
profiles that have two consecutive entries in the MemOPSize section for the
value 0. This causes the PGOMemOPSizeOpt pass to output an invalid switch
instruction with two cases for 0. As mentioned, we’re not quite sure what’s
causing this to happen, but this patch prevents llvm-profdata from outputting a
profile that has this problem and gives an error with a request for a
reproducible.

Differential Revision: https://reviews.llvm.org/D92074

show more ...

[CSSPGO] Support of CS profiles in extended binary format.

This change brings up support of context-sensitive profiles in the format of extended binary. Existing sample profile reader/writer/merger

[CSSPGO] Support of CS profiles in extended binary format.

This change brings up support of context-sensitive profiles in the format of extended binary. Existing sample profile reader/writer/merger code is being tweaked to reflect the fact of bracketed input contexts, like (`[...]`). The paired brackets are also needed in extbinary profiles because we don't yet have an otherwise good way to tell calling contexts apart from regular function names since the context delimiter `@` can somehow serve as a part of the C++ mangled names.

Reviewed By: wmi, wenlei

Differential Revision: https://reviews.llvm.org/D95547

show more ...

[llvm] Remove redundant return and continue statements (NFC)

Identified with readability-redundant-control-flow.

[llvm] Simplify string comparisons (NFC)

Identified with readability-string-compare.

[tools] Mark output of tools as text if it is really text

This is a continuation of https://reviews.llvm.org/D67696. The following tools also need to set the OF_Text flag correctly.

- llvm-prof

[tools] Mark output of tools as text if it is really text

This is a continuation of https://reviews.llvm.org/D67696. The following tools also need to set the OF_Text flag correctly.

- llvm-profdata
- llvm-link

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D94313

show more ...

[CSSPGO] Consume pseudo-probe-based AutoFDO profile

This change enables pseudo-probe-based sample counts to be consumed by the sample profile loader under the regular `-fprofile-sample-use` switch w

[CSSPGO] Consume pseudo-probe-based AutoFDO profile

This change enables pseudo-probe-based sample counts to be consumed by the sample profile loader under the regular `-fprofile-sample-use` switch with minimal adjustments to the existing sample file formats. After the counts are imported, a probe helper, aka, a `PseudoProbeManager` object, is automatically launched to verify the CFG checksum of every function in the current compilation against the corresponding checksum from the profile. Mismatched checksums will cause a function profile to be slipped. A `SampleProfileProber` pass is scheduled before any of the `SampleProfileLoader` instances so that the CFG checksums as well as probe mappings are available during the profile loading time. The `PseudoProbeManager` object is set up right after the profile reading is done. In the future a CFG-based fuzzy matching could be done in `PseudoProbeManager`.

Samples will be applied only to pseudo probe instructions as well as probed callsites once the checksum verification goes through. Those instructions are processed in the same way that regular instructions would be processed in the line-number-based scenario. In other words, a function is processed in a regular way as if it was reduced to just containing pseudo probes (block probes and callsites).

**Adjustment to profile format **

A CFG checksum field is being added to the existing AutoFDO profile formats. So far only the text format and the extended binary format are supported. For the text format, a new line like
```
!CFGChecksum: 12345
```
is added to the end of the body sample lines. For the extended binary profile format, we introduce a metadata section to store the checksum map from function names to their CFG checksums.

Differential Revision: https://reviews.llvm.org/D92347

show more ...

[llvm-profdata]Fix llvm-profdata crash on compact binary profile

llvm-profdata `show` and `overlap` will crash in `getFuncName` on compact binary profile. This change fixed this by switching to use

[llvm-profdata]Fix llvm-profdata crash on compact binary profile

llvm-profdata `show` and `overlap` will crash in `getFuncName` on compact binary profile. This change fixed this by switching to use `getName`.

`getFuncName` is misused in llvm-profdata. As showed below, `GUIDToFuncNameMap` is only supported in compilation mode, there is no initialization in llvm-profdata. Compact profile whose MD5 is true would try to query `GUIDToFuncNameMap` then caused the crash. So fix this by switching to `getName`

Reviewed By: MaskRay, wmi, wenlei, weihe, hoy

Differential Revision: https://reviews.llvm.org/D87740

show more ...

[llvm-profdata] Implement llvm-profdata overlap for sample profiles

Implemented the `llvm-profdata overlap` feature for sample profiles. It reports weighted //similarity// and unweighted //overlap//

[llvm-profdata] Implement llvm-profdata overlap for sample profiles

Implemented the `llvm-profdata overlap` feature for sample profiles. It reports weighted //similarity// and unweighted //overlap// metrics at program and function level for two input profiles. Similarity metrics are symmetric with regards to the order of two input profiles. By default, the tool only reports program-level summary. Users can look into function-level details via additional options `--function`, `--similarity-cutoff`, and `--value-cutoff`.

The similarity metrics are designed as follows:
* Program-level summary
* Whole program profile similarity is an aggregate over function-level similarity `FS`: `PS = sum(FS(A) * avg_weight(A))` for all function `A`.
* Whole program sample overlap: `PSO = common_samples / total_samples`.
* Function overlap: `FO = #common_function / #total_function`.
* Hot-function overlap: `HFO = #common_hot_function / #total_hot_function`.
* Hot-block overlap: `HBO = #common_hot_block / #total_hot_block`.
* Function-level details
* Function-level similarity is an aggregate over line/block-level similarities `BS` of all sample lines/blocks in the function, weighted by the closeness of the function's weights in two profiles: `FS = sum(BS(i)) * (1 - weight_distance(A))`.
* Function-level sample overlap: `FSO = common_samples / total_samples` for samples in the function.

Reviewed By: wenlei, hoyFB, wmi

Differential Revision: https://reviews.llvm.org/D83852

show more ...

Supplement instr profile with sample profile.

PGO profile is usually more precise than sample profile. However, PGO profile
needs to be collected from loadtest and loadtest may not be representative

Supplement instr profile with sample profile.

PGO profile is usually more precise than sample profile. However, PGO profile
needs to be collected from loadtest and loadtest may not be representative
enough to the production workload. Sample profile collected from production
can be used as a supplement -- for functions cold in loadtest but warm/hot
in production, we can scale up the related function in PGO profile if the
function is warm or hot in sample profile.

The implementation contains changes in compiler side and llvm-profdata side.
Given an instr profile and a sample profile, for a function cold in PGO
profile but warm/hot in sample profile, llvm-profdata will either mark
all the counters in the profile to be -1 or scale up the max count in the
function to be above hot threshold, depending on the zero counter ratio in
the profile. The assumption is if there are too many counters being zero
in the function profile, the profile is more likely to cause harm than good,
then llvm-profdata will mark all the counters to be -1 indicating the
function is hot but the profile is unaccountable. In compiler side, if a
function profile with all -1 counters is seen, the function entry count will
be set to be above hot threshold but its internal profile will be dropped.

In the long run, it may be useful to let compiler support using PGO profile
and sample profile at the same time, but that requires more careful design
and more substantial changes to make two profiles work seamlessly. The patch
here serves as a simple intermediate solution.

Differential Revision: https://reviews.llvm.org/D81981

show more ...

[PGO] Supporting code for always instrumenting entry block

This patch includes the supporting code that enables always
instrumenting the function entry block by default.

This patch will NOT the def

[PGO] Supporting code for always instrumenting entry block

This patch includes the supporting code that enables always
instrumenting the function entry block by default.

This patch will NOT the default behavior.

It adds a variant bit in the profile version, adds new directives in
text profile format, and changes llvm-profdata tool accordingly.

This patch is a split of D83024 (https://reviews.llvm.org/D83024)
Many test changes from D83024 are also included.

Differential Revision: https://reviews.llvm.org/D84261

show more ...

[NFC] Extract the code to write instr profile into function writeInstrProfile

So that the function writeInstrProfile can be used in other places.

Differential Revision: https://reviews.llvm.org/D83

[NFC] Extract the code to write instr profile into function writeInstrProfile

So that the function writeInstrProfile can be used in other places.

Differential Revision: https://reviews.llvm.org/D83521

show more ...

[llvm-profdata] --hot-func-list: fix some style issues in D81800

Reviewed By: wenlei, hoyFB

Differential Revision: https://reviews.llvm.org/D82500

Add --hot-func-list to llvm-profdata show for sample profiles

Summary:
Add the --hot-func-list feature to llvm-profdata show for sample profiles. This feature prints a list of hot functions whose ma

Add --hot-func-list to llvm-profdata show for sample profiles

Summary:
Add the --hot-func-list feature to llvm-profdata show for sample profiles. This feature prints a list of hot functions whose max sample count are above the 99% threshold, with their numbers of total samples, total samples percentage, max samples, entry samples, and their function names.

Test Plan:

Reviewers: wenlei, hoyFB

Reviewed By: wenlei, hoyFB

Subscribers: hoyFB, wenlei, weihe, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D82355

show more ...

Revert "Add --hot-func-list to llvm-profdata show for sample profiles"

This reverts commit 7348b951fe74f306970f6ac567fe5dddbb1c42d4.
It is causing Asan failures.

Add --hot-func-list to llvm-profdata show for sample profiles

Summary: Add the --hot-func-list feature to llvm-profdata show for sample profiles. This feature prints a list of hot functions whose ma

Add --hot-func-list to llvm-profdata show for sample profiles

Summary: Add the --hot-func-list feature to llvm-profdata show for sample profiles. This feature prints a list of hot functions whose max sample count are above the 99% threshold, with their numbers of total samples, total samples percentage, max samples, entry samples, and their function names.

Reviewers: wmi, hoyFB, wenlei

Reviewed By: wmi

Subscribers: hoyFB, wenlei, llvm-commits, weihe

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81800

show more ...

[SampleFDO] Rename llvm-profdata flag -partial-profile to -gen-partial-profile.

The internal flag -partial-profile in llvm conflicts with the flag with
the same name in llvm-profdata. The conflict h

[SampleFDO] Rename llvm-profdata flag -partial-profile to -gen-partial-profile.

The internal flag -partial-profile in llvm conflicts with the flag with
the same name in llvm-profdata. The conflict happens in builds with
LLVM_LINK_LLVM_DYLIB enabled. In this case the tools are linked with libLLVM
and we end up with two definitions for the same cl::opt.

The patch renames llvm-profdata flag -partial-profile to -gen-partial-profile.

show more ...

[llvm-profdata] Support -detailed-summary for Sample Profile

Summary: Add -detailed-summary support for sample profile dump to match that of instrumentation profile.

Reviewers: wmi, davidxl, hoyFB

[llvm-profdata] Support -detailed-summary for Sample Profile

Summary: Add -detailed-summary support for sample profile dump to match that of instrumentation profile.

Reviewers: wmi, davidxl, hoyFB

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79291

show more ...

Recommit [SampleFDO] Add flag for partial profile.

Fix the error of show-prof-info.test on some platforms without zlib.

The common profile usage is to collect profile from a target and then use the

Recommit [SampleFDO] Add flag for partial profile.

Fix the error of show-prof-info.test on some platforms without zlib.

The common profile usage is to collect profile from a target and then use the profile to guide the optimized build for the same target. There are some cases that no profile can be collected for a target. In those cases, although no full profile is available, it is possible to have some partial profile collected from other targets to optimize common libraries and utilities. A flag is needed to tell the partial profile from the full profile apart, so compiler can use different strategy for them.

Differential Revision: https://reviews.llvm.org/D77426

show more ...

Revert "[SampleFDO] Add flag for partial profile." show-prof-info.test breaks on some platforms.

This reverts commit e3ba652a1440794eff0b43ce747f1b0488585d22.