Reapply: Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries (#106165)

This retries #90692 which was reverted previously due to issues with
lld-available being set, even if the

Reapply: Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries (#106165)

This retries #90692 which was reverted previously due to issues with
lld-available being set, even if the copy of lld is not built from
source.

This does not change any code compared to #90692 to address the
lld-available issue.
The main change w.r.t, lld-available is xfailing tests in PR #99056
(until a longer term fix is available).

show more ...

Revert "Reapply "Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries" (#90610)" (#91194)

Reverts llvm/llvm-project#90692

Breaking PPC buildbots. The bots are not meant to test

Revert "Reapply "Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries" (#90610)" (#91194)

Reverts llvm/llvm-project#90692

Breaking PPC buildbots. The bots are not meant to test LLD, but are
running a test that is using an old version of LLD without the change
(so is incompatible). Revert until a fix is found.

show more ...

Reapply "Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries" (#90610) (#90692)

This reverts commit 2aabfc811670beb843074c765c056fff4a7b443b.

Add fixes to LLD and Gold tests m

Reapply "Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries" (#90610) (#90692)

This reverts commit 2aabfc811670beb843074c765c056fff4a7b443b.

Add fixes to LLD and Gold tests missed in original change.

Co-authored-by: Jan Voung <jvoung@google.com>

show more ...

Revert "Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries" (#90610)

Reverts llvm/llvm-project#90497
Broke some LLD tests.

Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries (#90497)

GUID often have content in the higher bits of a 64-bit entry so using
the unabbrev encoding is inefficient (lots of

Use an abbrev to reduce size of VALUE_GUID records in ThinLTO summaries (#90497)

GUID often have content in the higher bits of a 64-bit entry so using
the unabbrev encoding is inefficient (lots of VBR control bits).
Instead, use an abbrev with two 32-bit fixed width chunks.
The abbrev also helps encode the "count" in one place instead of
in every record.

Reduces size of distributed backend summary files by 8.7% in one
example app.

Co-authored-by: Jan Voung <jvoung@google.com>

show more ...

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of missing
frames from tail calls in profiled call stacks for MemProf of profiled
binaries that did not disable tail call elimination. A follow on change
will add the use of this new flag during MemProf context disambiguation.

The new flag is encoded in the bitcode along with either the hotness
flag from the profile, or the relative block frequency under the
-write-relbf-to-summary flag when there is no profile data.
Because we now will always have some additional call edge information, I
have removed the non-profile function summary record format, and we
simply encode the tail call flag along with a hotness type of none when
there is no profile information or relative block frequency. The change
of record format and name caused most of the test case changes.

I have added explicit testing of generation of the new tail call flag
into the bitcode and IR assembly format as part of the changes to
llvm/test/Bitcode/thinlto-function-summary-refgraph.ll. I have also
added round trip testing through assembly and bitcode to
llvm/test/Assembler/thinlto-summary.ll.

show more ...

Revert "[ThinLTO] Disable partial sample profile scaling by default"

This reverts commit aae8524bcc26cf04729f2bbc02ecb54233a587e4, which was
found to cause a few unexpected benchmark performance dif

Revert "[ThinLTO] Disable partial sample profile scaling by default"

This reverts commit aae8524bcc26cf04729f2bbc02ecb54233a587e4, which was
found to cause a few unexpected benchmark performance differences that
need investigation.

show more ...

[ThinLTO] Disable partial sample profile scaling by default

As pointed out in
https://discourse.llvm.org/t/undeterministic-thin-index-file/69985, the
block count added to distributed ThinLTO index f

[ThinLTO] Disable partial sample profile scaling by default

As pointed out in
https://discourse.llvm.org/t/undeterministic-thin-index-file/69985, the
block count added to distributed ThinLTO index files breaks incremental
builds on ThinLTO - if any linked file has a different number of BBs,
then the accumulated sum placed in the index files will change, causing
all ThinLTO backend compiles to be redone.

This was only used for partial sample profiles, and was therefore
removed for other cases (3adc6e03080c6d38a51f5c5b6744b7c0d9c7541b).

Subsequent testing did not show a performance effect of disabling this
feature even for partial sample profiles. Therefore, switch the default
to false. If this does not cause a noticeable performance degradation
after the default flip, we can remove this support completely.

Differential Revision: https://reviews.llvm.org/D151249

show more ...

[ThinLTO] Remove BlockCount for non partial sample profile builds

As pointed out in
https://discourse.llvm.org/t/undeterministic-thin-index-file/69985, the
block count added to distributed ThinLTO i

[ThinLTO] Remove BlockCount for non partial sample profile builds

As pointed out in
https://discourse.llvm.org/t/undeterministic-thin-index-file/69985, the
block count added to distributed ThinLTO index files breaks incremental
builds on ThinLTO - if any linked file has a different number of BBs,
then the accumulated sum placed in the index files will change, causing
all ThinLTO backend compiles to be redone.

The block count is only used for scaling of partial sample profiles, and
was added in D80403 for D79831.

This patch simply removes this field from the index files of non partial
sample profile compiles, which is NFC on the output of the compiler.

We subsequently need to see if this can be removed for partial sample
profiles without signficant performance loss, or redesigned in a way
that does not destroy caching.

Differential Revision: https://reviews.llvm.org/D148746

show more ...