[GlobalMerge] Fix inaccurate debug print. (#124377)

This message was not updated when MinSize was added.

[GlobalMerge] Use constructor to set all bits in BitVector. NFC (#124375)

The constructor has an optional bool for the starting value for each
bit. Use that instead of calling set().

[GlobalMerge][NFC] Reland "Skip sorting by profitability when it is not needed"

Relands #124146 but without changes to the sorting algorithm and the following
reverse.

Revert "[GlobalMerge][NFC] Skip sorting by profitability when it is not needed" (#124411)

Reverts llvm/llvm-project#124146 -- new comparator is not a strict-weak
as required by stable_sort.

Co-a

Revert "[GlobalMerge][NFC] Skip sorting by profitability when it is not needed" (#124411)

Reverts llvm/llvm-project#124146 -- new comparator is not a strict-weak
as required by stable_sort.

Co-authored-by: Michael Maitland <michaeltmaitland@gmail.com>

show more ...

[GlobalMerge][NFC] Skip sorting by profitability when it is not needed (#124146)

We were previously sorting by profitability even if we were choosing to
merge all globals together, which is not imp

[GlobalMerge][NFC] Skip sorting by profitability when it is not needed (#124146)

We were previously sorting by profitability even if we were choosing to
merge all globals together, which is not impacted by UsedGlobalSet
order.

We can also remove iteration of UsedGlobalSets in reverse order in both
cases. In the first csae, the order does not matter. In the second case,
we just sort by the order we need instead of sorting in the opposite
direction and calling reverse.

This change should only be an improvement on compile time. I have not
measured it, but I think it would never make things worse.

show more ...

[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNo

[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites (#123737)

As part of the "RemoveDIs" project, BasicBlock::iterator now carries a
debug-info bit that's needed when getFirstNonPHI and similar feed into
instruction insertion positions. Call-sites where that's necessary were
updated a year ago; but to ensure some type safety however, we'd like to
have all calls to getFirstNonPHI use the iterator-returning version.

This patch changes a bunch of call-sites calling getFirstNonPHI to use
getFirstNonPHIIt, which returns an iterator. All these call sites are
where it's obviously safe to fetch the iterator then dereference it. A
follow-up patch will contain less-obviously-safe changes.

We'll eventually deprecate and remove the instruction-pointer
getFirstNonPHI, but not before adding concise documentation of what
considerations are needed (very few).

---------

Co-authored-by: Stephen Tozer <Melamoto@gmail.com>

show more ...

[GlobalMerge][NFC] Fix inaccurate comments (#124136)

I was studying the code here and realized that the comments were talking
about grouping by basic blocks when the code was grouping by Function.

[GlobalMerge][NFC] Fix inaccurate comments (#124136)

I was studying the code here and realized that the comments were talking
about grouping by basic blocks when the code was grouping by Function.
Fix the comments so they reflect what the code is actually doing.

show more ...

[PPC] Replace PPCMergeStringPool with GlobalMerge for Linux (#114850)

Enable merging all constants without looking at use in GlobalMerge by
default to replace PPCMergeStringPool pass on Linux.

Enable aggressive constant merge in GlobalMerge for AIX (#113956)

Enable merging all constants without looking at use in GlobalMerge by
default to replace PPCMergeStringPool pass on AIX.

[GlobalMerge] Aggressively merge constants to reduce TOC entries (#111756)

Symbols that get mapped into the read-only section are loaded as part of
the text segment and will always need a TOC entry

[GlobalMerge] Aggressively merge constants to reduce TOC entries (#111756)

Symbols that get mapped into the read-only section are loaded as part of
the text segment and will always need a TOC entry to be addressable. Add
an option to aggressively merge these read only globals to reduce TOC
usage.

show more ...

Fix issues with GlobalMerge on Mach-O. (#110046)

As a side-effect of PR #101222, GlobalMerge started making transforms
which are unsafe on Mach-O platforms.

Two issues, in particular, are fixed

Fix issues with GlobalMerge on Mach-O. (#110046)

As a side-effect of PR #101222, GlobalMerge started making transforms
which are unsafe on Mach-O platforms.

Two issues, in particular, are fixed here:

1. We must never merge symbols in the `__cfstring` section, as the
linker assumes each object in this section is only ever referenced
directly, and that it can split the section as it likes.

Previously, we avoided this problem because CFString literals are
identified by private-linkage symbols. This patch adds a list of
section-names with special behavior, to avoid merging under Mach-O.

2. When GlobalMerge code was originally written, it had to be careful
about emitting symbol aliases, due to issues with Mach-O's subsection
splitting in the linker with `-dead_strip` enabled. The underlying cause
of this problem was fixed in 2016, via creation of the `.alt_entry`
assembler directive, which allows a symbol to not also imply the start
of a new subsection. GlobalMerge's workaround for that issue was never
removed.

In the meantime, Apple's new ld-prime linker was written, and has a bug
in `.alt_entry` handling. Therefore, even though the original issue was
fixed, we must _continue_ to be careful not to emit any such symbol
aliases. The existing workaround avoided it for InternalLinkage symbols,
but after the above-mentioned PR, we also must avoid emitting aliases
for PrivateLinkage symbols.

I will file an Apple bug-report about this issue, so that it can be
fixed in a future version of ld-prime. But, in the meantime, the
workaround is sufficient for GlobalMerge, unless
`-global-merge-on-externals` is enabled (which it is already not by
default, on MachO platforms, due to the original issue).

Fixes #104625

show more ...

[PowerPC][GlobalMerge] Reduce TOC usage by merging internal and private global data (#101224)

This patch aims to reduce TOC usage by merging internal and private
global data.

Moreover, we also a

[PowerPC][GlobalMerge] Reduce TOC usage by merging internal and private global data (#101224)

This patch aims to reduce TOC usage by merging internal and private
global data.

Moreover, we also add the GlobalMerge pass within the PPCTargetMachine
pipeline, which is disabled by default. This transformation can be
enabled by -ppc-global-merge.

show more ...

[GlobalMerge] Update the GlobalMerge pass to merge private global variables. (#101222)

This patch updates the GlobalMerge pass to be able to handle private
global variables, which is required for a

[GlobalMerge] Update the GlobalMerge pass to merge private global variables. (#101222)

This patch updates the GlobalMerge pass to be able to handle private
global variables, which is required for a follow-up PowerPC specific
GlobalMerge patch to merge internal and private globals.

A new LIT test is also added to exhibit the ability to merge private
globals.

show more ...

[GlobalMerge] Use MapVector to stabilize iteration order

DenseMap iteration order is not guaranteed to be deterministic.

Without the change,
llvm/test/Transforms/GlobalMerge/basic.ll could fail whe

[GlobalMerge] Use MapVector to stabilize iteration order

DenseMap iteration order is not guaranteed to be deterministic.

Without the change,
llvm/test/Transforms/GlobalMerge/basic.ll could fail when
`combineHashValue` changes (#95970).

show more ...

[GlobalMerge] Add MinSize feature to the GlobalMerge Pass. (#93686)

We add a feature that prevents the GlobalMerge pass from considering
data smaller than a minimum size in bytes for merging.

Th

[GlobalMerge] Add MinSize feature to the GlobalMerge Pass. (#93686)

We add a feature that prevents the GlobalMerge pass from considering
data smaller than a minimum size in bytes for merging.

The MinSize is set in 3 ways:
1. If global-merge-min-data-size is explicitly set, then it uses that
value.
2. If SmallDataLimit is set and non-zero, then SmallDataLimit + 1 is
used.
3. Otherwise, 0 is used, which means all sizes are considered for
merging.

We found that this feature allowed us to see the benefit of the
GlobalMerge pass while eliminating some merging that was not beneficial.
This feature allowed us to enable the GlobalMerge pass on RISC-V in our
downstream by default because it led to improvements on multiple
benchmark suites.

I plan to post a separate patch to propose enabling this by default on
RISC-V. But I do not want that discussion to be part of the discussion
of adding this feature, so I am keeping the patches separate.

show more ...

[GlobalMerge] Use vector::assign in place of fill+resize. NFC (#85723)

Noticed while reviewing the code.

If the resize causes a new allocation, this will fill the new allocation
with zeroes dire

[GlobalMerge] Use vector::assign in place of fill+resize. NFC (#85723)

Noticed while reviewing the code.

If the resize causes a new allocation, this will fill the new allocation
with zeroes directly. Previously, we would fill the old allocation with
zeroes, then copy them to the new allocation before filling the
additional space with zeros.

show more ...

[NFC] Remove unused parameter from shouldAssumeDSOLocal()

[CodeGen] Port GlobalMerge to new pass manager (#77474)

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::

[llvm] Use StringRef::{starts,ends}_with (NFC) (#74956)

This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.

I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.

show more ...

[llvm] Stop including llvm/ADT/SmallPtrSet.h (NFC)

Identified with clangd.

Fix uninitialized scalar members in CodeGen

This change fixes some static code analysis warnings.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D148811

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

Re-land: [MTE] Add AArch64GlobalsTagging Pass

Adds an IR pass for -fsanitize=memtag-globals. This pass goes over the
tag-capable global variables, and replaces them with a tagged global
variable of

Re-land: [MTE] Add AArch64GlobalsTagging Pass

Adds an IR pass for -fsanitize=memtag-globals. This pass goes over the
tag-capable global variables, and replaces them with a tagged global
variable of the same contents. This new global variable will have its
size and alignment adjusted if neccesary so that they're both a multiple
of the tag granule size (16 bytes).

Global merge must also be suppressed for tagged globals, as each global
variable must have a unique tag. This can possibly be relaxed in future;
globals that are identical in size, alignment, and content can possibly
be merged. The major problem comes from tail- or head-merging, which if
left unchecked, could have partially-overlapping global variables with
different memory tags, leading to crashes at runtime.

Reviewed By: fmayer, eugenis

Differential Revision: https://reviews.llvm.org/D133392

show more ...

Revert "[MTE] Add AArch64GlobalsTagging Pass"

This reverts commit 4edfcff71e150770675a19576f698c7bbe788ee2.

Broke the non-aarch64-containing target builds.
https://reviews.llvm.org/D133392 has more

Revert "[MTE] Add AArch64GlobalsTagging Pass"

This reverts commit 4edfcff71e150770675a19576f698c7bbe788ee2.

Broke the non-aarch64-containing target builds.
https://reviews.llvm.org/D133392 has more context.

show more ...

[MTE] Add AArch64GlobalsTagging Pass

Adds an IR pass for -fsanitize=memtag-globals. This pass goes over the
tag-capable global variables, and replaces them with a tagged global
variable of the same

[MTE] Add AArch64GlobalsTagging Pass

Adds an IR pass for -fsanitize=memtag-globals. This pass goes over the
tag-capable global variables, and replaces them with a tagged global
variable of the same contents. This new global variable will have its
size and alignment adjusted if neccesary so that they're both a multiple
of the tag granule size (16 bytes).

Global merge must also be suppressed for tagged globals, as each global
variable must have a unique tag. This can possibly be relaxed in future;
globals that are identical in size, alignment, and content can possibly
be merged. The major problem comes from tail- or head-merging, which if
left unchecked, could have partially-overlapping global variables with
different memory tags, leading to crashes at runtime.

Reviewed By: fmayer, eugenis

Differential Revision: https://reviews.llvm.org/D133392

show more ...