[CodeGen] Fix -Wcovered-switch-default in Combiner.cpp (NFC)

/llvm-project/llvm/lib/CodeGen/GlobalISel/Combiner.cpp:220:3:
error: default label in switch which covers all enumeration values [-Werror

[CodeGen] Fix -Wcovered-switch-default in Combiner.cpp (NFC)

/llvm-project/llvm/lib/CodeGen/GlobalISel/Combiner.cpp:220:3:
error: default label in switch which covers all enumeration values [-Werror,-Wcovered-switch-default]
default:
^
1 error generated.

show more ...

[GlobalISel] Combiner: Observer-based DCE and retrying of combines

Continues the work for disabling fixed-point iteration in the Combiner
(#94291).

This introduces improved Observer-based heuristic

[GlobalISel] Combiner: Observer-based DCE and retrying of combines

Continues the work for disabling fixed-point iteration in the Combiner
(#94291).

This introduces improved Observer-based heuristics in the
GISel Combiner to retry combining defs/uses of modified instructions and
for performing sparse dead code elimination.

I have experimented a lot with the heuristics and this seems to be the
minimal set of heuristics that allows disabling fixed-point iteration
for AArch64 CTMark O2 without regressions.
Enabling this globally would pass all regression tests for all official
targets (apart from small benign diffs), but I have made this fully
opt-in for now, because I can't quantify the impact for other targets.

This should mostly be on-par with how the WorkList-aware functions
in the InstCombiner and DAGCombiner handle rescheduling instructions
for recombining.

For performance numbers see my follow-up patch for AArch64 (#102167)

Pull Request: https://github.com/llvm/llvm-project/pull/102163

show more ...

[GlobalISel] Don't remove from unfinalized GISelWorkList

Remove a hack from GISelWorkList caused by the Combiner removing
instructions from an unfinalized GISelWorkList during the DCE phase.
This is

[GlobalISel] Don't remove from unfinalized GISelWorkList

Remove a hack from GISelWorkList caused by the Combiner removing
instructions from an unfinalized GISelWorkList during the DCE phase.
This is in preparation for larger changes to the WorkListMaintainer.

Pull Request: https://github.com/llvm/llvm-project/pull/102158

show more ...

[GlobalISel] Combiner: Install Observer into MachineFunction

The Combiner doesn't install the Observer into the MachineFunction.
This probably went unnoticed, because MachineFunction::getObserver()

[GlobalISel] Combiner: Install Observer into MachineFunction

The Combiner doesn't install the Observer into the MachineFunction.
This probably went unnoticed, because MachineFunction::getObserver() is
currently only used in constrainOperandRegClass(), but this might cause
issues down the line.

Pull Request: https://github.com/llvm/llvm-project/pull/102156

show more ...

[GlobalISel][AArch64] AArch64O0PreLegalizerCombiner: Disable fixed-point iteration (#94291)

This adds an option to CombinerInfo to limit the number of iterations in the
Combiner. This option is the

[GlobalISel][AArch64] AArch64O0PreLegalizerCombiner: Disable fixed-point iteration (#94291)

This adds an option to CombinerInfo to limit the number of iterations in the
Combiner. This option is then used to disable fixed-point iteration for the
AArch64O0PreLegalizerCombiner. The combines there are simple enough that
code quality impact should be minimal with the current heuristics
(instructions are processed from top to bottom of the basic block,
new/changed instructions are added back to the worklist). Test changes
are due to some instructions not being DCE'd, which has no actual impact
because InstructionSelect performs DCE as well.

AArch64 CTMark O0:
-0.9% geomean compile-time (instruction count)
no change in size..text for any of the benchmarks

show more ...

[GlobalISel] Refactor Combiner API

Remove CodeGen leftovers from the old combiner backend and adapt the API to fit the new backend better.
It's now quite a bit closer to how InstructionSelector work

[GlobalISel] Refactor Combiner API

Remove CodeGen leftovers from the old combiner backend and adapt the API to fit the new backend better.
It's now quite a bit closer to how InstructionSelector works.

- `CombinerInfo` is now a simple "options" struct.
- `Combiner` is now the base class of all TableGen'd combiner implementation.
- Many fields have been moved from derived classes into that class.
- It has been refactored to create & own the Observer and Builder.
- `tryCombineAll` TableGen'd method can now be renamed, which allows targets to implement the actual `tryCombineAll` call manually and do whatever they want to do before/after it.

Note: `CombinerHelper` needs to be mutable because none of its methods are const. This can be revisited later.

Depends on D158710

Reviewed By: aemerson, dsanders

Differential Revision: https://reviews.llvm.org/D158713

show more ...

GISel/Combiner: maintain created instructions in a SetVector

This is not a correctness fix because the set is only used for debug
output. However, it helps avoid noise when looking at diffs between

GISel/Combiner: maintain created instructions in a SetVector

This is not a correctness fix because the set is only used for debug
output. However, it helps avoid noise when looking at diffs between
compiler runs.

The set is only maintained with debug output enabled, so the added cost
should be acceptable.

Differential Revision: https://reviews.llvm.org/D139465

show more ...

[GlobalISel][DebugInfo] salvageDebugInfo analogue for gMIR

Salvage debug info of instruction that is about to be deleted as dead in
Combiner pass. Currently supported instructions are COPY and G_TRU

[GlobalISel][DebugInfo] salvageDebugInfo analogue for gMIR

Salvage debug info of instruction that is about to be deleted as dead in
Combiner pass. Currently supported instructions are COPY and G_TRUNC.

It allows to salvage debug info of some dead arguments of functions, by putting
DWARF expression corresponding to the instruction being deleted into related
DBG_VALUE instruction.

Here is an example of missing variables location https://godbolt.org/z/K48osb9dK.
We see that arguments x, y of function foo are not available in debugger, and
corresponding DBG_VALUE instructions have undefined register operand instead of
variables locaton after Aarch64PreLegalizerCombiner pass. The reason is that
registers where variables are located are removed as dead (with instruction
G_TRUNC). We can use salvageDebugInfo analogue for gMIR to preserve debug
locations of dead variables.

Statistics of llvm object files built with vs without this commit on -O2
optimization level (CMAKE_BUILD_TYPE=RelWithDebInfo, -fglobal-isel) on Aarch64 (macOS):

Number of variables with 100% of parent scope covered by DW_AT_location has been increased by 7,9%.
Number of variables with 0% coverage of parent scope has been decreased by 1,2%.
Number of variables processed by location statistics has been increased by 2,9%.
Average PC ranges coverage has been increased by 1,8 percentage points.

Coverage can be improved by supporting more instructions, or by calling
salvageDebugInfo for instructions that are deleted during Combiner rules exection.

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D129909

show more ...

[CodeGen] Apply clang-tidy fixes for readability-redundant-smartptr-get (NFC)

Cleanup includes: DebugInfo & CodeGen

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121332

[CodeGen] Use = default (NFC)

Identified with modernize-use-equals-default

[llvm] Remove redundant member initialization (NFC)

Identified with readability-redundant-member-init.

[GlobalISel] Allow DBG_VALUE to use undefined vregs before LiveDebugValues.

Expanding on D109750.

Since `DBG_VALUE` instructions have final register validity determined in
`LDVImpl::handleDebugValu

[GlobalISel] Allow DBG_VALUE to use undefined vregs before LiveDebugValues.

Expanding on D109750.

Since `DBG_VALUE` instructions have final register validity determined in
`LDVImpl::handleDebugValue`, there is no apparent reason to immediately prune
unused register operands as their defs are erased. Consequently, this renders
`MachineInstr::eraseFromParentAndMarkDBGValuesForRemoval` moot; gaining a
substantial performance improvement.

The only necessary changes involve making relevant passes consider invalid
DBG_VALUE vregs uses as valid.

Reviewed By: MatzeB

Differential Revision: https://reviews.llvm.org/D112852

show more ...

[CodeGen] Use make_early_inc_range (NFC)

[GISel] Print better error messages for missing Combiner Observer calls

Differential revision: https://reviews.llvm.org/D105290

[GlobalISel] fix a compilation error with gcc 6.3.0

With gcc 6.3.0, I hit the following compilation error:
../lib/CodeGen/GlobalISel/Combiner.cpp: In member function
‘bool llvm::Combiner::co

[GlobalISel] fix a compilation error with gcc 6.3.0

With gcc 6.3.0, I hit the following compilation error:
../lib/CodeGen/GlobalISel/Combiner.cpp: In member function
‘bool llvm::Combiner::combineMachineInstrs(llvm::MachineFunction&,
llvm::GISelCSEInfo*)’:
../lib/CodeGen/GlobalISel/Combiner.cpp:156:54: error: suggest parentheses
around ‘&&’ within ‘||’ [-Werror=parentheses]
assert(!CSEInfo || !errorToBool(CSEInfo->verify()) &&
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
"CSEInfo is not consistent. Likely missing calls to "
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"observer on mutations");

Fix the code as suggested by the compiler.

show more ...

[GISel]: Fix one more CSE Non determinism

https://reviews.llvm.org/D86676

Sometimes we can have the following code

x:gpr(s32) = G_OP

Say we build G_OP2 to the same x and then delete the previous

[GISel]: Fix one more CSE Non determinism

https://reviews.llvm.org/D86676

Sometimes we can have the following code

x:gpr(s32) = G_OP

Say we build G_OP2 to the same x and then delete the previous instruction. Using something like

Register X = ...;
auto NewMIB = CSEBuilder.buildOp2(X, ... args);

Currently there's a mismatch in how NewMIB is profiled and inserted into the CSEMap (ie it doesn't consider register bank/register class along with type).Unify the profiling by refactoring and calling the common method.

This was found by turning on the CSEInfo::verify in at the end of each of our GISel passes which turns inconsistent state/non determinism in CSEing into crashes which likely usually indicates missing calls to Observer on mutations (the most common case). Here non determinism usually means not cseing sometimes, but almost never about producing incorrect code.
Also this patch adds this verification at the end of the combiners as well.

show more ...

GlobalISel: Remove redundant check for empty blocks

[gicombiner] Add the run-time rule disable option

Summary:
Each generated helper can be configured to generate an option that disables
rules in that helper. This can be used to bisect rulesets.

The

[gicombiner] Add the run-time rule disable option

Summary:
Each generated helper can be configured to generate an option that disables
rules in that helper. This can be used to bisect rulesets.

The disable bits are stored in a SparseVector as this is very cheap for the
common case where nothing is disabled. It gets more expensive the more rules
are disabled but you're generally doing that for debug purposes where
performance is less of a concern.

Depends on D68426

Reviewers: volkan, bogner

Reviewed By: volkan

Subscribers: hiraditya, Petar.Avramovic, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68438

llvm-svn: 375067

show more ...

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of

[llvm] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013

show more ...

[GISel][NFC]: Add methods to speed up insertion into GISelWorklist

https://reviews.llvm.org/D58073

Speed up insertion during the initial populating phase into the
GISelWorkList by deferring repeate

[GISel][NFC]: Add methods to speed up insertion into GISelWorklist

https://reviews.llvm.org/D58073

Speed up insertion during the initial populating phase into the
GISelWorkList by deferring repeatedly resizing the DenseMap.
This results in ~10% improvement in the combiner passes, and
~3% speedup in the Legalizer.

reviewed by: aemerson.

llvm-svn: 354093

show more ...

[globalisel] Correct string emitted by GISelChangeObserver::erasingInstr()

The API indicates that the MI is about to be erased rather than it has been erased.

llvm-svn: 353746

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[GISel]: Add support for CSEing continuously during GISel passes.

https://reviews.llvm.org/D52803

This patch adds support to continuously CSE instructions during
each of the GISel passes. It consis

[GISel]: Add support for CSEing continuously during GISel passes.

https://reviews.llvm.org/D52803

This patch adds support to continuously CSE instructions during
each of the GISel passes. It consists of a GISelCSEInfo analysis pass
that can be used by the CSEMIRBuilder.

llvm-svn: 351283

show more ...

[globalisel][combiner] Make the CombinerChangeObserver a MachineFunction::Delegate

Summary:
This allows us to register it with the MachineFunction delegate and be
notified automatically about erasur

[globalisel][combiner] Make the CombinerChangeObserver a MachineFunction::Delegate

Summary:
This allows us to register it with the MachineFunction delegate and be
notified automatically about erasure and creation of instructions. However,
we still need explicit notification for modifications such as those caused
by setReg() or replaceRegWith().

There is a catch with this though. The notification for creation is
delivered before any operands can be added. While appropriate for
scheduling combiner work. This is unfortunate for debug output since an
opcode by itself doesn't provide sufficient information on what happened.
As a result, the work list remembers the instructions (when debug output is
requested) and emits a more complete dump later.

Another nit is that the MachineFunction::Delegate provides const pointers
which is inconvenient since we want to use it to schedule future
modification. To resolve this GISelWorkList now has an optional pointer to
the MachineFunction which describes the scope of the work it is permitted
to schedule. If a given MachineInstr* is in this function then it is
permitted to schedule work to be performed on the MachineInstr's. An
alternative to this would be to remove the const from the
MachineFunction::Delegate interface, however delegates are not permitted
to modify the MachineInstr's they receive.

In addition to this, the observer has three interface changes.
* erasedInstr() is now erasingInstr() to indicate it is about to be erased
but still exists at the moment.
* changingInstr() and changedInstr() have been added to report changes
before and after they are made. This allows us to trace the changes
in the debug output.
* As a convenience changingAllUsesOfReg() and
finishedChangingAllUsesOfReg() will report changingInstr() and
changedInstr() for each use of a given register. This is primarily useful
for changes caused by MachineRegisterInfo::replaceRegWith()

With this in place, both combine rules have been updated to report their
changes to the observer.

Finally, make some cosmetic changes to the debug output and make Combiner
and CombinerHelp

Reviewers: aditya_nandakumar, bogner, volkan, rtereshin, javed.absar

Reviewed By: aditya_nandakumar

Subscribers: mgorny, rovka, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D52947

llvm-svn: 349167

show more ...