Add OffsetIsScalable to getMemOperandWithOffset

Summary:
Making `Scale` a `TypeSize` in AArch64InstrInfo::getMemOpInfo,
has the effect that all places where this information is used
(notably, Target

Add OffsetIsScalable to getMemOperandWithOffset

Summary:
Making `Scale` a `TypeSize` in AArch64InstrInfo::getMemOpInfo,
has the effect that all places where this information is used
(notably, TargetInstrInfo::getMemOperandWithOffset) will need
to consider Scale - and derived, Offset - possibly being scalable.

This patch adds a new operand `bool &OffsetIsScalable` to
TargetInstrInfo::getMemOperandWithOffset and fixes up all
the places where this function is used, to consider the
offset possibly being scalable.

In most cases, this means bailing out because the algorithm does not
(or cannot) support scalable offsets in places where it does some
form of alias checking for example.

Reviewers: rovka, efriedma, kristof.beyls

Reviewed By: efriedma

Subscribers: wuzish, kerbowa, MatzeB, arsenm, nemanjai, jvesely, nhaehnle, hiraditya, kbarton, javed.absar, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, jsji, Jim, lenary, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D72758

show more ...

PostRA Machine Sink should take care of COPY defining register that is a sub-register by another COPY source operand

Differential Revision: https://reviews.llvm.org/D71132

Revert 30e8f80fd5a4 "[DebugInfo] Don't create multiple DBG_VALUEs when sinking"

This caused non-determinism in the compiler, see command on the Phabricator
code review.

> This patch addresses a per

Revert 30e8f80fd5a4 "[DebugInfo] Don't create multiple DBG_VALUEs when sinking"

This caused non-determinism in the compiler, see command on the Phabricator
code review.

> This patch addresses a performance problem reported in PR43855, and
> present in the reapplication in in 001574938e5. It turns out that
> MachineSink will (often) move instructions to the first block that
> post-dominates the current block, and then try to sink further. This
> means if we have a lot of conditionals, we can needlessly create large
> numbers of DBG_VALUEs, one in each block the sunk instruction passes
> through.
>
> To fix this, rather than immediately sinking DBG_VALUEs, record them in
> a pass structure. When sinking is complete and instructions won't be
> sunk any further, new DBG_VALUEs are added, avoiding lots of
> intermediate DBG_VALUE $noregs being created.
>
> Differential revision: https://reviews.llvm.org/D70676

show more ...

[DebugInfo] Don't create multiple DBG_VALUEs when sinking

This patch addresses a performance problem reported in PR43855, and
present in the reapplication in in 001574938e5. It turns out that
Machin

[DebugInfo] Don't create multiple DBG_VALUEs when sinking

This patch addresses a performance problem reported in PR43855, and
present in the reapplication in in 001574938e5. It turns out that
MachineSink will (often) move instructions to the first block that
post-dominates the current block, and then try to sink further. This
means if we have a lot of conditionals, we can needlessly create large
numbers of DBG_VALUEs, one in each block the sunk instruction passes
through.

To fix this, rather than immediately sinking DBG_VALUEs, record them in
a pass structure. When sinking is complete and instructions won't be
sunk any further, new DBG_VALUEs are added, avoiding lots of
intermediate DBG_VALUE $noregs being created.

Differential revision: https://reviews.llvm.org/D70676

show more ...

[DebugInfo] Don't reorder DBG_VALUEs when sunk

Fix part of PR43855, resolving a problem that comes from the reapplication
in 001574938e5. If we have two DBG_VALUE insts in a block that specify
the l

[DebugInfo] Don't reorder DBG_VALUEs when sunk

Fix part of PR43855, resolving a problem that comes from the reapplication
in 001574938e5. If we have two DBG_VALUE insts in a block that specify
the location of the same variable, for example:

%0 = someinst
DBG_VALUE %0, !123, !DIExpression()
%1 = anotherinst
DBG_VALUE %1, !123, !DIExpression()

if %0 were to sink, the corresponding DBG_VALUE would sink too, past the
next DBG_VALUE, effectively re-ordering assignments. To fix this, I've
added a SeenDbgVars set recording what variable locations have been seen in
a block already (working bottom up), and now flag DBG_VALUEs that would
pass a later DBG_VALUE for the same variable.

NB, this only works for repeated DBG_VALUEs in the same basic block, the
general case involving control flow is much harder, which I've written
up in PR44117.

Differential revision: https://reviews.llvm.org/D70672

show more ...

[DebugInfo] Re-apply two patches to MachineSink

These were:
* D58386 / f5e1b718a67 / reverted in d382a8a768b
* D58238 / ee50590e168 / reverted in a8db456b53a

Of which the latter has a performance

[DebugInfo] Re-apply two patches to MachineSink

These were:
* D58386 / f5e1b718a67 / reverted in d382a8a768b
* D58238 / ee50590e168 / reverted in a8db456b53a

Of which the latter has a performance regression tracked in PR43855,
fixed by D70672 / D70676, which will be committed atomically with this
reapplication.

Contains a minor difference to account for a change in the IsCopyInstr
signature.

show more ...

Sink all InitializePasses.h includes

This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of reco

Sink all InitializePasses.h includes

This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of recompilation.

I found this fact by looking at this table, which is sorted by the
number of times a file was changed over the last 100,000 git commits
multiplied by the number of object files that depend on it in the
current checkout:
recompiles touches affected_files header
342380 95 3604 llvm/include/llvm/ADT/STLExtras.h
314730 234 1345 llvm/include/llvm/InitializePasses.h
307036 118 2602 llvm/include/llvm/ADT/APInt.h
213049 59 3611 llvm/include/llvm/Support/MathExtras.h
170422 47 3626 llvm/include/llvm/Support/Compiler.h
162225 45 3605 llvm/include/llvm/ADT/Optional.h
158319 63 2513 llvm/include/llvm/ADT/Triple.h
140322 39 3598 llvm/include/llvm/ADT/StringRef.h
137647 59 2333 llvm/include/llvm/Support/Error.h
131619 73 1803 llvm/include/llvm/Support/FileSystem.h

Before this change, touching InitializePasses.h would cause 1345 files
to recompile. After this change, touching it only causes 550 compiles in
an incremental rebuild.

Reviewers: bkramer, asbirlea, bollu, jdoerfert

Differential Revision: https://reviews.llvm.org/D70211

show more ...

Revert "[DebugInfo] MachineSink: Insert undef DBG_VALUEs when sinking instructions"

This reverts commit ee50590e1684c197bc4336984795e48bf53c7a4e.

PR43855 reports a performance regression from this

Revert "[DebugInfo] MachineSink: Insert undef DBG_VALUEs when sinking instructions"

This reverts commit ee50590e1684c197bc4336984795e48bf53c7a4e.

PR43855 reports a performance regression from this commit, which I'll
look into.

show more ...

Revert "[DebugInfo] MachineSink: find more DBG_VALUEs to sink"

This reverts commit f5e1b718a675a4449b71423f04d38e1e93045105.

PR43855 reports a performance regression with commit ee50590e. This comm

Revert "[DebugInfo] MachineSink: find more DBG_VALUEs to sink"

This reverts commit f5e1b718a675a4449b71423f04d38e1e93045105.

PR43855 reports a performance regression with commit ee50590e. This commit
depends on the faulty one, so has to come out too.

show more ...

[DebugInfo] MachineSink: find more DBG_VALUEs to sink

In the Pre-RA machine sinker, previously we were relying on all DBG_VALUEs
being immediately after the instruction that defined their operands.

[DebugInfo] MachineSink: find more DBG_VALUEs to sink

In the Pre-RA machine sinker, previously we were relying on all DBG_VALUEs
being immediately after the instruction that defined their operands. This
isn't a valid assumption, as a variable location change doesn't
necessarily correspond to where the value is computed. In this patch, we
collect DBG_VALUEs that might need sinking as we walk through a block,
and sink all of them if their defining instruction is sunk.

This patch adds some copy propagation too, so that if we sink a copy inst,
the now non-dominated paths can use the copy source for the variable
location.

Differential Revision: https://reviews.llvm.org/D58386

show more ...

[DebugInfo] MachineSink: Insert undef DBG_VALUEs when sinking instructions

When we sink DBG_VALUEs between blocks, we simply move the DBG_VALUE
instruction to below the sunk instruction. However, we

[DebugInfo] MachineSink: Insert undef DBG_VALUEs when sinking instructions

When we sink DBG_VALUEs between blocks, we simply move the DBG_VALUE
instruction to below the sunk instruction. However, we should also mark
the variable as being undef at the original location, to terminate any
earlier variable location. This patch does that -- plus, if the
instruction being sunk is a copy, it attempts to propagate the copy
through the DBG_VALUE, replacing the destination with the source.

Differential Revision: https://reviews.llvm.org/D58238

show more ...

[Dominators][CodeGen] Add MachinePostDominatorTree verification

Summary:
This patch implements Machine PostDominator Tree verification and ensures that the verification doesn't fail the in-tree test

[Dominators][CodeGen] Add MachinePostDominatorTree verification

Summary:
This patch implements Machine PostDominator Tree verification and ensures that the verification doesn't fail the in-tree tests.

MPDT verification can be enabled using `verify-machine-dom-info` -- the same flag used by Machine Dominator Tree verification.

Flipping the flag revealed that MachineSink falsely claimed to preserve CFG and MDT/MPDT. This patch fixes that.

Reviewers: arsenm, hliao, rampitec, vpykhtin, grosser

Reviewed By: hliao

Subscribers: wdng, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68235

llvm-svn: 373341

show more ...

[DebugInfo] Make postra sinking of DBG_VALUEs subregister-safe

Currently the machine instruction sinker identifies DBG_VALUE insts that
also need to sink by comparing register numbers. Unfortunately

[DebugInfo] Make postra sinking of DBG_VALUEs subregister-safe

Currently the machine instruction sinker identifies DBG_VALUE insts that
also need to sink by comparing register numbers. Unfortunately this isn't
safe, because (after register allocation) a DBG_VALUE may read a register
that aliases what's being sunk. To fix this, identify the DBG_VALUEs that
need to sink by recording & examining their register units. Register units
gives us the following guarantee:

"Two registers overlap if and only if they have a common register unit"
[MCRegisterInfo.h]

Thus we can always identify aliasing DBG_VALUEs if the set of register
units read by the DBG_VALUE, and the register units of the instruction
being sunk, intersect. (MachineSink already uses classes like
"LiveRegUnits" for determining sinking validity anyway).

The test added checks for super and subregister DBG_VALUE reads of a sunk
copy being sunk as well.

Differential Revision: https://reviews.llvm.org/D58191

llvm-svn: 369247

show more ...

Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM

Summary:
This clang-tidy check is looking for unsigned integer variables whose initializer
starts with an implicit cast from llvm::Re

Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVM

Summary:
This clang-tidy check is looking for unsigned integer variables whose initializer
starts with an implicit cast from llvm::Register and changes the type of the
variable to llvm::Register (dropping the llvm:: where possible).

Partial reverts in:
X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister
X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister
X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister
HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned&
MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register
PPCFastISel.cpp - No Register::operator-=()
PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned&
MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor

Manual fixups in:
ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned&
HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register
HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register.
PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned&

Depends on D65919

Reviewers: arsenm, bogner, craig.topper, RKSimon

Reviewed By: arsenm

Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65962

llvm-svn: 369041

show more ...

Finish moving TargetRegisterInfo::isVirtualRegister() and friends to llvm::Register as started by r367614. NFC

llvm-svn: 367633

Use llvm::stable_sort

While touching the code, simplify if feasible.

llvm-svn: 358996

[CodeGen] Add "const" to MachineInstr::mayAlias

Summary:
The basic idea here is to make it possible to use
MachineInstr::mayAlias also when the MachineInstr
is const (or the "Other" MachineInstr is

[CodeGen] Add "const" to MachineInstr::mayAlias

Summary:
The basic idea here is to make it possible to use
MachineInstr::mayAlias also when the MachineInstr
is const (or the "Other" MachineInstr is const).

The addition of const in MachineInstr::mayAlias
then rippled down to the need for adding const
in several other places, such as
TargetTransformInfo::getMemOperandWithOffset.

Reviewers: hfinkel

Reviewed By: hfinkel

Subscribers: hfinkel, MatzeB, arsenm, jvesely, nhaehnle, hiraditya, javed.absar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60856

llvm-svn: 358744

show more ...

Add skipFunction to PostRA machine sinking pass.

Summary: Add skipFunction to PostRA machine sinking pass.

Reviewers: junbuml

Subscribers: arsenm, llvm-commits

Tags: #llvm

Differential Revision:

Add skipFunction to PostRA machine sinking pass.

Summary: Add skipFunction to PostRA machine sinking pass.

Reviewers: junbuml

Subscribers: arsenm, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D57847

llvm-svn: 354541

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[CodeGen][NFC] Make `TII::getMemOpBaseImmOfs` return a base operand

Currently, instructions doing memory accesses through a base operand that is
not a register can not be analyzed using `TII::getMem

[CodeGen][NFC] Make `TII::getMemOpBaseImmOfs` return a base operand

Currently, instructions doing memory accesses through a base operand that is
not a register can not be analyzed using `TII::getMemOpBaseRegImmOfs`.

This means that functions such as `TII::shouldClusterMemOps` will bail
out on instructions using an FI as a base instead of a register.

The goal of this patch is to refactor all this to return a base
operand instead of a base register.

Then in a separate patch, I will add FI support to the mem op clustering
in the MachineScheduler.

Differential Revision: https://reviews.llvm.org/D54846

llvm-svn: 347746

show more ...

[MachineSink][DebugInfo] Correctly sink DBG_VALUEs

As reported in PR38952, postra-machine-sink relies on DBG_VALUE insns being
adjacent to the def of the register that they reference. This is not al

[MachineSink][DebugInfo] Correctly sink DBG_VALUEs

As reported in PR38952, postra-machine-sink relies on DBG_VALUE insns being
adjacent to the def of the register that they reference. This is not always
true, leading to register copies being sunk but not the associated DBG_VALUEs,
which gives the debugger a bad variable location.

This patch collects DBG_VALUEs as we walk through a BB looking for copies to
sink, then passes them down to performSink. Compile-time impact should be
negligable.

Differential Revision: https://reviews.llvm.org/D53992

llvm-svn: 345996

show more ...

[PostRASink] Make sure to remove subregisters from live-ins as well

llvm-svn: 342492

[DWARF] Missing location debug information with -O2.

Check that Machine CSE correctly handles during the transformation, the
debug location information for local variables.

Differential Revision: h

[DWARF] Missing location debug information with -O2.

Check that Machine CSE correctly handles during the transformation, the
debug location information for local variables.

Differential Revision: https://reviews.llvm.org/D50887

llvm-svn: 341025

show more ...

Remove trailing space

sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h}

llvm-svn: 338293

[DebugInfo] Ignore DBG_VALUE instructions in PostRA Machine Sink

Summary:
The logic for handling the sinking of COPY instructions was generating
different code when building with debug flags.

The o

[DebugInfo] Ignore DBG_VALUE instructions in PostRA Machine Sink

Summary:
The logic for handling the sinking of COPY instructions was generating
different code when building with debug flags.

The original code did not take into consideration debug instructions. This
resulted in the registers in the DBG_VALUE instructions being treated as used,
and prevented the COPY from being sunk. This patch avoids analyzing debug
instructions when trying to sink COPY instructions.

This patch also creates a routine from the code in MachineSinking::SinkInstruction to
perform the logic of sinking an instruction along with its debug instructions.
This functionality is used in multiple places, including the code for sinking COPY instrs.


Reviewers: junbuml, javed.absar, MatzeB, bjope

Reviewed By: bjope

Subscribers: aprantl, probinson, thegameg, jonpa, bjope, vsk, kristof.beyls, JDevlieghere, llvm-commits

Tags: #debug-info

Differential Revision: https://reviews.llvm.org/D45637

llvm-svn: 335264

show more ...