Sort the remaining #include lines in include/... and lib/....

I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line

Sort the remaining #include lines in include/... and lib/....

I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787

show more ...

This patch closes PR#32216: Better testing of schedule model instruction latencies/throughputs.
The details are here: https://reviews.llvm.org/D30941

llvm-svn: 300311

Improve machine schedulers for in-order processors

This patch enables schedulers to specify instructions that
cannot be issued with any other instructions.
It also fixes BeginGroup/EndGroup.

Revie

Improve machine schedulers for in-order processors

This patch enables schedulers to specify instructions that
cannot be issued with any other instructions.
It also fixes BeginGroup/EndGroup.

Reviewed by: Andrew Trick
Differential Revision: https://reviews.llvm.org/D30744

llvm-svn: 298885

show more ...

[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).

llvm-svn: 296404

TargetSchedule: Do not consider subregister definitions as reads.

We should not consider subregister definitions as reads for schedule
model purposes (they are just modeled as reads of the overal vr

TargetSchedule: Do not consider subregister definitions as reads.

We should not consider subregister definitions as reads for schedule
model purposes (they are just modeled as reads of the overal vreg for
liveness calculation purposes, the CPU instructions are not actually
reading).

Unfortunately I cannot submit a test for this as it requires a target
which uses ReadAdvance annotation in the scheduling model and has
subregister liveness enabled at the same time, which is only the case on
an out of tree target.

llvm-svn: 279604

show more ...

CodeGen: Use MachineInstr& in TargetInstrInfo, NFC

This is mostly a mechanical change to make TargetInstrInfo API take
MachineInstr& (instead of MachineInstr* or MachineBasicBlock::iterator)
when th

CodeGen: Use MachineInstr& in TargetInstrInfo, NFC

This is mostly a mechanical change to make TargetInstrInfo API take
MachineInstr& (instead of MachineInstr* or MachineBasicBlock::iterator)
when the argument is expected to be a valid MachineInstr. This is a
general API improvement.

Although it would be possible to do this one function at a time, that
would demand a quadratic amount of churn since many of these functions
call each other. Instead I've done everything as a block and just
updated what was necessary.

This is mostly mechanical fixes: adding and removing `*` and `&`
operators. The only non-mechanical change is to split
ARMBaseInstrInfo::getOperandLatencyImpl out from
ARMBaseInstrInfo::getOperandLatency. Previously, the latter took a
`MachineInstr*` which it updated to the instruction bundle leader; now,
the latter calls the former either with the same `MachineInstr&` or the
bundle leader.

As a side effect, this removes a bunch of MachineInstr* to
MachineBasicBlock::iterator implicit conversions, a necessary step
toward fixing PR26753.

Note: I updated WebAssembly, Lanai, and AVR (despite being
off-by-default) since it turned out to be easy. I couldn't run tests
for AVR since llc doesn't link with it turned on.

llvm-svn: 274189

show more ...

[TargetSchedule] Use 'isOutOfOrder' as possible to avoid magic number. NFC.

Summary:
Using isOutOfOrder makes the code more clear.

Reviewers: rengolin, atrick, hfinkel.

Differential Revision: http

[TargetSchedule] Use 'isOutOfOrder' as possible to avoid magic number. NFC.

Summary:
Using isOutOfOrder makes the code more clear.

Reviewers: rengolin, atrick, hfinkel.

Differential Revision: http://reviews.llvm.org/D21548

llvm-svn: 273255

show more ...

CodeGen: TII: Take MachineInstr& in predicate API, NFC

Change TargetInstrInfo API to take `MachineInstr&` instead of
`MachineInstr*` in the functions related to predicated instructions
(I'll try to

CodeGen: TII: Take MachineInstr& in predicate API, NFC

Change TargetInstrInfo API to take `MachineInstr&` instead of
`MachineInstr*` in the functions related to predicated instructions
(I'll try to come back later and get some of the rest). All of these
functions require non-null parameters already, so references are more
clear. As a bonus, this happens to factor away a host of implicit
iterator => pointer conversions.

No functionality change intended.

llvm-svn: 261605

show more ...

[MISched] Explanatory error message when machine model is not complete. NFC

When not all instructions have a scheduling class,
the error message now provides a possible solution.

Differential Revis

[MISched] Explanatory error message when machine model is not complete. NFC

When not all instructions have a scheduling class,
the error message now provides a possible solution.

Differential Revision: http://reviews.llvm.org/D15854

llvm-svn: 256839

show more ...

Use llvm_unreachable() instead of report_fatal_error() if the machine model is incomplete

This error is for developers only so it makes sense to abort and get a
backtrace.

llvm-svn: 242551

Turn effective assert(0) into llvm_unreachable

llvm-svn: 237379

TargetSchedule: factor out common code; NFC

llvm-svn: 237376

Remove unnecessary TargetMachine.h includes.

llvm-svn: 219672

Change MCSchedModel to be a struct of statically initialized data.

This removes static initializers from the backends which generate this data, and also makes this struct match the other Tablegen ge

Change MCSchedModel to be a struct of statically initialized data.

This removes static initializers from the backends which generate this data, and also makes this struct match the other Tablegen generated structs in behaviour

Reviewed by Andy Trick and Chandler C

llvm-svn: 216919

show more ...

Have MachineFunction cache a pointer to the subtarget to make lookups
shorter/easier and have the DAG use that to do the same lookup. This
can be used in the future for TargetMachine based caching lo

Have MachineFunction cache a pointer to the subtarget to make lookups
shorter/easier and have the DAG use that to do the same lookup. This
can be used in the future for TargetMachine based caching lookups from
the MachineFunction easily.

Update the MIPS subtarget switching machinery to update this pointer
at the same time it runs.

llvm-svn: 214838

show more ...

Remove the TargetMachine forwards for TargetSubtargetInfo based
information and update all callers. No functional change.

llvm-svn: 214781

MachineCombiner Pass for selecting faster instruction
sequence - target independent framework

When the DAGcombiner selects instruction sequences
it could increase the critical path or resource l

MachineCombiner Pass for selecting faster instruction
sequence - target independent framework

When the DAGcombiner selects instruction sequences
it could increase the critical path or resource len.

For example, on arm64 there are multiply-accumulate instructions (madd,
msub). If e.g. the equivalent multiply-add sequence is not on the
crictial path it makes sense to select it instead of the combined,
single accumulate instruction (madd/msub). The reason is that the
conversion from add+mul to the madd could lengthen the critical path
by the latency of the multiply.

But the DAGCombiner would always combine and select the madd/msub
instruction.

This patch uses machine trace metrics to estimate critical path length
and resource length of an original instruction sequence vs a combined
instruction sequence and picks the faster code based on its estimates.

This patch only commits the target independent framework that evaluates
and selects code sequences. The machine instruction combiner is turned
off for all targets and expected to evolve over time by gradually
handling DAGCombiner pattern in the target specific code.

This framework lays the groundwork for fixing
rdar://16319955

llvm-svn: 214666

show more ...

Revert "Introduce a string_ostream string builder facilty"

Temporarily back out commits r211749, r211752 and r211754.

llvm-svn: 211814

Introduce a string_ostream string builder facilty

string_ostream is a safe and efficient string builder that combines opaque
stack storage with a built-in ostream interface.

small_string_ostream<by

Introduce a string_ostream string builder facilty

string_ostream is a safe and efficient string builder that combines opaque
stack storage with a built-in ostream interface.

small_string_ostream<bytes> additionally permits an explicit stack storage size
other than the default 128 bytes to be provided. Beyond that, storage is
transferred to the heap.

This convenient class can be used in most places an
std::string+raw_string_ostream pair or SmallString<>+raw_svector_ostream pair
would previously have been used, in order to guarantee consistent access
without byte truncation.

The patch also converts much of LLVM to use the new facility. These changes
include several probable bug fixes for truncated output, a programming error
that's no longer possible with the new interface.

llvm-svn: 211749

show more ...

IfConverter: Use TargetSchedule for instruction latencies

For targets that have instruction itineraries this means no change. Targets
that move over to the new schedule model will use be able the ne

IfConverter: Use TargetSchedule for instruction latencies

For targets that have instruction itineraries this means no change. Targets
that move over to the new schedule model will use be able the new schedule
module for instruction latencies in the if-converter (the logic is such that if
there is no itineary we will use the new sched model for the latencies).

Before, we queried "TTI->getInstructionLatency()" for the instruction latency
and the extra prediction cost. Now, we query the TargetSchedule abstraction for
the instruction latency and TargetInstrInfo for the extra predictation cost. The
TargetSchedule abstraction will internally call "TTI->getInstructionLatency" if
an itinerary exists, otherwise it will use the new schedule model.

ATTENTION: Out of tree targets!

(I will also send out an email later to LLVMDev)

This means, if your target implements

unsigned getInstrLatency(const InstrItineraryData *ItinData,
const MachineInstr *MI,
unsigned *PredCost);

and returns a value for "PredCost", you now also need to implement

unsigned getPredictationCost(const MachineInstr *MI);

(if your target uses the IfConversion.cpp pass)

radar://15077010

llvm-svn: 191671

show more ...

Mark the x86 machine model as incomplete. PR17367.

Ideally, the machinel model is added at the time the instructions are
defined. But many instructions in X86InstrSSE.td still need a model.

Without

Mark the x86 machine model as incomplete. PR17367.

Ideally, the machinel model is added at the time the instructions are
defined. But many instructions in X86InstrSSE.td still need a model.

Without this workaround the scheduler asserts because x86 already has
itinerary classes for these instructions, indicating they should be
modeled by the scheduler. Since we use the new machine model for other
instructions, it expects a new machine model for these too.

llvm-svn: 191391

show more ...

MI-Sched: handle ReadAdvance latencies as used by Swift.

llvm-svn: 184135

Machine Model: Add MicroOpBufferSize and resource BufferSize.

Replace the ill-defined MinLatency and ILPWindow properties with
with straightforward buffer sizes:
MCSchedMode::MicroOpBufferSize
MCPro

Machine Model: Add MicroOpBufferSize and resource BufferSize.

Replace the ill-defined MinLatency and ILPWindow properties with
with straightforward buffer sizes:
MCSchedMode::MicroOpBufferSize
MCProcResourceDesc::BufferSize

These can be used to more precisely model instruction execution if desired.

Disabled some misched tests temporarily. They'll be reenabled in a few commits.

llvm-svn: 184032

show more ...

MI-Sched cleanup. If an instruction has no valid sched class, do not attempt to check for a variant.

llvm-svn: 179451

Change the default latency for implicit defs.

Implicit defs are not currently positional and not modeled by the
per-operand machine model. Unfortunately, we treat defs that are part
of the architect

Change the default latency for implicit defs.

Implicit defs are not currently positional and not modeled by the
per-operand machine model. Unfortunately, we treat defs that are part
of the architectural instruction description, like flags, the same as
other implicit defs. Really, they should have a fixed MachineInstr
layout and probably shouldn't be "implicit" at all.

For now, we'll change the default latency to be the max operand
latency. That will give flag setting operands full latency for x86
folded loads. Other kinds of "fake" implicit defs don't occur prior to
regalloc anyway, and we would like them to go away postRegAlloc as
well.

llvm-svn: 177227

show more ...