[NFC][DebugInfo] Switch more call-sites to using iterator-insertion (#124283)

To finalise the "RemoveDIs" work removing debug intrinsics, we're
updating call sites that insert instructions to use i

[NFC][DebugInfo] Switch more call-sites to using iterator-insertion (#124283)

To finalise the "RemoveDIs" work removing debug intrinsics, we're
updating call sites that insert instructions to use iterators instead.
This set of changes are those where it's not immediately obvious that
just calling getIterator to fetch an iterator is correct, and one or two
places where more than one line needs to change.

Overall the same rule holds though: iterators generated for the start of
a block such as getFirstNonPHIIt need to be passed into insert/move
methods without being unwrapped/rewrapped, everything else can use
getIterator.

show more ...

[TTI] Add SCEVExpansionBudget to loop unrolling options. (#118316)

Add an extra know to UnrollingPreferences to let backends control the
maximum budget for SCEV expansions.

This gives backends m

[TTI] Add SCEVExpansionBudget to loop unrolling options. (#118316)

Add an extra know to UnrollingPreferences to let backends control the
maximum budget for SCEV expansions.

This gives backends more fine-grained control on the cost of the runtime
checks for runtime unrolling.

PR: https://github.com/llvm/llvm-project/pull/118316

show more ...

[Transforms] Avoid repeated hash looksup (NFC) (#107727)

[RemoveDIs] Reapply 3fda50d3915, insert instructions using iterators

I'd reverted this in 6c7805d5d1 after a bad stage. Original commit
messsage follows:

[NFC][RemoveDIs] Bulk update utilities to i

[RemoveDIs] Reapply 3fda50d3915, insert instructions using iterators

I'd reverted this in 6c7805d5d1 after a bad stage. Original commit
messsage follows:

[NFC][RemoveDIs] Bulk update utilities to insert with iterators

As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.

There are two general flavours of update:
* Almost all call-sites just call getIterator on an instruction
* Several make use of an existing iterator (scenarios where the code is
actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.

I've also switched DemotePHIToStack to take an optional iterator: it needs
to take an iterator, and having a no-insert-location behaviour appears to
be important. The constructors for ICmpInst and FCmpInst have been updated
too. They're the only instructions that take block _references_ rather than
pointers for certain calls, and a future patch is going to make use of
default-null block insertion locations.

All of this should be NFC.

show more ...

Revert "[NFC][RemoveDIs] Bulk update utilities to insert with iterators"

This reverts commit 3fda50d3915b2163a54a37b602be7783a89dd808.

Apparently I've missed a hunk while staging this; will back ou

Revert "[NFC][RemoveDIs] Bulk update utilities to insert with iterators"

This reverts commit 3fda50d3915b2163a54a37b602be7783a89dd808.

Apparently I've missed a hunk while staging this; will back out for now.

Picked up here: https://lab.llvm.org/buildbot/#/builders/139/builds/60429/steps/6/logs/stdio

show more ...

[NFC][RemoveDIs] Bulk update utilities to insert with iterators

As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carr

[NFC][RemoveDIs] Bulk update utilities to insert with iterators

As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.

There are two general flavours of update:
* Almost all call-sites just call getIterator on an instruction
* Several make use of an existing iterator (scenarios where the code is
actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.

I've also switched DemotePHIToStack to take an optional iterator: it needs
to take an iterator, and having a no-insert-location behaviour appears to
be important. The constructors for ICmpInst and FCmpInst have been updated
too. They're the only instructions that take block _references_ rather than
pointers for certain calls, and a future patch is going to make use of
default-null block insertion locations.

All of this should be NFC.

show more ...

[RemoveDIs] Use iterators for moving PHIs in loop-unroll-and-jam (#83003)

With no debug intrinsics, correctly identifying the start of a block
with iterators becomes important. We need to use the i

[RemoveDIs] Use iterators for moving PHIs in loop-unroll-and-jam (#83003)

With no debug intrinsics, correctly identifying the start of a block
with iterators becomes important. We need to use the iterator-returning
methods here in loop-unroll-and-jam where we're shifting PHIs around.
Otherwise they can be inserted after debug-info records, leading to
debug-info attached to PHIs, which is ill formed.

Fixes #83000

show more ...

[llvm] Stop including llvm/ADT/iterator_range.h (NFC)

Identified with misc-include-cleaner.

[PseudoProbe] Clean up dwarf discriminator and avoid duplicating factor.

A pseudo probe is created with dwarf line information shared with its nearest instruction. If the instruction comes with a dw

[PseudoProbe] Clean up dwarf discriminator and avoid duplicating factor.

A pseudo probe is created with dwarf line information shared with its nearest instruction. If the instruction comes with a dwarf discriminator, it will be shared with the probe as well. This can confuse the later FS-AFDO discriminator assignment pass. To fix this, I'm cleaning up the discriminator fields for probes when they are inserted.

I also notice another possibility to change the discriminator field of pseudo probes in the pipeline before the FS discriminator assignment pass. That is the loop unroller, which assigns duplication factor to instruction being vectorized. I'm disabling that for pseudo probe intrinsics specifically, also for callsites with probes.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D148569

show more ...

[Transforms] Use *{Set,Map}::contains (NFC)

[LoopUnrollAndJam] Visit phi operand dependencies in post-order

Fixes https://github.com/llvm/llvm-project/issues/58565

The previous implementation visits operands in pre-order, but this does
not g

[LoopUnrollAndJam] Visit phi operand dependencies in post-order

Fixes https://github.com/llvm/llvm-project/issues/58565

The previous implementation visits operands in pre-order, but this does
not guarantee an instruction is visited before its uses. This can cause
instructions to be copied in the incorrect order. For example:

```
a = ...
b = add a, 1
c = add a, b
d = add b, a
```

Pre-order visits does not guarantee the order in which `a` and `b` are
visited. LoopUnrollAndJam may incorrectly insert `b` before `a`.

This patch implements post-order visits. By visiting dependencies first,
we guarantee that an instruction's dependencies are visited first.

Differential Revision: https://reviews.llvm.org/D140255

show more ...

[LoopUnrollAndJam] Forget scalar evolution dispositions. Do no explicitly forget subloop.

Fixes https://github.com/llvm/llvm-project/issues/58454

Scalar evolution dispositions need to be forgotten

[LoopUnrollAndJam] Forget scalar evolution dispositions. Do no explicitly forget subloop.

Fixes https://github.com/llvm/llvm-project/issues/58454

Scalar evolution dispositions need to be forgotten to pass verification.

We do not need to forget the subloop since it is automatically forgotten
when forgetting the parent loop.

Differential Revision: https://reviews.llvm.org/D140953

show more ...

[NFC] Rename Function::isDebugInfoForProfiling to shouldEmit[...]

The function name was misleading - the expectation set both by the name
and by other members of Function (like isDeclaration or isIn

[NFC] Rename Function::isDebugInfoForProfiling to shouldEmit[...]

The function name was misleading - the expectation set both by the name
and by other members of Function (like isDeclaration or isIntrinsic)
would be that the function somehow would "be" "debug info for
profiling". But that's not the case - the property indicates (as the
comment over the declaration also explains) whether debug info should be
emitted (for profiling).

show more ...

[NFC] Rename Function::insertBasicBlockAt() to Function::insert().

I think this is a better name because it is what STL uses.

Differential Revision: https://reviews.llvm.org/D140068

[IR][NFC] Adds Function::insertBasicBlockAt() to replace things like F->getBasicBlockList().insert()

This is part of a series of patches that aim at making Function::getBasicBlockList() private.

Di

[IR][NFC] Adds Function::insertBasicBlockAt() to replace things like F->getBasicBlockList().insert()

This is part of a series of patches that aim at making Function::getBasicBlockList() private.

Differential Revision: https://reviews.llvm.org/D139906

show more ...

Remove unused #include "llvm/ADT/Optional.h"

[NFC] Replaced BB->getInstList().{erase(),pop_front(),pop_back()} with eraseFromParent().

Differential Revision: https://reviews.llvm.org/D138617

[llvm] Don't use Optional::getValue (NFC)

[iwyu] Handle regressions in libLLVM header include

Running iwyu-diff on LLVM codebase since 7030654296a0416bd9402a0278 detected a few
regressions, fixing them.

Differential Revision: https://revie

[iwyu] Handle regressions in libLLVM header include

Running iwyu-diff on LLVM codebase since 7030654296a0416bd9402a0278 detected a few
regressions, fixing them.

Differential Revision: https://reviews.llvm.org/D126417

show more ...

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120741

show more ...

[LoopUnroll] Push runtime unrolling decision up into tryToUnrollLoop()

Currently, UnrollLoop() is passed an AllowRuntime flag and decides
itself whether runtime unrolling should be used or not. This

[LoopUnroll] Push runtime unrolling decision up into tryToUnrollLoop()

Currently, UnrollLoop() is passed an AllowRuntime flag and decides
itself whether runtime unrolling should be used or not. This patch
pushes the decision into the caller and allows us to eliminate the
ULO.TripCount and ULO.TripMultiple parameters.

Differential Revision: https://reviews.llvm.org/D104487

show more ...

Revert "[LoopUnrollAndJam] Change LoopUnrollAndJamPass to LoopNest pass"

This reverts commit d65c32fb41b03a35a2a16330ba1ea15cf6818f04.

[LoopUnrollAndJam] Change LoopUnrollAndJamPass to LoopNest pass

This patch changes LoopUnrollAndJamPass from FunctionPass to LoopNest pass.
The next patch will utilize LoopNest to effectively handle

[LoopUnrollAndJam] Change LoopUnrollAndJamPass to LoopNest pass

This patch changes LoopUnrollAndJamPass from FunctionPass to LoopNest pass.
The next patch will utilize LoopNest to effectively handle loop nests.

Reviewed By: Whitney

Differential Revision: https://reviews.llvm.org/D99149

show more ...

[SampleFDO] New hierarchical discriminator for Flow Sensitive SampleFDO

This patch implements first part of Flow Sensitive SampleFDO (FSAFDO).
It has the following changes:
(1) disable current discr

[SampleFDO] New hierarchical discriminator for Flow Sensitive SampleFDO

This patch implements first part of Flow Sensitive SampleFDO (FSAFDO).
It has the following changes:
(1) disable current discriminator encoding scheme,
(2) new hierarchical discriminator for FSAFDO.

For this patch, option "-enable-fs-discriminator=true" turns on the new
functionality. Option "-enable-fs-discriminator=false" (the default)
keeps the current SampleFDO behavior. When the fs-discriminator is
enabled, we insert a flag variable, namely, llvm_fs_discriminator, to
the object. This symbol will checked by create_llvm_prof tool, and used
to generate a profile with FS-AFDO discriminators enabled. If this
happens, for an extbinary format profile, create_llvm_prof tool
will add a flag to profile summary section.

Differential Revision: https://reviews.llvm.org/D102246

show more ...

[LoopUnrollAndJam] Avoid repeated instructions for UAJ analysis

Avoid visiting repeated instructions for processHeaderPhiOperands as it can cause a scenario of endless loop. Test case is attached an

[LoopUnrollAndJam] Avoid repeated instructions for UAJ analysis

Avoid visiting repeated instructions for processHeaderPhiOperands as it can cause a scenario of endless loop. Test case is attached and can be ran with `opt -basic-aa -tbaa -loop-unroll-and-jam -allow-unroll-and-jam -unroll-and-jam-count=4`.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D97407

show more ...