[CostModel] Replace getUserCost with getInstructionCost
* Replace getUserCost with getInstructionCost, covering all cost kinds.
* Remove getInstructionLatency, it's not implemented by any backends, a

[CostModel] Replace getUserCost with getInstructionCost
* Replace getUserCost with getInstructionCost, covering all cost kinds.
* Remove getInstructionLatency, it's not implemented by any backends, and we should fold the functionality into getUserCost (now getInstructionCost) to make it easier for targets to handle the cost kinds with their existing cost callbacks.

Original Patch by @samparker (Sam Parker)

Differential Revision: https://reviews.llvm.org/D79483

show more ...

[Transforms] Qualify auto in range-based for loops (NFC)

Identified with readability-qualified-auto.

[SimpleLoopUnswitch] Skip non-trivial unswitching of cold loops

With profile data, non-trivial LoopUnswitch will only apply on non-cold loops, as unswitching cold loops may not gain much benefit but

[SimpleLoopUnswitch] Skip non-trivial unswitching of cold loops

With profile data, non-trivial LoopUnswitch will only apply on non-cold loops, as unswitching cold loops may not gain much benefit but significantly increase the code size.

Reviewed By: aeubanks, asbirlea

Differential Revision: https://reviews.llvm.org/D129599

show more ...

[NFC] Switch a few uses of undef to poison as placeholders for unreachble code

Don't use Optional::hasValue (NFC)

[Scalar] Use llvm::append_range (NFC)

Remove unneeded cl::ZeroOrMore for cl::opt/cl::list options

[SimpleLoopUnswitch] Always skip trivial select and set condition.

When updating the branch instruction outside the loopduring non-trivial
unswitching, always skip trivial selects and update the co

[SimpleLoopUnswitch] Always skip trivial select and set condition.

When updating the branch instruction outside the loopduring non-trivial
unswitching, always skip trivial selects and update the condition.

Otherwise we might create invalid IR, because the trivial select is
inside the loop, while the condition is outside the loop.

Fixes #55697.

show more ...

[SimpleLoopUnswitch] Skip trivial selects during trivial unswitching.

Update the remaining places in unswitchTrivialBranch to properly skip
trivial selects.

Fixes #55526.

Recommit "[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both."

This reverts commit 7211d5ce07830ebfa2cfc30818cd7155375f7e47.

This version fixes a crash that caused buildbot failures

Recommit "[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both."

This reverts commit 7211d5ce07830ebfa2cfc30818cd7155375f7e47.

This version fixes a crash that caused buildbot failures with the first
version.

show more ...

Revert "[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both."

This reverts commit db7a87ed4fa79a7c366438ae62d8a1088a11f29c.

This seems to cause a PPC buildbot failure:
https://lab.llv

Revert "[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both."

This reverts commit db7a87ed4fa79a7c366438ae62d8a1088a11f29c.

This seems to cause a PPC buildbot failure:
https://lab.llvm.org/buildbot#builders/93/builds/8787

show more ...

[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both.

After D97756, collectHomogenousInstGraphLoopInvariants may collect
conditions for both logical ANDs and logical ORs in case the roo

[SimpleLoopUnswitch] Collect either logical ANDs/ORs but not both.

After D97756, collectHomogenousInstGraphLoopInvariants may collect
conditions for both logical ANDs and logical ORs in case the root is a
select that matches both logical AND & OR.

This means the function won't return invariant values of either AND/OR
chains, but both. This can result in incorrect transformations.

See llvm/test/Transforms/SimpleLoopUnswitch/trivial-unswitch-logical-and-or.ll.
Without the patch, Alive2 rejects the modified tests with:
Source and target don't have the same return domain.

Note that this also applies to the test case added in D97756
(@test_partial_condition_unswitch_or_select). We can't unswitch on
%cond6, because the graph leading to it contains and AND and an OR.

This only fixes trivial unswitching for now, but a similar problem
likely exists with non-trivial unswitching.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D124526

show more ...

[SimpleLoopUnswitch] Add freeze if branch execs for partial unswitching.

We cannot skip the freezing the condition if the unswitched branch
executes, if the condition is a chain of ANDs/ORs. For exa

[SimpleLoopUnswitch] Add freeze if branch execs for partial unswitching.

We cannot skip the freezing the condition if the unswitched branch
executes, if the condition is a chain of ANDs/ORs. For example, if if we
have an AND %c1, %c2 with %c1 == undef and %c2 == 0, there would be no
branch on undef in the original code, but a branch on undef if we
unswitch %c1.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D124603

show more ...

[SimpleLoopUnswitch] Freeze individual OR/AND operands.

In some cases, it is not enough to freeze the final AND/OR operation
when chaining a number of invariant conditions together.

After creating

[SimpleLoopUnswitch] Freeze individual OR/AND operands.

In some cases, it is not enough to freeze the final AND/OR operation
when chaining a number of invariant conditions together.

After creating a chain of ANDs/ORs, we assume all unswitched operands to
be either true or false. But if any of the operands is poison, the rest
of the operands could have any value after branching on the frozen
condition.

To avoid that, freeze individual operands, if needed. In some cases this
may lead to unnecessary freezes, but it seems required at least for some
cases (see trivial-unswitch-freeze-individual-conditions.ll)

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D124554

show more ...

[SimpleLoopUnswitch] Freeze trivial conditions if needed.

Trivial unswitching can also introduce new branches on undef/poison.
Freeze the conditions if needed.

Reviewed By: nikic

Differential Revi

[SimpleLoopUnswitch] Freeze trivial conditions if needed.

Trivial unswitching can also introduce new branches on undef/poison.
Freeze the conditions if needed.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D124549

show more ...

[SimpleLoopUnswitch] Enable freezing of conditions by default.

This fixes a series of mis-compiles by SimpleLoopUnswitch.

My measurements showed no performance regression with -O3 on AArch64
in SPE

[SimpleLoopUnswitch] Enable freezing of conditions by default.

This fixes a series of mis-compiles by SimpleLoopUnswitch.

My measurements showed no performance regression with -O3 on AArch64
in SPEC2006, SPEC2017 and a set of internal benchmarks.

Fixes #50387, #50430

Depends on D124251.

Reviewed By: nikic, aqjune

Differential Revision: https://reviews.llvm.org/D124252

show more ...

[SimpleLoopUnswitch] Check if freeze is needed for partial unswitching.

We only need to insert a Freeze instruction if any of the conditions
may be poison. Similar checks are already done in the oth

[SimpleLoopUnswitch] Check if freeze is needed for partial unswitching.

We only need to insert a Freeze instruction if any of the conditions
may be poison. Similar checks are already done in the other places
SimpleLoopUnswitch creates Freeze instruction.

Reviewed By: aeubanks, efriedma

Differential Revision: https://reviews.llvm.org/D124259

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-

Cleanup includes: TransformsUtils

Estimation on the impact on preprocessor output:
before: 1065307662
after: 1064800684

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120741

show more ...

[SimpleLoopUnswitch] Remove duplicate include.

Header "llvm/Transforms/Scalar/SimpleLoopUnswitch.h" is currently
included twice. This commit removes the duplicate 'include' line.

Previous commit 69

[SimpleLoopUnswitch] Remove duplicate include.

Header "llvm/Transforms/Scalar/SimpleLoopUnswitch.h" is currently
included twice. This commit removes the duplicate 'include' line.

Previous commit 693eedb13833245e2670308fa0e6fe47324ce553
seems to have mistakenly added the duplicate 'include'.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D112979

show more ...

[SimpleLoopUnswitch] Re-fix introduction of UB when hoisted condition may be undef or poison

https://bugs.llvm.org/show_bug.cgi?id=27506
https://bugs.llvm.org/show_bug.cgi?id=31652
https://bugs.llvm

[SimpleLoopUnswitch] Re-fix introduction of UB when hoisted condition may be undef or poison

https://bugs.llvm.org/show_bug.cgi?id=27506
https://bugs.llvm.org/show_bug.cgi?id=31652
https://bugs.llvm.org/show_bug.cgi?id=51043

Problems with SimpleLoopUnswitch cause the bug reports above.

```
while (...) {
if (C) { A }
else { B }
}
Into:

C' = freeze(C)
if (C') {
while (...) { A }
} else {
while (...) { B }
}
```
This problem can be solved by adding a freeze on hoisted branches(above transform) and has been solved by D29015.
However, D29015 is now reverted by performance regression(https://github.com/llvm/llvm-project/commit/2b5a8976514de326bb84f0913d9d451089c11d22)

It is not the first time that an added freeze has caused performance regression.
SimplifyCFG also had a problem with UB caused by branching-on-undef, which was solved by adding freeze to the branching condition. (D104569)
Performance regression occurred in D104569, and patches such as D105344 and D105392 were written to minimize it.

This patch will correct the SimpleLoopUnswitch as D104569 handles the SimplyCFG while minimizing performance loss by introducing patches like D105344 and D105392(This patch was rebased with the author's permission)

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D106041

show more ...

[SimpleLoopUnswitch] Allow threshold to be specified zero or more times

Differential Revision: https://reviews.llvm.org/D110594

[SimpleLoopUnswitch] Don't unswitch constant conditions

Added an additional check for constants after simplification of
"select _, true, false" pattern. We need to prevent attempts to unswitch const

[SimpleLoopUnswitch] Don't unswitch constant conditions

Added an additional check for constants after simplification of
"select _, true, false" pattern. We need to prevent attempts to unswitch constant
conditions for two reasons:
a) Doing that doesn't make any sense, in the best case it will just burn
some compile time.
b) SimpleLoopUnswitch isn't designed to unswitch constant conditions
(due to (a)), so attempting that can cause miscompiles. The attached
testcase is an example of such miscompile.

Also added an assertion that'll make sure we aren't trying to replace
constants, so it will help us prevent such bugs in future. The assertion
from D110751 is another layer of protection against such cases.

Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D110752

show more ...

[NPM] Added -print-pipeline-passes print params for a few passes.

Added '-print-pipeline-passes' printing of parameters for those passes
declared with *_WITH_PARAMS macro in PassRegistry.def.

Note

[NPM] Added -print-pipeline-passes print params for a few passes.

Added '-print-pipeline-passes' printing of parameters for those passes
declared with *_WITH_PARAMS macro in PassRegistry.def.

Note that it only prints the parameters declared inside *_WITH_PARAMS as
in a few cases there appear to be additional parameters not parsable.

The following passes are now covered (i.e. all of those with *_WITH_PARAMS in
PassRegistry.def).

LoopExtractorPass - loop-extract
HWAddressSanitizerPass - hwsan
EarlyCSEPass - early-cse
EntryExitInstrumenterPass - ee-instrument
LowerMatrixIntrinsicsPass - lower-matrix-intrinsics
LoopUnrollPass - loop-unroll
AddressSanitizerPass - asan
MemorySanitizerPass - msan
SimplifyCFGPass - simplifycfg
LoopVectorizePass - loop-vectorize
MergedLoadStoreMotionPass - mldst-motion
GVN - gvn
StackLifetimePrinterPass - print<stack-lifetime>
SimpleLoopUnswitchPass - simple-loop-unswitch

Differential Revision: https://reviews.llvm.org/D109310

show more ...

[SimpleLoopUnswitch] Inform pass manager when child loops are deleted

As part of the nontrivial unswitching we could end up removing child
loops. This patch add a notification to the pass manager wh

[SimpleLoopUnswitch] Inform pass manager when child loops are deleted

As part of the nontrivial unswitching we could end up removing child
loops. This patch add a notification to the pass manager when
that happens (using the markLoopAsDeleted callback).

Without this there could be stale LoopAccessAnalysis results cached
in the analysis manager. Those analysis results are cached based on
a Loop* as key. Since the BumpPtrAllocator used to allocate
Loop objects could be resetted between different runs of for
example the loop-distribute pass (running on different functions),
a new Loop object could be created using the same Loop pointer.
And then when requiring the LoopAccessAnalysis for the loop we
got the stale (corrupt) result from the destroyed loop.

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D109257

show more ...