EarlyCSE: fix CmpPredicate duplicate-hashing (#119902)

Strip hash_value() for CmpPredicate, as different callers have different
hashing use-cases. In this case, there is just one caller, namely
Ea

EarlyCSE: fix CmpPredicate duplicate-hashing (#119902)

Strip hash_value() for CmpPredicate, as different callers have different
hashing use-cases. In this case, there is just one caller, namely
EarlyCSE, which calls hash_combine() on a CmpPredicate, which used to
call hash_combine() on a CmpInst::Predicate prior to 4a0d53a
(PatternMatch: migrate to CmpPredicate). This has uncovered a bug where
two icmp instructions differing in just the fact that one of them has
the samesign flag on it are hashed differently, leading to divergent
hashing, and a crash. Fix this crash by dropping samesign information on
icmp instructions before hashing them, preserving the former behavior.

Fixes #119893.

show more ...

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key p

PatternMatch: migrate to CmpPredicate (#118534)

With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key pieces of infrastructure that must be updated to match a CmpInst
respecting samesign information. Implement this change to Cmp-matchers.

This is a preparatory step in migrating the codebase over to
CmpPredicate. Since we no functional changes are desired at this stage,
we have chosen not to migrate CmpPredicate::operator==(CmpPredicate)
calls to use CmpPredicate::getMatching(), as that would have visible
impact on tests that are not yet written: instead, we call
CmpPredicate::operator==(Predicate), preserving the old behavior, while
also inserting a few FIXME comments for follow-ups.

show more ...

EarlyCSE: refactor getOrCreateResult (NFC) (#113339)

[EarlyCSE] De-Duplicate callsites with differing attrs

We only do this if the attributes of the two callsites are compatible
(intersectable) which is probably not in fact necessary.

Closes #110929

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, re

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, replacing the
current `getParent()->getDataLayout()` pattern.

show more ...

[EarlyCSE] Support CSE for commutative intrinsics with over 2 args (#67255)

Extends EarlyCSE to support commutative intrinsics with over 2 args.

[EarlyCSE] Compare GEP instructions based on offset (#65875)

Closes #65763.
This will provide more opportunities for constant propagation for
subsequent optimizations.

fix unused variable warnings in conditionals

for 92023b15099012a657da07ebf49dd7d94a260f84

[EarlyCSE] Do not CSE convergent calls with memory effects

D149348 did this for readnone calls, which are handled by SimpleValue.
This patch does the same for all other CSEable calls, which are hand

[EarlyCSE] Do not CSE convergent calls with memory effects

D149348 did this for readnone calls, which are handled by SimpleValue.
This patch does the same for all other CSEable calls, which are handled
by CallValue.

Differential Revision: https://reviews.llvm.org/D153151

show more ...

[IR] Add getAccessType to Instruction

There are multiple places in the code where the type of memory being accessed from an instruction needs to be obtained, including an upcoming patch to improve G

[IR] Add getAccessType to Instruction

There are multiple places in the code where the type of memory being accessed from an instruction needs to be obtained, including an upcoming patch to improve GEP cost modeling. This deduplicates the logic between them. It's not strictly NFC as EarlyCSE/LoopStrengthReduce may catch more intrinsics now.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D150583

show more ...

[EarlyCSE] Only combine metadata for load CSE

There is no need to combine metadata if we're performing store to
load forwarding. In that case we would end up combining metadata
on an unrelated load

[EarlyCSE] Only combine metadata for load CSE

There is no need to combine metadata if we're performing store to
load forwarding. In that case we would end up combining metadata
on an unrelated load instruction.

show more ...

[EarlyCSE] Do not CSE convergent calls in different basic blocks

"convergent" is documented as meaning that the call cannot be made
control-dependent on more values, but in practice we also require

[EarlyCSE] Do not CSE convergent calls in different basic blocks

"convergent" is documented as meaning that the call cannot be made
control-dependent on more values, but in practice we also require that
it cannot be made control-dependent on fewer values, e.g. it cannot be
hoisted out of the body of an "if" statement.

In code like this, if we allow CSE to combine the two calls:

x = convergent_call();
if (cond) {
y = convergent_call();
use y;
}

then we get this:

x = convergent_call();
if (cond) {
use x;
}

This is conceptually equivalent to moving the second call out of the
body of the "if", up to the location of the first call, so it should be
disallowed.

Differential Revision: https://reviews.llvm.org/D149348

show more ...

[EarlyCSE] Call combineMetadataForCSE() when CSEing loads

We may have to adjust metadata on the replacement load if the
metadata is poison-generating.

[NFC] Use single quotes for single char output during `printPipline`

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D144365

[EarlyCSE] Fix crash when optimizing masked loads/stores

With opaque pointers, it is possible for EarlyCSE to encounter masked
load/store intrinsics which access the same pointer value but with
diff

[EarlyCSE] Fix crash when optimizing masked loads/stores

With opaque pointers, it is possible for EarlyCSE to encounter masked
load/store intrinsics which access the same pointer value but with
different incompatible types. These cannot form valid replacements
(without explicit casting, which we don't yet do even for regular
load/store instructions) so should be prevented.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D141613

show more ...

[NFC] Removed call to getInstList() from range loops on BBs.

Differential Revision: https://reviews.llvm.org/D138605

[Coroutines] Don't merge readnone calls in presplit coroutines

Another alternative to fix the thread identification problem in
coroutines.

We plan to fix this problem by unifying memory effecting

[Coroutines] Don't merge readnone calls in presplit coroutines

Another alternative to fix the thread identification problem in
coroutines.

We plan to fix this problem by unifying memory effecting attributes. See
https://discourse.llvm.org/t/rfc-unify-memory-effect-attributes/65579.
But it may be a long-term project. And it is a pity that the coroutines
can't resume in different threads for years. So this one is temporary
fix. It may cause unnecessary performance regression for coroutines. But
correctness are more important. And this one is planned to be reverted
after we are able to unify the memory effecting attributes actually.

Reviewed By: jdoerfert, rjmccall

Differential Revision: https://reviews.llvm.org/D135550

show more ...

Fix build error: [FPEnv][EarlyCSE] Support for CSE when exception behavior is "ignore" or "maytrap" and the rounding mode is known.

This should fix these build bot errors:

Step 6 (build-check-mlir-

Fix build error: [FPEnv][EarlyCSE] Support for CSE when exception behavior is "ignore" or "maytrap" and the rounding mode is known.

This should fix these build bot errors:

Step 6 (build-check-mlir-build-only) failure: build (failure)
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Transforms\Scalar\EarlyCSE.cpp(124): error C2220: the following warning is treated as an error
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Transforms\Scalar\EarlyCSE.cpp(124): warning C4996: 'llvm::Optional<llvm::fp::ExceptionBehavior>::getValue': Use value instead.
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Transforms\Scalar\EarlyCSE.cpp(129): warning C4996: 'llvm::Optional<llvm::RoundingMode>::getValue': Use value instead.
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Transforms\Scalar\EarlyCSE.cpp(1386): warning C4996: 'llvm::Optional<llvm::fp::ExceptionBehavior>::getValue': Use value instead.
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Transforms\Scalar\EarlyCSE.cpp(1388): warning C4996: 'llvm::Optional<llvm::RoundingMode>::getValue': Use value instead.

show more ...

[FPEnv][EarlyCSE] Support for CSE when exception behavior is "ignore" or "maytrap" and the rounding mode is known.

Previously we would only CSE constrained FP intrinsics in the default
floating poin

[FPEnv][EarlyCSE] Support for CSE when exception behavior is "ignore" or "maytrap" and the rounding mode is known.

Previously we would only CSE constrained FP intrinsics in the default
floating point environment. Exception behavior of "strict" is still not
allowed since we are not allowed to remove any traps in that case.

There are no restrictions on CSE across function calls inside a function.

Differential Revision: https://reviews.llvm.org/D112256

show more ...

[EarlyCSE][OpaquePointers]Replace assert with return for mask type check.

When EarlyCSE tries to common vector masked loads/stores, it first checks that
they have same base operand and then assumes

[EarlyCSE][OpaquePointers]Replace assert with return for mask type check.

When EarlyCSE tries to common vector masked loads/stores, it first checks that
they have same base operand and then assumes that this is enough for mask types
to be equal. This is true for typed pointers but false for opaque ones -
two loads of different vector sizes from same base pointer '%b' are the same,
`ptr %b`. (For typed pointers, `%b` was cast to vector pointer type so bases
were different).
Change assert to return from lambda `isSubmask` so this transformation properly
works with opaque pointers.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D131251

show more ...

[llvm] Qualify auto (NFC)

Identified with readability-qualified-auto.

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with

[NFC] format InstructionSimplify & lowerCaseFunctionNames

Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.

This is the alternative to the less invasive clang-format only patch: D126783

Reviewed By: spatel, rengolin

Differential Revision: https://reviews.llvm.org/D126889

show more ...

Revert f7381a795ab235d34c94eaf01dc880eb5b89619d

Roll-forward 29fada4a3d3db309f11f7fa7a0c61cd4021e9947.
Issue triggered was due to UB.

Differential Revision: https://reviews.llvm.org/D121987

Revert 29fada4a3d3db309f11f7fa7a0c61cd4021e9947

Seeing a test failure with asan in Halide generated code, reverting
while I investigate.

Differential Revision: https://reviews.llvm.org/D121987

[EarlyCSE] Don't eagerly optimize MemoryUses

EarlyCSE currently optimizes all MemoryUses upfront. However,
EarlyCSE only actually queries the clobbering memory access for
a subset of uses, namely th

[EarlyCSE] Don't eagerly optimize MemoryUses

EarlyCSE currently optimizes all MemoryUses upfront. However,
EarlyCSE only actually queries the clobbering memory access for
a subset of uses, namely those where a CSE candidate has already
been identified. Delaying use optimization to the clobber query
improves compile-time in practice.

This change is not NFC because EarlyCSE has a limit on the number
of clobber queries (EarlyCSEMssaOptCap), in which case it falls
back to the defining access. The defining access for uses will now
no longer coincide with the optimized access.

If there are performance regressions from this change, we should
be able to address them by raising this limit.

Differential Revision: https://reviews.llvm.org/D121987

show more ...