[InstCombine][NFC] Fold variable into assert

Avoids unused variable warnings when asserts are disabled.

[InstCombine] Don't replace unused `atomicrmw xchg` with `atomic store`

Following the discussion from https://reviews.llvm.org/D141277 and in
particular Ralf Jung's comment at
https://reviews.llvm.o

[InstCombine] Don't replace unused `atomicrmw xchg` with `atomic store`

Following the discussion from https://reviews.llvm.org/D141277 and in
particular Ralf Jung's comment at
https://reviews.llvm.org/D141277#inline-1365148, replacing an unused `atomicrmw
xchg` into an `atomic store` is illegal even for release ordering.

Quoting Connor Horman from the rust lang discussion linked in that comment:
"An acquire operation A only synchronizes-with a release operation R if it
takes its value from R, or any store in the release sequence headed by R, which
is R, followed by the longest continuous sequence of read-modify-write
operations.
A regular store following R in the modification order would break the release
sequence, and if an acquire operation reads that store or something later, then
it loses any synchronization it might have already had."

This fixes https://github.com/llvm/llvm-project/issues/60418

Differential Revision: https://reviews.llvm.org/D142097

show more ...

[InstCombine] Don't optimize idempotent `atomicrmw <op>, 0` into `load atomic`

Turning idempotent `atomicrmw`s into `load atomic` is perfectly legal
with respect to how the loading happens, but it m

[InstCombine] Don't optimize idempotent `atomicrmw <op>, 0` into `load atomic`

Turning idempotent `atomicrmw`s into `load atomic` is perfectly legal
with respect to how the loading happens, but it may not be legal for the
whole program semantic.

Indeed, this optimization removes a store that may have some effects on
the legality of other optimizations.
Essentially, we lose some information and depending on the backend
it may or may not produce incorrect code, so don't do it!

This fixes llvm.org/PR56450.

Differential Revision: https://reviews.llvm.org/D141277

show more ...

[InstCombine] Preserve alignment in atomicrmw -> store fold

Preserve the alignment of the original atomicrmw, rather than using
the ABI alignment.

The same problem exists for loads, but that code i

[InstCombine] Preserve alignment in atomicrmw -> store fold

Preserve the alignment of the original atomicrmw, rather than using
the ABI alignment.

The same problem exists for loads, but that code is being removed
in D141277 anyway.

show more ...

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instr

[LLVM] Add the support for fmax and fmin in atomicrmw instruction

This patch adds the support for `fmax` and `fmin` operations in `atomicrmw`
instruction. For now (at least in this patch), the instruction will be expanded
to CAS loop. There are already a couple of targets supporting the feature. I'll
create another patch(es) to enable them accordingly.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D127041

show more ...

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cl

Cleanup includes: Transform/Scalar

Estimated impact on preprocessor output line:
before: 1062981579
after: 1062494547

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120817

show more ...

Add missing namespace closure comments. NFCI.

Fixes some clang-tidy llvm-namespace-comment warnings.

[InstCombine] Move target-specific inst combining

For a long time, the InstCombine pass handled target specific
intrinsics. Having target specific code in general passes was noted as
an area for imp

[InstCombine] Move target-specific inst combining

For a long time, the InstCombine pass handled target specific
intrinsics. Having target specific code in general passes was noted as
an area for improvement for a long time.

D81728 moves most target specific code out of the InstCombine pass.
Applying the target specific combinations in an extra pass would
probably result in inferior optimizations compared to the current
fixed-point iteration, therefore the InstCombine pass resorts to newly
introduced functions in the TargetTransformInfo when it encounters
unknown intrinsics.
The patch should not have any effect on generated code (under the
assumption that code never uses intrinsics from a foreign target).

This introduces three new functions:
TargetTransformInfo::instCombineIntrinsic
TargetTransformInfo::simplifyDemandedUseBitsIntrinsic
TargetTransformInfo::simplifyDemandedVectorEltsIntrinsic

A few target specific parts are left in the InstCombine folder, where
it makes sense to share code. The largest left-over part in
InstCombineCalls.cpp is the code shared between arm and aarch64.

This allows to move about 3000 lines out from InstCombine to the targets.

Differential Revision: https://reviews.llvm.org/D81728

show more ...

[NFC] Modernize misc. uses of Align/MaybeAlign APIs.

Use the current getAlign() APIs where it makes sense, and use Align
instead of MaybeAlign when we know the value is non-zero.

[InstCombine] Use replaceOperand() in more places

This is a followup to D73803, which uses the replaceOperand()
helper in more places.

This should be NFC apart from changes to worklist order.

Diff

[InstCombine] Use replaceOperand() in more places

This is a followup to D73803, which uses the replaceOperand()
helper in more places.

This should be NFC apart from changes to worklist order.

Differential Revision: https://reviews.llvm.org/D73919

show more ...

[Alignment][NFC] Remove StoreInst::setAlignment(unsigned)

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm

[Alignment][NFC] Remove StoreInst::setAlignment(unsigned)

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, bollu, jdoerfert

Subscribers: hiraditya, asbirlea, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D68268

llvm-svn: 373595

show more ...

[Alignment][NFC] Remove LoadInst::setAlignment(unsigned)

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-

[Alignment][NFC] Remove LoadInst::setAlignment(unsigned)

Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790

Reviewers: courbet, jdoerfert

Subscribers: hiraditya, asbirlea, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D68142

llvm-svn: 373195

show more ...

[InstCombine] Extend saturating idempotent atomicrmw transform to FP

I'm assuming that the nan propogation logic for InstructonSimplify's handling of fadd and fsub is correct, and applying the same

[InstCombine] Extend saturating idempotent atomicrmw transform to FP

I'm assuming that the nan propogation logic for InstructonSimplify's handling of fadd and fsub is correct, and applying the same to atomicrmw.

Differential Revision: https://reviews.llvm.org/D58836

llvm-svn: 355222

show more ...

[InstCombine] Extend "idempotent" atomicrmw optimizations to floating point

An idempotent atomicrmw is one that does not change memory in the process of execution. We have already added handling fo

[InstCombine] Extend "idempotent" atomicrmw optimizations to floating point

An idempotent atomicrmw is one that does not change memory in the process of execution. We have already added handling for the various integer operations; this patch extends the same handling to floating point operations which were recently added to IR.

Note: At the moment, we canonicalize idempotent fsub to fadd when ordering requirements prevent us from using a load. As discussed in the review, I will be replacing this with canonicalizing both floating point ops to integer ops in the near future.

Differential Revision: https://reviews.llvm.org/D58251

llvm-svn: 355210

show more ...

[InstCombine] Address a couple stylistic issues pointed out by reviewer [NFC]

Better addressing comments from https://reviews.llvm.org/D58290.

llvm-svn: 354171

[InstCombine] Convert atomicrmws to xchg or store where legal

Implement two more transforms of atomicrmw:
1) We can convert an atomicrmw which produces a known value in memory into an xchg instead.

[InstCombine] Convert atomicrmws to xchg or store where legal

Implement two more transforms of atomicrmw:
1) We can convert an atomicrmw which produces a known value in memory into an xchg instead.
2) We can convert an atomicrmw xchg w/o users into a store for some orderings.

Differential Revision: https://reviews.llvm.org/D58290

llvm-svn: 354170

show more ...

[InstCombine] Add todos for possible atomicrmw transforms

llvm-svn: 354059

Canonicalize all integer "idempotent" atomicrmw ops

For "idempotent" atomicrmw instructions which we can't simply turn into load, canonicalize the operation and constant. This reduces the matching n

Canonicalize all integer "idempotent" atomicrmw ops

For "idempotent" atomicrmw instructions which we can't simply turn into load, canonicalize the operation and constant. This reduces the matching needed elsewhere in the optimizer, but doesn't directly impact codegen.

For any architecture where OR/Zero is not a good default choice, you can extend the AtomicExpand lowerIdempotentRMWIntoFencedLoad mechanism. I reviewed X86 to make sure this works well, haven't audited other backends.

Differential Revision: https://reviews.llvm.org/D58244

llvm-svn: 354058

show more ...

Teach instcombine about remaining "idempotent" atomicrmw types

Expand on Quentin's r353471 patch which converts some atomicrmws into loads. Handle remaining operation types, and fix a slight bug. At

Teach instcombine about remaining "idempotent" atomicrmw types

Expand on Quentin's r353471 patch which converts some atomicrmws into loads. Handle remaining operation types, and fix a slight bug. Atomic loads are required to have alignment. Since this was within the InstCombine fixed point, somewhere else in InstCombine was adding alignment before the verifier saw it, but still, we should fix.

Terminology wise, I'm using the "idempotent" naming that is used for the same operations in AtomicExpand and X86ISelLoweringInfo. Once this lands, I'll add similar tests for AtomicExpand, and move the pattern match function to a common location. In the review, there was seemingly consensus that "idempotent" was slightly incorrect for this context. Once we setle on a better name, I'll update all uses at once.

Differential Revision: https://reviews.llvm.org/D58242

llvm-svn: 354046

show more ...

[InstCombine] Optimize `atomicrmw <op>, 0` into `load atomic` when possible

This commit teaches InstCombine how to replace an atomicrmw operation
into a simple load atomic.
For a given `atomicrmw <o

[InstCombine] Optimize `atomicrmw <op>, 0` into `load atomic` when possible

This commit teaches InstCombine how to replace an atomicrmw operation
into a simple load atomic.
For a given `atomicrmw <op>`, this is possible when:
1. The ordering of that operation is compatible with a load (i.e.,
anything that doesn't have a release semantic).
2. <op> does not modify the value being stored

Differential Revision: https://reviews.llvm.org/D57854

llvm-svn: 353471

show more ...