[IR] Drop poison-generating return attributes when necessary (#89138)

Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align a

[IR] Drop poison-generating return attributes when necessary (#89138)

Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.

show more ...

[ValueTracking] Restore isKnownNonZero parameter order. (#88873)

Prior to #85863, the required parameters of llvm::isKnownNonZero were
Value and DataLayout. After, they are Value, Depth, and Simpli

[ValueTracking] Restore isKnownNonZero parameter order. (#88873)

Prior to #85863, the required parameters of llvm::isKnownNonZero were
Value and DataLayout. After, they are Value, Depth, and SimplifyQuery,
where SimplifyQuery is implicitly constructible from DataLayout. The
change to move Depth before SimplifyQuery needed callers to be updated
unnecessarily, and as commented in #85863, we actually want Depth to be
after SimplifyQuery anyway so that it can be defaulted and the caller
does not need to specify it.

show more ...

[ValueTracking] Convert `isKnownNonZero` to use SimplifyQuery (#85863)

This patch converts `isKnownNonZero` to use SimplifyQuery. Then we can
use the context information from `DomCondCache`.

Fix

[ValueTracking] Convert `isKnownNonZero` to use SimplifyQuery (#85863)

This patch converts `isKnownNonZero` to use SimplifyQuery. Then we can
use the context information from `DomCondCache`.

Fixes https://github.com/llvm/llvm-project/issues/85823.
Alive2: https://alive2.llvm.org/ce/z/QUvHVj

show more ...

[SCEV] Add range attribute handling (#88449)

[SCEV] Fix BinomialCoefficient Iteration to fit in W bits (#88010)

BinomialCoefficient computes the value of W-bit IV at iteration It of a loop. When W is 1, we can call multiplicative inverse on 0

[SCEV] Fix BinomialCoefficient Iteration to fit in W bits (#88010)

BinomialCoefficient computes the value of W-bit IV at iteration It of a loop. When W is 1, we can call multiplicative inverse on 0 which triggers an assert since 1b76120.

Since the arithmetic is supposed to wrap if It or K does not fit in W bits, do the truncation into W bits after we do the shift.

Fixes #87798

show more ...

[APInt] Add a simpler overload of multiplicativeInverse (#87610)

The current APInt::multiplicativeInverse takes a modulus which can be
any value, but all in-tree callers use a power of two. Moreove

[APInt] Add a simpler overload of multiplicativeInverse (#87610)

The current APInt::multiplicativeInverse takes a modulus which can be
any value, but all in-tree callers use a power of two. Moreover, most
callers want to use two to the power of the width of an existing APInt,
which is awkward because 2^N is not representable as an N-bit APInt.

Add a new overload of multiplicativeInverse which implicitly uses
2^BitWidth as the modulus.

show more ...

Add llvm::min/max_element and use it in llvm/ and mlir/ directories. (#84678)

For some reason this was missing from STLExtras.

[SCEV] Match both (-1)b + a and a + (-1)b as a - b (#84247)

In our analysis of guarding conditions, we were converting a-b == 0 into
a == b alternate form, but we were only checking for one of the

[SCEV] Match both (-1)b + a and a + (-1)b as a - b (#84247)

In our analysis of guarding conditions, we were converting a-b == 0 into
a == b alternate form, but we were only checking for one of the two
forms for the sub. There's no requirement that the multiply only be on
the LHS of the add.

show more ...

[SCEV] Extend type hint in analysis output to all backedge kinds

This extends the work from 7755c26 to all of the different backend
taken count kinds that we print for the scev analysis printer. As

[SCEV] Extend type hint in analysis output to all backedge kinds

This extends the work from 7755c26 to all of the different backend
taken count kinds that we print for the scev analysis printer. As
before, the goal is to cut down on confusion as i4 -1 is a very
different (unsigned) value from i32 -1.

show more ...

[SCEV] Print predicate backedge count only if new information available

When printing the result of SCEV's analysis, we can avoid printing
the predicated backedge taken count and the predicates if t

[SCEV] Print predicate backedge count only if new information available

When printing the result of SCEV's analysis, we can avoid printing
the predicated backedge taken count and the predicates if the predicates
are empty and no new information is provided. This helps to reduce the
verbosity of the output.

show more ...

[SCEV] Include type when printing constant max backedge taken count

When printing the result of the analysis, i8 -1 and i64 -1 are quite
different in terms of analysis quality. In a recent conversi

[SCEV] Include type when printing constant max backedge taken count

When printing the result of the analysis, i8 -1 and i64 -1 are quite
different in terms of analysis quality. In a recent conversion with
a new contributor, we ran into exactly this confusion.

Adding the type for constant scevs more globally seems worthwhile, but
introduces a much larger test diff. I'm splitting this off first since
it addresses the immediate need, and then going to do some further
changes to clarify a few related bits of analysis result output.

show more ...

[LV] Handle scalable VFs in optimizeForVFAndUF (#82669)

Given a scalable VF of the form <NumElts * VScale>, this patch adds the
ability to discharge a backedge test for a loop whose trip count is

[LV] Handle scalable VFs in optimizeForVFAndUF (#82669)

Given a scalable VF of the form <NumElts * VScale>, this patch adds the
ability to discharge a backedge test for a loop whose trip count is
between (NumElts, MinVScale*NumElts).

A couple of notes on this:
* Annoyingly, I could not figure out to write a test for this case. My
attempt is checked in as test32_i8 in f67ef1a, but LV uses a fixed
vector in that case, and ignored the force flags.
* This depends on 9eb5f94f to avoid appearing like a regression. Since
SCEV doesn't know any upper bound on vscale without the vscale_range
attribute (it doesn't query TTI), the ranges overflow on the multiply.
Arguably, this is fixing a bug in the current LV code since in theory
vscale can be large enough to overflow for real, but no actual target is
going to see that case.

show more ...

[SCEV] Move canReuseInstruction() helper into SCEV (NFC)

To allow reusing it in IndVars.

[SCEV] Special case sext in isKnownNonZero (#77834)

The existing logic in isKnownNonZero relies on unsigned ranges, which
can be problematic when our range calculation is imprecise. Consider the
f

[SCEV] Special case sext in isKnownNonZero (#77834)

The existing logic in isKnownNonZero relies on unsigned ranges, which
can be problematic when our range calculation is imprecise. Consider the
following:
%offset.nonzero = or i32 %offset, 1
--> %offset.nonzero U: [1,0) S: [1,0)
%offset.i64 = sext i32 %offset.nonzero to i64
--> (sext i32 %offset.nonzero to i64) U: [-2147483648,2147483648)
S: [-2147483648,2147483648)

Note that the unsigned range for the sext does contain zero in this case
despite the fact that it can never actually be zero.

Instead, we can push the query down one level - relying on the fact that
the sext is an invertible operation and that the result can only be zero
if the input is. We could likely generalize this reasoning for other
invertible operations, but special casing sext seems worthwhile.

show more ...

[SCEV] Ensure shift amount is in range before calling getZExtValue()

Fixes #76234

[SCEV] Use loop guards when checking that RHS >= Start (#75039)

Loop guards tend to provide better results when it comes to reasoning
about ranges than isLoopEntryGuardedByCond(). See the test chan

[SCEV] Use loop guards when checking that RHS >= Start (#75039)

Loop guards tend to provide better results when it comes to reasoning
about ranges than isLoopEntryGuardedByCond(). See the test change for
the motivating case.

I have retained both the loop guard check and the implied cond based
check for now, though the latter only seems to impact a single test and
only via side effects (nowrap flag calculation) at that.

show more ...

[SCEV] Use or disjoint flag (#74467)

Use the disjoint flag to convert or to add instead of calling the
haveNoCommonBitsSet() ValueTracking query. This ensures that we can
reliably undo add -> or c

[SCEV] Use or disjoint flag (#74467)

Use the disjoint flag to convert or to add instead of calling the
haveNoCommonBitsSet() ValueTracking query. This ensures that we can
reliably undo add -> or canonicalization, even in cases where the
necessary information has been lost or is too complex to reinfer in
SCEV.

I have updated the bulk of the test coverage to add the necessary
disjoint flags in advance.

show more ...

[SCEV][LV] Invalidate LCSSA exit phis more thoroughly (#69909)

This an alternative to #69886. The basic problem is that SCEV can look
through trivial LCSSA phis. When the phi node later becomes non

[SCEV][LV] Invalidate LCSSA exit phis more thoroughly (#69909)

This an alternative to #69886. The basic problem is that SCEV can look
through trivial LCSSA phis. When the phi node later becomes non-trivial,
we do invalidate it, but this doesn't catch uses that are not covered by
the IR use-def walk, such as those in BECounts.

Fix this by adding a special invalidation method for LCSSA phis, which
will also invalidate all the SCEVUnknowns/SCEVAddRecExprs used by the
LCSSA phi node and defined in the loop.

We should probably also use this invalidation method in other places
that add predecessors to exit blocks, such as loop unrolling and loop
peeling.

Fixes #69097.
Fixes #66616.
Fixes #63970.

show more ...

[SCEV] Support larger than 64-bit types in ashr(add(shl(x, n), c), m) (#71600)

In commit 5a9a02f67b771fb2edcf06 scalar evolution got support for
computing SCEV:s for (ashr(add(shl(x, n), c), m)) co

[SCEV] Support larger than 64-bit types in ashr(add(shl(x, n), c), m) (#71600)

In commit 5a9a02f67b771fb2edcf06 scalar evolution got support for
computing SCEV:s for (ashr(add(shl(x, n), c), m)) constructs. The code
however used APInt::getZExtValue without first checking that the APInt
would fit inside an uint64_t. When for example using 128-bit types we
ended up in assertion failures (or maybe miscompiles in non-assert
builds).
This patch simply avoid converting from APInt to uint64_t when creating
the truncated constant. We can just truncate the APInt instead.

show more ...

[SCEV] Extend isImpliedCondOperandsViaRanges to independent predicates (#71110)

As far as I can tell, there's nothing in this code which actually
assumes the two predicates in (FoundLHS FoundPred F

[SCEV] Extend isImpliedCondOperandsViaRanges to independent predicates (#71110)

As far as I can tell, there's nothing in this code which actually
assumes the two predicates in (FoundLHS FoundPred FoundRHS) => (LHS Pred
RHS) are the same.

Noticed while investigating something else, this is purely an
oppurtunistic optimization while I'm looking at the code. Unfortunately,
this doesn't solve my original problem. :)

show more ...

[SCEV] Remove mul handling from BuildConstantFromSCEV()

We can't support this once mul constant expressions are removed,
and this is not useful in any practical sense (as this code is
primarily inte

[SCEV] Remove mul handling from BuildConstantFromSCEV()

We can't support this once mul constant expressions are removed,
and this is not useful in any practical sense (as this code is
primarily intended for GEP expressions).

show more ...

[SCEV] Remove newline after predicates in dump

update_analyze_test_checks.py will now insert check lines for
empty lines, which means that all the existing test coverage will
have a spurious change

[SCEV] Remove newline after predicates in dump

update_analyze_test_checks.py will now insert check lines for
empty lines, which means that all the existing test coverage will
have a spurious change to check for the newline after "Predicates:".

I don't think we actually want to have that newline, so drop it
before it gets into more test coverage.

show more ...

[SCEV] Fix incorrect NUW inference (#70521)

This patch fixes a miscompile in LSR caused by incorrect inference of
NUW flag for AddRec: we shouldn't infer no-wrap flags based on a
comparison which

[SCEV] Fix incorrect NUW inference (#70521)

This patch fixes a miscompile in LSR caused by incorrect inference of
NUW flag for AddRec: we shouldn't infer no-wrap flags based on a
comparison which doesn't fully control the loop exit.

show more ...

[SCEV] Fix "quick and dirty" difference that could lead to assert (#70688)

The old algorithm would remove all operands matching %step SCEV when it
intended to only remove a single one. This lead to

[SCEV] Fix "quick and dirty" difference that could lead to assert (#70688)

The old algorithm would remove all operands matching %step SCEV when it
intended to only remove a single one. This lead to assert when
SCEVAddExpr was of the form %step + %step and potential miscompiles in
similar cases. Such SCEVs could be created when construction reached
depth thresholds.

Fixes #70348

show more ...

[ADT] Rename llvm::erase_value to llvm::erase (NFC) (#70156)

C++20 comes with std::erase to erase a value from std::vector. This
patch renames llvm::erase_value to llvm::erase for consistency with

[ADT] Rename llvm::erase_value to llvm::erase (NFC) (#70156)

C++20 comes with std::erase to erase a value from std::vector. This
patch renames llvm::erase_value to llvm::erase for consistency with
C++20.

We could make llvm::erase more similar to std::erase by having it
return the number of elements removed, but I'm not doing that for now
because nobody seems to care about that in our code base.

Since there are only 50 occurrences of erase_value in our code base,
this patch replaces all of them with llvm::erase and deprecates
llvm::erase_value.

show more ...