[IR] Fix accumulateConstantOffset() on zero-index GEP

These are degenerate but not malformed, so make sure we don't
crash.

[LLVM][IR] When evaluating GEP offsets don't assume ConstantInt is a scalar. (#117162)

[Operator] Truncate large type sizes in GEP calculations

If the size is larger than the index width, truncate it instead
of asserting.

Longer-term we should consider rejecting types larger than the

[Operator] Truncate large type sizes in GEP calculations

If the size is larger than the index width, truncate it instead
of asserting.

Longer-term we should consider rejecting types larger than the
index size in the verifier, though this is probably tricky in
practice (it's address space dependent, and types are owned by
the context, not the module).

Fixes https://github.com/llvm/llvm-project/issues/116960.

show more ...

[IR] Add support for `samesign` in `Operator::hasPoisonGeneratingFlags` (#112358)

Fix https://github.com/llvm/llvm-project/issues/112356.

[NFC] Use initial-stack-allocations for more data structures (#110544)

This replaces some of the most frequent offenders of using a DenseMap that
cause a malloc, where the typical element-count is

[NFC] Use initial-stack-allocations for more data structures (#110544)

This replaces some of the most frequent offenders of using a DenseMap that
cause a malloc, where the typical element-count is small enough to fit in
an initial stack allocation.

Most of these are fairly obvious, one to highlight is the collectOffset
method of GEP instructions: if there's a GEP, of course it's going to have
at least one offset, but every time we've called collectOffset we end up
calling malloc as well for the DenseMap in the MapVector.

show more ...

[IR] Add getelementptr nusw and nuw flags (#90824)

This implements the `nusw` and `nuw` flags for `getelementptr` as
proposed at
https://discourse.llvm.org/t/rfc-add-nusw-and-nuw-flags-for-getelem

[IR] Add getelementptr nusw and nuw flags (#90824)

This implements the `nusw` and `nuw` flags for `getelementptr` as
proposed at
https://discourse.llvm.org/t/rfc-add-nusw-and-nuw-flags-for-getelementptr/78672.

The three possible flags are encapsulated in the new `GEPNoWrapFlags`
class. Currently this class has a ctor from bool, interpreted as the
InBounds flag. This ctor should be removed in the future, as code gets
migrated to handle all flags.

There are a few places annotated with `TODO(gep_nowrap)`, where I've had
to touch code but opted to not infer or precisely preserve the new
flags, so as to keep this as NFC as possible and make sure any changes
of that kind get test coverage when they are made.

show more ...

[IR] Drop poison-generating return attributes when necessary (#89138)

Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align a

[IR] Drop poison-generating return attributes when necessary (#89138)

Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.

show more ...

Add support for `nneg` flag with `uitofp`

As noted when #82404 was pushed (canonicalizing `sitofp` -> `uitofp`),
different signedness on fp casts can have dramatic performance
implications on differ

Add support for `nneg` flag with `uitofp`

As noted when #82404 was pushed (canonicalizing `sitofp` -> `uitofp`),
different signedness on fp casts can have dramatic performance
implications on different backends.

So, it makes to create a reliable means for the backend to pick its
cast signedness if either are correct.

Further, this allows us to start canonicalizing `sitofp`- > `uitofp`
which may easy middle end analysis.

Closes #86141

show more ...

[IR] Fix crashes caused by #85592 (#87169)

This patch fixes the crash caused by the pull request:
https://github.com/llvm/llvm-project/pull/85592

[IR] Add nowrap flags for trunc instruction (#85592)

This patch adds the nuw (no unsigned wrap) and nsw (no signed wrap)
poison-generating flags to the trunc instruction.

Discourse thread:
htt

[IR] Add nowrap flags for trunc instruction (#85592)

This patch adds the nuw (no unsigned wrap) and nsw (no signed wrap)
poison-generating flags to the trunc instruction.

Discourse thread:
https://discourse.llvm.org/t/rfc-add-nowrap-flags-to-trunc/77453

show more ...

[IR] Change representation of getelementptr inrange (#84341)

As part of the migration to ptradd
(https://discourse.llvm.org/t/rfc-replacing-getelementptr-with-ptradd/68699),
we need to change the

[IR] Change representation of getelementptr inrange (#84341)

As part of the migration to ptradd
(https://discourse.llvm.org/t/rfc-replacing-getelementptr-with-ptradd/68699),
we need to change the representation of the `inrange` attribute, which
is used for vtable splitting.

Currently, inrange is specified as follows:

```
getelementptr inbounds ({ [4 x ptr], [4 x ptr] }, ptr @vt, i64 0, inrange i32 1, i64 2)
```

The `inrange` is placed on a GEP index, and all accesses must be "in
range" of that index. The new representation is as follows:

```
getelementptr inbounds inrange(-16, 16) ({ [4 x ptr], [4 x ptr] }, ptr @vt, i64 0, i32 1, i64 2)
```

This specifies which offsets are "in range" of the GEP result. The new
representation will continue working when canonicalizing to ptradd
representation:

```
getelementptr inbounds inrange(-16, 16) (i8, ptr @vt, i64 48)
```

The inrange offsets are relative to the return value of the GEP. An
alternative design could make them relative to the source pointer
instead. The result-relative format was chosen on the off-chance that we
want to extend support to non-constant GEPs in the future, in which case
this variant is more expressive.

This implementation "upgrades" the old inrange representation in bitcode
by simply dropping it. This is a very niche feature, and I don't think
trying to upgrade it is worthwhile. Let me know if you disagree.

show more ...

[IR] Add ptradd fast path in accumulateConstantOffset() (NFC)

For getelementptr i8 (aka ptradd) we can skip the whole logic and
directly use the offset. As we're now canonicalizing to this form,
it'

[IR] Add ptradd fast path in accumulateConstantOffset() (NFC)

For getelementptr i8 (aka ptradd) we can skip the whole logic and
directly use the offset. As we're now canonicalizing to this form,
it's pretty common and worth having a fast-path for.

show more ...

[IR] Fix GEP offset computations for vector GEPs (#75448)

Vectors are always bit-packed and don't respect the elements' alignment
requirements. This is different from arrays. This means offsets of

[IR] Fix GEP offset computations for vector GEPs (#75448)

Vectors are always bit-packed and don't respect the elements' alignment
requirements. This is different from arrays. This means offsets of
vector GEPs need to be computed differently than offsets of array GEPs.

This PR fixes many places that rely on an incorrect pattern
that always relies on `DL.getTypeAllocSize(GTI.getIndexedType())`.
We replace these by usages of `GTI.getSequentialElementStride(DL)`,
which is a new helper function added in this PR.

This changes behavior for GEPs into vectors with element types for which
the (bit) size and alloc size is different. This includes two cases:

* Types with a bit size that is not a multiple of a byte, e.g. i1.
GEPs into such vectors are questionable to begin with, as some elements
are not even addressable.
* Overaligned types, e.g. i16 with 32-bit alignment.

Existing tests are unaffected, but a miscompilation of a new test is fixed.

---------

Co-authored-by: Nikita Popov <github@npopov.com>

show more ...

[IR] Avoid redundant map lookup [NFC]

Use the iterator returned by MapVector::insert to update
the value in the map, instead of a second redundant map lookup.

[IR] Support or disjoint in hasPoisonGeneratingFlags()

This fixed incorrect removal of freeze instructions.

[IR] Add zext nneg flag (#67982)

Add an nneg flag to the zext instruction, which specifies that the
argument is non-negative. Otherwise, the result is a poison value.

The primary use-case for th

[IR] Add zext nneg flag (#67982)

Add an nneg flag to the zext instruction, which specifies that the
argument is non-negative. Otherwise, the result is a poison value.

The primary use-case for the flag is to preserve information when sext
gets replaced with zext due to range-based canonicalization. The nneg
flag allows us to convert the zext back into an sext later. This is
useful for some optimizations (e.g. a signed icmp can fold with sext but
not zext), as well as some targets (e.g. RISCV prefers sext over zext).

Discourse thread: https://discourse.llvm.org/t/rfc-add-zext-nneg-flag/73914

This patch is based on https://reviews.llvm.org/D156444 by
@Panagiotis156, with some implementation simplifications and additional
tests.

---------

Co-authored-by: Panagiotis K <karouzakispan@gmail.com>

show more ...

[IR] Enable load/store/alloca for arrays of scalable vectors.

Differential Revision: https://reviews.llvm.org/D158517

[ValueTracking] Take poison-generating metadata into account (PR59888)

In canCreateUndefOrPoison(), take not only poison-generating flags,
but also poison-generating metadata into account. The helpe

[ValueTracking] Take poison-generating metadata into account (PR59888)

In canCreateUndefOrPoison(), take not only poison-generating flags,
but also poison-generating metadata into account. The helpers are
written generically, but I believe the only case that can actually
matter is !range on calls -- !nonnull and !align are only valid on
loads, and those can create undef/poison anyway.

Unfortunately, this negatively impacts logical to bitwise and/or
conversion: For ctpop/ctlz/cttz we always attach !range metadata,
which will now block the transform, because it might introduce
poison. It would be possible to recover this regression by supporting
a ConsiderFlagsAndMetadata=false mode in impliesPoison() and clearing
flags/metadata on visited instructions.

Fixes https://github.com/llvm/llvm-project/issues/59888.

Differential Revision: https://reviews.llvm.org/D142115

show more ...

[NFC][IR] Remove unused assignment to Offset

This value is overwritten anyway, so let's remove it

[IR] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manu

[IR] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avo

Cleanup header dependencies in LLVMCore

Based on the output of include-what-you-use.

This is a big chunk of changes. It is very likely to break downstream code
unless they took a lot of care in avoiding hidden ehader dependencies, something
the LLVM codebase doesn't do that well :-/

I've tried to summarize the biggest change below:

- llvm/include/llvm-c/Core.h: no longer includes llvm-c/ErrorHandling.h
- llvm/IR/DIBuilder.h no longer includes llvm/IR/DebugInfo.h
- llvm/IR/IRBuilder.h no longer includes llvm/IR/IntrinsicInst.h
- llvm/IR/LLVMRemarkStreamer.h no longer includes llvm/Support/ToolOutputFile.h
- llvm/IR/LegacyPassManager.h no longer include llvm/Pass.h
- llvm/IR/Type.h no longer includes llvm/ADT/SmallPtrSet.h
- llvm/IR/PassManager.h no longer includes llvm/Pass.h nor llvm/Support/Debug.h

And the usual count of preprocessed lines:
$ clang++ -E -Iinclude -I../llvm/include ../llvm/lib/IR/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 6400831
after: 6189948

200k lines less to process is no that bad ;-)

Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup

Differential Revision: https://reviews.llvm.org/D118652

show more ...

[llvm] Construct SmallVector with iterator ranges (NFC)

Add FMF to hasPoisonGeneratingFlags/dropPoisonGeneratingFlags

These flags are documented as generating poison values for particular input values. As such, we should really be consistent about their

Add FMF to hasPoisonGeneratingFlags/dropPoisonGeneratingFlags

These flags are documented as generating poison values for particular input values. As such, we should really be consistent about their handling with how we handle nsw/nuw/exact/inbounds.

Differential Revision: https://reviews.llvm.org/D115460

show more ...

[LoopVectorize] Propagate fast-math flags for VPInstruction

In-loop vector reductions which use the llvm.fmuladd intrinsic involve
the creation of two recipes; a VPReductionRecipe for the fadd and a

[LoopVectorize] Propagate fast-math flags for VPInstruction

In-loop vector reductions which use the llvm.fmuladd intrinsic involve
the creation of two recipes; a VPReductionRecipe for the fadd and a
VPInstruction for the fmul. If the call to llvm.fmuladd has fast-math flags
these should be propagated through to the fmul instruction, so an
interface setFastMathFlags has been added to the VPInstruction class to
enable this.

Differential Revision: https://reviews.llvm.org/D113125

show more ...

[Operator] Add hasPoisonGeneratingFlags [mostly NFC]

This method parallels the dropPoisonGeneratingFlags on Instruction, but is hoisted to operator to handle constant expressions as well.

This is m

[Operator] Add hasPoisonGeneratingFlags [mostly NFC]

This method parallels the dropPoisonGeneratingFlags on Instruction, but is hoisted to operator to handle constant expressions as well.

This is mostly code movement, but I did go ahead and add the inrange constexpr gep case. This had been discussed previously, but apparently never followed up o.

show more ...