[RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

If --write-experimental-debuginfo-iterators-to-bitcode is true (default false)
and --expermental-debuginfo-iterators is also true

[RemoveDIs] Read/write DbgRecords directly from/to bitcode (#83251)

If --write-experimental-debuginfo-iterators-to-bitcode is true (default false)
and --expermental-debuginfo-iterators is also true then the new debug info
format (non-instruction records) is written to bitcode directly.

Added the following records:

FUNC_CODE_DEBUG_RECORD_LABEL
FUNC_CODE_DEBUG_RECORD_VALUE
FUNC_CODE_DEBUG_RECORD_DECLARE
FUNC_CODE_DEBUG_RECORD_ASSIGN
FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE

The last one has an abbrev in FUNCTION_BLOCK BLOCK_INFO. Incidentally, this uses
the last value available without widening the code-length for FUNCTION_BLOCK
from 4 to 5 bits.

Records are formatted as follows:

All DbgRecord start with:
1. DILocation

FUNC_CODE_DEBUG_RECORD_LABEL
2. DILabel

DPValues then share common fields:
2. DILocalVariable
3. DIExpression

FUNC_CODE_DEBUG_RECORD_VALUE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_DECLARE
4. Location Metadata

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE
4. Location Value (single)

FUNC_CODE_DEBUG_RECORD_ASSIGN
4. Location Metadata
5. DIAssignID
6. DIExpression (address)
7. Location Metadata (address)

Encoding the DILocation metadata reference directly appeared to yield smaller
bitcode files than encoding the operands seperately (as is done with instruction
DILocations).

FUNC_CODE_DEBUG_RECORD_VALUE_SIMPLE is by far the most common DbgRecord record
in optimized code (order of 5x-10x over other kinds). Unoptimized code should
only contain FUNC_CODE_DEBUG_RECORD_DECLARE.

show more ...

Reapply [IR] Add new Range attribute using new ConstantRange Attribute type (#84617)

The only change from https://github.com/llvm/llvm-project/pull/83171 is the
change of the allocator so the destr

Reapply [IR] Add new Range attribute using new ConstantRange Attribute type (#84617)

The only change from https://github.com/llvm/llvm-project/pull/83171 is the
change of the allocator so the destructor is called for
ConstantRangeAttributeImpl.

reverts https://github.com/llvm/llvm-project/pull/84549

show more ...

Revert "[IR] Add new Range attribute using new ConstantRange Attribute type" (#84549)

Reverts llvm/llvm-project#83171

broke sanitizer buildbot
https://lab.llvm.org/buildbot/#/builders/168/builds

Revert "[IR] Add new Range attribute using new ConstantRange Attribute type" (#84549)

Reverts llvm/llvm-project#83171

broke sanitizer buildbot
https://lab.llvm.org/buildbot/#/builders/168/builds/19110/steps/10/logs/stdio

show more ...

[IR] Add new Range attribute using new ConstantRange Attribute type (#83171)

implementation as discussed in
https://discourse.llvm.org/t/rfc-metadata-attachments-for-function-arguments/76420

Revert "[llvm][AArch64] Autoupgrade function attributes from Module attributes. (#80640)"

This reverts commit 531e8c26b3f2626e7f1a997e0e8b61d67d10aded.

[llvm][AArch64] Autoupgrade function attributes from Module attributes. (#80640)

`sign-return-address` and similar module attributes should be propagated
to the function level before modules got me

[llvm][AArch64] Autoupgrade function attributes from Module attributes. (#80640)

`sign-return-address` and similar module attributes should be propagated
to the function level before modules got merged because module flags may
contradict and this information is not recoverable.
Generated code will match with the normal linking flow.

show more ...

[LLVM][IR] Add native vector support to ConstantInt & ConstantFP. (#74502)

NOTE: For brevity the following talks about ConstantInt but
everything extends to cover ConstantFP as well.

Whilst Cons

[LLVM][IR] Add native vector support to ConstantInt & ConstantFP. (#74502)

NOTE: For brevity the following talks about ConstantInt but
everything extends to cover ConstantFP as well.

Whilst ConstantInt::get() supports the creation of vectors whereby
each lane has the same value, it achieves this via other constants:

* ConstantVector for fixed-length vectors
* ConstantExprs for scalable vectors

However, ConstantExprs are being deprecated and ConstantVector is
not space efficient for larger vector types. By extending ConstantInt
we can represent vector splats by only storing the underlying scalar
value.

More specifically:

* ConstantInt gains an ElementCount variant of get().
* LLVMContext is extended to map <EC,APInt>->ConstantInt.
* BitcodeReader/Writer support is extended to allow vector types.

Whilst this patch adds the base support, more work is required
before it's production ready. For example, there's likely to be
many places where isa<ConstantInt> assumes a scalar type. Accordingly
the default behaviour of ConstantInt::get() remains unchanged but a
set of flags are added to allow wider testing and thus help with the
migration:

--use-constant-int-for-fixed-length-splat
--use-constant-fp-for-fixed-length-splat
--use-constant-int-for-scalable-splat
--use-constant-fp-for-scalable-splat

NOTE: No change is required to the bitcode format because types and
values are handled separately.

NOTE: For similar reasons as above, code generation doesn't work
out-the-box.

show more ...

[RemoveDIs] Don't convert debug-info in bitcode-loading just now (#80865)

We've been building and testing this no-debug-intrinsic work inside of
the pass manager for a while, so that optimisation p

[RemoveDIs] Don't convert debug-info in bitcode-loading just now (#80865)

We've been building and testing this no-debug-intrinsic work inside of
the pass manager for a while, so that optimisation passes get exercised
and tested when we turn it on. However, by converting to the
non-intrinsic form in the bitcode loader, we accidentally caused all
parts of LLVM to potentially see non-intrinsic debug-info.

Seeing how we're trying to turn things on incrementally, it was a
mistake to go this far this fast: we can instead just focus on enabling
during optimisations for the moment, then all the other parts of LLVM
later.

show more ...

Reapply 215b8f1e252, reverted in c3f7fb1421e

Turns out I was using DbgMarker::getDbgValueRange rather than the helper
utility in Instruction::getDbgValueRange, which checks for null-ness.
Original c

Reapply 215b8f1e252, reverted in c3f7fb1421e

Turns out I was using DbgMarker::getDbgValueRange rather than the helper
utility in Instruction::getDbgValueRange, which checks for null-ness.
Original commit message follows.

[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)

As part of eliminating debug-intrinsics in LLVM, we'll shortly be
pushing the conversion from "old" dbg.value mode to "new" DPValue mode
out from when the pass manager runs, to when modules are loaded. This
patch adds that conversion process and some (temporary) options to
llvm-lto{,2} to help test it.

Specifically: now whenever we load a bitcode module, consider a flag of
whether to "upgrade" it into the new debug-info mode, and if we're
lazily materializing functions then do that lazily too. Doing this
exposes an error in the IRLinker/materializer handling of DPValues,
where we need to transfer the debug-info format flag correctly, and in
ValueMapper we need to remap the Values that DPValues point at.

I've added some test coverage in the modified tests; these will be
exercised by our llvm-new-debug-iterators buildbot.

This upgrading of debug-info won't be happening for the llvm18 release,
instead we'll turn it on after the branch date, thenbe push the boundary
of where "new" debug-info starts and ends down into the existing
debug-info upgrade path over the course of the next release.

show more ...

Revert "[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)"

This reverts commit 215b8f1e252b4f30cf1b734faa370c0ac4b88659.

Numerous builders exploded from this X_X, for ex

Revert "[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)"

This reverts commit 215b8f1e252b4f30cf1b734faa370c0ac4b88659.

Numerous builders exploded from this X_X, for example

https://lab.llvm.org/buildbot/#/builders/46/builds/62657

show more ...

[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)

As part of eliminating debug-intrinsics in LLVM, we'll shortly be
pushing the conversion from "old" dbg.value mode to "

[DebugInfo][RemoveDIs] Convert debug-info modes when loading bitcode (#78967)

As part of eliminating debug-intrinsics in LLVM, we'll shortly be
pushing the conversion from "old" dbg.value mode to "new" DPValue mode
out from when the pass manager runs, to when modules are loaded. This
patch adds that conversion process and some (temporary) options to
llvm-lto{,2} to help test it.

Specifically: now whenever we load a bitcode module, consider a flag of
whether to "upgrade" it into the new debug-info mode, and if we're
lazily materializing functions then do that lazily too. Doing this
exposes an error in the IRLinker/materializer handling of DPValues,
where we need to transfer the debug-info format flag correctly, and in
ValueMapper we need to remap the Values that DPValues point at.

I've added some test coverage in the modified tests; these will be
exercised by our llvm-new-debug-iterators buildbot.

This upgrading of debug-info won't be happening for the llvm18 release,
instead we'll turn it on after the branch date, thenbe push the boundary
of where "new" debug-info starts and ends down into the existing
debug-info upgrade path over the course of the next release.

show more ...

[BitcodeReader] Add bounds checking on Strtab (#76403)

This is needed to protect against global overflows, which was found by a
fuzzer recently.

Fixes: https://bugs.chromium.org/p/oss-fuzz/issue

[BitcodeReader] Add bounds checking on Strtab (#76403)

This is needed to protect against global overflows, which was found by a
fuzzer recently.

Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65283

---------

Signed-off-by: David Korczynski <david@adalogics.com>

show more ...

[IR] Add dead_on_unwind attribute (#74289)

Add the `dead_on_unwind` attribute, which states that the caller will
not read from this argument if the call unwinds. This allows eliding
stores that co

[IR] Add dead_on_unwind attribute (#74289)

Add the `dead_on_unwind` attribute, which states that the caller will
not read from this argument if the call unwinds. This allows eliding
stores that could otherwise be visible on the unwind path, for example:

```
declare void @may_unwind()

define void @src(ptr noalias dead_on_unwind %out) {
store i32 0, ptr %out
call void @may_unwind()
store i32 1, ptr %out
ret void
}

define void @tgt(ptr noalias dead_on_unwind %out) {
call void @may_unwind()
store i32 1, ptr %out
ret void
}
```

The optimization is not valid without `dead_on_unwind`, because the `i32
0` value might be read if `@may_unwind` unwinds.

This attribute is primarily intended to be used on sret arguments. In
fact, I previously wanted to change the semantics of sret to include
this "no read after unwind" property (see D116998), but based on the
feedback there it is better to keep these attributes orthogonal (sret is
an ABI attribute, dead_on_unwind is an optimization attribute). This is
a reboot of that change with a separate attribute.

show more ...

[Bitcode] Check validity of fcmp/icmp predicate

The predicate should match the operand types.

[Bitcode] Check type of alloca size argument

This must be an integer. This check is present in LLParser but not
the BitcodeReader.

[Bitcode] Add missing getValue() return value checks

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of missing
frames from tail calls in profiled call stacks for MemProf of profiled
binaries that did not disable tail call elimination. A follow on change
will add the use of this new flag during MemProf context disambiguation.

The new flag is encoded in the bitcode along with either the hotness
flag from the profile, or the relative block frequency under the
-write-relbf-to-summary flag when there is no profile data.
Because we now will always have some additional call edge information, I
have removed the non-profile function summary record format, and we
simply encode the tail call flag along with a hotness type of none when
there is no profile information or relative block frequency. The change
of record format and name caused most of the test case changes.

I have added explicit testing of generation of the new tail call flag
into the bitcode and IR assembly format as part of the changes to
llvm/test/Bitcode/thinlto-function-summary-refgraph.ll. I have also
added round trip testing through assembly and bitcode to
llvm/test/Assembler/thinlto-summary.ll.

show more ...

[Bitcode] Check for missing load type

We perform do this check independently of whether this is an old
or new style load.

Fixes https://github.com/llvm/llvm-project/issues/74556.

[llvm][IR] Add per-global code model attribute (#72077)

This adds a per-global code model attribute, which can override the
target's code model to access global variables.

Suggested-by: Arthur E

[llvm][IR] Add per-global code model attribute (#72077)

This adds a per-global code model attribute, which can override the
target's code model to access global variables.

Suggested-by: Arthur Eubanks <aeubanks@google.com>
Link: https://discourse.llvm.org/t/how-to-best-implement-code-model-overriding-for-certain-values/71816
Link: https://discourse.llvm.org/t/rfc-add-per-global-code-model-attribute/74944

show more ...

[IR] Add disjoint flag for Or instructions. (#72583)

This flag indicates that every bit is known to be zero in at least one
of the inputs. This allows the Or to be treated as an Add since there is

[IR] Add disjoint flag for Or instructions. (#72583)

This flag indicates that every bit is known to be zero in at least one
of the inputs. This allows the Or to be treated as an Add since there is
no possibility of a carry from any bit.

If the flag is present and this property does not hold, the result is
poison.

This makes it easier to reverse the InstCombine transform that turns Add
into Or.

This is inspired by a comment here
https://github.com/llvm/llvm-project/pull/71955#discussion_r1391614578

Discourse thread
https://discourse.llvm.org/t/rfc-add-or-disjoint-flag/75036

show more ...

[llvm] Remove no-op ptr-to-ptr bitcasts (NFC)

Opaque ptr cleanup effort (NFC).

[Coroutines] Introduce [[clang::coro_only_destroy_when_complete]] (#71014)

Close https://github.com/llvm/llvm-project/issues/56980.

This patch tries to introduce a light-weight optimization attri

[Coroutines] Introduce [[clang::coro_only_destroy_when_complete]] (#71014)

Close https://github.com/llvm/llvm-project/issues/56980.

This patch tries to introduce a light-weight optimization attribute for
coroutines which are guaranteed to only be destroyed after it reached
the final suspend.

The rationale behind the patch is simple. See the example:

```C++
A foo() {
dtor d;
co_await something();
dtor d1;
co_await something();
dtor d2;
co_return 43;
}
```

Generally the generated .destroy function may be:

```C++
void foo.destroy(foo.Frame *frame) {
switch(frame->suspend_index()) {
case 1:
frame->d.~dtor();
break;
case 2:
frame->d.~dtor();
frame->d1.~dtor();
break;
case 3:
frame->d.~dtor();
frame->d1.~dtor();
frame->d2.~dtor();
break;
default: // coroutine completed or haven't started
break;
}

frame->promise.~promise_type();
delete frame;
}
```

Since the compiler need to be ready for all the cases that the coroutine
may be destroyed in a valid state.

However, from the user's perspective, we can understand that certain
coroutine types may only be destroyed after it reached to the final
suspend point. And we need a method to teach the compiler about this.
Then this is the patch. After the compiler recognized that the
coroutines can only be destroyed after complete, it can optimize the
above example to:

```C++
void foo.destroy(foo.Frame *frame) {
frame->promise.~promise_type();
delete frame;
}
```

I spent a lot of time experimenting and experiencing this in the
downstream. The numbers are really good. In a real-world coroutine-heavy
workload, the size of the build dir (including .o files) reduces 14%.
And the size of final libraries (excluding the .o files) reduces 8% in
Debug mode and 1% in Release mode.

show more ...

[IR] Remove zext and sext constant expressions (#71040)

Remove support for zext and sext constant expressions. All places
creating them have been removed beforehand, so this just removes the
APIs

[IR] Remove zext and sext constant expressions (#71040)

Remove support for zext and sext constant expressions. All places
creating them have been removed beforehand, so this just removes the
APIs and uses of these constant expressions in tests.

There is some additional cleanup that can be done on top of this, e.g.
we can remove the ZExtInst vs ZExtOperator footgun.

This is part of
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.

show more ...

[IR] Add writable attribute

This adds a writable attribute, which in conjunction with
dereferenceable(N) states that a spurious store of N bytes is
introduced on function entry. This implies that th

[IR] Add writable attribute

This adds a writable attribute, which in conjunction with
dereferenceable(N) states that a spurious store of N bytes is
introduced on function entry. This implies that this many bytes
are writable without trapping or introducing data races. See
https://llvm.org/docs/Atomics.html#optimization-outside-atomic for
why the second point is important.

This attribute can be added to sret arguments. I believe Rust will
also be able to use it for by-value (moved) arguments. Rust likely
won't be able to use it for &mut arguments (tree borrows does not
appear to allow spurious stores).

In this patch the new attribute is only used by LICM scalar promotion.
However, the actual motivation for this is to fix a correctness issue
in call slot optimization, which needs this attribute to avoid
optimization regressions.

Followup to the discussion on D157499.

Differential Revision: https://reviews.llvm.org/D158081

show more ...

[IR] Add zext nneg flag (#67982)

Add an nneg flag to the zext instruction, which specifies that the
argument is non-negative. Otherwise, the result is a poison value.

The primary use-case for th

[IR] Add zext nneg flag (#67982)

Add an nneg flag to the zext instruction, which specifies that the
argument is non-negative. Otherwise, the result is a poison value.

The primary use-case for the flag is to preserve information when sext
gets replaced with zext due to range-based canonicalization. The nneg
flag allows us to convert the zext back into an sext later. This is
useful for some optimizations (e.g. a signed icmp can fold with sext but
not zext), as well as some targets (e.g. RISCV prefers sext over zext).

Discourse thread: https://discourse.llvm.org/t/rfc-add-zext-nneg-flag/73914

This patch is based on https://reviews.llvm.org/D156444 by
@Panagiotis156, with some implementation simplifications and additional
tests.

---------

Co-authored-by: Panagiotis K <karouzakispan@gmail.com>

show more ...