Don't rely on undefined behavior to store how a `User` object's allocation is laid out (#105714)

In `User::operator new` a single allocation is created to store the
`User` object itself, "intrusive

Don't rely on undefined behavior to store how a `User` object's allocation is laid out (#105714)

In `User::operator new` a single allocation is created to store the
`User` object itself, "intrusive" operands or a pointer for "hung off"
operands, and the descriptor. After allocation, details about the layout
(number of operands, how the operands are stored, if there is a
descriptor) are stored in the `User` object by settings its fields. The
`Value` and `User` constructors are then very careful not to initialize
these fields so that the values set during allocation can be
subsequently read. However, when the `User` object is returned from
`operator new` [its value is technically "indeterminate" and so reading
a field without first initializing it is undefined behavior (and will be
erroneous in
C++26)](https://en.cppreference.com/w/cpp/language/default_initialization#Indeterminate_and_erroneous_values).

We discovered this issue when trying to build LLVM using MSVC's [`/sdl`
flag](https://learn.microsoft.com/en-us/cpp/build/reference/sdl-enable-additional-security-checks?view=msvc-170)
which clears class fields after allocation (the docs say that this
feature shouldn't be turned on for custom allocators and should only
clear pointers, but that doesn't seem to match the implementation).
MSVC's behavior both with and without the `/sdl` flag is standards
conforming since a program is supposed to initialize storage before
reading from it, thus the compiler implementation changing any values
will never be observed in a well-formed program. The standard also
provides no provisions for making storage bytes not indeterminate by
setting them during allocation or `operator new`.

The fix for this is to create a set of types that encode the layout and
provide these to both `operator new` and the constructor:
* The `AllocMarker` types are used to select which `operator new` to
use.
* `AllocMarker` can then be implicitly converted to a `AllocInfo` which
tells the constructor how the type was laid out.

show more ...

[LoopDeletion] Unblock loop deletion with `llvm.experimental.noalias.scope.decl` (#108144)

Since `llvm.experimental.noalias.scope.decl` is marked as
`memory(inaccessiblemem: readwrite)`, we cannot

[LoopDeletion] Unblock loop deletion with `llvm.experimental.noalias.scope.decl` (#108144)

Since `llvm.experimental.noalias.scope.decl` is marked as
`memory(inaccessiblemem: readwrite)`, we cannot treat this annotation
intrinsic as having no side effects. It will block loop deletion when
this intrinsic exists inside a dead loop:

https://github.com/llvm/llvm-project/blob/3dad29b677e427bf69c035605a16efd065576829/llvm/lib/Transforms/Scalar/LoopDeletion.cpp#L103-L110

This patch marks `llvm.experimental.noalias.scope.decl` as droppable to
address the issue.

Fixes https://github.com/llvm/llvm-project/issues/108052.

show more ...

[IR] Report whether replaceUsesOfWith() changed something (NFC)

With change reporting in transformation passes in mind.

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 3.

This patch continues unblocking optimizations that are blocked by pseudo probe instrumentation.

Not exactly like DbgIntrinsi

[CSSPGO] Unblock optimizations with pseudo probe instrumentation part 3.

This patch continues unblocking optimizations that are blocked by pseudo probe instrumentation.

Not exactly like DbgIntrinsics, PseudoProbe intrinsic has other attributes (such as mayread, maywrite, mayhaveSideEffect) that can block optimizations. The issues fixed are:
- Flipped default param of getFirstNonPHIOrDbg API to skip pseudo probes
- Unblocked CSE by avoiding pseudo probe from clobbering memory SSA
- Unblocked induction variable simpliciation
- Allow empty loop deletion by treating probe intrinsic isDroppable
- Some refactoring.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D110847

show more ...

Use AssumeInst in a few more places [nfc]

Follow up to a6d2a8d6f5. These were found by simply grepping for "::assume", and are the subset of that result which looked cleaner to me using the isa/dyn

Use AssumeInst in a few more places [nfc]

Follow up to a6d2a8d6f5. These were found by simply grepping for "::assume", and are the subset of that result which looked cleaner to me using the isa/dyn_cast patterns.

show more ...

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic

Reapply "[DebugInfo] Handle multiple variable location operands in IR"

Fixed section of code that iterated through a SmallDenseMap and added
instructions in each iteration, causing non-deterministic code; replaced
SmallDenseMap with MapVector to prevent non-determinism.

This reverts commit 01ac6d1587e8613ba4278786e8341f8b492ac941.

show more ...

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to co

Revert "[DebugInfo] Handle multiple variable location operands in IR"

This caused non-deterministic compiler output; see comment on the
code review.

> This patch updates the various IR passes to correctly handle dbg.values with a
> DIArgList location. This patch does not actually allow DIArgLists to be produced
> by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
> Other than that, it should cover every IR pass.
>
> Most of the changes simply extend code that operated on a single debug value to
> operate on the list of debug values in the style of any_of, all_of, for_each,
> etc. Instances of setOperand(0, ...) have been replaced with with
> replaceVariableLocationOp, which takes the value that is being replaced as an
> additional argument. In places where this value isn't readily available, we have
> to track the old value through to the point where it gets replaced.
>
> Differential Revision: https://reviews.llvm.org/D88232

This reverts commit df69c69427dea7f5b3b3a4d4564bc77b0926ec88.

show more ...

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIA

[DebugInfo] Handle multiple variable location operands in IR

This patch updates the various IR passes to correctly handle dbg.values with a
DIArgList location. This patch does not actually allow DIArgLists to be produced
by salvageDebugInfo, and it does not affect any pass after codegen-prepare.
Other than that, it should cover every IR pass.

Most of the changes simply extend code that operated on a single debug value to
operate on the list of debug values in the style of any_of, all_of, for_each,
etc. Instances of setOperand(0, ...) have been replaced with with
replaceVariableLocationOp, which takes the value that is being replaced as an
additional argument. In places where this value isn't readily available, we have
to track the old value through to the point where it gets replaced.

Differential Revision: https://reviews.llvm.org/D88232

show more ...

[NFC] Edit the comment in User::replaceUsesOfWith

Fix some clang-tidy namespace closing comments warnings. NFC.

[NFC] Remove waymarking because it improves performances

Summary:
This patch remove waymarking and replaces it with storing a pointer to the User in the Use.
here are the results on the measurements

[NFC] Remove waymarking because it improves performances

Summary:
This patch remove waymarking and replaces it with storing a pointer to the User in the Use.
here are the results on the measurements for the CTMark tests of the test suite.
```
Metric: instructions_count

Program baseline patched diff
test-suite :: CTMark/ClamAV/clamscan.test 72557942065 71733653521 -1.1%
test-suite :: CTMark/sqlite3/sqlite3.test 76281422939 75484840636 -1.0%
test-suite :: CTMark/consumer-typeset/consumer-typeset.test 51364676366 50862185614 -1.0%
test-suite :: CTMark/SPASS/SPASS.test 60476106505 59908437767 -0.9%
test-suite :: CTMark/tramp3d-v4/tramp3d-v4.test 112578442329 111725050856 -0.8%
test-suite :: CTMark/mafft/pairlocalalign.test 50846133013 50473644539 -0.7%
test-suite :: CTMark/kimwitu++/kc.test 54692641250 54349070299 -0.6%
test-suite :: CTMark/7zip/7zip-benchmark.test 182216614747 181216091230 -0.5%
test-suite :: CTMark/Bullet/bullet.test 123459210616 122905866767 -0.4%
Geomean difference -0.8%

Metric: peak_memory_use

Program baseline patched diff
test-suite :: CTMark/tramp3d-v4/tramp3d-v4.test 326864 338524 3.6%
test-suite :: CTMark/sqlite3/sqlite3.test 216412 221240 2.2%
test-suite :: CTMark/7zip/7zip-benchmark.test 11808284 12022604 1.8%
test-suite :: CTMark/Bullet/bullet.test 6831752 6945988 1.7%
test-suite :: CTMark/SPASS/SPASS.test 2682552 2721820 1.5%
test-suite :: CTMark/ClamAV/clamscan.test 5037256 5107936 1.4%
test-suite :: CTMark/consumer-typeset/consumer-typeset.test 2752728 2790768 1.4%
test-suite :: CTMark/mafft/pairlocalalign.test 1517676 1537244 1.3%
test-suite :: CTMark/kimwitu++/kc.test 1090748 1103448 1.2%
Geomean difference 1.8%

Metric: compile_time

Program baseline patched diff
test-suite :: CTMark/consumer-typeset/consumer-typeset.test 14.71 14.38 -2.2%
test-suite :: CTMark/sqlite3/sqlite3.test 23.18 22.73 -2.0%
test-suite :: CTMark/7zip/7zip-benchmark.test 57.96 56.99 -1.7%
test-suite :: CTMark/ClamAV/clamscan.test 20.75 20.49 -1.2%
test-suite :: CTMark/kimwitu++/kc.test 18.35 18.15 -1.1%
test-suite :: CTMark/SPASS/SPASS.test 18.72 18.57 -0.8%
test-suite :: CTMark/mafft/pairlocalalign.test 14.09 14.00 -0.6%
test-suite :: CTMark/Bullet/bullet.test 37.38 37.19 -0.5%
test-suite :: CTMark/tramp3d-v4/tramp3d-v4.test 33.81 33.76 -0.2%
Geomean difference -1.1%
```

i believe that it is worth trading +1.8% peak memory use for -1.1% compile time.
also this patch removes waymarking which simplifies the Use and User classes.

Reviewers: nikic, lattner

Reviewed By: lattner

Subscribers: russell.gallop, foad, ggreif, rriddle, ekatz, fhahn, lebedev.ri, mgorny, hiraditya, george.burgess.iv, asbirlea, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77144

show more ...

Basis of dropping uses in llvm.assume.

Summary: This patch adds the basic utilities to deal with dropable uses. dropable uses are uses that we rather drop than prevent transformations, for now they

Basis of dropping uses in llvm.assume.

Summary: This patch adds the basic utilities to deal with dropable uses. dropable uses are uses that we rather drop than prevent transformations, for now they are limited to uses in llvm.assume.

Reviewers: jdoerfert, sstefan1

Reviewed By: jdoerfert

Subscribers: uenoku, lebedev.ri, mgorny, hiraditya, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73404

show more ...

Reinstate MSan suppression of PR24578.

Summary:
Revert "Rollback of commit "Repress sanitization on User dtor.""

There is no point in keeping an active MSan error in the codebase.
PR24578 tracks th

Reinstate MSan suppression of PR24578.

Summary:
Revert "Rollback of commit "Repress sanitization on User dtor.""

There is no point in keeping an active MSan error in the codebase.
PR24578 tracks the actual UB in LLVM code; this change enables testing
of LLVM with MSAN + -fsanitize-memory-use-after-dtor.

This reverts commit 21c1bc46aee2b69c2c48db8e961f0ce8394f21e1.

Reviewers: vitalybuka

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70611

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Remove redundant includes from lib/IR.

llvm-svn: 320622

[IR] De-virtualize ~Value to save a vptr

Summary:
Implements PR889

Removing the virtual table pointer from Value saves 1% of RSS when doing
LTO of llc on Linux. The impact on time was positive, but

[IR] De-virtualize ~Value to save a vptr

Summary:
Implements PR889

Removing the virtual table pointer from Value saves 1% of RSS when doing
LTO of llc on Linux. The impact on time was positive, but too noisy to
conclusively say that performance improved. Here is a link to the
spreadsheet with the original data:

https://docs.google.com/spreadsheets/d/1F4FHir0qYnV0MEp2sYYp_BuvnJgWlWPhWOwZ6LbW7W4/edit?usp=sharing

This change makes it invalid to directly delete a Value, User, or
Instruction pointer. Instead, such code can be rewritten to a null check
and a call Value::deleteValue(). Value objects tend to have their
lifetimes managed through iplist, so for the most part, this isn't a big
deal. However, there are some places where LLVM deletes values, and
those places had to be migrated to deleteValue. I have also created
llvm::unique_value, which has a custom deleter, so it can be used in
place of std::unique_ptr<Value>.

I had to add the "DerivedUser" Deleter escape hatch for MemorySSA, which
derives from User outside of lib/IR. Code in IR cannot include MemorySSA
headers or call the MemoryAccess object destructors without introducing
a circular dependency, so we need some level of indirection.
Unfortunately, no class derived from User may have any virtual methods,
because adding a virtual method would break User::getHungOffOperands(),
which assumes that it can find the use list immediately prior to the
User object. I've added a static_assert to the appropriate OperandTraits
templates to help people avoid this trap.

Reviewers: chandlerc, mehdi_amini, pete, dberlin, george.burgess.iv

Reviewed By: chandlerc

Subscribers: krytarowski, eraman, george.burgess.iv, mzolotukhin, Prazek, nlewycky, hans, inglorion, pcc, tejohnson, dberlin, llvm-commits

Differential Revision: https://reviews.llvm.org/D31261

llvm-svn: 303362

show more ...

Retire llvm::alignOf in favor of C++11 alignof.

No functionality change intended.

llvm-svn: 284733

[IR] Teach `llvm::User` to co-allocate a descriptor.

Summary:
With this change, subclasses of `llvm::User` will be able to co-allocate
a variable number of bytes (called a "descriptor") with the `ll

[IR] Teach `llvm::User` to co-allocate a descriptor.

Summary:
With this change, subclasses of `llvm::User` will be able to co-allocate
a variable number of bytes (called a "descriptor") with the `llvm::User`
instance. The co-allocated descriptor can later be accessed using
`llvm::User::getDescriptor`. This will be used in later changes to
implement operand bundles.

This change steals one bit from `NumUserOperands`, but given that it is
still 28 bits wide I don't think this will be a practical issue.

This change does not allow allocating hung off uses with descriptors.
This only for simplicity, not for any fundamental reason; and we can
easily add this functionality later if needed.

Reviewers: reames, chandlerc, dexonsmith, kmod, majnemer, pete, JosephTremoulet

Subscribers: pete, sanjoy, llvm-commits

Differential Revision: http://reviews.llvm.org/D12455

llvm-svn: 248453

show more ...

Rollback of commit "Repress sanitization on User dtor."
This would have suppressed bug 24578, about use-after-
destroy on User and MDNode. Rolled back suppression for
the sake of code cleanliness, in

Rollback of commit "Repress sanitization on User dtor."
This would have suppressed bug 24578, about use-after-
destroy on User and MDNode. Rolled back suppression for
the sake of code cleanliness, in preferance for bug
tracking to keep track of this issue.

This reverts commit 6ff2baabc4625d5b0a8dccf76aa0f72d930ea6c0.

llvm-svn: 246484

show more ...

Undo reversion on commit: Revert "Revert "Repress sanitization on User dtor.
Modify msan macros for applying attribute""

This reverts commit 020e70a79878c96457e6882bcdfaf6628baf32b7.

llvm-svn: 2464

Undo reversion on commit: Revert "Revert "Repress sanitization on User dtor.
Modify msan macros for applying attribute""

This reverts commit 020e70a79878c96457e6882bcdfaf6628baf32b7.

llvm-svn: 246470

show more ...

Revert "Repress sanitization on User dtor. Modify msan macros for applying attribute"

This reverts commit 5e3bfbb38eb3fb6f568b107f6b239e0aa4c5f334.

llvm-svn: 246450

Repress sanitization on User dtor. Modify msan macros for applying attribute
to repress sanitization. Move attribute for repressing sanitization to
operator delete for User, MDNode.

Summary: In resp

Repress sanitization on User dtor. Modify msan macros for applying attribute
to repress sanitization. Move attribute for repressing sanitization to
operator delete for User, MDNode.

Summary: In response to bug 24578, reported against failing LLVM test.

Reviewers: chandlerc, rsmith, eugenis

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D12335

llvm-svn: 246449

show more ...

Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)

Apparently, the style needs to be agreed upon first.

llvm-svn: 240390

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-c

Fixed/added namespace ending comments using clang-tidy. NFC

The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
llvm/lib/


Thanks to Eugene Kosov for the original patch!

llvm-svn: 240137

show more ...

Tweak wording of alignment static_assert messages.

llvm-svn: 239907