[WPD]Provide branch weight for checking mode. (#124084)

Checking mode aims to help diagnose and confirm undefined behavior. In
most cases, source code don't cast pointers between unrelated types for

[WPD]Provide branch weight for checking mode. (#124084)

Checking mode aims to help diagnose and confirm undefined behavior. In
most cases, source code don't cast pointers between unrelated types for
virtual calls, so we expect direct calls in the frequent branch and
debug trap in the unlikely branch.

This way, the overhead of checking mode is not higher than an indirect
call promotion for a hot callsite as long as the callsite doesn't run the debug trap
branch.

show more ...

[IR] Replace of PointerType::getUnqual(Type) with opaque version (NFC) (#123909)

Follow up to https://github.com/llvm/llvm-project/issues/123569

[WPD]Regard unreachable function as a possible devirtualizable target (#115668)

https://reviews.llvm.org/D115492 skips unreachable functions and
potentially allows more static de-virtualizations. T

[WPD]Regard unreachable function as a possible devirtualizable target (#115668)

https://reviews.llvm.org/D115492 skips unreachable functions and
potentially allows more static de-virtualizations. The motivation is to
ignore virtual deleting destructor of abstract class (e.g.,
`Base::~Base()` in https://gcc.godbolt.org/z/dWMsdT9Kz).
* Note WPD already handles most pure virtual functions (like `Base::x()`
in the godbolt example above), which becomes a `__cxa_pure_virtual` in
the vtable slot.

This PR proposes to undo the change, because it turns out there are
other unreachable functions that a general program wants to run and fail
intentionally, with `LOG(FATAL)` or `CHECK` [1] for example. While many
real-world applications are encouraged to check-fail sparingly, they are
allowed to do so on critical errors (e.g., misconfiguration or bug is
detected during server startup).
* Implementation-wise, this PR keeps the one-bit 'unreachable' state in
bitcode and updates WPD analysis.

https://gcc.godbolt.org/z/T1aMhczYr is a minimum reproducible example
extracted from unit test. `Base::func` is a one-liner of `LOG(FATAL) <<
"message"`, and lowered to one basic block ending with `unreachable`. A
real-world program is _allowed_ to invoke Base::func to terminate the
program as a way to report errors (in server initialization stage for
example), even if errors on the serving path should be handled more
gracefully.

[1] https://abseil.io/docs/cpp/guides/logging#CHECK and
https://abseil.io/docs/cpp/guides/logging#configuration-and-flags

show more ...

[IPO] Remove unused includes (NFC) (#114716)

Identified with misc-include-cleaner.

[WPD][ThinLTO]Add cutoff option for WPD (#113383)

This option applies for _import_ WPD (i.e., when `DevirtModule` pass
de-virtualizes according to an imported summary, in ThinLTO backend
pipeline)

[WPD][ThinLTO]Add cutoff option for WPD (#113383)

This option applies for _import_ WPD (i.e., when `DevirtModule` pass
de-virtualizes according to an imported summary, in ThinLTO backend
pipeline). It's meant for debugging (e.g., bisection).

show more ...

[LLVM] Add `Intrinsic::getDeclarationIfExists` (#112428)

Add `Intrinsic::getDeclarationIfExists` to lookup an existing
declaration of an intrinsic in a `Module`.

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is a

[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)

Rename the function to reflect its correct behavior and to be consistent
with `Module::getOrInsertFunction`. This is also in preparation of
adding a new `Intrinsic::getDeclaration` that will have behavior similar
to `Module::getFunction` (i.e, just lookup, no creation).

show more ...

[LLVM] Use {} instead of std::nullopt to initialize empty ArrayRef (#109133)

It is almost always simpler to use {} instead of std::nullopt to
initialize an empty ArrayRef. This patch changes all oc

[LLVM] Use {} instead of std::nullopt to initialize empty ArrayRef (#109133)

It is almost always simpler to use {} instead of std::nullopt to
initialize an empty ArrayRef. This patch changes all occurrences I could
find in LLVM itself. In future the ArrayRef(std::nullopt_t) constructor
could be deprecated or removed.

show more ...

[llvm][Transforms] Avoid 'raw_string_ostream::str' (NFC)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the str

[llvm][Transforms] Avoid 'raw_string_ostream::str' (NFC)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.

Work towards TODO comment to remove `raw_string_ostream::str()`.

show more ...

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, re

[IR] Add getDataLayout() helpers to Function and GlobalValue (#96919)

Similar to https://github.com/llvm/llvm-project/pull/96902, this adds
`getDataLayout()` helpers to Function and GlobalValue, replacing the
current `getParent()->getDataLayout()` pattern.

show more ...

[WPD] Directly create geteleementptr inbounds (NFCI)

We know that this GEP is inbounds, so make it explicit. NFCI
because constant expression construction already infers this.

[NFCI][sanitizers][metadata] Exctract create{Unlikely,Likely}BranchWeights (#89464)

We have a lot of repeated code with random constants.
Particular values are not important, the one just needs to

[NFCI][sanitizers][metadata] Exctract create{Unlikely,Likely}BranchWeights (#89464)

We have a lot of repeated code with random constants.
Particular values are not important, the one just needs to be
bigger then another.

UR_NONTAKEN_WEIGHT is selected as it's the most common one.

show more ...

[NFC][RemoveDIs] Insert instruction using iterators in Transforms/

As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can c

[NFC][RemoveDIs] Insert instruction using iterators in Transforms/

As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.

There are two general flavours of update:
* Almost all call-sites just call getIterator on an instruction
* Several make use of an existing iterator (scenarios where the code is
actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.

Noteworthy changes:
* FindInsertedValue now takes an optional iterator rather than an
instruction pointer, as we need to always insert with iterators,
* I've added a few iterator-taking versions of some value-tracking and
DomTree methods -- they just unwrap the iterator. These are purely
convenience methods to avoid extra syntax in some passes.
* A few calls to getNextNode become std::next instead (to keep in the
theme of using iterators for positions),
* SeparateConstOffsetFromGEP has it's insertion-position field changed.
Noteworthy because it's not a purely localised spelling change.

All this should be NFC.

show more ...

[CallPromotionUtil] See through function alias when devirtualizing a virtual call on an alloca. (#80736)

- Extract utility function from
`DevirtModule::tryFindVirtualCallTargets` [1], which sees th

[CallPromotionUtil] See through function alias when devirtualizing a virtual call on an alloca. (#80736)

- Extract utility function from
`DevirtModule::tryFindVirtualCallTargets` [1], which sees through an alias to a function. Call this utility function in
the WPD callsite.
- For type profiling work, this helper function will be used by indirect-call-promotion pass to find the function pointer at a specified vtable offset (an example in [2])

[1] https://github.com/llvm/llvm-project/blob/b99163fe8feeacba7797d5479bbcd5d8f327dd2d/llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp#L1069-L1082
[2] https://github.com/minglotus-6/llvm-project/blob/77a0ef12de82d11f448f7f9de6f2dcf87d9b74af/llvm/lib/Transforms/Instrumentation/IndirectCallPromotion.cpp#L347

show more ...

[IRBuilder] Add CreatePtrAdd() method (NFC) (#77582)

This abstracts over the common pattern of creating a gep with i8 element
type.

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of

[ThinLTO] Add tail call flag to call edges in summary (#74043)

This adds support for a HasTailCall flag on function call edges in the
ThinLTO summary. It is intended for use in aiding discovery of missing
frames from tail calls in profiled call stacks for MemProf of profiled
binaries that did not disable tail call elimination. A follow on change
will add the use of this new flag during MemProf context disambiguation.

The new flag is encoded in the bitcode along with either the hotness
flag from the profile, or the relative block frequency under the
-write-relbf-to-summary flag when there is no profile data.
Because we now will always have some additional call edge information, I
have removed the non-profile function summary record format, and we
simply encode the tail call flag along with a hotness type of none when
there is no profile information or relative block frequency. The change
of record format and name caused most of the test case changes.

I have added explicit testing of generation of the new tail call flag
into the bitcode and IR assembly format as part of the changes to
llvm/test/Bitcode/thinlto-function-summary-refgraph.ll. I have also
added round trip testing through assembly and bitcode to
llvm/test/Assembler/thinlto-summary.ll.

show more ...

[Transforms] Use StringRef::starts_with/ends_with instead of startswith/endswith. NFC.

startswith/endswith wrap starts_with/ends_with and will eventually go away (to more closely match string_view)

[IPO] Remove unnecessary bitcasts (NFC)

[llvm] Stop including llvm/ADT/iterator_range.h (NFC)

Identified with misc-include-cleaner.

[WPD][LLD] Add option to validate RTTI is enabled on all native types and prevent devirtualization on types with native RTTI

Discussion about this approach: https://discourse.llvm.org/t/rfc-safer-wh

[WPD][LLD] Add option to validate RTTI is enabled on all native types and prevent devirtualization on types with native RTTI

Discussion about this approach: https://discourse.llvm.org/t/rfc-safer-whole-program-class-hierarchy-analysis/65144/18

When enabling WPD in an environment where native binaries are present, types we want to optimize can be derived from inside these native files and devirtualizing them can lead to correctness issues. RTTI can be used as a way to determine all such types in native files and exclude them from WPD providing a safe checked way to enable WPD.

The approach is:
1. In the linker, identify if RTTI is available for all native types. If not, under `--lto-validate-all-vtables-have-type-infos` `--lto-whole-program-visibility` is automatically disabled. This is done by examining all .symtab symbols in object files and .dynsym symbols in DSOs for vtable (_ZTV) and typeinfo (_ZTI) symbols and ensuring there's always a match for every vtable symbol.
2. During thinlink, if `--lto-validate-all-vtables-have-type-infos` is set and RTTI is available for all native types, identify all typename (_ZTS) symbols via their corresponding typeinfo (_ZTI) symbols that are used natively or outside of our summary and exclude them from WPD.

Testing:
ninja check-all
large Meta service that uses boost, glog and libstdc++.so runs successfully with WPD via --lto-whole-program-visibility. Previously, native types in boost caused incorrect devirtualization that led to crashes.

Reviewed By: MaskRay, tejohnson

Differential Revision: https://reviews.llvm.org/D155659

show more ...

[WholeProgramDevirt] Use llvm:: qualifier to implement declared functions. NFC

[llvm] Use range-based for loops (NFC)

[llvm] Drop some typed pointer handling/bitcasts

Differential Revision: https://reviews.llvm.org/D157016

Stop using legacy helpers indicating typed pointer types. NFC

Since we no longer support typed LLVM IR pointer types, the code can
be simplified into for example using PointerType::get directly inst

Stop using legacy helpers indicating typed pointer types. NFC

Since we no longer support typed LLVM IR pointer types, the code can
be simplified into for example using PointerType::get directly instead
of using Type::getInt8PtrTy and Type::getInt32PtrTy etc.

Differential Revision: https://reviews.llvm.org/D156733

show more ...

Add a type_checked_load_relative to support relative function pointer tables

This adds a type_checked_load_relative intrinsic whose semantics it is to
load a relative function pointer.

A relative f

Add a type_checked_load_relative to support relative function pointer tables

This adds a type_checked_load_relative intrinsic whose semantics it is to
load a relative function pointer.

A relative function pointer is a pointer to a 32bit value that when
added to its address yields the address of the function.

Differential Revision: https://reviews.llvm.org/D143204

show more ...