[RS4GC] Cache BDVs and bases alogn with IsKnownBase flag (NFC)

This refactors RS4GC to cache results returned findBaseDefiningValue
and also gets rid of BaseDefiningValueResult by caching the
IsKnow

[RS4GC] Cache BDVs and bases alogn with IsKnownBase flag (NFC)

This refactors RS4GC to cache results returned findBaseDefiningValue
and also gets rid of BaseDefiningValueResult by caching the
IsKnownBase flag for BDVs and bases.

Differential Revision: https://reviews.llvm.org/D125000

show more ...

[RS4GC] Add support for 'freeze' instruction to findBaseDefiningValue

Because this instruction is a noop, we can simply go through it in
search of the base.

[NFC] Fix typo in assert message

[RS4GC] Prune inputs of BDV if they are BDV themselves

Don't check whether an input of BDV can be pruned if the input
is the BDV itself. BDV is present in the states map, so in case
the input is the

[RS4GC] Prune inputs of BDV if they are BDV themselves

Don't check whether an input of BDV can be pruned if the input
is the BDV itself. BDV is present in the states map, so in case
the input is the BDV itself, we'd return false. So explicitly check this case.

Differential Revision: https://reviews.llvm.org/D123846

show more ...

Fix warnings about variables that are set but only used in debug mode

Add void casts to mark the variables used, next to the places where
they are used in assert or `LLVM_DEBUG()` expressions.

Diff

Fix warnings about variables that are set but only used in debug mode

Add void casts to mark the variables used, next to the places where
they are used in assert or `LLVM_DEBUG()` expressions.

Differential Revision: https://reviews.llvm.org/D123117

show more ...

[RewriteStatepointsForGC] Fix an incorrect assertion

The assertion verifying that a newly computed value matches what is
already cached used stripPointerCasts() to strip bitcasts, however the
values

[RewriteStatepointsForGC] Fix an incorrect assertion

The assertion verifying that a newly computed value matches what is
already cached used stripPointerCasts() to strip bitcasts, however the
values can be not only pointers, but also vectors of pointers. That is
problematic because stripPointerCasts() doesn't handle vectors of
pointers. This patch introduces an ad-hoc utility function to strip all
bitcasts regardless of the value type.

Reviewed By: skatkov, reames

Differential Revision: https://reviews.llvm.org/D119994

show more ...

[IRBuilder][RS4GC] Require FunctionCallee when creating statepoint

This makes the statepoint methods in IRBuilder accept a
FunctionCallee, which carries both the callee and function type.
This is us

[IRBuilder][RS4GC] Require FunctionCallee when creating statepoint

This makes the statepoint methods in IRBuilder accept a
FunctionCallee, which carries both the callee and function type.
This is used to add the elementtype attribute to the statepoint call.

RS4GC requires an additional tweak to actually preserve that attribute
-- previously the attributes on the call were completely overwritten.

Differential Revision: https://reviews.llvm.org/D118886

show more ...

[RS4GC] Extract rematerilazable candidate search. NFC.

Finding re-materialization chain for derived pointer does not depend on
call site. To avoid this finding for each call site it can be extracted

[RS4GC] Extract rematerilazable candidate search. NFC.

Finding re-materialization chain for derived pointer does not depend on
call site. To avoid this finding for each call site it can be extracted in
a separate routine.

Reviewers: reames, dantrushin
Reviewed By: reames
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D118676

show more ...

[RS4GC] Make PointerToBase mapping be independent on call site. NFC.

PointerToBase is a mapping between potentially derived pointer to its base.
As soon as we are in SSA form if there is a base of d

[RS4GC] Make PointerToBase mapping be independent on call site. NFC.

PointerToBase is a mapping between potentially derived pointer to its base.
As soon as we are in SSA form if there is a base of derived pointer and it
is available at def of derived pointer, the same base will be available at any
point where derived pointer is alive.

So the mapping of derived pointer to base pointer is not a property
of a call site but the same on function level.

Reviewers: reames, yrouban
Reviewed By: reames
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D118604

show more ...

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor sligh

Use a sorted array instead of a map to store AttrBuilder string attributes

Using and std::map<SmallString, SmallString> for target dependent attributes is
inefficient: it makes its constructor slightly heavier, and involves extra
allocation for each new string attribute. Storing the attribute key/value as
strings implies extra allocation/copy step.

Use a sorted vector instead. Given the low number of attributes generally
involved, this is cheaper, as showcased by

https://llvm-compile-time-tracker.com/compare.php?from=5de322295f4ade692dc4f1823ae4450ad3c48af2&to=05bc480bf641a9e3b466619af43a2d123ee3f71d&stat=instructions

Differential Revision: https://reviews.llvm.org/D116599

show more ...

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for bu

Introduce the AttributeMask class

This class is solely used as a lightweight and clean way to build a set of
attributes to be removed from an AttrBuilder. Previously AttrBuilder was used
both for building and removing, which introduced odd situation like creation of
Attribute with dummy value because the only relevant part was the attribute
kind.

Differential Revision: https://reviews.llvm.org/D116110

show more ...

[RS4GC] Clean up attribute removal (NFC)

It is not necessary to explicitly check which attributes are
present, and only add those to the builder. We can simply list
all attributes that need to be st

[RS4GC] Clean up attribute removal (NFC)

It is not necessary to explicitly check which attributes are
present, and only add those to the builder. We can simply list
all attributes that need to be stripped and remove them
unconditionally. This also allows us to use some nicer APIs that
don't require mucking about with attribute list indices.

show more ...

[LLVM] Inclusive terms: remove/replace references to sanity in RewriteStatepointsForGC.cpp and test

Part of work to have the LLVM backend to use more inclusive terms.

Reviewed By: reames

Different

[LLVM] Inclusive terms: remove/replace references to sanity in RewriteStatepointsForGC.cpp and test

Part of work to have the LLVM backend to use more inclusive terms.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D112461

show more ...

Fixed some errors detected by PVS Studio

RewriteStatepointsForGC - Use const-ref iterator in for-range loops. NFCI.

Avoid unnecessary copies, reported by MSVC static analyzer.

[NFC] Use newly introduced *AtIndex methods

Introduced in D108788. These are clearer.

[NFC] Replace some attribute methods that use confusing indexes

[NFC] Cleanup calls to AttributeList::getAttribute(FunctionIndex)

getAttribute() is confusing, use a clearer method.

[NFC] Clean up users of AttributeList::hasAttribute()

AttributeList::hasAttribute() is confusing, use clearer methods like
hasParamAttr()/hasRetAttr().

Add hasRetAttr() since it was missing from At

[NFC] Clean up users of AttributeList::hasAttribute()

AttributeList::hasAttribute() is confusing, use clearer methods like
hasParamAttr()/hasRetAttr().

Add hasRetAttr() since it was missing from AttributeList.

show more ...

[NFC] Rename AttributeList::getParam/Ret/FnAttributes() -> get*Attributes()

This is more consistent with similar methods.

[RS4GC] Use one DVCache for both inlineGetBaseAndOffset() and insertParsePoints()

This new test demonstrates a case where a base ptr is generated
twice for the same value: the first one is generated

[RS4GC] Use one DVCache for both inlineGetBaseAndOffset() and insertParsePoints()

This new test demonstrates a case where a base ptr is generated
twice for the same value: the first one is generated while
the gc.get.pointer.base() is inlined, the second is generated
for the statepoint. This happens because the methods
inlineGetBaseAndOffset() and insertParsePoints() do not share
their defining value cache used by the findBasePointer() method.

Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D103240

show more ...

[RS4GC] Treat inttoptr as base pointer

This is a modified version of a patch by tolziplohu with a style change, and most importantly, a revised commit message.

inttoptr for a non-integral address s

[RS4GC] Treat inttoptr as base pointer

This is a modified version of a patch by tolziplohu with a style change, and most importantly, a revised commit message.

inttoptr for a non-integral address space is currently ill defined in the LangRef. Figuring out exactly what the dynamic semantics of such a cast would be is hard, and not yet settled. Despite that, we still need to go ahead and implement something in RS4GC for a couple of reasons.

First, as a simple consistency argument. We're apparently added support for constexpr inttoptrs a while back, and even have tests which exercised them. Having a lack of constant folding trigger a crash during lowering is non-ideal.

Second, and more fundementally, the optimizer is allowed to insert undefined constructs in unreachable code. At the same time, we can't assume that dynamically dead code is always pruned before lowering. As a result, we must assume that inttoptrs can occur (even if completely ill defined) along dead paths. We need the lowering to not crash. The stackmaps produced can be garbage (as the assumption is the code is dynamically dead), but the lowering itself can't crash.

Differential Revision: https://reviews.llvm.org/D103492

show more ...

[RS4GC] Introduce intrinsics to get base ptr and offset

There can be a need for some optimizations to get (base, offset)
for any GC pointer. The base can be calculated by generating
needed instructi

[RS4GC] Introduce intrinsics to get base ptr and offset

There can be a need for some optimizations to get (base, offset)
for any GC pointer. The base can be calculated by generating
needed instructions as it is done by the
RewriteStatepointsForGC::findBasePointer() function. The offset
can be calculated in the same way. Though to not expose the base
calculation and to make the offset calculation as simple as
ptrtoint(derived_ptr) - ptrtoint(base_ptr), which is illegal
outside RS4GC, this patch introduces 2 intrinsics:

@llvm.experimental.gc.get.pointer.base(%derived_ptr)
@llvm.experimental.gc.get.pointer.offset(%derived_ptr)

These intrinsics are inlined by RS4GC along with generation of
statepoint sequences.

With these new intrinsics the GC parseable lowering for atomic
memcpy intrinsics (6ec2c5e402a724ba99bce82a9cac7a3006d660f4)
could be implemented as a separate pass.

Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D100445

show more ...

[rs4gc] Strip memory related attributes consistently

I noticed that rs4gc is not stripping a number of memory aliasing related attributes. We do strip some from call sites, but don't strip the same

[rs4gc] Strip memory related attributes consistently

I noticed that rs4gc is not stripping a number of memory aliasing related attributes. We do strip some from call sites, but don't strip the same ones from declarations or parameters.

Why do we need to strip these? Two answers:

Safepoints conceptually read and write to the entire garbage collected heap in the physical model. We need this to preserve ordering of all loads and stores with respect to possible relocation.
We can infer other attributes from these. For instance, readnone can imply both nofree and nosync. Both of which don't hold after physical rewriting.

Note: This exposed a latent issue which was fixed a couple weeks back in 01801d5274.

Differential Revision: https://reviews.llvm.org/D99802

show more ...

[rs4gc] Fix a latent bug around attribute stripping for intrinsics

This change fixes a latent bug which was exposed by a change currently in review (https://reviews.llvm.org/D99802#2685032).

The st

[rs4gc] Fix a latent bug around attribute stripping for intrinsics

This change fixes a latent bug which was exposed by a change currently in review (https://reviews.llvm.org/D99802#2685032).

The story on this is a bit involved. Without this change, what ended up happening with the pending review was that we'd strip attributes off intrinsics, and then selectiondag would fail to lower the intrinsic. Why? Because the lowering of the intrinsic relies on the presence of the readonly attribute. We don't have a matcher to select the case where there's a glue node needed.

Now, on the surface, this still seems like a codegen bug. However, here it gets fun. I was unable to reproduce this with a standalone test at all, and was pretty much struck until skatkov provided the critical detail. This reproduces only when RS4GC and codegen are run in the same process and context. Why? Because it turns out we can't roundtrip the stripped attribute through serialized IR!

We'll happily print out the missing attribute, but when we parse it back, the auto-upgrade logic has a side effect of blindly overwriting attributes on intrinsics with those specified in Intrinsics.td. This makes it impossible to exercise SelectionDAG from a standalone test case.

At this point, I decided to treat this an RS4GC bug as a) we don't need to strip in this case, and b) I could write a test which shows the correct behavior to ensure this doesn't break again in the future.

As an aside, I'd originally set out to handle libfuncs too - since in theory they might have the same issues - but backed away quickly when I realized how the semantics of builtin, nobuiltin, and no-builtin-x all interacted. I'm utterly convinced that no part of the optimizer handles that correctly, and decided not to open that can of worms here.

show more ...