[CallGraph] Teach the CallGraph about non-leaf intrinsics.

Summary:
Currently intrinsics don't affect the creation of the call graph.
This is not accurate with respect to statepoint and patchpoint
i

[CallGraph] Teach the CallGraph about non-leaf intrinsics.

Summary:
Currently intrinsics don't affect the creation of the call graph.
This is not accurate with respect to statepoint and patchpoint
intrinsics -- these do call (or invoke) LLVM level functions.

This change fixes this inconsistency by adding a call to the external
node for call sites that call these non-leaf intrinsics. This coupled
with the fact that these intrinsics also escape the function pointer
they call gives us a conservatively correct call graph.

Reviewers: reames, chandlerc, atrick, pgavlin

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10526

llvm-svn: 240039

show more ...

Move the personality function from LandingPadInst to Function

The personality routine currently lives in the LandingPadInst.

This isn't desirable because:
- All LandingPadInsts in the same function

Move the personality function from LandingPadInst to Function

The personality routine currently lives in the LandingPadInst.

This isn't desirable because:
- All LandingPadInsts in the same function must have the same
personality routine. This means that each LandingPadInst beyond the
first has an operand which produces no additional information.

- There is ongoing work to introduce EH IR constructs other than
LandingPadInst. Moving the personality routine off of any one
particular Instruction and onto the parent function seems a lot better
than have N different places a personality function can sneak onto an
exceptional function.

Differential Revision: http://reviews.llvm.org/D10429

llvm-svn: 239940

show more ...

ArgumentPromotion: Drop sret attribute on functions that are only called directly.

If the first argument to a function is a 'this' argument and the second
has the sret attribute, the ArgumentPromoti

ArgumentPromotion: Drop sret attribute on functions that are only called directly.

If the first argument to a function is a 'this' argument and the second
has the sret attribute, the ArgumentPromotion pass may promote the 'this'
argument to more than one argument, violating the IR constraint that 'sret'
may only be applied to the first or second argument.

Although this IR constraint is arguably unnecessary, it highlighted the fact
that ArgPromotion does not need to preserve this attribute. Dropping the
attribute reduces register pressure in the backend by avoiding the register
copy required by sret. Because sret implies noalias, we also replace the
former with the latter.

Differential Revision: http://reviews.llvm.org/D10353

llvm-svn: 239488

show more ...

This patch adds support for the vector quadword add/sub instructions introduced
in POWER8:

vadduqm
vaddeuqm
vaddcuq
vaddecuq
vsubuqm
vsubeuqm
vsubcuq
vsubecuq
In addition to adding the instructions

This patch adds support for the vector quadword add/sub instructions introduced
in POWER8:

vadduqm
vaddeuqm
vaddcuq
vaddecuq
vsubuqm
vsubeuqm
vsubcuq
vsubecuq
In addition to adding the instructions themselves, it also adds support for the
v1i128 type for intrinsics (Intrinsics.td, Function.cpp, and
IntrinsicEmitter.cpp).

http://reviews.llvm.org/D9081

llvm-svn: 238144

show more ...

Simplify and rename function overrideFunctionAttributes. NFC.

This is in preparation to making changes needed to stop resetting
NoFramePointerElim in resetTargetOptions.

llvm-svn: 238079

Store intrinsic ID by value in Function instead of a string lookup. NFC.

On 64-bit targets, Function has 4-bytes of padding in its struct layout.

This uses the space for the intrinsic ID. It is se

Store intrinsic ID by value in Function instead of a string lookup. NFC.

On 64-bit targets, Function has 4-bytes of padding in its struct layout.

This uses the space for the intrinsic ID. It is set and recalculated whenever the function name is set. This is similar to the current behavior which clears the function from the intrinsic ID cache when its renamed.

The intrinsic cache itself is removed as the only purpose was to speedup calls to getIntrinsicID() which now just reading the new field in the struct.

Reviewed by Duncan. http://reviews.llvm.org/D9836

llvm-svn: 237642

show more ...

Move Function::lookupIntrinsicID to a static method. NFC

llvm-svn: 237641

Add function entry count metadata.

Summary:
This adds three Function methods to handle function entry counts:
setEntryCount() and getEntryCount().

Entry counts are stored under the MD_prof metadata

Add function entry count metadata.

Summary:
This adds three Function methods to handle function entry counts:
setEntryCount() and getEntryCount().

Entry counts are stored under the MD_prof metadata node with the name
"function_entry_count". They are unsigned 64 bit values set by profilers
(instrumentation and sample profiler changes coming up).

Added documentation for new profile metadata and tests.

Reviewers: dexonsmith, bogner

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9628

llvm-svn: 237260

show more ...

Replace branch-to-unreachable with assertion.

llvm-svn: 236893

Let llc and opt override "-target-cpu" and "-target-features" via command line
options.

This commit fixes a bug in llc and opt where "-mcpu" and "-mattr" wouldn't
override function attributes "-targ

Let llc and opt override "-target-cpu" and "-target-features" via command line
options.

This commit fixes a bug in llc and opt where "-mcpu" and "-mattr" wouldn't
override function attributes "-target-cpu" and "-target-features" in the IR.

Differential Revision: http://reviews.llvm.org/D9537

llvm-svn: 236677

show more ...

Add missing dereferenceable_or_null getters

Summary: Add missing dereferenceable_or_null getters required for
http://reviews.llvm.org/D9253 change. Separated from the D9253 review.

Patch by Artur P

Add missing dereferenceable_or_null getters

Summary: Add missing dereferenceable_or_null getters required for
http://reviews.llvm.org/D9253 change. Separated from the D9253 review.

Patch by Artur Pilipenko!

Reviewers: sanjoy

Reviewed By: sanjoy

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9499

llvm-svn: 236615

show more ...

IR: Add Function metadata attachments

Add IR support for `Metadata` attachments. Assembly and bitcode support
will follow shortly, but for now we just have unit tests. This is part
of PR23340.

ll

IR: Add Function metadata attachments

Add IR support for `Metadata` attachments. Assembly and bitcode support
will follow shortly, but for now we just have unit tests. This is part
of PR23340.

llvm-svn: 235783

show more ...

IR: Use a bitmask to access GlobalObject subclass data

Make room for more than just `Function::isMaterializable()` in the
`GlobalObject` subclass data bitfield. Since we're treating it like a
bitfi

IR: Use a bitmask to access GlobalObject subclass data

Make room for more than just `Function::isMaterializable()` in the
`GlobalObject` subclass data bitfield. Since we're treating it like a
bitfield, change `Function::Function()` to zero-out the whole thing.

llvm-svn: 235770

show more ...

[IR] Introduce a dereferenceable_or_null(N) attribute.

Summary:
If a pointer is marked as dereferenceable_or_null(N), LLVM assumes it
is either `null` or `dereferenceable(N)` or both. This change o

[IR] Introduce a dereferenceable_or_null(N) attribute.

Summary:
If a pointer is marked as dereferenceable_or_null(N), LLVM assumes it
is either `null` or `dereferenceable(N)` or both. This change only
introduces the attribute and adds a token test case for the `llvm-as`
/ `llvm-dis`. It does not hook up other parts of the optimizer to
actually exploit the attribute -- those changes will come later.

For pointers in address space 0, `dereferenceable(N)` is now exactly
equivalent to `dereferenceable_or_null(N)` && `nonnull`. For other
address spaces, `dereferenceable(N)` is potentially weaker than
`dereferenceable_or_null(N)` && `nonnull` (since we could have a null
`dereferenceable(N)` pointer).

The motivating case for this change is Java (and other managed
languages), where pointers are either `null` or dereferenceable up to
some usually known-at-compile-time constant offset.

Reviewers: rafael, hfinkel

Reviewed By: hfinkel

Subscribers: nicholas, llvm-commits

Differential Revision: http://reviews.llvm.org/D8650

llvm-svn: 235132

show more ...

[opaque pointer type] Avoid using PointerType::getElementType when parsing IR

A few calls are left in for error checking - but I'm commenting those
out & trying to build some IR tests (aiming for Ar

[opaque pointer type] Avoid using PointerType::getElementType when parsing IR

A few calls are left in for error checking - but I'm commenting those
out & trying to build some IR tests (aiming for Argument Promotion to
start with). When I get any of these tests passing I may add flag to
disable the checking so I can add tests that pass with the assertion in
place.

llvm-svn: 234206

show more ...

InstCombine: propagate deref via new addDereferenceableAttr

The "dereferenceable" attribute cannot be added via .addAttribute(),
since it also expects a size in bytes. AttrBuilder#addAttribute or
At

InstCombine: propagate deref via new addDereferenceableAttr

The "dereferenceable" attribute cannot be added via .addAttribute(),
since it also expects a size in bytes. AttrBuilder#addAttribute or
AttributeSet#addAttribute is wrapped by classes Function, InvokeInst,
and CallInst. Add corresponding wrappers to
AttrBuilder#addDereferenceableAttr.

Having done this, propagate the dereferenceable attribute via
gc.relocate, adding a test to exercise it. Note that -datalayout is
required during execution over and above -instcombine, because
InstCombine only optionally requires DataLayoutPass.

Differential Revision: http://reviews.llvm.org/D7510

llvm-svn: 229265

show more ...

Masked Gather and Scatter Intrinsics.

Gather and Scatter are new introduced intrinsics, comming after recently implemented masked load and store.
This is the first patch for Gather and Scatter intri

Masked Gather and Scatter Intrinsics.

Gather and Scatter are new introduced intrinsics, comming after recently implemented masked load and store.
This is the first patch for Gather and Scatter intrinsics. It includes only the syntax, parsing and verification.

Gather and Scatter intrinsics allow to perform multiple memory accesses (read/write) in one vector instruction.
The intrinsics are not target specific and will have the following syntax:
Gather:
declare <16 x i32> @llvm.masked.gather.v16i32(<16 x i32*> <vector of ptrs>, i32 <alignment>, <16 x i1> <mask>, <16 x i32> <passthru>)
declare <8 x float> @llvm.masked.gather.v8f32(<8 x float*><vector of ptrs>, i32 <alignment>, <8 x i1> <mask>, <8 x float><passthru>)

Scatter:
declare void @llvm.masked.scatter.v8i32(<8 x i32><vector value to be stored> , <8 x i32*><vector of ptrs> , i32 <alignment>, <8 x i1> <mask>)
declare void @llvm.masked.scatter.v16i32(<16 x i32> <vector value to be stored> , <16 x i32*> <vector of ptrs>, i32 <alignment>, <16 x i1><mask> )

Vector of ptrs - a set of source/destination addresses, to load/store the value.
Mask - switches on/off vector lanes to prevent memory access for switched-off lanes
vector of ptrs, value and mask should have the same vector width.

These are code examples where gather / scatter should be used and will allow function vectorization
;void foo1(int * restrict A, int * restrict B, int * restrict C) {
; for (int i=0; i<SIZE; i++) {
; A[i] = B[C[i]];
; }
;}

;void foo3(int * restrict A, int * restrict B) {
; for (int i=0; i<SIZE; i++) {
; A[B[i]] = i+5;
; }
;}

Tests will come in the following patches, with CodeGen and Vectorizer.

http://reviews.llvm.org/D7433

llvm-svn: 228521

show more ...

Revert GCStrategy ownership changes

This change reverts the interesting parts of 226311 (and 227046). This change introduced two problems, and I've been convinced that an alternate approach is pref

Revert GCStrategy ownership changes

This change reverts the interesting parts of 226311 (and 227046). This change introduced two problems, and I've been convinced that an alternate approach is preferrable anyways.

The bugs were:
- Registery appears to require all users be within the same linkage unit. After this change, asking for "statepoint-example" in Transform/ would sometimes get you nullptr, whereas asking the same question in CodeGen would return the right GCStrategy. The correct long term fix is to get rid of the utter hack which is Registry, but I don't have time for that right now. 227046 appears to have been an attempt to fix this, but I don't believe it does so completely.
- GCMetadataPrinter::finishAssembly was being called more than once per GCStrategy. Each Strategy was being added to the GCModuleInfo multiple times.

Once I get time again, I'm going to split GCModuleInfo into the gc.root specific part and a GCStrategy owning Analysis pass. I'm probably also going to kill off the Registry. Once that's done, I'll move the new GCStrategyAnalysis and all built in GCStrategies into Analysis. (As original suggested by Chandler.) This will accomplish my original goal of being able to access GCStrategy from Transform/ without adding all of the builtin GCs to IR/.

llvm-svn: 227109

show more ...

Move ownership of GCStrategy objects to LLVMContext

Note: This change ended up being slightly more controversial than expected. Chandler has tentatively okayed this for the moment, but I may be rev

Move ownership of GCStrategy objects to LLVMContext

Note: This change ended up being slightly more controversial than expected. Chandler has tentatively okayed this for the moment, but I may be revisiting this in the near future after we settle some high level questions.

Rather than have the GCStrategy object owned by the GCModuleInfo - which is an immutable analysis pass used mainly by gc.root - have it be owned by the LLVMContext. This simplifies the ownership logic (i.e. can you have two instances of the same strategy at once?), but more importantly, allows us to access the GCStrategy in the middle end optimizer. To this end, I add an accessor through Function which becomes the canonical way to get at a GCStrategy instance.

In the near future, this will allows me to move some of the checks from http://reviews.llvm.org/D6808 into the Verifier itself, and to introduce optimization legality predicates for some of the recent additions to InstCombine. (These will follow as separate changes.)

Differential Revision: http://reviews.llvm.org/D6811

llvm-svn: 226311

show more ...

getMangledTypeStr: clarify how it mangles types, and add tests

"Write a set of tests that show how name mangling is done for overloaded intrinsics." These happen to use gc.relocates to exercise the

getMangledTypeStr: clarify how it mangles types, and add tests

"Write a set of tests that show how name mangling is done for overloaded intrinsics." These happen to use gc.relocates to exercise the codepath in question, but is not a GC specific test.

Patch by: artagnon@gmail.com
Differential Revision: http://reviews.llvm.org/D6915

llvm-svn: 226056

show more ...

Masked Load/Store - Changed the order of parameters in intrinsics.
No functional changes.
The documentation is coming.

llvm-svn: 224829

The leak detector is dead, long live asan and valgrind.

In resent times asan and valgrind have found way more memory management bugs
in llvm than the special purpose leak detector.

llvm-svn: 224703

Masked Load / Store Intrinsics - the CodeGen part.
I'm recommiting the codegen part of the patch.
The vectorizer part will be send to review again.

Masked Vector Load and Store Intrinsics.
Introduce

Masked Load / Store Intrinsics - the CodeGen part.
I'm recommiting the codegen part of the patch.
The vectorizer part will be send to review again.

Masked Vector Load and Store Intrinsics.
Introduced new target-independent intrinsics in order to support masked vector loads and stores. The loop vectorizer optimizes loops containing conditional memory accesses by generating these intrinsics for existing targets AVX2 and AVX-512. The vectorizer asks the target about availability of masked vector loads and stores.
Added SDNodes for masked operations and lowering patterns for X86 code generator.
Examples:
<16 x i32> @llvm.masked.load.v16i32(i8* %addr, <16 x i32> %passthru, i32 4 /* align */, <16 x i1> %mask)
declare void @llvm.masked.store.v8f64(i8* %addr, <8 x double> %value, i32 4, <8 x i1> %mask)

Scalarizer for other targets (not AVX2/AVX-512) will be done in a separate patch.

http://reviews.llvm.org/D6191

llvm-svn: 223348

show more ...

Prologue support

Patch by Ben Gamari!

This redefines the `prefix` attribute introduced previously and
introduces a `prologue` attribute. There are a two primary usecases
that these attributes aim

Prologue support

Patch by Ben Gamari!

This redefines the `prefix` attribute introduced previously and
introduces a `prologue` attribute. There are a two primary usecases
that these attributes aim to serve,

1. Function prologue sigils

2. Function hot-patching: Enable the user to insert `nop` operations
at the beginning of the function which can later be safely replaced
with a call to some instrumentation facility

3. Runtime metadata: Allow a compiler to insert data for use by the
runtime during execution. GHC is one example of a compiler that
needs this functionality for its tables-next-to-code functionality.

Previously `prefix` served cases (1) and (2) quite well by allowing the user
to introduce arbitrary data at the entrypoint but before the function
body. Case (3), however, was poorly handled by this approach as it
required that prefix data was valid executable code.

Here we redefine the notion of prefix data to instead be data which
occurs immediately before the function entrypoint (i.e. the symbol
address). Since prefix data now occurs before the function entrypoint,
there is no need for the data to be valid code.

The previous notion of prefix data now goes under the name "prologue
data" to emphasize its duality with the function epilogue.

The intention here is to handle cases (1) and (2) with prologue data and
case (3) with prefix data.

References
----------

This idea arose out of discussions[1] with Reid Kleckner in response to a
proposal to introduce the notion of symbol offsets to enable handling of
case (3).

[1] http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/073235.html

Test Plan: testsuite

Differential Revision: http://reviews.llvm.org/D6454

llvm-svn: 223189

show more ...

Revert "Masked Vector Load and Store Intrinsics."

This reverts commit r222632 (and follow-up r222636), which caused a host
of LNT failures on an internal bot. I'll respond to the commit on the
list

Revert "Masked Vector Load and Store Intrinsics."

This reverts commit r222632 (and follow-up r222636), which caused a host
of LNT failures on an internal bot. I'll respond to the commit on the
list with a reproduction of one of the failures.

Conflicts:
lib/Target/X86/X86TargetTransformInfo.cpp

llvm-svn: 222936

show more ...