Masked gather and scatter: Added code for SelectionDAG.
All other patches, including tests will follow.

http://reviews.llvm.org/D7665

llvm-svn: 235970

Switch lowering: use uint32_t for weights everywhere

I previously thought switch clusters would need to use uint64_t in case
the weights of multiple cases overflowed a 32-bit int. It turns
out that

Switch lowering: use uint32_t for weights everywhere

I previously thought switch clusters would need to use uint64_t in case
the weights of multiple cases overflowed a 32-bit int. It turns
out that the weights on a terminator instruction are capped to allow for
being added together, so using a uint32_t should be safe.

llvm-svn: 235945

show more ...

Switch lowering: Take branch weight into account when ordering for fall-through

Previously, the code would try to put a fall-through case last,
even if that meant moving a case with much higher bran

Switch lowering: Take branch weight into account when ordering for fall-through

Previously, the code would try to put a fall-through case last,
even if that meant moving a case with much higher branch weight
further down the chain.

Ordering by branch weight is most important, putting a fall-through
block last is secondary.

llvm-svn: 235942

show more ...

Switch lowering: order bit tests by branch weight.

llvm-svn: 235912

[SEH] Implement GetExceptionCode in __except blocks

This introduces an intrinsic called llvm.eh.exceptioncode. It is lowered
by copying the EAX value live into whatever basic block it is called
from

[SEH] Implement GetExceptionCode in __except blocks

This introduces an intrinsic called llvm.eh.exceptioncode. It is lowered
by copying the EAX value live into whatever basic block it is called
from. Obviously, this only works if you insert it late during codegen,
because otherwise mid-level passes might reschedule it.

llvm-svn: 235768

show more ...

Switch lowering: fix APInt overflow causing infinite loop / OOM

llvm-svn: 235729

Re-commit r235560: Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

Third time's the charm. The previous commit was reverted as a
reverse for-loop in Selectio

Re-commit r235560: Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

Third time's the charm. The previous commit was reverted as a
reverse for-loop in SelectionDAGBuilder::lowerWorkItem did 'I--'
on an iterator at the beginning of a vector, causing asserts
when using debugging iterators. This commit fixes that.

llvm-svn: 235608

show more ...

Revert r235560; this commit was causing several failed assertions in Debug builds using MSVC's STL. The iterator is being used outside of its valid range.

llvm-svn: 235597

Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

This is a re-commit of r235101, which also fixes the problems with the previous patch:

- Switches with only

Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

This is a re-commit of r235101, which also fixes the problems with the previous patch:

- Switches with only a default case and non-fallthrough were handled incorrectly

- The previous patch tickled a bug in PowerPC Early-Return Creation which is fixed here.

> This is a major rewrite of the SelectionDAG switch lowering. The previous code
> would lower switches as a binary tre, discovering clusters of cases
> suitable for lowering by jump tables or bit tests as it went along. To increase
> the likelihood of finding jump tables, the binary tree pivot was selected to
> maximize case density on both sides of the pivot.
>
> By not selecting the pivot in the middle, the binary trees would not always
> be balanced, leading to performance problems in the generated code.
>
> This patch rewrites the lowering to search for clusters of cases
> suitable for jump tables or bit tests first, and then builds the binary
> tree around those clusters. This way, the binary tree will always be balanced.
>
> This has the added benefit of decoupling the different aspects of the lowering:
> tree building and jump table or bit tests finding are now easier to tweak
> separately.
>
> For example, this will enable us to balance the tree based on profile info
> in the future.
>
> The algorithm for finding jump tables is quadratic, whereas the previous algorithm
> was O(n log n) for common cases, and quadratic only in the worst-case. This
> doesn't seem to be major problem in practice, e.g. compiling a file consisting
> of a 10k-case switch was only 30% slower, and such large switches should be rare
> in practice. Compiling e.g. gcc.c showed no compile-time difference. If this
> does turn out to be a problem, we could limit the search space of the algorithm.
>
> This commit also disables all optimizations during switch lowering in -O0.
>
> Differential Revision: http://reviews.llvm.org/D8649

llvm-svn: 235560

show more ...

[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the
X86 backend.

The code generated for symbolic targets is identical to the code generated for
constant targets, except

[patchpoint] Add support for symbolic patchpoint targets to SelectionDAG and the
X86 backend.

The code generated for symbolic targets is identical to the code generated for
constant targets, except that a relocation is emitted to fix up the actual
target address at link-time. This allows IR and object files containing
patchpoints to be cached across JIT-invocations where the target address may
change.

llvm-svn: 235483

show more ...

DebugInfo: Drop rest of DIDescriptor subclasses

Delete the remaining subclasses of (the already deleted) `DIDescriptor`.
Part of PR23080.

llvm-svn: 235404

DebugInfo: Assert dbg.declare/value insts are valid

Remove early returns for when `getVariable()` is null, and just assert
that it never happens. The Verifier already confirms that there's a
valid

DebugInfo: Assert dbg.declare/value insts are valid

Remove early returns for when `getVariable()` is null, and just assert
that it never happens. The Verifier already confirms that there's a
valid variable on these intrinsics, so we should assume the debug info
isn't broken. I also updated a check for a `!dbg` attachment, which the
Verifier similarly guarantees.

llvm-svn: 235400

show more ...

Revert the switch lowering change (r235101, r235103, r235106)

Looks like it broke the sanitizer-ppc64-linux1 build. Reverting for now.

llvm-svn: 235108

Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

This is a major rewrite of the SelectionDAG switch lowering. The previous code
would lower switches as a bina

Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)

This is a major rewrite of the SelectionDAG switch lowering. The previous code
would lower switches as a binary tre, discovering clusters of cases
suitable for lowering by jump tables or bit tests as it went along. To increase
the likelihood of finding jump tables, the binary tree pivot was selected to
maximize case density on both sides of the pivot.

By not selecting the pivot in the middle, the binary trees would not always
be balanced, leading to performance problems in the generated code.

This patch rewrites the lowering to search for clusters of cases
suitable for jump tables or bit tests first, and then builds the binary
tree around those clusters. This way, the binary tree will always be balanced.

This has the added benefit of decoupling the different aspects of the lowering:
tree building and jump table or bit tests finding are now easier to tweak
separately.

For example, this will enable us to balance the tree based on profile info
in the future.

The algorithm for finding jump tables is O(n^2), whereas the previous algorithm
was O(n log n) for common cases, and quadratic only in the worst-case. This
doesn't seem to be major problem in practice, e.g. compiling a file consisting
of a 10k-case switch was only 30% slower, and such large switches should be rare
in practice. Compiling e.g. gcc.c showed no compile-time difference. If this
does turn out to be a problem, we could limit the search space of the algorithm.

This commit also disables all optimizations during switch lowering in -O0.

Differential Revision: http://reviews.llvm.org/D8649

llvm-svn: 235101

show more ...

DebugInfo: Gut DIVariable and DIGlobalVariable

Gut all the non-pointer API from the variable wrappers, except an
implicit conversion from `DIGlobalVariable` to `DIDescriptor`. Note
that if you're u

DebugInfo: Gut DIVariable and DIGlobalVariable

Gut all the non-pointer API from the variable wrappers, except an
implicit conversion from `DIGlobalVariable` to `DIDescriptor`. Note
that if you're updating out-of-tree code, `DIVariable` wraps
`MDLocalVariable` (`MDVariable` is a common base class shared with
`MDGlobalVariable`).

llvm-svn: 234840

show more ...

SelectionDAG: Stop using DIVariable::isInlinedFnArgument()

Instead of calling the somewhat confusingly-named
`DIVariable::isInlinedFnArgument()`, do the check directly here.
There's possibly a small

SelectionDAG: Stop using DIVariable::isInlinedFnArgument()

Instead of calling the somewhat confusingly-named
`DIVariable::isInlinedFnArgument()`, do the check directly here.
There's possibly a small functionality change here: instead of
`dyn_cast<>`'ing `DV->getScope()` to `MDSubprogram`, I'm looking up the
scope chain for the actual subprogram. I suspect that this is a no-op
for function arguments so in practise there isn't a real difference.

I've also added a `FIXME` to check the `inlinedAt:` chain instead, since
I wonder if that would be more reliable than the
`MDSubprogram::describes()` function.

Since this was the only user of `DIVariable::isInlinedFnArgument()`,
delete it.

llvm-svn: 234799

show more ...

Allow memory intrinsics to be tail calls

llvm-svn: 234764

CodeGen: Stop using DIDescriptor::is*() and auto-casting

Same as r234255, but for lib/CodeGen and lib/Target.

llvm-svn: 234258

[WinEH] Don't sink allocas into child handlers

The uselist isn't enough to infer anything about the lifetime of such
allocas. If we want to re-add this optimization, we will need to
leverage lifetim

[WinEH] Don't sink allocas into child handlers

The uselist isn't enough to infer anything about the lifetime of such
allocas. If we want to re-add this optimization, we will need to
leverage lifetime markers to do it.

Fixes PR23122.

llvm-svn: 234196

show more ...

[WinEH] Fill out .xdata for catch objects

This add support for catching an exception such that an exception object
available to the catch handler will be initialized by the runtime.

llvm-svn: 234062

[WinEH] Sink UnwindHelp completely out of IR

We don't need to represent UnwindHelp in IR. Instead, we can use the
knowledge that we are emitting the parent function to decide if we
should create th

[WinEH] Sink UnwindHelp completely out of IR

We don't need to represent UnwindHelp in IR. Instead, we can use the
knowledge that we are emitting the parent function to decide if we
should create the UnwindHelp stack object.

llvm-svn: 234061

show more ...

CodeGen: Assert that inlined-at locations agree

As a follow-up to r234021, assert that a debug info intrinsic variable's
`MDLocalVariable::getInlinedAt()` always matches the
`MDLocation::getInlinedA

CodeGen: Assert that inlined-at locations agree

As a follow-up to r234021, assert that a debug info intrinsic variable's
`MDLocalVariable::getInlinedAt()` always matches the
`MDLocation::getInlinedAt()` of its `!dbg` attachment.

The goal here is to get rid of `MDLocalVariable::getInlinedAt()`
entirely (PR22778), but I'll let these assertions bake for a while
first.

If you have an out-of-tree backend that just broke, you're probably
attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one
you want is the location that was attached to the corresponding
`@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with.

llvm-svn: 234038

show more ...

SelectionDAG: Use specialized metadata nodes in EmitFuncArgumentDbgValue(), NFC

Use `MDLocalVariable` and `MDExpression` directly for the arguments of
`EmitFuncArgumentDbgValue()` to simplify a foll

SelectionDAG: Use specialized metadata nodes in EmitFuncArgumentDbgValue(), NFC

Use `MDLocalVariable` and `MDExpression` directly for the arguments of
`EmitFuncArgumentDbgValue()` to simplify a follow-up patch.

llvm-svn: 234026

show more ...

[WinEH] Generate .xdata for catch handlers

This lets us catch exceptions in simple cases.

N.B. Things that do not work include (but are not limited to):
- Throwing from within a catch handler.
- Ca

[WinEH] Generate .xdata for catch handlers

This lets us catch exceptions in simple cases.

N.B. Things that do not work include (but are not limited to):
- Throwing from within a catch handler.
- Catching an object with a named catch parameter.
- 'CatchHigh' is fictitious, we aren't sure of its purpose.
- We aren't entirely efficient with regards to the number of EH states
that we generate.
- IP-to-State tables are sensitive to the order of emission.

llvm-svn: 233767

show more ...

[WinEH] Run cleanup handlers when an exception is thrown

Generate tables in the .xdata section representing what actions to take
when an exception is thrown. This currently fills in state for
clean

[WinEH] Run cleanup handlers when an exception is thrown

Generate tables in the .xdata section representing what actions to take
when an exception is thrown. This currently fills in state for
cleanups, catch handlers are still unfinished.

llvm-svn: 233636

show more ...