Don't use Optional::hasValue (NFC)

[AMDGPU] Flush the vmcnt counter in loop preheaders when necessary

waitcnt vmcnt instructions are currently generated in loop bodies before using
values loaded outside of the loop. In some cases, it

[AMDGPU] Flush the vmcnt counter in loop preheaders when necessary

waitcnt vmcnt instructions are currently generated in loop bodies before using
values loaded outside of the loop. In some cases, it is better to flush the
vmcnt counter in a loop preheader before entering the loop body. This patch
detects these cases and generates waitcnt instructions to flush the counter.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D115747

show more ...

[machinesink] fix debug invariance issue

Do not include debug instructions when comparing block sizes with
thresholds.

Differential Revision: https://reviews.llvm.org/D127208

[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated comments

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-in

Cleanup codegen includes

This is a (fixed) recommit of https://reviews.llvm.org/D121169

after: 1061034926
before: 1063332844

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121681

show more ...

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https:/

Revert "Cleanup codegen includes"

This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https://reviews.llvm.org/D121169

show more ...

Cleanup codegen includes

after: 1061034926
before: 1063332844

Differential Revision: https://reviews.llvm.org/D121169

[CodeGen] Use = default (NFC)

Identified with modernize-use-equals-default

[NFC] Rename MachineFunction::deleteMachineInstr (coding style)

[CodeGen] Use range-based for loops (NFC)

[llvm] Use range-for loops (NFC)

[CodeGen] Use MachineInstr::operands (NFC)

[llvm] Use MachineBasicBlock::{successors,predecessors} (NFC)

[MIRParser] Add support for IsInlineAsmBrIndirectTarget

Print this basic block flag as inlineasm-br-indirect-target and parse
it. This allows you to write MIR test cases for INLINEASM_BR. The test
c

[MIRParser] Add support for IsInlineAsmBrIndirectTarget

Print this basic block flag as inlineasm-br-indirect-target and parse
it. This allows you to write MIR test cases for INLINEASM_BR. The test
case I added is one that I wanted to precommit anyway for D110834.

Differential Revision: https://reviews.llvm.org/D111291

show more ...

[CSSPGO] Undoing the concept of dangling pseudo probe

As a follow-up to https://reviews.llvm.org/D104129, I'm cleaning up the danling probe related code in both the compiler and llvm-profgen.

I'm s

[CSSPGO] Undoing the concept of dangling pseudo probe

As a follow-up to https://reviews.llvm.org/D104129, I'm cleaning up the danling probe related code in both the compiler and llvm-profgen.

I'm seeing a 5% size win for the pseudo_probe section for SPEC2017 and 10% for Ciner. Certain benchmark such as 602.gcc has a 20% size win. No obvious difference seen on build time for SPEC2017 and Cinder.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D104477

show more ...

MachineBasicBlock: add liveout iterator aware of which liveins are defined by the runtime.

Using this in RegAlloc fast reduces register pressure, and in some cases allows
x86 code to compile that wo

MachineBasicBlock: add liveout iterator aware of which liveins are defined by the runtime.

Using this in RegAlloc fast reduces register pressure, and in some cases allows
x86 code to compile that wouldn't before.

show more ...

[CSSPGO] Exclude pseudo probes from slot index

Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because o

[CSSPGO] Exclude pseudo probes from slot index

Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because of enlarged lifetime length with pseudo probe instructions. As a consequence, program could get different code generated w/ and w/o pseudo probes. I'm closing the gap by excluding pseudo probes from stack index and downstream register allocation related passes.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D100334

show more ...

[CSSPGO] Flip SkipPseudoOp to true for MIR APIs.

Flipping the default value of SkipPseudoOp to true for those MIR APIs to favor maximum performance. Note that certain spots like branch folding and M

[CSSPGO] Flip SkipPseudoOp to true for MIR APIs.

Flipping the default value of SkipPseudoOp to true for those MIR APIs to favor maximum performance. Note that certain spots like branch folding and MIR if-conversion is are disabled for better counts quality. For these two optimizations, this is a no-diff change.

The counts quality with SPEC2017 before/after this change is unchanged.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D100332

show more ...

[M68k][MIR](2/8) Changes in the target-independent MIR part

- Add new callback in `TargetInstrInfo` --
`isPCRelRegisterOperandLegal` -- to query whether pc-rel
register MachineOperand is legal

[M68k][MIR](2/8) Changes in the target-independent MIR part

- Add new callback in `TargetInstrInfo` --
`isPCRelRegisterOperandLegal` -- to query whether pc-rel
register MachineOperand is legal.
- Add new function to search DebugLoc in a reverse ordering

Authors: myhsu, m4yers, glaubitz

Differential Revision: https://reviews.llvm.org/D88386

show more ...

[CSSPGO] Unblocking optimizations by dangling pseudo probes.

This change fixes a couple places where the pseudo probe intrinsic blocks optimizations because they are not naturally removable. To unbl

[CSSPGO] Unblocking optimizations by dangling pseudo probes.

This change fixes a couple places where the pseudo probe intrinsic blocks optimizations because they are not naturally removable. To unblock those optimizations, the blocking pseudo probes are moved out of the original blocks and tagged dangling, instead of allowing pseudo probes to be literally removed. The reason is that when the original block is removed, we won't be able to sample it. Instead of assigning it a zero weight, moving all its pseudo probes into another block and marking them dangling should allow the counts inference a chance to assign them a more reasonable weight. We have not seen counts quality degradation from our experiments.

The optimizations being unblocked are:

1. Removing conditional probes for if-converted branches. Conditional probes are tagged dangling when their homing branch arms are folded so that they will not be over-counted.
2. Unblocking jump threading from removing empty blocks. Pseudo probe prevents jump threading from removing logically empty blocks that only has one unconditional jump instructions.
3. Unblocking SimplifyCFG and MIR tail duplicate to thread empty blocks and blocks with redundant branch checks.

Since dangling probes are logically deleted, they should not consume any samples in LTO postLink. This can be achieved by setting their distribution factors to zero when dangled.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D97481

show more ...

[CodeGen] Use range-based for loops (NFC)

Add ehcont section support

In the future Windows will enable Control-flow Enforcement Technology (CET aka shadow stacks). To protect the path where the context is updated during exception handling,

Add ehcont section support

In the future Windows will enable Control-flow Enforcement Technology (CET aka shadow stacks). To protect the path where the context is updated during exception handling, the binary is required to enumerate valid unwind entrypoints in a dedicated section which is validated when the context is being set during exception handling.

This change allows llvm to generate the section that contains the appropriate symbol references in the form expected by the msvc linker.

This feature is enabled through a new module flag, ehcontguard, which was modelled on the cfguard flag.

The change includes a test that when the module flag is enabled the section is correctly generated.

The set of exception continuation information includes returns from exceptional control flow (catchret in llvm).

In order to collect catchret we:
1) Includes an additional flag on machine basic blocks to indicate that the given block is the target of a catchret operation,
2) Introduces a new machine function pass to insert and collect symbols at the start of each block, and
3) Combines these targets with the other EHCont targets that were already being collected.

Change originally authored by Daniel Frampton <dframpto@microsoft.com>

For more details, see MSVC documentation for `/guard:ehcont`
https://docs.microsoft.com/en-us/cpp/build/reference/guard-enable-eh-continuation-metadata

Reviewed By: pengfei

Differential Revision: https://reviews.llvm.org/D94835

show more ...

[StringExtras] Rename SubsequentDelim to ListSeparator

This patch renames SubsequentDelim to ListSeparator to clarify the
purpose of the class.

Differential Revision: https://reviews.llvm.org/D94649

[SlotIndexes] Fix and simplify basic block splitting

Remove the InsertionPoint argument from SlotIndexes::insertMBBInMaps
because it was confusing: what does it mean to insert a new block
between tw

[SlotIndexes] Fix and simplify basic block splitting

Remove the InsertionPoint argument from SlotIndexes::insertMBBInMaps
because it was confusing: what does it mean to insert a new block
between two instructions, in the middle of an existing block?

Instead, support the case that MachineBasicBlock::splitAt really needs,
where the new block contains some instructions that are already in the
maps because they have been moved there from the tail of the previous
block.

In all other use cases the new block is empty.

Based on work by Carl Ritson!

Differential Revision: https://reviews.llvm.org/D94311

show more ...

[StringExtras] Add a helper class for comma-separated lists

This patch introduces a helper class SubsequentDelim to simplify loops
that generate a comma-separated lists.

For example, consider the f

[StringExtras] Add a helper class for comma-separated lists

This patch introduces a helper class SubsequentDelim to simplify loops
that generate a comma-separated lists.

For example, consider the following loop, taken from
llvm/lib/CodeGen/MachineBasicBlock.cpp:

for (auto I = pred_begin(), E = pred_end(); I != E; ++I) {
if (I != pred_begin())
OS << ", ";
OS << printMBBReference(**I);
}

The new class allows us to rewrite the loop as:

SubsequentDelim SD;
for (auto I = pred_begin(), E = pred_end(); I != E; ++I)
OS << SD << printMBBReference(**I);

where SD evaluates to the empty string for the first time and ", " for
subsequent iterations.

Unlike interleaveComma, defined in llvm/include/llvm/ADT/STLExtras.h,
SubsequentDelim can accommodate a wider variety of loops, including:

- those that conditionally skip certain items,
- those that need iterators to call getSuccProbability(I), and
- those that iterate over integer ranges.

As an example, this patch cleans up MachineBasicBlock::print.

Differential Revision: https://reviews.llvm.org/D94377

show more ...