[CodeGen] [ARM] Make RISC-V Init Undef Pass Target Independent and add support for the ARM Architecture. (#77770)

When using Greedy Register Allocation, there are times where
early-clobber values

[CodeGen] [ARM] Make RISC-V Init Undef Pass Target Independent and add support for the ARM Architecture. (#77770)

When using Greedy Register Allocation, there are times where
early-clobber values are ignored, and assigned the same register. This
is illeagal behaviour for these intructions. To get around this, using
Pseudo instructions for early-clobber registers gives them a definition
and allows Greedy to assign them to a different register. This then
meets the ARM Architecture Reference Manual and matches the defined
behaviour.

This patch takes the existing RISC-V patch and makes it target
independent, then adds support for the ARM Architecture. Doing this will
ensure early-clobber restraints are followed when using the ARM
Architecture. Making the pass target independent will also open up
possibility that support other architectures can be added in the future.

show more ...

[WebAssembly] Demote PHIs in catchswitch BB only (#81570)

`DemoteCatchSwitchPHIOnly` option in `WinEHPrepare` pass was added in
https://github.com/llvm/llvm-project/commit/99d60e0dabcf20f4db683da83

[WebAssembly] Demote PHIs in catchswitch BB only (#81570)

`DemoteCatchSwitchPHIOnly` option in `WinEHPrepare` pass was added in
https://github.com/llvm/llvm-project/commit/99d60e0dabcf20f4db683da83cde905b7a1373de,
because Wasm EH uses `WinEHPrepare`, but it doesn't need to demote all
PHIs. PHIs in `catchswitch` BBs have to be removed (= demoted) because
`catchswitch`s are removed in ISel and `catchswitch` BBs are removed as
well, so they can't have other instructions.

But because Wasm EH doesn't use funclets, so PHIs in `catchpad` or
`cleanuppad` BBs don't need to be demoted. That was the reason
`DemoteCatchSwitchPHIOnly` option was added, in order not to demote more
instructions unnecessarily.

The problem is it should have been set to `true` for Wasm EH. (Its
default value is `false` for WinEH) And I mistakenly set it to `false`
and wasn't aware about this for more than 5 years. This was not the end
of the world; it just means we've been demoting more instructions than
we should, possibly huting code size. In practice I think it would've
had hardly any effect in real performance given that the occurrence of
PHIs in `catchpad` or `cleanuppad` BBs are not very frequent and many
people run other optimizers like Binaryen anyway.

show more ...

[SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (#74128)

Today `-split-machine-functions` and `-fbasic-block-sections={a

[SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (#74128)

Today `-split-machine-functions` and `-fbasic-block-sections={all,list}`
cannot be combined with `-basic-block-sections=labels` (the labels
option will be ignored).
The inconsistency comes from the way basic block address map -- the
underlying mechanism for basic block labels -- encodes basic block
addresses
(https://lists.llvm.org/pipermail/llvm-dev/2020-July/143512.html).
Specifically, basic block offsets are computed relative to the function
begin symbol. This relies on functions being contiguous which is not the
case for MFS and basic block section binaries. This means Propeller
cannot use binary profiles collected from these binaries, which limits
the applicability of Propeller for iterative optimization.

To make the `SHT_LLVM_BB_ADDR_MAP` feature work with basic block section
binaries, we propose modifying the encoding of this section as follows.

First let us review the current encoding which emits the address of each
function and its number of basic blocks, followed by basic block entries
for each basic block.

| | |
|--|--|
| Address of the function | Function Address |
| Number of basic blocks in this function | NumBlocks |
| BB entry 1
| BB entry 2
| ...
| BB entry #NumBlocks

To make this work for basic block sections, we treat each basic block
section similar to a function, except that basic block sections of the
same function must be encapsulated in the same structure so we can map
all of them to their single function.

We modify the encoding to first emit the number of basic block sections
(BB ranges) in the function. Then we emit the address map of each basic
block section section as before: the base address of the section, its
number of blocks, and BB entries for its basic block. The first section
in the BB address map is always the function entry section.
| | |
|--|--|
| Number of sections for this function | NumBBRanges |
| Section 1 begin address | BaseAddress[1] |
| Number of basic blocks in section 1 | NumBlocks[1] |
| BB entries for Section 1
|..................|
| Section #NumBBRanges begin address | BaseAddress[NumBBRanges] |
| Number of basic blocks in section #NumBBRanges |
NumBlocks[NumBBRanges] |
| BB entries for Section #NumBBRanges

The encoding of basic block entries remains as before with the minor
change that each basic block offset is now computed relative to the
begin symbol of its containing BB section.

This patch adds a new boolean codegen option `-basic-block-address-map`.
Correspondingly, the front-end flag `-fbasic-block-address-map` and LLD
flag `--lto-basic-block-address-map` are introduced.
Analogously, we add a new TargetOption field `BBAddrMap`. This means BB
address maps are either generated for all functions in the compiling
unit, or for none (depending on `TargetOptions::BBAddrMap`).

This patch keeps the functionality of the old
`-fbasic-block-sections=labels` option but does not remove it. A
subsequent patch will remove the obsolete option.

We refactor the `BasicBlockSections` pass by separating the BB address
map and BB sections handing to their own functions (named
`handleBBAddrMap` and `handleBBSections`). `handleBBSections` renumbers
basic blocks and places them in their assigned sections.
`handleBBAddrMap` is invoked after `handleBBSections` (if requested) and
only renumbers the blocks.
- New tests added:
- Two tests basic-block-address-map-with-basic-block-sections.ll and
basic-block-address-map-with-mfs.ll to exercise the combination of
`-basic-block-address-map` with `-basic-block-sections=list` and
'-split-machine-functions`.
- A driver sanity test for the `-fbasic-block-address-map` option
(basic-block-address-map.c).
- An LLD test for testing the `--lto-basic-block-address-map` option.
This reuses the LLVM IR from `lld/test/ELF/lto/basic-block-sections.ll`.
- Renamed and modified the two existing codegen tests for basic block
address map (`basic-block-sections-labels-functions-sections.ll` and
`basic-block-sections-labels.ll`)
- Removed `SHT_LLVM_BB_ADDR_MAP_V0` tests. Full deprecation of
`SHT_LLVM_BB_ADDR_MAP_V0` and `SHT_LLVM_BB_ADDR_MAP` version less than 2
will happen in a separate PR in a few months.

show more ...

[NewPM][CodeGen][llc] Add NPM support (#70922)

Add new pass manager support to `llc`. Users can use
`--passes=pass1,pass2...` to run mir passes, and use `--enable-new-pm`
to run default codegen pi

[NewPM][CodeGen][llc] Add NPM support (#70922)

Add new pass manager support to `llc`. Users can use
`--passes=pass1,pass2...` to run mir passes, and use `--enable-new-pm`
to run default codegen pipeline.
This patch is taken from [D83612](https://reviews.llvm.org/D83612), the
original author is @yuanfang-chen.

---------

Co-authored-by: Yuanfang Chen <455423+yuanfang-chen@users.noreply.github.com>

show more ...

Fix MFS warning format

WithColor::warning() does not append new line automatically.

Reland "[CodeGen] Support start/stop in CodeGenPassBuilder (#70912)" (#78570)

Unfortunately the legacy pass system can't recognize `no-op-module` and
`no-op-function` so it causes test failure in `

Reland "[CodeGen] Support start/stop in CodeGenPassBuilder (#70912)" (#78570)

Unfortunately the legacy pass system can't recognize `no-op-module` and
`no-op-function` so it causes test failure in `CodeGenTests`. Add a
workaround in function `PassInfo *getPassInfo(StringRef PassName)`,
`TargetPassConfig.cpp`.

show more ...

Revert "[CodeGen] Support start/stop in CodeGenPassBuilder" (#78567)

Reverts llvm/llvm-project#70912. This breaks some bazel tests.

[CodeGen] Support start/stop in CodeGenPassBuilder (#70912)

Add `-start/stop-before/after` support for CodeGenPassBuilder.
Part of #69879.

Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#77182)

Port CodeGenPrepare to new pass manager and dependency
BasicBlockSectionsProfileReader
Fixes: #75380

Co-authored-

Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#77182)

Port CodeGenPrepare to new pass manager and dependency
BasicBlockSectionsProfileReader
Fixes: #75380

Co-authored-by: Krishna-13-cyber <84722531+Krishna-13-cyber@users.noreply.github.com>

show more ...

Revert 4d7c5ad58467502fcbc433591edff40d8a4d697d "[NewPM] Update CodeGenPreparePass reference in CodeGenPassBuilder (#77054)"
Revert e0c554ad87d18dcbfcb9b6485d0da800ae1338d1 "Port CodeGenPrepare to ne

Revert 4d7c5ad58467502fcbc433591edff40d8a4d697d "[NewPM] Update CodeGenPreparePass reference in CodeGenPassBuilder (#77054)"
Revert e0c554ad87d18dcbfcb9b6485d0da800ae1338d1 "Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#75380)"

Revert #75380 and #77054 as they were breaking EXPENSIVE_CHECKS buildbots: https://lab.llvm.org/buildbot/#/builders/104

show more ...

Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#75380)

Port CodeGenPrepare to new pass manager and dependency
BasicBlockSectionsProfileReader
Fixes: #64560

Co-authored-

Port CodeGenPrepare to new pass manager (and BasicBlockSectionsProfil… (#75380)

Port CodeGenPrepare to new pass manager and dependency
BasicBlockSectionsProfileReader
Fixes: #64560

Co-authored-by: Krishna-13-cyber <84722531+Krishna-13-cyber@users.noreply.github.com>

show more ...

[SystemZ][z/OS] Add exception handling for XPLINK (#74638)

Adds emitting the exception table and the EH registers for XPLINK.

---------

Co-authored-by: Yusra Syeda <yusra.syeda@ibm.com>

[CodeGen][NewPM] Add necessary codegen options (#70904)

These options are used by `TargetPassConfig` to build CodeGen pass
pipeline, add them to `CGPassBuilderOption` so `CodeGenPassBuilder` can
u

[CodeGen][NewPM] Add necessary codegen options (#70904)

These options are used by `TargetPassConfig` to build CodeGen pass
pipeline, add them to `CGPassBuilderOption` so `CodeGenPassBuilder` can
use them. Currently not all options are added, but it is enough to build
a prototype of `CodeGenPassBuilder`. Part of #69879.

show more ...

[CodeGen] Port `ExpandMemCmp` to new pass manager (#74050)

[CodeGen][GC] Remove `GCInfoPrinter` (#75033)

This pass is broken and looks like no one uses it for the last 15+ years.

```c++
bool Printer::runOnFunction(Function &F) {
if (F.hasGC())
r

[CodeGen][GC] Remove `GCInfoPrinter` (#75033)

This pass is broken and looks like no one uses it for the last 15+ years.

```c++
bool Printer::runOnFunction(Function &F) {
if (F.hasGC())
return false;

GCFunctionInfo *FD = &getAnalysis<GCModuleInfo>().getFunctionInfo(F);
```
```c++
GCFunctionInfo &GCModuleInfo::getFunctionInfo(const Function &F) {
assert(!F.isDeclaration() && "Can only get GCFunctionInfo for a definition!");
assert(F.hasGC()); // Equivalent to `assert(false);` when called by `Printer::runOnFunction`
```

See also #74972.

show more ...

[BasicBlockSections] Apply path cloning with -basic-block-sections. (#68860)

https://github.com/llvm/llvm-project/commit/28b912687900bc0a67cd61c374fce296b09963c4
introduced the path cloning format

[BasicBlockSections] Apply path cloning with -basic-block-sections. (#68860)

https://github.com/llvm/llvm-project/commit/28b912687900bc0a67cd61c374fce296b09963c4
introduced the path cloning format in the basic-block-sections profile.

This PR validates and applies path clonings.
A path cloning is valid if all of these conditions hold:
1. All bb ids in the path are mapped to existing blocks.
2. Each two consecutive bb ids in the path have a successor relationship
in the CFG.
3. The path does not include a block with indirect branches, except
possibly as the last block.

Applying a path cloning involves cloning all blocks in the path (except
the first one) and setting up their branches.
Once all clonings are applied, the cluster information is used to guide
block layout in the modified function.

show more ...

Revert "[ARM] Always lower direct calls as direct when the outliner is enabled (#66434)"

This reverts commit 003bcad9a8b21e15e3786a52b1dafa844075ab84.

ARM folks say it regresses some of their bench

Revert "[ARM] Always lower direct calls as direct when the outliner is enabled (#66434)"

This reverts commit 003bcad9a8b21e15e3786a52b1dafa844075ab84.

ARM folks say it regresses some of their benchmarks:
https://github.com/llvm/llvm-project/pull/66434#issuecomment-1722424162

show more ...

[ARM] Always lower direct calls as direct when the outliner is enabled (#66434)

The indirect lowering hinders the outliner's ability to see that
sequences are in fact common, since the sequence sim

[ARM] Always lower direct calls as direct when the outliner is enabled (#66434)

The indirect lowering hinders the outliner's ability to see that
sequences are in fact common, since the sequence similarity is rendered
opaque by the register callee. The size savings from making them
indirect seems to be dwarfed by the outliner's savings from
de-duplication.

rdar://115178034
rdar://115459865

show more ...

[NFC][CodeGen] Change CodeGenOpt::Level/CodeGenFileType into enum classes (#66295)

This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future chang

[NFC][CodeGen] Change CodeGenOpt::Level/CodeGenFileType into enum classes (#66295)

This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.

This matches other nearby enums.

For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::

show more ...

Add a pass to garbage-collect empty basic blocks after code generation.

Propeller and pseudo-probes map profiles back to Machine IR via basic block addresses that are stored in metadata sections.
Em

Add a pass to garbage-collect empty basic blocks after code generation.

Propeller and pseudo-probes map profiles back to Machine IR via basic block addresses that are stored in metadata sections.
Empty basic blocks (basic blocks without real code) obfuscate the profile mapping because their addresses collide with their next basic blocks.
For instance, the fallthrough block of an empty block should always be adjacent to it. Otherwise, a completely unnecessary jump would be added.
This patch adds a MachineFunction pass named `GCEmptyBasicBlocks` which attempts to garbage-collect the empty blocks before the `BasicBlockSections` and pass.
This pass removes each empty basic block after redirecting its incoming edges to its fall-through block.
The garbage-collection is not complete. We keep the empty block in 4 cases:
1. The empty block is an exception handling pad.
2. The empty block has its address taken.
3. The empty block is the last block of the function and it has
predecessors.
4. The empty block is the only block of the function.
The first three cases are extremely rare in normal code (no cases for the clang binary). Removing the blocks under the first two cases requires modifying exception handling structures and operands of non-terminator instructions -- which is doable but not worth the additional complexity in the pass.

Reviewed By: tmsriram

Differential Revision: https://reviews.llvm.org/D107534

show more ...

[AArch64] Move branch relaxation after bbsection assignment

Because branch relaxation needs to factor in if branches target
a block in the same section or a different one, it needs to run
after the

[AArch64] Move branch relaxation after bbsection assignment

Because branch relaxation needs to factor in if branches target
a block in the same section or a different one, it needs to run
after the Basic Block Sections / Machine Function Splitting passes.

Because Jump table compression relies on block offsets remaining
fixed after the table is compressed, we must also move the JT
compression pass.

The only tests affected are ones enforcing just the ordering and
the a few that have basic block ids changed because RenumberBlocks
hasn't run yet.

Differential Revision: https://reviews.llvm.org/D153829

show more ...

[CodeGen] Part II of "Fine tune MachineFunctionSplitPass (MFS) for FSAFDO".

This CL adds a new discriminator pass. Also adds a new sample profile
loading pass when MFS is enabled.

Differential Revi

[CodeGen] Part II of "Fine tune MachineFunctionSplitPass (MFS) for FSAFDO".

This CL adds a new discriminator pass. Also adds a new sample profile
loading pass when MFS is enabled.

Differential Revision: https://reviews.llvm.org/D152577

show more ...

CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering

Expand large or unknown size memory intrinsics into loops in the
default lowering pipeline if the target doesn't have the corresponding

CodeGen: Expand memory intrinsics in PreISelIntrinsicLowering

Expand large or unknown size memory intrinsics into loops in the
default lowering pipeline if the target doesn't have the corresponding
libfunc. Previously AMDGPU had a custom pass which existed to call the
expansion utilities.

With a default no-libcall option, we can remove the libfunc checks in
LoopIdiomRecognize for these, which never made any sense. This also
provides a path to lifting the immarg restriction on
llvm.memcpy.inline.

There seems to be a bug where TLI reports functions as available if
you use -march and not -mtriple.

show more ...

[MachO] Disable atexit()-based lowering when LTO'ing kernel/kext code

The kernel and kext environments do not provide the `__cxa_atexit()`
function, so we can't use it for lowering global module des

[MachO] Disable atexit()-based lowering when LTO'ing kernel/kext code

The kernel and kext environments do not provide the `__cxa_atexit()`
function, so we can't use it for lowering global module destructors.

Unfortunately, just querying for "compiling for kernel/kext?" in the LTO
pipeline isn't possible (kernel/kext identifier isn't part of the triple
yet) so we need to pass down a CodeGen flag.

rdar://93536111

Differential Revision: https://reviews.llvm.org/D148967

show more ...

Remove -lower-global-dtors-via-cxa-atexit flag

Remove the `-lower-global-dtors-via-cxa-atexit` escape hatch introduced
in D121736 [1], which switched the default lowering of global
destructors on Ma

Remove -lower-global-dtors-via-cxa-atexit flag

Remove the `-lower-global-dtors-via-cxa-atexit` escape hatch introduced
in D121736 [1], which switched the default lowering of global
destructors on MachO to use `__cxa_atexit()` to avoid emitting
deprecated `__mod_term_func` sections.

I added this flag as an escape hatch in case the switch causes any
problems. We didn't discover any problems so now we can remove it.

[1] https://reviews.llvm.org/D121736

rdar://90277838

Differential Revision: https://reviews.llvm.org/D145715

show more ...