[aarch64][win] Update Called Globals info when updating Call Site info (#122762)

Fixes the "use after poison" issue introduced by #121516 (see
<https://github.com/llvm/llvm-project/pull/121516#issue

[aarch64][win] Update Called Globals info when updating Call Site info (#122762)

Fixes the "use after poison" issue introduced by #121516 (see
<https://github.com/llvm/llvm-project/pull/121516#issuecomment-2585912395>).

The root cause of this issue is that #121516 introduced "Called Global"
information for call instructions modeling how "Call Site" info is
stored in the machine function, HOWEVER it didn't copy the
copy/move/erase operations for call site information.

The fix is to rename and update the existing copy/move/erase functions
so they also take care of Called Global info.

show more ...

[AMDGPU] Add target hook to isGlobalMemoryObject (#112781)

We want special handing for IGLP instructions in the scheduler but they
should still be treated like they have side effects by other passe

[AMDGPU] Add target hook to isGlobalMemoryObject (#112781)

We want special handing for IGLP instructions in the scheduler but they
should still be treated like they have side effects by other passes. Add
a target hook to the ScheduleDAGInstrs DAG builder so that we have more
control over this.

show more ...

[TII][RISCV] Add renamable bit to copyPhysReg (#91179)

The renamable flag is useful during MachineCopyPropagation but renamable
flag will be dropped after lowerCopy in some case.

This patch intr

[TII][RISCV] Add renamable bit to copyPhysReg (#91179)

The renamable flag is useful during MachineCopyPropagation but renamable
flag will be dropped after lowerCopy in some case.

This patch introduces extra arguments to pass the renamable flag to
copyPhysReg.

show more ...

MachineOutliner: Use PM to query MachineModuleInfo (#99688)

Avoid getting this from the MachineFunction

[llvm][CodeGen] Avoid 'raw_string_ostream::str' (NFC) (#97318)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference

[llvm][CodeGen] Avoid 'raw_string_ostream::str' (NFC) (#97318)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.

Work towards TODO comment to remove `raw_string_ostream::str()`.

show more ...

[CodeGen][TII] Allow reassociation on custom operand indices (#88306)

This opens up a door for reusing reassociation optimizations on
target-specific binary operations with non-standard operand lis

[CodeGen][TII] Allow reassociation on custom operand indices (#88306)

This opens up a door for reusing reassociation optimizations on
target-specific binary operations with non-standard operand list.

This is effectively a NFC.

show more ...

[clang][llvm] Remove "implicit-section-name" attribute (#87906)

D33412/D33413 introduced this to support a clang pragma to set section
names for a symbol depending on if it would be placed in
bss/

[clang][llvm] Remove "implicit-section-name" attribute (#87906)

D33412/D33413 introduced this to support a clang pragma to set section
names for a symbol depending on if it would be placed in
bss/data/rodata/text, which may not be known until the backend. However,
for text we know that only functions will go there, so just directly set
the section in clang instead of going through a completely separate
attribute.

Autoupgrade the "implicit-section-name" attribute to directly setting
the section on a Fuction.

show more ...

[MachineCombiner][NFC] Split target-dependent patterns

We split target-dependent MachineCombiner patterns into their target
folder.

This makes MachineCombiner much more target-independent.

Reviewe

[MachineCombiner][NFC] Split target-dependent patterns

We split target-dependent MachineCombiner patterns into their target
folder.

This makes MachineCombiner much more target-independent.

Reviewers:
davemgreen, asavonic, rotateright, RKSimon, lukel97, LuoYuanke, topperc, mshockwave, asi-sc

Reviewed By: topperc, mshockwave

Pull Request: https://github.com/llvm/llvm-project/pull/87991

show more ...

[CodeGen] Use LocationSize for MMO getSize (#84751)

This is part of #70452 that changes the type used for the external
interface of MMO to LocationSize as opposed to uint64_t. This means the
const

[CodeGen] Use LocationSize for MMO getSize (#84751)

This is part of #70452 that changes the type used for the external
interface of MMO to LocationSize as opposed to uint64_t. This means the
constructors take LocationSize, and convert ~UINT64_C(0) to
LocationSize::beforeOrAfter(). The getSize methods return a
LocationSize.

This allows us to be more precise with unknown sizes, not accidentally
treating them as unsigned values, and in the future should allow us to
add proper scalable vector support but none of that is included in this
patch. It should mostly be an NFC.

Global ISel is still expected to use the underlying LLT as it needs, and
are not expected to see unknown sizes for generic operations. Most of
the changes are hopefully fairly mechanical, adding a lot of getValue()
calls and protecting them with hasValue() where needed.

show more ...

[Codegen] Make Width in getMemOperandsWithOffsetWidth a LocationSize. (#83875)

This is another part of #70452 which makes getMemOperandsWithOffsetWidth
use a LocationSize for Width, as opposed to t

[Codegen] Make Width in getMemOperandsWithOffsetWidth a LocationSize. (#83875)

This is another part of #70452 which makes getMemOperandsWithOffsetWidth
use a LocationSize for Width, as opposed to the unsigned it currently
uses. The advantages on it's own are not super high if
getMemOperandsWithOffsetWidth usually uses known sizes, but if the
values can come from an MMO it can help be more accurate in case they
are Unknown (and in the future, scalable).

show more ...

TargetInstrInfo, TargetSchedule: fix non-NFC parts of 9468de4 (#74338)

Follow up on a post-commit review of 9468de4 (TargetInstrInfo: make
getOperandLatency return optional (NFC)) by Bjorn Petterss

TargetInstrInfo, TargetSchedule: fix non-NFC parts of 9468de4 (#74338)

Follow up on a post-commit review of 9468de4 (TargetInstrInfo: make
getOperandLatency return optional (NFC)) by Bjorn Pettersson to fix a
couple of things that are not NFC:

- std::optional<T>::operator<= returns true if the first operand is a
std::nullopt and second operand is T. Fix a couple of places where we
assumed it would return false.
- In TargetSchedule, computeInstrCost could take another codepath,
returning InstrLatency instead of DefaultDefLatency. Fix one instance
not accounting for this behavior.

show more ...

[TargetInstrInfo] update INLINEASM memoperands once (#74135)

In commit b05335989239 ("[X86InstrInfo] support memfold on spillable
inline asm
(#70832)"), I had a last minute fix to update the memoper

[TargetInstrInfo] update INLINEASM memoperands once (#74135)

In commit b05335989239 ("[X86InstrInfo] support memfold on spillable
inline asm
(#70832)"), I had a last minute fix to update the memoperands. I
originally
did this in the parent foldInlineAsmMemOperand call, updated the mir
test via
update_mir_test_checks.py, but then decided to move it to the child call
of
foldInlineAsmMemOperand.

But I forgot to rerun update_mir_test_checks.py. That last minute change
caused
the same memoperand to be added twice when recursion occurred (for tied
operands). I happened to get lucky that trailing content omitted from
the
CHECK line doesn't result in test failure.

But rerunning update_mir_test_checks.py on the mir test added in that
commit
produces updated output. This is resulting in updates to the test that:
1. conflate additions to the test in child commits with simply updating
the
test as it should have been when first committed.
2. look wrong because the same memoperand is specified twice (we don't
deduplicate memoperands when added). Example:

INLINEASM ... :: (load (s32) from %stack.0) (load (s32) from %stack.0)

Fix the bug, so that in child commits, we don't have additional
unrelated test
changes (which would be wrong anyways) from simply running
update_mir_test_checks.py.

Link: #20571

show more ...

Fix MSVC signed/unsigned mismatch warning. NFC.

TargetInstrInfo: make getOperandLatency return optional (NFC) (#73769)

getOperandLatency has the following behavior: it returns -1 as a special
value, negative numbers other than -1 on some target-

TargetInstrInfo: make getOperandLatency return optional (NFC) (#73769)

getOperandLatency has the following behavior: it returns -1 as a special
value, negative numbers other than -1 on some target-specific overrides,
or a valid non-negative latency. This behavior can be surprising, as
some callers do arithmetic on these negative values. Change the
interface of getOperandLatency to return a std::optional<unsigned> to
prevent surprises in callers. While at it, change the interface of
getInstrLatency to return unsigned instead of int.

This change was inspired by a refactoring in
TargetSchedModel::computeOperandLatency.

show more ...

[X86InstrInfo] support memfold on spillable inline asm (#70832)

This enables -regalloc=greedy to memfold spillable inline asm
MachineOperands.

Because no instruction selection framework marks Machi

[X86InstrInfo] support memfold on spillable inline asm (#70832)

This enables -regalloc=greedy to memfold spillable inline asm
MachineOperands.

Because no instruction selection framework marks MachineOperands as
spillable, no language frontend can observe functional changes from this
patch. That will change once instruction selection frameworks are
updated.

Link: https://github.com/llvm/llvm-project/issues/20571

show more ...

[AArch64] Use the same fast math preservation for MachineCombiner reassociation as X86/PowerPC/RISCV. (#72820)

Don't blindly copy the original flags from the pre-reassociated
instrutions.
This cop

[AArch64] Use the same fast math preservation for MachineCombiner reassociation as X86/PowerPC/RISCV. (#72820)

Don't blindly copy the original flags from the pre-reassociated
instrutions.
This copied the integer poison flags which are not safe to preserve
after reassociation.

For the FP flags, I think we should only keep the intersection of
the flags. Override setSpecialOperandAttr to do this.

Fixes #72777.

show more ...

reapply "[TargetInstrInfo] enable foldMemoryOperand for InlineAsm (#70743)" (#72910)


This reverts commit 42204c94ba9fcb0b4b1335e648ce140a3eef8a9d.

It was accidentally backed out.

#20571
#70743

Revert "[TargetInstrInfo] enable foldMemoryOperand for InlineAsm (#70743)"

This reverts commit 99ee2db198d86f685bcb07a1495a7115ffc31d7e.

It's causing ICEs in the ARM tests. See the comment here:

h

Revert "[TargetInstrInfo] enable foldMemoryOperand for InlineAsm (#70743)"

This reverts commit 99ee2db198d86f685bcb07a1495a7115ffc31d7e.

It's causing ICEs in the ARM tests. See the comment here:

https://github.com/llvm/llvm-project/commit/99ee2db198d86f685bcb07a1495a7115ffc31d7e

show more ...

[TargetInstrInfo] enable foldMemoryOperand for InlineAsm (#70743)


foldMemoryOperand looks at pairs of instructions (generally a load to
virt reg then use of the virtreg, or def of a virtreg then a

[TargetInstrInfo] enable foldMemoryOperand for InlineAsm (#70743)


foldMemoryOperand looks at pairs of instructions (generally a load to
virt reg then use of the virtreg, or def of a virtreg then a store) and
attempts to combine them. This can reduce register pressure.

A prior commit added the ability to mark such a MachineOperand as
foldable. In terms of INLINEASM, this means that "rm" was used (rather
than just "r") to denote that the INLINEASM may use a memory operand
rather than a register operand. This effectively undoes decisions made
by the instruction selection framework. Callers will be added in the
register allocation frameworks. This has been tested with all of the
above (which will come as follow up patches).

Thanks to @topperc who suggested this at last years LLVM US Dev Meeting
and @qcolombet who confirmed this was the right approach.

Link: https://github.com/llvm/llvm-project/issues/20571

show more ...

[InlineAsm] Steal a bit to denote a register is foldable (#70738)


When using the inline asm constraint string "rm" (or "g"), we generally
would like the compiler to choose "r", but it is permitted

[InlineAsm] Steal a bit to denote a register is foldable (#70738)


When using the inline asm constraint string "rm" (or "g"), we generally
would like the compiler to choose "r", but it is permitted to choose "m"
if there's register pressure. This is distinct from "r" in which the
register is not permitted to be spilled to the stack.

The decision of which to use must be made at some point. Currently, the
instruction selection frameworks (ISELs) make the choice, and the
register allocators had better be able to handle the result.

Steal a bit from Storage when using register operands to disambiguate
between the two cases. Add helpers/getters/setters, and print in MIR
when such a register is foldable.

The getter will later be used by the register allocation frameworks (and
asserted by the ISELs) while the setters will be used by the instruction
selection frameworks.

Link: https://github.com/llvm/llvm-project/issues/20571

show more ...

[BasicBlockSections] Apply path cloning with -basic-block-sections. (#68860)

https://github.com/llvm/llvm-project/commit/28b912687900bc0a67cd61c374fce296b09963c4
introduced the path cloning format

[BasicBlockSections] Apply path cloning with -basic-block-sections. (#68860)

https://github.com/llvm/llvm-project/commit/28b912687900bc0a67cd61c374fce296b09963c4
introduced the path cloning format in the basic-block-sections profile.

This PR validates and applies path clonings.
A path cloning is valid if all of these conditions hold:
1. All bb ids in the path are mapped to existing blocks.
2. Each two consecutive bb ids in the path have a successor relationship
in the CFG.
3. The path does not include a block with indirect branches, except
possibly as the last block.

Applying a path cloning involves cloning all blocks in the path (except
the first one) and setting up their branches.
Once all clonings are applied, the cluster information is used to guide
block layout in the modified function.

show more ...

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66264)

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

This reverts commit ee643b706be2b6bef9980b25cc9cc988dab94bb5.

Fi

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66264)

reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

This reverts commit ee643b706be2b6bef9980b25cc9cc988dab94bb5.

Fix up build failures in targets I missed in #66003

Kept as 3 commits for reviewers to see better what's changed. Will
squash when
merging.

- reland [InlineAsm] wrap ConstraintCode in enum class NFC (#66003)
- fix all the targets I missed in #66003
- fix off by one found by llvm/test/CodeGen/SystemZ/inline-asm-addr.ll

show more ...

Revert "[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)"

This reverts commit 2ca4d136124d151216aac77a0403dcb5c5835bcd.

Also revert the followup, "[InlineAsm] fix botched merge conflict r

Revert "[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)"

This reverts commit 2ca4d136124d151216aac77a0403dcb5c5835bcd.

Also revert the followup, "[InlineAsm] fix botched merge conflict resolution"

This reverts commit 8b9bf3a9f715ee5dce96eb1194441850c3663da1.

There were SystemZ and Mips build errors, too many to fix forward.

show more ...

[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

Similar to
commit 2fad6e69851e ("[InlineAsm] wrap Kind in enum class NFC")

Fix the TODOs added in
commit 93bd428742f9 ("[InlineAsm] refact

[InlineAsm] wrap ConstraintCode in enum class NFC (#66003)

Similar to
commit 2fad6e69851e ("[InlineAsm] wrap Kind in enum class NFC")

Fix the TODOs added in
commit 93bd428742f9 ("[InlineAsm] refactor InlineAsm class NFC
(#65649)")

show more ...

[InlineAsm] refactor InlineAsm class NFC (#65649)

I would like to steal one of these bits to denote whether a kind may be
spilled by the register allocator or not, but I'm afraid to touch of any
thi

[InlineAsm] refactor InlineAsm class NFC (#65649)

I would like to steal one of these bits to denote whether a kind may be
spilled by the register allocator or not, but I'm afraid to touch of any
this code using bitwise operands.

Make flags a first class type using bitfields, rather than launder data
around via `unsigned`.

show more ...