[AArch64] SME implementation for agnostic-ZA functions (#120150)

This implements the lowering of calls from agnostic-ZA functions to
non-agnostic-ZA functions, using the ABI routines
`__arm_sme_st

[AArch64] SME implementation for agnostic-ZA functions (#120150)

This implements the lowering of calls from agnostic-ZA functions to
non-agnostic-ZA functions, using the ABI routines
`__arm_sme_state_size`, `__arm_sme_save` and `__arm_sme_restore`.

This implements the proposal described in the following PRs:
* https://github.com/ARM-software/acle/pull/336
* https://github.com/ARM-software/abi-aa/pull/264

show more ...

[AArch64][SME] Allow memory operations lowering to custom SME functions. (#79263)

This change allows to lower memcpy, memset, memmove to custom SME
version provided by LibRT.

[AArch64] Replace LLVM IR function attributes for PSTATE.ZA. (#79166)

Since https://github.com/ARM-software/acle/pull/276 the ACLE
defines attributes to better describe the use of a given SME state

[AArch64] Replace LLVM IR function attributes for PSTATE.ZA. (#79166)

Since https://github.com/ARM-software/acle/pull/276 the ACLE
defines attributes to better describe the use of a given SME state.

Previously the attributes merely described the possibility of it being
'shared' or 'preserved', whereas the new attributes have more semantics
and also describe how the data flows through the program.

For ZT0 we already had to add new LLVM IR attributes:
* aarch64_new_zt0
* aarch64_in_zt0
* aarch64_out_zt0
* aarch64_inout_zt0
* aarch64_preserves_zt0

We have now done the same for ZA, such that we add:
* aarch64_new_za (previously `aarch64_pstate_za_new`)
* aarch64_in_za (more specific variation of `aarch64_pstate_za_shared`)
* aarch64_out_za (more specific variation of `aarch64_pstate_za_shared`)
* aarch64_inout_za (more specific variation of
`aarch64_pstate_za_shared`)
* aarch64_preserves_za (previously `aarch64_pstate_za_shared,
aarch64_pstate_za_preserved`)

This explicitly removes 'pstate' from the name, because with SME2 and
the new ACLE attributes there is a difference between "sharing ZA"
(sharing
the ZA matrix register with the caller) and "sharing PSTATE.ZA" (sharing
either the ZA or ZT0 register, both part of PSTATE.ZA with the caller).

show more ...

[AArch64][SME2] Extend SMEABIPass to handle functions with new ZT0 state (#78848)

updateNewZAFunctions is extended to generate the following on entry to a
function with either the "aarch64_pstate_z

[AArch64][SME2] Extend SMEABIPass to handle functions with new ZT0 state (#78848)

updateNewZAFunctions is extended to generate the following on entry to a
function with either the "aarch64_pstate_za_new" or "arm_new_zt0"
attribute:
- Private-ZA interface: commit any active lazy-saves & enable PSTATE.ZA.
- "aarch64_pstate_za_new": zero ZA.
- "arm_new_zt0": zero ZT0.

Additionally, PSTATE.ZA should disabled before returning if the function
has a private-ZA interface.

show more ...

[AArch64] NFC: Simplify discombobulating 'requiresSMChange' interface (#78703)

Having it return a `std::optional<bool>` is unnecessarily confusing.
This patch changes it to a simple 'bool'.

This

[AArch64] NFC: Simplify discombobulating 'requiresSMChange' interface (#78703)

Having it return a `std::optional<bool>` is unnecessarily confusing.
This patch changes it to a simple 'bool'.

This patch also removes the 'BodyOverridesInterface' operand because
there is only a single use for this which is easily rewritten.

show more ...

[AArch64][SME2] Add ZT0 attributes to SMEAttrs (#77607)

This patch extends SMEAttrs to interpret the following new attributes,
which are mutually exclusive and apply to SME2 only:
- aarch64_sme_

[AArch64][SME2] Add ZT0 attributes to SMEAttrs (#77607)

This patch extends SMEAttrs to interpret the following new attributes,
which are mutually exclusive and apply to SME2 only:
- aarch64_sme_zt0_in (ZT0_In)
- aarch64_sme_zt0_out (ZT0_Out)
- aarch64_sme_zt0_inout (ZT0_InOut)
- aarch64_sme_zt0_new (ZT0_New)
- aarch64_sme_zt0_preserved (ZT0_Preserved)

ZT0_In, ZT0_Out, ZT0_InOut & ZT0_Preserved are all considered to share
ZT0. These attributes will be required by later patches to determine
if ZT0 should be preserved around function calls, or cleared on entry
to the function.

show more ...

[AArch64][SME] Enable TPIDR2 lazy-save for za_preserved

This change makes callees with the __arm_preserves_za
type attribute comply with the dormant state requirements
when it's caller has the __arm

[AArch64][SME] Enable TPIDR2 lazy-save for za_preserved

This change makes callees with the __arm_preserves_za
type attribute comply with the dormant state requirements
when it's caller has the __arm_shared_za type attribute.
Several external SME functions also do not need to lazy
save.

https://github.com/ARM-software/abi-aa/blob/5e67092434b50c04f8ad178a9c272ce3c6ada7fd/aapcs64/aapcs64.rst?plain=1#L1381

Differential Revision: https://reviews.llvm.org/D159186

show more ...

[AArch64][SME] NFC: Rename hasNewZAInterface to hasNewZABody.

__arm_new_za is a declaration attribution, not a type attribute,
and is therefore not part of the interface of a function.

[ADT] Don't including None.h (NFC)

These source files no longer use None, so they do not need to include
None.h.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://

[ADT] Don't including None.h (NFC)

These source files no longer use None, so they do not need to include
None.h.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[Target] llvm::Optional => std::optional

The updated functions are mostly internal with a few exceptions (virtual functions in
TargetInstrInfo.h, TargetRegisterInfo.h).
To minimize changes to LLVMCo

[Target] llvm::Optional => std::optional

The updated functions are mostly internal with a few exceptions (virtual functions in
TargetInstrInfo.h, TargetRegisterInfo.h).
To minimize changes to LLVMCodeGen, GlobalISel files are skipped.

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[Target] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of

[Target] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[AArch64][SME] Add utility class for handling SME attributes.

This patch adds a utility class that will be used in subsequent patches
for parsing the function/callsite attributes and determining whe

[AArch64][SME] Add utility class for handling SME attributes.

This patch adds a utility class that will be used in subsequent patches
for parsing the function/callsite attributes and determining whether
changes to PSTATE.SM are needed, or whether a lazy-save mechanism is
required.

It also implements some of the restrictions on the SME attributes
in the IR Verifier pass.

More details about the SME attributes and design can be found
in D131562.

Reviewed By: david-arm, aemerson

Differential Revision: https://reviews.llvm.org/D131570

show more ...