[AMDGPU] Enable unaligned scratch accesses (#110219)

This allows us to emit wide generic and scratch memory accesses when we
do not have alignment information. In cases where accesses happen to be

[AMDGPU] Enable unaligned scratch accesses (#110219)

This allows us to emit wide generic and scratch memory accesses when we
do not have alignment information. In cases where accesses happen to be
properly aligned or where generic accesses do not go to scratch memory,
this improves performance of the generated code by a factor of up to 16x
and reduces code size, especially when lowering memcpy and memmove
intrinsics.

Also: Make the use of the FeatureUnalignedScratchAccess feature more
consistent: FeatureUnalignedScratchAccess and EnableFlatScratch are now
orthogonal, whereas, before, code assumed that the latter implies the
former at some places.

Part of SWDEV-455845.

show more ...

[AMDGPU][DAG] Enable ganging up of memcpy loads/stores for AMDGPU (#96185)

In the SelectionDAG lowering of the memcpy intrinsic, this optimization
introduces additional chains between fixed-size gr

[AMDGPU][DAG] Enable ganging up of memcpy loads/stores for AMDGPU (#96185)

In the SelectionDAG lowering of the memcpy intrinsic, this optimization
introduces additional chains between fixed-size groups of loads and the
corresponding stores. While initially introduced to ensure that wider
load/store-pair instructions are generated on AArch64, this optimization
also improves code generation for AMDGPU: Ganged loads are scheduled
into a clause; stores only await completion of their corresponding load.

The chosen value of 16 performed good in microbenchmarks, values of 8,
32, or 64 would perform similarly.
The testcase updates are autogenerated by
utils/update_llc_test_checks.py.

See also:
- PR introducing this optimization: https://reviews.llvm.org/D46477

Part of SWDEV-455845.

show more ...