[MLIR][Affine] Rewrite fusion helper hasNonAffineUsersOnPath for efficiency (#115588)

The hasNonAffineUsersOnPath utility used during fusion was terribly
inefficient in its approach. Rewrite it eff

[MLIR][Affine] Rewrite fusion helper hasNonAffineUsersOnPath for efficiency (#115588)

The hasNonAffineUsersOnPath utility used during fusion was terribly
inefficient in its approach. Rewrite it efficiently to simply work based
on use lists (sparse) instead of having to traverse all nodes of an MDG
repeatedly and all operands of all ops of each node in the relevant
range.

On large models (with 10s of thousands of loop nests), this reduces
fusion pass time by nearly 2x (cutting down several tens of seconds).

show more ...

[mlir] Use llvm::unique (NFC) (#96415)

[mlir] fix affine-loop-fusion crash (#76351)

If `user` not lies in `Region` `findAncestorOpInRegion` will return
`nullptr`.

Fixes https://github.com/llvm/llvm-project/issues/76281.

Generalize affine fusion to work at all depths and inside other region-holding ops (#72288)

Generalize affine fusion to work at any inner depth; fusing loops inside
other
affine.for or even inside

Generalize affine fusion to work at all depths and inside other region-holding ops (#72288)

Generalize affine fusion to work at any inner depth; fusing loops inside
other
affine.for or even inside scf.for or scf.while nests. Apply in post
order on
all affine nests on the pass' top-level operation.

Fix MDG init for blocks inside other affine nests.

Relax unnecessary requirement for unique vars during merge and align of
FlatLinearValueConstraints. There are several cases where
FlatLinearValueConstraints need to have duplicate Values for the
dimensions:
for eg. in dependence relation systems with source and destination
accesses
could have common loop IVs. `mergeAndAlign` can be done even in the
presence
of Values reappearing by simply aligning from left to right in that
order.

While at this, drop outdated comments; improve some debug messages.

show more ...

[MLIR][Affine] NFC. Move misplaced MDG init method (#71665)

MemRefDependenceGraph::init should have been in affine analysis utils
since MemRefDependenceGraph is part of the affine analysis library;

[MLIR][Affine] NFC. Move misplaced MDG init method (#71665)

MemRefDependenceGraph::init should have been in affine analysis utils
since MemRefDependenceGraph is part of the affine analysis library; its
move was missed. Move it. NFC.

show more ...

[MLIR][Affine] NFC. Fix stale comments and style in affine libraries (#71660)

Fix stale comments in affine libraries; fix clang-tidy warnings/style.
NFC.

fix unused variable warnings in conditionals

warning was updated in 92023b15099012a657da07ebf49dd7d94a260f84

[mlir] Move casting calls from methods to function calls

The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionali

[mlir] Move casting calls from methods to function calls

The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.

Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.

Caveats include:
- This clang-tidy script probably has more problems.
- This only touches C++ code, so nothing that is being generated.

Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443

Implementation:
This first patch was created with the following steps. The intention is
to only do automated changes at first, so I waste less time if it's
reverted, and so the first mass change is more clear as an example to
other teams that will need to follow similar steps.

Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
4. Some changes have been deleted for the following reasons:
- Some files had a variable also named cast
- Some files had not included a header file that defines the cast
functions
- Some files are definitions of the classes that have the casting
methods, so the code still refers to the method instead of the
function without adding a prefix or removing the method declaration
at the same time.

```
ninja -C $BUILD_DIR clang-tidy

run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix

rm -rf $BUILD_DIR/tools/mlir/**/*.inc

git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\
mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\
mlir/lib/**/IR/\
mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\
mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\
mlir/test/lib/Dialect/Test/TestTypes.cpp\
mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\
mlir/test/lib/Dialect/Test/TestAttributes.cpp\
mlir/unittests/TableGen/EnumsGenTest.cpp\
mlir/test/python/lib/PythonTestCAPI.cpp\
mlir/include/mlir/IR/
```

Differential Revision: https://reviews.llvm.org/D150123

show more ...

[mlir][Affine][NFC] Wrap dialect in "affine" namespace

This cleanup aligns the affine dialect with all the other dialects.

Differential Revision: https://reviews.llvm.org/D148687

[MLIR] NFC. Expose/move MDG out of Affine fusion into Affine Analysis

Move out MemRefDependenceGraph analysis structure out of LoopFusion into
the Affine Analysis library. This had been a long pendi

[MLIR] NFC. Expose/move MDG out of Affine fusion into Affine Analysis

Move out MemRefDependenceGraph analysis structure out of LoopFusion into
the Affine Analysis library. This had been a long pending TODO. Moving
MDG out allows its use in other affine passes as well as allows building
custom affine fusion passes downstream while reusing upstream fusion
utilties. The file LoopFusion.cpp had also become lengthy and this
change makes things more modular. This change is a pure NFC and is a
code movement.

NFC.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D147105

show more ...

[MLIR][Affine] Fix/improve affine sibling fusion

The sibling fusion profitability checks shouldn't rely on the presence
of a store op in the sibling. The reuse is between the loads.

Fixes issues ra

[MLIR][Affine] Fix/improve affine sibling fusion

The sibling fusion profitability checks shouldn't rely on the presence
of a store op in the sibling. The reuse is between the loads.

Fixes issues raised at https://discourse.llvm.org/t/understanding-the-affine-loop-fusion-pass/69452

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D146763

show more ...

[MLIR][Affine] Fix assumption on int type in memref elt size method

Fix assumption on memref element type being int/float in memref elt size
related method and affine data copy generate.

Fixes http

[MLIR][Affine] Fix assumption on int type in memref elt size method

Fix assumption on memref element type being int/float in memref elt size
related method and affine data copy generate.

Fixes https://github.com/llvm/llvm-project/issues/61310

Differential Revision: https://reviews.llvm.org/D146495

show more ...

[MLIR] Remove unnecessary Block argument on MemRefDependenceGraph::init

Remove unnecessary Block argument on MemRefDependenceGraph::init.
`block` is already a field on MDG.

Reviewed By: dcaballe

D

[MLIR] Remove unnecessary Block argument on MemRefDependenceGraph::init

Remove unnecessary Block argument on MemRefDependenceGraph::init.
`block` is already a field on MDG.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D142597

show more ...

NFC. Refactor affine fusion code for readability

Replace a couple of check instances with llvm::any_of (clang-tidy
warnings). Factor out "canCreatePrivateMemRef" and
"performFusionsIntoDest" into s

NFC. Refactor affine fusion code for readability

Replace a couple of check instances with llvm::any_of (clang-tidy
warnings). Factor out "canCreatePrivateMemRef" and
"performFusionsIntoDest" into separate methods to reduce the
length/indent of the containing methods. Add doc comments and debug messages.

Mark some of the methods that should have been const const.

NFC.

Reviewed By: vinayaka-polymage

Differential Revision: https://reviews.llvm.org/D142076

show more ...

[mlir] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".

This is pa

[mlir] Use std::optional instead of llvm::Optional (NFC)

This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[MLIR] Fix affine analysis methods for affine.parallel

Drop unnecessary bailout in checkMemRefAccessDependence in the presence of
surrounding affine.parallel ops. When the affine.parallel op was add

[MLIR] Fix affine analysis methods for affine.parallel

Drop unnecessary bailout in checkMemRefAccessDependence in the presence of
surrounding affine.parallel ops. When the affine.parallel op was added, affine
analysis methods weren't extended to account for it. Fix this and allow memref
dependence check to work in the presence of affine.parallel ops in the mix.

Rename isForInductionVar -> isAffineForInductionVar, getLoopIVs ->
getAffineForIVs to avoid confusion since that's what they were.

Differential Revision: https://reviews.llvm.org/D141254

show more ...

[MLIR][Affine] Make fusion helper check method significantly more efficient

The `hasDependencePath` method in affine fusion is quite inefficient as
it does a DFS on the complete graph for what is a

[MLIR][Affine] Make fusion helper check method significantly more efficient

The `hasDependencePath` method in affine fusion is quite inefficient as
it does a DFS on the complete graph for what is a small part of the
checks before fusion can be performed. Make this efficient by using the
fact that the nodes involved are all at the top-level of the same block.
With this change, for large graphs with about 10,000 nodes, the check
runs in a few seconds instead of not terminating even in a few hours.

This is NFC from a functionality standpoint; it only leads to an
improvement in pass running time on large IR.

Differential Revision: https://reviews.llvm.org/D140522

show more ...

[mlir] std::optional::value => operator*/operator->

value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBC

[mlir] std::optional::value => operator*/operator->

value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).

show more ...

[MLIR] Generalize affine fusion to work on `Block` instead of `FuncOp`

The affine fusion pass can actually work on the top-level of a `Block`
and doesn't require to be called on a `FuncOp`. Remove t

[MLIR] Generalize affine fusion to work on `Block` instead of `FuncOp`

The affine fusion pass can actually work on the top-level of a `Block`
and doesn't require to be called on a `FuncOp`. Remove this restriction
and generalize the pass to work on any `Block`. This allows fusion to be
performed, for example, on multiple blocks of a FuncOp or any
region-holding op like an scf.while, scf.if or even at an inner depth of
an affine.for or affine.if op. This generalization has no effect on
existing functionality. No changes to the fusion logic or its
transformational power were needed.

Update fusion pass to be a generic operation pass (instead of FuncOp
pass) and remove references and assumptions on the parent being a
FuncOp.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D139293

show more ...

[MLIR] Fix/check for aliases for escaping memrefs in affine fusion

Check for aliases for escaping memrefs check in affine fusion pass. Fix
`isEscapingMemRef` to handle unknown defining ops for the

[MLIR] Fix/check for aliases for escaping memrefs in affine fusion

Check for aliases for escaping memrefs check in affine fusion pass. Fix
`isEscapingMemRef` to handle unknown defining ops for the memref.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D139268

show more ...

NFC. Refactor/update some affine fusion pass code for readability

NFC. Refactor some affine fusion pass code for readability. Some of its
methods are too long. This is the first among some NFC chang

NFC. Refactor/update some affine fusion pass code for readability

NFC. Refactor some affine fusion pass code for readability. Some of its
methods are too long. This is the first among some NFC changes before new
features/related updates are posted. Add missing code comments at a couple of
places.

Differential Revision: https://reviews.llvm.org/D139255

show more ...

[Transforms] Use std::optional in LoopFusion.cpp (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-g

[Transforms] Use std::optional in LoopFusion.cpp (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[mlir] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of ma

[mlir] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[MLIR] NFC. Introduce mlir::hasEffect and refactor usages dialect util

Introduce mlir::hasEffect and refactor existing usage to use utility.
NFC.

Reviewed By: rriddle, mehdi_amini

Differential Rev

[MLIR] NFC. Introduce mlir::hasEffect and refactor usages dialect util

Introduce mlir::hasEffect and refactor existing usage to use utility.
NFC.

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D132117

show more ...

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow drop

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure.

Reviewed By: mehdi_amini, rriddle

Differential Review: https://reviews.llvm.org/D132838

show more ...