[mlir] Enable decoupling two kinds of greedy behavior. (#104649)

The greedy rewriter is used in many different flows and it has a lot of
convenience (work list management, debugging actions, tracin

[mlir] Enable decoupling two kinds of greedy behavior. (#104649)

The greedy rewriter is used in many different flows and it has a lot of
convenience (work list management, debugging actions, tracing, etc). But
it combines two kinds of greedy behavior 1) how ops are matched, 2)
folding wherever it can.

These are independent forms of greedy and leads to inefficiency. E.g.,
cases where one need to create different phases in lowering and is
required to applying patterns in specific order split across different
passes. Using the driver one ends up needlessly retrying folding/having
multiple rounds of folding attempts, where one final run would have
sufficed.

Of course folks can locally avoid this behavior by just building their
own, but this is also a common requested feature that folks keep on
working around locally in suboptimal ways.

For downstream users, there should be no behavioral change. Updating
from the deprecated should just be a find and replace (e.g., `find ./
-type f -exec sed -i
's|applyPatternsAndFoldGreedily|applyPatternsGreedily|g' {} \;` variety)
as the API arguments hasn't changed between the two.

show more ...

[mlir] Add bubbling patterns for non intersecting reshapes (#103401)

Refactored @Max191's PR https://github.com/llvm/llvm-project/pull/94637
to move it to `Tensor`

From the original PR
>This PR

[mlir] Add bubbling patterns for non intersecting reshapes (#103401)

Refactored @Max191's PR https://github.com/llvm/llvm-project/pull/94637
to move it to `Tensor`

From the original PR
>This PR adds fusion by expansion patterns to push a tensor.expand_shape
up through a tensor.collapse_shape with non-intersecting reassociations.
Sometimes parallel collapse_shape ops like this can block propagation of
expand_shape ops, so this allows them to pass through each other.

I'm not sure if I put the code/tests in the right places, so let me know
where those go if they aren't.

cc @MaheshRavishankar @hanhanW

---------

Co-authored-by: Max Dawkins <max.dawkins@gmail.com>

show more ...

[mlir] split transform interfaces into a separate library (#85221)

Transform interfaces are implemented, direction or via extensions, in
libraries belonging to multiple other dialects. Those dialec

[mlir] split transform interfaces into a separate library (#85221)

Transform interfaces are implemented, direction or via extensions, in
libraries belonging to multiple other dialects. Those dialects don't
need to depend on the non-interface part of the transform dialect, which
includes the growing number of ops and transitive dependency footprint.

Split out the interfaces into a separate library. This in turn requires
flipping the dependency from the interface on the dialect that has crept
in because both co-existed in one library. The interface shouldn't
depend on the transform dialect either.

As a consequence of splitting, the capability of the interpreter to
automatically walk the payload IR to identify payload ops of a certain
kind based on the type used for the entry point symbol argument is
disabled. This is a good move by itself as it simplifies the interpreter
logic. This functionality can be trivially replaced by a
`transform.structured.match` operation.

show more ...

[mlir][tensor] Centralize pack/unpack related patterns. (#76603)

The revision moves pack/unpack related patterns to
PackAndUnpackPatterns.cpp. This follows the convention like other tensor
ops.

[mlir][tensor] Centralize pack/unpack related patterns. (#76603)

The revision moves pack/unpack related patterns to
PackAndUnpackPatterns.cpp. This follows the convention like other tensor
ops.

It also renames `populateSimplifyTensorPack` to
`populateSimplifyPackAndUnpackPatterns` and adds a TODO item for
tensor.unpack op.

show more ...

Reapply "[mlir][transform] Improve error message of tracking listener. (#66987)"

This commit reapplies #66987, which got original contained a memory leak
and got reverted by 78c8ab5844e618162c4cf398

Reapply "[mlir][transform] Improve error message of tracking listener. (#66987)"

This commit reapplies #66987, which got original contained a memory leak
and got reverted by 78c8ab5844e618162c4cf3982d05102d4da10d23. The leak
is now fixed.

Original description:

This PR extends the error message of the tracking listener when
replacement ops cannot be found. That may happen if the applied patterns
replace an op by an op of a different kind or by block arguments.
However, this only matters if there are alive handles to the replaced
op. The new error message mentions that explicitly and reports the alive
handles.

show more ...

Revert "[mlir][transform] Improve error message of tracking listener. (#66987)"

Breaks https://lab.llvm.org/buildbot/#/builders/5/builds/36953

This reverts commit a7530452fd163c84e83e662b549ade7b0f

Revert "[mlir][transform] Improve error message of tracking listener. (#66987)"

Breaks https://lab.llvm.org/buildbot/#/builders/5/builds/36953

This reverts commit a7530452fd163c84e83e662b549ade7b0fae9edf.

show more ...

[mlir][transform] Improve error message of tracking listener. (#66987)

This PR extends the error message of the tracking listener when
replacement ops cannot be found. That may happen if the applie

[mlir][transform] Improve error message of tracking listener. (#66987)

This PR extends the error message of the tracking listener when
replacement ops cannot be found. That may happen if the applied patterns
replace an op by an op of a different kind or by block arguments.
However, this only matters if there are alive handles to the replaced
op. The new error message mentions that explicitly and reports the alive
handles.

show more ...

[MLIR] Fix compiler warnings (NFC)

In `TestTensorTransforms.cpp` `replaced` is nullptr I assumed the intent
was to emit the error for the `rootOp`.

In `TransformInterfaces.cpp` there were some unin

[MLIR] Fix compiler warnings (NFC)

In `TestTensorTransforms.cpp` `replaced` is nullptr I assumed the intent
was to emit the error for the `rootOp`.

In `TransformInterfaces.cpp` there were some uninitialized variables.

In `NVGPUTransformOps.cpp` `matmulOp` was never used.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D154439

show more ...

[mlir][tensor] Add option to fold only tensor.empty with a single use

This is useful for transformations such as bufferization, which is looking for tensor.extract_slice/insert_slice pairs.

Also fi

[mlir][tensor] Add option to fold only tensor.empty with a single use

This is useful for transformations such as bufferization, which is looking for tensor.extract_slice/insert_slice pairs.

Also fix the documentation of the corresponding tranform op.

Differential Revision: https://reviews.llvm.org/D152455

show more ...

[mlir][transform] TrackingListener: Distinguish between failure and "should be dropped"

When looking for replacement ops (`findReplacementOp`) distinguish between "no replacement could be found" and

[mlir][transform] TrackingListener: Distinguish between failure and "should be dropped"

When looking for replacement ops (`findReplacementOp`) distinguish between "no replacement could be found" and "this op should be dropped from the mapping". The latter case will be utilized in a subsequent revision when a payload op is mapped to a consumed handle.

Differential Revision: https://reviews.llvm.org/D152375

show more ...

[mlir][transform] Utilize op interface instead of tensor::TrackingListener

Add a new interface `FindPayloadReplacementOpInterface` to specify ops that should be skipped when looking for payload repl

[mlir][transform] Utilize op interface instead of tensor::TrackingListener

Add a new interface `FindPayloadReplacementOpInterface` to specify ops that should be skipped when looking for payload replacement ops. Such ops are typically metadata-only ops.

With this change, we no longer need to maintain a custom TrackingListener in the tensor dialect.

Note: `CastOpInterface` by itself is not sufficient. Some metadata-only ops such as "tensor.reshape" are not casts, and it would be incorrect for them to implement the `CastOpInterface`.

Differential Revision: https://reviews.llvm.org/D151888

show more ...

[mlir][tensor] Add pattern to drop redundant insert_slice rank expansion

Drop insert_slice rank expansions if they are directly followed by an inverse rank reduction.

Differential Revision: https:/

[mlir][tensor] Add pattern to drop redundant insert_slice rank expansion

Drop insert_slice rank expansions if they are directly followed by an inverse rank reduction.

Differential Revision: https://reviews.llvm.org/D151800

show more ...

[mlir][tensor] TrackingListener: Find replacement ops through cast-like InsertSliceOps

Certain InsertSliceOps, that do not use elements from the destination, are treated like casts when looking for

[mlir][tensor] TrackingListener: Find replacement ops through cast-like InsertSliceOps

Certain InsertSliceOps, that do not use elements from the destination, are treated like casts when looking for replacement ops. Such InsertSliceOps are typically rank expansions.

Tensors with dynamic shape are not supported at the moment.

Also adds test cases for the TrackingListener.

Differential Revision: https://reviews.llvm.org/D151422

show more ...

[mlir] Move casting calls from methods to function calls

The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionali

[mlir] Move casting calls from methods to function calls

The MLIR classes Type/Attribute/Operation/Op/Value support
cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast
functionality in addition to defining methods with the same name.
This change begins the migration of uses of the method to the
corresponding function call as has been decided as more consistent.

Note that there still exist classes that only define methods directly,
such as AffineExpr, and this does not include work currently to support
a functional cast/isa call.

Caveats include:
- This clang-tidy script probably has more problems.
- This only touches C++ code, so nothing that is being generated.

Context:
- https://mlir.llvm.org/deprecation/ at "Use the free function variants
for dyn_cast/cast/isa/…"
- Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443

Implementation:
This first patch was created with the following steps. The intention is
to only do automated changes at first, so I waste less time if it's
reverted, and so the first mass change is more clear as an example to
other teams that will need to follow similar steps.

Steps are described per line, as comments are removed by git:
0. Retrieve the change from the following to build clang-tidy with an
additional check:
https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check
1. Build clang-tidy
2. Run clang-tidy over your entire codebase while disabling all checks
and enabling the one relevant one. Run on all header files also.
3. Delete .inc files that were also modified, so the next build rebuilds
them to a pure state.
4. Some changes have been deleted for the following reasons:
- Some files had a variable also named cast
- Some files had not included a header file that defines the cast
functions
- Some files are definitions of the classes that have the casting
methods, so the code still refers to the method instead of the
function without adding a prefix or removing the method declaration
at the same time.

```
ninja -C $BUILD_DIR clang-tidy

run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\
-header-filter=mlir/ mlir/* -fix

rm -rf $BUILD_DIR/tools/mlir/**/*.inc

git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\
mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\
mlir/lib/**/IR/\
mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\
mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\
mlir/test/lib/Dialect/Test/TestTypes.cpp\
mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\
mlir/test/lib/Dialect/Test/TestAttributes.cpp\
mlir/unittests/TableGen/EnumsGenTest.cpp\
mlir/test/python/lib/PythonTestCAPI.cpp\
mlir/include/mlir/IR/
```

Differential Revision: https://reviews.llvm.org/D150123

show more ...

[mlir][Tensor] Drop SplitPaddingPatterns.

These old patterns are not in use in either MLIR or downstream projects except for one test.
Additionally this is redundant with logic in the tensor.pad til

[mlir][Tensor] Drop SplitPaddingPatterns.

These old patterns are not in use in either MLIR or downstream projects except for one test.
Additionally this is redundant with logic in the tensor.pad tiling implementation.

Drop SplitPaddingPatterns to reduce entropy.

Differential Revision: https://reviews.llvm.org/D148207

show more ...

[mlir][NFC] reifyResultShapes: Add extra error checking

This change adds a new helper function `mlir::reifyResultShapes` that calls the corresponding interface method and also checks the result prod

[mlir][NFC] reifyResultShapes: Add extra error checking

This change adds a new helper function `mlir::reifyResultShapes` that calls the corresponding interface method and also checks the result produced by the implementation when running in debug mode. Bugs due to incorrect interface implementations can be difficult to debug.

This helper function also reduces the amount of code needed at call sites: the cast to `ReifyRankedShapedTypeOpInterface` is done in the helper function.

Differential Revision: https://reviews.llvm.org/D145777

show more ...

[mlir][Interfaces] ReifyRankedShapedTypeOpInterface returns OpFoldResults

`reifyResultShapes` now returns `OpFoldResult`s instead of `Value`s. This is often more efficient because many transformatio

[mlir][Interfaces] ReifyRankedShapedTypeOpInterface returns OpFoldResults

`reifyResultShapes` now returns `OpFoldResult`s instead of `Value`s. This is often more efficient because many transformations immediately attempt to extract a constant from the reified values.

Differential Revision: https://reviews.llvm.org/D145250

show more ...

[mlir][scf] Rename ForeachThreadOp->ForallOp, PerformConcurrentlyOp->InParallelOp.

Differential Revision: https://reviews.llvm.org/D144242

[mlir] Add loop bounds to scf.foreach_thread.

https://discourse.llvm.org/t/rfc-parallel-loops-on-tensors-in-mlir/68332

Differential Revision: https://reviews.llvm.org/D144072

[MLIR][Tensor] Add canonicalization patterns for `tensor.pack`

- Fold an unpack(pack(x)) to x.

- Rewrite a `tensor.pack` to an `tensor.expand_shape` if only one
dimension is packed.

Reviewed By:

[MLIR][Tensor] Add canonicalization patterns for `tensor.pack`

- Fold an unpack(pack(x)) to x.

- Rewrite a `tensor.pack` to an `tensor.expand_shape` if only one
dimension is packed.

Reviewed By: tyb0807, hanchung, mravishankar

Differential Revision: https://reviews.llvm.org/D141123

show more ...

[mlir][tensor] Add patterns that fold ops into pack and unpack ops.

The tensor.pack ops have pad semantic, so we can fold pad + pack into
pack when

1. They have the same padding values or the pack

[mlir][tensor] Add patterns that fold ops into pack and unpack ops.

The tensor.pack ops have pad semantic, so we can fold pad + pack into
pack when

1. They have the same padding values or the pack op does not have
padding values.
2. The pad op does not have low paddings.

The tensor.unpack ops have extract_slice semantic, so we can fold unpack
+ extract_slice into unpack when

1. All the offsets are 0s.
2. All the strides are 1s.

Reviewed By: tyb0807

Differential Revision: https://reviews.llvm.org/D141099

show more ...

[mlir] Make patterns for folding tensor.empty optional.

At the moment, they are a part of EmptyOp::getCanonicalizationPatterns. When
extract_slice(tensor.empty) is rewritten as a new tensor.empty, i

[mlir] Make patterns for folding tensor.empty optional.

At the moment, they are a part of EmptyOp::getCanonicalizationPatterns. When
extract_slice(tensor.empty) is rewritten as a new tensor.empty, it could
happen that we end up with two tensor.empty ops, since the original
tensor.empty can have two users. After bufferization such cases result in two
allocations.

Differential Revision: https://reviews.llvm.org/D139308

show more ...

[mlir][tensor] Fold rank-reducing extract_slice with inverse expand_shape

Differential Revision: https://reviews.llvm.org/D139220

Revert "[mlir][tensor] Fold rank-reducing extract_slice with inverse expand_shape"

This reverts commit a076f57a1a6b6d775aa4f11ac678d1c43ab33fb1.

[mlir][tensor] Fold rank-reducing extract_slice with inverse expand_shape

Differential Revision: https://reviews.llvm.org/D139103