[mlir] Enable decoupling two kinds of greedy behavior. (#104649)

The greedy rewriter is used in many different flows and it has a lot of
convenience (work list management, debugging actions, tracin

[mlir] Enable decoupling two kinds of greedy behavior. (#104649)

The greedy rewriter is used in many different flows and it has a lot of
convenience (work list management, debugging actions, tracing, etc). But
it combines two kinds of greedy behavior 1) how ops are matched, 2)
folding wherever it can.

These are independent forms of greedy and leads to inefficiency. E.g.,
cases where one need to create different phases in lowering and is
required to applying patterns in specific order split across different
passes. Using the driver one ends up needlessly retrying folding/having
multiple rounds of folding attempts, where one final run would have
sufficed.

Of course folks can locally avoid this behavior by just building their
own, but this is also a common requested feature that folks keep on
working around locally in suboptimal ways.

For downstream users, there should be no behavioral change. Updating
from the deprecated should just be a find and replace (e.g., `find ./
-type f -exec sed -i
's|applyPatternsAndFoldGreedily|applyPatternsGreedily|g' {} \;` variety)
as the API arguments hasn't changed between the two.

show more ...

[MLIR][SCF] Fix LoopPeelOp documentation (NFC) (#113179)

As an example, I added annotations to the peel_front unit test.

```
func.func @loop_peel_first_iter_op() {
// CHECK: %[[C0:.+]] = arit

[MLIR][SCF] Fix LoopPeelOp documentation (NFC) (#113179)

As an example, I added annotations to the peel_front unit test.

```
func.func @loop_peel_first_iter_op() {
// CHECK: %[[C0:.+]] = arith.constant 0
// CHECK: %[[C41:.+]] = arith.constant 41
// CHECK: %[[C5:.+]] = arith.constant 5
// CHECK: %[[C5_0:.+]] = arith.constant 5
// CHECK: scf.for %{{.+}} = %[[C0]] to %[[C5_0]] step %[[C5]]
// CHECK: arith.addi
// CHECK: scf.for %{{.+}} = %[[C5_0]] to %[[C41]] step %[[C5]]
// CHECK: arith.addi
%0 = arith.constant 0 : index
%1 = arith.constant 41 : index
%2 = arith.constant 5 : index
scf.for %i = %0 to %1 step %2 {
arith.addi %i, %i : index
}
return
}

module attributes {transform.with_named_sequence} {
transform.named_sequence @__transform_main(%arg1: !transform.any_op {transform.readonly}) {
%0 = transform.structured.match ops{["arith.addi"]} in %arg1 : (!transform.any_op) -> !transform.any_op
%1 = transform.get_parent_op %0 {op_name = "scf.for"} : (!transform.any_op) -> !transform.op<"scf.for">
%main_loop, %remainder = transform.loop.peel %1 {peel_front = true} : (!transform.op<"scf.for">) -> (!transform.op<"scf.for">, !transform.op<"scf.for">)
transform.annotate %main_loop "main_loop" : !transform.op<"scf.for">
transform.annotate %remainder "remainder" : !transform.op<"scf.for">
transform.yield
}
}
```
Gives :
```
func.func @loop_peel_first_iter_op() {
%c0 = arith.constant 0 : index
%c41 = arith.constant 41 : index
%c5 = arith.constant 5 : index
%c5_0 = arith.constant 5 : index
scf.for %arg0 = %c0 to %c5_0 step %c5 {
%0 = arith.addi %arg0, %arg0 : index
} {remainder} // The first iteration loop (second result) has been annotated remainder
scf.for %arg0 = %c5_0 to %c41 step %c5 {
%0 = arith.addi %arg0, %arg0 : index
} {main_loop} // The main loop (first result) has been annotated main_loop
return
}
```

---------

Co-authored-by: Andrzej Warzyński <andrzej.warzynski@gmail.com>

show more ...

Fix the condition for peeling the first iteration (#86350)

This PR fixes the condition used in loop peeling of the first iteration.
Using ceilDiv instead of floorDiv when calculating the loop count

Fix the condition for peeling the first iteration (#86350)

This PR fixes the condition used in loop peeling of the first iteration.
Using ceilDiv instead of floorDiv when calculating the loop counts, so
that the first iteration gets peeled as needed.

show more ...

[mlir][IR] Rename "update root" to "modify op" in rewriter API (#78260)

This commit renames 4 pattern rewriter API functions:
* `updateRootInPlace` -> `modifyOpInPlace`
* `startRootUpdate` -> `sta

[mlir][IR] Rename "update root" to "modify op" in rewriter API (#78260)

This commit renames 4 pattern rewriter API functions:
* `updateRootInPlace` -> `modifyOpInPlace`
* `startRootUpdate` -> `startOpModification`
* `finalizeRootUpdate` -> `finalizeOpModification`
* `cancelRootUpdate` -> `cancelOpModification`

The term "root" is a misnomer. The root is the op that a rewrite pattern
matches against
(https://mlir.llvm.org/docs/PatternRewriter/#root-operation-name-optional).
A rewriter must be notified of all in-place op modifications, not just
in-place modifications of the root
(https://mlir.llvm.org/docs/PatternRewriter/#pattern-rewriter). The old
function names were confusing and have contributed to various broken
rewrite patterns.

Note: The new function names use the term "modify" instead of "update"
for consistency with the `RewriterBase::Listener` terminology
(`notifyOperationModified`).

show more ...

[mlir][scf] Fix `for-loop-peeling` crash (#77697)

Before applying the peeling patterns, it can happen that the `ForOp`
gets a step of zero during folding. This leads to a division-by-zero
down the

[mlir][scf] Fix `for-loop-peeling` crash (#77697)

Before applying the peeling patterns, it can happen that the `ForOp`
gets a step of zero during folding. This leads to a division-by-zero
down the line.

This patch adds an additional check for a constant-zero step and a
test.

Fix https://github.com/llvm/llvm-project/issues/75758

show more ...

[mlir][SCF] Add support for peeling the first iteration out of the loop (#74015)

There is a use case that we need to peel the first iteration out of the
for loop so that the peeled forOp can be can

[mlir][SCF] Add support for peeling the first iteration out of the loop (#74015)

There is a use case that we need to peel the first iteration out of the
for loop so that the peeled forOp can be canonicalized away and the
fillOp can be fused into the inner forall loop. For example, we have
nested loops as below

```
linalg.fill ins(...) outs(...)
scf.for %arg = %lb to %ub step %step
scf.forall ...
```

After the peeling transform, it is expected to be

```
scf.forall ...
linalg.fill ins(...) outs(...)
scf.for %arg = %(lb + step) to %ub step %step
scf.forall ...
```

This patch makes the most use of the existing peeling functions and adds
support for peeling the first iteration out of the loop.

show more ...

[mlir][SCF] Do not peel already peeled loops (#71900)

Loop peeling is not beneficial if the step size already divides "ub -
lb". There are currently some simple checks to prevent peeling in such
c

[mlir][SCF] Do not peel already peeled loops (#71900)

Loop peeling is not beneficial if the step size already divides "ub -
lb". There are currently some simple checks to prevent peeling in such
cases when lb, ub, step are constants. This commit adds support for IR
that is the result of loop peeling in the general case; i.e., lb, ub,
step do not necessarily have to be constants.

This change adds a new affine_map simplification rule for semi-affine
maps that appear during loop peeling and are guaranteed to evaluate to a
constant zero. Affine maps such as:
```
(1) affine_map<()[ub, step] -> ((ub - ub mod step) mod step)
(2) affine_map<()[ub, lb, step] -> ((ub - (ub - lb) mod step - lb) mod step)
(3) ^ may contain additional summands
```
Other affine maps with modulo expressions are not supported by the new
simplification rule.

This fixes #71469.

show more ...

[mlir][affine][nfc] cleanup deprecated T.cast style functions (#71269)

detail see the docment: https://mlir.llvm.org/deprecation/

Not all changes are made manually, most of them are made through

[mlir][affine][nfc] cleanup deprecated T.cast style functions (#71269)

detail see the docment: https://mlir.llvm.org/deprecation/

Not all changes are made manually, most of them are made through a clang
tool I wrote https://github.com/lipracer/cpp-refactor.

show more ...

[mlir][Affine][NFC] Wrap dialect in "affine" namespace

This cleanup aligns the affine dialect with all the other dialects.

Differential Revision: https://reviews.llvm.org/D148687

[mlir][Linalg] NFC - Apply cleanups to transforms

Depends on: D144656

Differential Revision: https://reviews.llvm.org/D144717

[mlir][SCF] Fix incorrect API usage in RewritePatterns

Incorrect API usage was detected by D144552.

Differential Revision: https://reviews.llvm.org/D144636

[mlir] Add operations to BlockAndValueMapping and rename it to IRMapping

The patch adds operations to `BlockAndValueMapping` and renames it to `IRMapping`. When operations are cloned, old operations

[mlir] Add operations to BlockAndValueMapping and rename it to IRMapping

The patch adds operations to `BlockAndValueMapping` and renames it to `IRMapping`. When operations are cloned, old operations are mapped to the cloned operations. This allows mapping from an operation to a cloned operation. Example:

```
Operation *opWithRegion = ...
Operation *opInsideRegion = &opWithRegion->front().front();

IRMapping map
Operation *newOpWithRegion = opWithRegion->clone(map);
Operation *newOpInsideRegion = map.lookupOrNull(opInsideRegion);
```

Migration instructions:
All includes to `mlir/IR/BlockAndValueMapping.h` should be replaced with `mlir/IR/IRMapping.h`. All uses of `BlockAndValueMapping` need to be renamed to `IRMapping`.

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D139665

show more ...

[mlir][affine][NFC] Extract core functionality of `canonicalizeMinMaxOp`

Move code from SCF to Affine: Add a new helper function `simplifyConstrainedMinMaxOp` to Affine/Analysis/Utils.h. `canonicali

[mlir][affine][NFC] Extract core functionality of `canonicalizeMinMaxOp`

Move code from SCF to Affine: Add a new helper function `simplifyConstrainedMinMaxOp` to Affine/Analysis/Utils.h. `canonicalizeMinMaxOp` was originally designed for loop peeling, but it is not SCF-specific and can be used to simplify any affine.min/max ops.

Various functions in SCF/Transforms are simplified by dropping unnecessary parameters.

Differential Revision: https://reviews.llvm.org/D140962

show more ...

[mlir] Remove support for non-prefixed accessors

This finishes off a year long pursuit to LLVMify the generated
operation accessors, prefixing them with get/set. Support for
any other accessor namin

[mlir] Remove support for non-prefixed accessors

This finishes off a year long pursuit to LLVMify the generated
operation accessors, prefixing them with get/set. Support for
any other accessor naming is fully removed after this commit.

https://discourse.llvm.org/t/psa-raw-accessors-are-being-removed/65629

Differential Revision: https://reviews.llvm.org/D136727

show more ...

[mlir][arith] Change dialect name from Arithmetic to Arith

Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-ml

[mlir][arith] Change dialect name from Arithmetic to Arith

Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples`

and `bazel build --config=generic_clang @llvm-project//mlir:all`.

Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D134762

show more ...

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow drop

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure.

Reviewed By: mehdi_amini, rriddle

Differential Review: https://reviews.llvm.org/D132838

show more ...

Revert "[MLIR] Update pass declarations to new autogenerated files"

This reverts commit 2be8af8f0e0780901213b6fd3013a5268ddc3359.

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow drop

[MLIR] Update pass declarations to new autogenerated files

The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure.

Reviewed By: mehdi_amini, rriddle

Differential Review: https://reviews.llvm.org/D132838

show more ...

[mlir] Update flipped accessors (NFC)

Follow up with memref flipped and flipping any intermediate changes
made.

[mlir] Generalize SCF passes to not have to run on FuncOp.

Seems to have been an accident of history and none of these had any reason to be restricted to FuncOp.

Differential Revision: https://revi

[mlir] Generalize SCF passes to not have to run on FuncOp.

Seems to have been an accident of history and none of these had any reason to be restricted to FuncOp.

Differential Revision: https://reviews.llvm.org/D128614

show more ...

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.

[mlir] move SCF headers to SCF/{IR,Transforms} respectively

This aligns the SCF dialect file layout with the majority of the dialects.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D128049

show more ...

[mlir:NFC] Remove the forward declaration of FuncOp in the mlir namespace

FuncOp has been moved to the `func` namespace for a little over a month, the
using directive can be dropped now.

[mlir] Trim a huge number of unnecessary dependencies on the Func dialect

The Func has a large number of legacy dependencies carried over from the old
Standard dialect, which was pervasive and conta

[mlir] Trim a huge number of unnecessary dependencies on the Func dialect

The Func has a large number of legacy dependencies carried over from the old
Standard dialect, which was pervasive and contained a large number of varied
operations. With the split of the standard dialect and its demise, a lot of lingering
dead dependencies have survived to the Func dialect. This commit removes a
large majority of then, greatly reducing the dependence surface area of the
Func dialect.

show more ...

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the init

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the initial
renaming (which by itself is already huge), but there are a large number
of cleanups unlocked/necessary afterwards:

* Removing a bunch of unnecessary dependencies on Func
* Cleaning up the From/ToStandard conversion passes
* Preparing for the move of FuncOp to the Func dialect

See the discussion at https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Differential Revision: https://reviews.llvm.org/D120624

show more ...

[mlir] Move SCF utils implementations to SCF/Utils.

BEGIN_PUBLIC
[mlir] Move SCF utils implementations to SCF/Utils.
END_PUBLIC