[mlir][Linalg] Add support for fusion between indexed_generic ops and generic ops on tensors.

Summary:
Different from the fusion between generic ops, indices are involved. In this
context, we need t

[mlir][Linalg] Add support for fusion between indexed_generic ops and generic ops on tensors.

Summary:
Different from the fusion between generic ops, indices are involved. In this
context, we need to re-map the indices for producer since the fused op is built
on consumer's perspective. This patch supports all combination of the fusion
between indexed_generic ops and generic ops, which includes tests case:
1) generic op as producer and indexed_generic op as consumer.
2) indexed_generic op as producer and generic op as consumer.
3) indexed_generic op as producer and indexed_generic op as consumer.

Differential Revision: https://reviews.llvm.org/D80347

show more ...

[mlir][Linalg] Add pass to remove unit-extent dims from tensor
operands of Generic ops.

Unit-extent dimensions are typically used for achieving broadcasting
behavior. The pattern added (along with c

[mlir][Linalg] Add pass to remove unit-extent dims from tensor
operands of Generic ops.

Unit-extent dimensions are typically used for achieving broadcasting
behavior. The pattern added (along with canonicalization patterns
added previously) removes the use of unit-extent dimensions, and
instead uses a more canonical representation of the computation. This
new pattern is not added as a canonicalization for now since it
entails adding additional reshape operations. A pass is added to
exercise these patterns, along with an API entry to populate a
patterns list with these patterns.

Differential Revision: https://reviews.llvm.org/D79766

show more ...

[mlir][Linalg] Add producer-consumer fusion when producer is a ConstantOp
and Consumer is a GenericOp.

Differential Revision: https://reviews.llvm.org/D79838

[mlir][Linalg] Refactor linalg tiling

Summary:
This revision refactors the Linalg tiling pass to be written as pattern applications and retires the use of the folder in Linalg tiling.
In the early d

[mlir][Linalg] Refactor linalg tiling

Summary:
This revision refactors the Linalg tiling pass to be written as pattern applications and retires the use of the folder in Linalg tiling.
In the early days, tiling was written as a pass that would create (partially) folded and canonicalized operations on the fly for better composability.
As this evolves towards composition of patterns, the pass-specific folder is counter-productive and is retired.
The tiling options struct evolves to take a tile size creation function which allows materializing tile sizes on the fly (in particular constant tile sizes). This plays better with folding and DCE.

With the folder going away in Tiling, the check on whether subviews are the same in linalg fusion needs to be more robust. This revision also implements such a check.

In the current form, there are still some canonicalizations missing due to AffineMin/Max ops fed by scf::ForOp. These will be improved at a later time.

Differential Revision: https://reviews.llvm.org/D80267

show more ...

[mlir] Add SubViewOp::getOrCreateRanges and fix folding pattern

The existing implementation of SubViewOp::getRanges relies on all
offsets/sizes/strides to be dynamic values and does not work in
comb

[mlir] Add SubViewOp::getOrCreateRanges and fix folding pattern

The existing implementation of SubViewOp::getRanges relies on all
offsets/sizes/strides to be dynamic values and does not work in
combination with canonicalization. This revision adds a
SubViewOp::getOrCreateRanges to create the missing constants in the
canonicalized case.

This allows reactivating the fused pass with staged pattern
applications.

However another issue surfaces that the SubViewOp verifier is now too
strict to allow folding. The existing folding pattern is turned into a
canonicalization pattern which rewrites memref_cast + subview into
subview + memref_cast.

The transform-patterns-matmul-to-vector can then be reactivated.

Differential Revision: https://reviews.llvm.org/D79759

show more ...

[mlir][Value] Add v.getDefiningOp<OpTy>()

Summary:
This makes a common pattern of
`dyn_cast_or_null<OpTy>(v.getDefiningOp())` more concise.

Differential Revision: https://reviews.llvm.org/D79681

[MLIR] Move Verifier and Dominance Analysis from /Analysis to /IR

These libraries are distinct from other things in Analysis in that they
operate only on core IR concepts. This also simplifies depe

[MLIR] Move Verifier and Dominance Analysis from /Analysis to /IR

These libraries are distinct from other things in Analysis in that they
operate only on core IR concepts. This also simplifies dependencies
so that Dialect -> Analysis -> Parser -> IR. Previously, the parser depended
on portions of the the Analysis directory as well, which sometimes
caused issues with the way the cmake makefile generator discovers
dependencies on generated files during compilation.

Differential Revision: https://reviews.llvm.org/D79240

show more ...

[mlir][linalg] Fix crash in tileAndFuseLinalgOpToParallelLoopsAndSetMarker

Instead of using llvm_unreachable to guard against fusing linalg.conv,
reject fusing linalg.conv in isFusableInto.

tileLin

[mlir][linalg] Fix crash in tileAndFuseLinalgOpToParallelLoopsAndSetMarker

Instead of using llvm_unreachable to guard against fusing linalg.conv,
reject fusing linalg.conv in isFusableInto.

tileLinalgOpImpl is a templated function now and it can operate on
loop.parellel. So we should avoid calling into getForInductionVarOwner
which always assumes loop.for.

Differential Revision: https://reviews.llvm.org/D78936

show more ...

[mlir][Linalg] Add support for fusing linalg.tensor_reshape with
linalg.generic operations.

Differential Revision: https://reviews.llvm.org/D78464

[mlir][Linalg] NFC: Refactor fusion on tensors to enable extending
it to fusing different kinds of linalg operations on tensors.

The implementation of fusion on tensor was initially planned for just

[mlir][Linalg] NFC: Refactor fusion on tensors to enable extending
it to fusing different kinds of linalg operations on tensors.

The implementation of fusion on tensor was initially planned for just
GenericOps (and maybe IndexedGenericOps). With addition of
linalg.tensor_reshape, and potentially other such non-structured ops,
refactor the existing implementation to allow easier specification of
fusion between different linalg operations on tensors.

Differential Revision: https://reviews.llvm.org/D78463

show more ...

[mlir][EDSC] Retire ValueHandle

The EDSC discussion [thread](https://llvm.discourse.group/t/evolving-builder-apis-based-on-lessons-learned-from-edsc/879) points out that ValueHandle has become an un

[mlir][EDSC] Retire ValueHandle

The EDSC discussion [thread](https://llvm.discourse.group/t/evolving-builder-apis-based-on-lessons-learned-from-edsc/879) points out that ValueHandle has become an unnecessary level of abstraction since MLIR switch from `Value *` to `Value` everywhere.

This revision removes this level of indirection.

show more ...

[MLIR] Move `replaceAllUsesExcept` from LoopUtil.h to Value.h.

Differential Revision: https://reviews.llvm.org/D78426

[mlir][Linalg] NFC - Split out EDSCs that require a Folder

Summary: This is an NFC cleanup in preparation for end-to-end named Linalg ops.

Differential Revision: https://reviews.llvm.org/D78331

[mlir][Linalg] Drop function attribute from generic ops.

The function attribute in generic ops is not paying for itself.
A region is the more standardized way of specifying a custom computation.
If

[mlir][Linalg] Drop function attribute from generic ops.

The function attribute in generic ops is not paying for itself.
A region is the more standardized way of specifying a custom computation.
If needed this region can call a function directly.
This is deemed more natural than managing a dedicated function attribute.

This also simplifies named ops generation by trimming unnecessary complexity.

Differential Revision: https://reviews.llvm.org/D78266

show more ...

[MLIR] Fix fusion of linalg.indexed_generic producer into tiled (Indexed)GenericOp.

Differential Revision: https://reviews.llvm.org/D78209

[mlir][NFC] Remove the STLExtras.h header file now that it has been merged into LLVM.

Now that no more utilities exist within, this file can be deleted.

Differential Revision: https://reviews.llvm.

[mlir][NFC] Remove the STLExtras.h header file now that it has been merged into LLVM.

Now that no more utilities exist within, this file can be deleted.

Differential Revision: https://reviews.llvm.org/D78079

show more ...

[mlir][Linalg] Handle null affine map returns from inversePermutation.

The inversePermutation method returns a null map on failure. Update
uses of this method within Linalg to handle this. In Linalg

[mlir][Linalg] Handle null affine map returns from inversePermutation.

The inversePermutation method returns a null map on failure. Update
uses of this method within Linalg to handle this. In LinalgToLoops the
null return value was used to emit scalar code. Modify that to return
failure, and emit scalar implementation when affine map is "empty",
i.e. 1 dims, 0 symbols and no result exprs.

Differential Revision: https://reviews.llvm.org/D77964

show more ...

[MLIR][NFC] applyPatternsGreedily -> applyPatternsAndFoldGreedily

Rename mlir::applyPatternsGreedily -> applyPatternsAndFoldGreedily. The
new name is a more accurate description of the method - it p

[MLIR][NFC] applyPatternsGreedily -> applyPatternsAndFoldGreedily

Rename mlir::applyPatternsGreedily -> applyPatternsAndFoldGreedily. The
new name is a more accurate description of the method - it performs
both, application of the specified patterns and folding of all ops in
the op's region irrespective of whether any patterns have been supplied.

Differential Revision: https://reviews.llvm.org/D77478

show more ...

[mlir][Pass] Update the PassGen to generate base classes instead of utilities

Summary:
This is much cleaner, and fits the same structure as many other tablegen backends. This was not done originally

[mlir][Pass] Update the PassGen to generate base classes instead of utilities

Summary:
This is much cleaner, and fits the same structure as many other tablegen backends. This was not done originally as the CRTP in the pass classes made it overly verbose/complex.

Differential Revision: https://reviews.llvm.org/D77367

show more ...

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cle

[mlir][Pass] Remove the use of CRTP from the Pass classes

This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cleaner interface in the C++ code, and naturally fits with the rest of the infrastructure. A new utility class, PassWrapper, is added to replicate the existing behavior for passes not suitable for using the tablegen backend.

Differential Revision: https://reviews.llvm.org/D77350

show more ...

[mlir][Pass] Add support for generating pass utilities via tablegen

This revision adds support for generating utilities for passes such as options/statistics/etc. that can be inferred from the table

[mlir][Pass] Add support for generating pass utilities via tablegen

This revision adds support for generating utilities for passes such as options/statistics/etc. that can be inferred from the tablegen definition. This removes additional boilerplate from the pass, and also makes it easier to remove the reliance on the pass registry to provide certain things(e.g. the pass argument).

Differential Revision: https://reviews.llvm.org/D76659

show more ...

[mlir][Pass] Move the registration of dialect passes to tablegen

This generates a Passes.td for all of the dialects that have transformation passes. This removes the need for global registration for

[mlir][Pass] Move the registration of dialect passes to tablegen

This generates a Passes.td for all of the dialects that have transformation passes. This removes the need for global registration for all of the dialect passes.

Differential Revision: https://reviews.llvm.org/D76657

show more ...

[mlir][Linalg] Extend fusion to support WAW atm on buffers.

Summary:
The RAW fusion happens only if the produecer block dominates the consumer block.
The WAW pattern also works with the precondition

[mlir][Linalg] Extend fusion to support WAW atm on buffers.

Summary:
The RAW fusion happens only if the produecer block dominates the consumer block.
The WAW pattern also works with the precondition. I.e., if a producer can
dominate the consumer, they can fairly fuse together.

Since they are all tilable, we can think the pattern like this way:

Input:
```
linalg_op1 view

tile_loop
subview_2
linalg_op2 subview_2
```

Tile the first Linalg op as same as the second Linalg.
```
tile_loop
subview_1
linalg_op1 subview_1

tile_loop
subview_2
liangl_op2 subview_2
```

Since the first Linalg op is tilable in the same way and the computation are
independently, it's fair to fuse it with the second Linalg op.
```
tile_loop
subview_1
linalg_op1 subview_1
linalg_op2 subview_2
```

In short, this patch includes:
- Handling both RAW and WAW pattern.
- Adding a interface method to get input and output buffers.
- Exposing a method to get a StringRef of a dependency type.
- Fixing existing WAW tests and add one more use case: initialize the buffer
before conv op.

Differential Revision: https://reviews.llvm.org/D76897

show more ...

[mlir] Move several static cl::opts to be pass options instead.

This removes the reliance on global options, and also simplifies the pass registration.

Differential Revision: https://reviews.llvm.o

[mlir] Move several static cl::opts to be pass options instead.

This removes the reliance on global options, and also simplifies the pass registration.

Differential Revision: https://reviews.llvm.org/D76552

show more ...

[mlir][NFC] Replace all usages of PatternMatchResult with LogicalResult

This also replaces usages of matchSuccess/matchFailure with success/failure respectively.

Differential Revision: https://revi

[mlir][NFC] Replace all usages of PatternMatchResult with LogicalResult

This also replaces usages of matchSuccess/matchFailure with success/failure respectively.

Differential Revision: https://reviews.llvm.org/D76313

show more ...