[mlir] remove test-tranfsorm-dialect-interpreter (#89931)

This pass has been deprecated for more than two months, alternative is
available via `-transform-interpreter` and `-transform-preload-libra

[mlir] remove test-tranfsorm-dialect-interpreter (#89931)

This pass has been deprecated for more than two months, alternative is
available via `-transform-interpreter` and `-transform-preload-library`.


https://discourse.llvm.org/t/psa-deprecating-test-transform-dialect-interpreter/76904

show more ...

[mlir] split transform interfaces into a separate library (#85221)

Transform interfaces are implemented, direction or via extensions, in
libraries belonging to multiple other dialects. Those dialec

[mlir] split transform interfaces into a separate library (#85221)

Transform interfaces are implemented, direction or via extensions, in
libraries belonging to multiple other dialects. Those dialects don't
need to depend on the non-interface part of the transform dialect, which
includes the growing number of ops and transitive dependency footprint.

Split out the interfaces into a separate library. This in turn requires
flipping the dependency from the interface on the dialect that has crept
in because both co-existed in one library. The interface shouldn't
depend on the transform dialect either.

As a consequence of splitting, the capability of the interpreter to
automatically walk the payload IR to identify payload ops of a certain
kind based on the type used for the entry point symbol argument is
disabled. This is a good move by itself as it simplifies the interpreter
logic. This functionality can be trivially replaced by a
`transform.structured.match` operation.

show more ...

[mlir] introduce debug transform dialect extension (#77595)

Introduce a new extension for simple print-debugging of the transform
dialect scripts. The initial version of this extension consists of

[mlir] introduce debug transform dialect extension (#77595)

Introduce a new extension for simple print-debugging of the transform
dialect scripts. The initial version of this extension consists of two
ops that are printing the payload objects associated with transform
dialect values. Similar ops were already available in the test extenion
and several downstream projects, and were extensively used for testing.

show more ...

[mlir][transform] Support for multiple top-level transform ops (#69615)

This adds a flag to the `TransformDialectInterpreter` that relaxes the
requirement for only a single top-level transform op.

[mlir][transform] Support for multiple top-level transform ops (#69615)

This adds a flag to the `TransformDialectInterpreter` that relaxes the
requirement for only a single top-level transform op.
This is useful for supporting transforms that take transform IR as
payload.

This also aligns the function `findTopLevelTransform`
[here](https://github.com/llvm/llvm-project/commit/7b0f4c9db55c355bffddf94d7710f40ee2c1e9db#diff-551f92bb609487ccf981daf9571f0f1b1703ab2330560a388a5f0d133e520be4L59)
with its documentation:
In the presence of multiple top-level transform ops it now correctly
returns the first of them after reporting the error instead of returning
a `nullptr`.

show more ...

[mlir][transform] Allow passing various library files to interpreter. (#67120)

The transfrom interpreter accepts an argument to a "library" file with
named sequences. This patch exteneds this funct

[mlir][transform] Allow passing various library files to interpreter. (#67120)

The transfrom interpreter accepts an argument to a "library" file with
named sequences. This patch exteneds this functionality such that (1)
several such individual files are accepted and (2) folders can be passed
in, in which all `*.mlir` files are loaded.

show more ...

[mlir][transform] Fix handling of transitive include in interpreter. (#67560)

Until now, the interpreter would only load those symbols from the
provided library files that were declared in the main

[mlir][transform] Fix handling of transitive include in interpreter. (#67560)

Until now, the interpreter would only load those symbols from the
provided library files that were declared in the main transform module.
However, sequences in the library may include other sequences on their
own. Until now, if such sequences were not *also* declared in the main
transform module, the interpreter would fail to resolve them. Forward
declaring all of them is undesirable as it defeats the purpose of
encapsulation into library modules.

This PR implements a kind of linker for transform scripts to solve this
problem. The linker merges all symbols of the library module into the
main module before interpreting the latter. Symbols whose names collide
are handled as follows: (1) if they are both functions (in the sense of
`FunctionOpInterface`) with compatible signatures, one is external, and
the other one is public, then they are merged; (2) of one of them is
private, that one is renamed; and (3) an error is raised otherwise.

One consequence of this change is that the loading of the library files
in the interpreter pass is not idempotent anymore, i.e., subsequent
interpreter passes cannot (and need not) load the same library files again
since would lead to doubly defined symbols.

show more ...

[mlir][transform] generate transform module on-the-fly

Add a TransformInterpreterPassBase capability to generate the (shared)
module containing the transform script during the pass initialization.
T

[mlir][transform] generate transform module on-the-fly

Add a TransformInterpreterPassBase capability to generate the (shared)
module containing the transform script during the pass initialization.
This is helpful to programmatically generate the script as opposed to
parsing it from the textual module.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D152185

show more ...

[mlir] support external named transform libraries

Introduce support for external definitions of named sequences in the
transform dialect by letting the TransformInterpreterPassBase read a
"library"

[mlir] support external named transform libraries

Introduce support for external definitions of named sequences in the
transform dialect by letting the TransformInterpreterPassBase read a
"library" MLIR file. This file is expected to contain definitions for
named sequences that are only declared in the main transformation
script. This allows for sharing non-trivial transform combinations
without duplication.

This patch provides only the minimal plumbing for a single textual IR
file. Further changes are possible to support multiple libraries and
bytecode files.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D146961

show more ...

[mlir] add support for transform dialect value handles

Introduce support for the third kind of values in the transform dialect:
value handles. Similarly to operation handles, value handles are
point

[mlir] add support for transform dialect value handles

Introduce support for the third kind of values in the transform dialect:
value handles. Similarly to operation handles, value handles are
pointing to a set of values in the payload IR. This enables
transformation to be targeted at specific values, such as individual
results of a multi-result payload operation without indirecting through
the producing op or block arguments that previously could not be easily
addressed. This is expected to support a broad class of memory-oriented
transformations such as selective bufferization, buffer assignment, and
memory transfer management.

Value handles are functionally similar to operation handles and require
similar implementation logic. The most important change concerns the
handle invalidation mechanism where operation and value handles can
affect each other.

This patch includes two cleanups that make it easier to introduce value
handles:

- `RaggedArray` structure that encapsulates the SmallVector of
ArrayRef backed by flat SmallVector logic, frequently used in the
transform interfaces implementation;

- rewrite the tests that associated payload handles with an integer
value `reinterpret_cast`ed as a pointer, which were a frequent
source of confusion and crashes when adding more debugging
facilities that can inspect the payload.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D143385

show more ...

[mlir] properly fix concurrent transform interpreter pass base

The original implementation of the transform interpreter pass base was
cloning the entire transform IR in presence of PDL-related opera

[mlir] properly fix concurrent transform interpreter pass base

The original implementation of the transform interpreter pass base was
cloning the entire transform IR in presence of PDL-related operations to
avoid concurrency issues when running the pass with the same transform
IR on multiple operations of the payload IR. The root cause of those
issues is the `transform.pdl_match` operation that was moving the PDL
pattern definition operation into a new module, consumed by the PDL
interpreter and leading to a race. Clone the pattern operation instead.
This avoids the race as well as the cost for transform IR that doesn't
use PDL.

Depends on D142729.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D142962

show more ...

[mlir] provide a base class for transform interpreter passes

The transform dialect infrastructure does not provide a default
interpreter pass and instead expects users to create their own to ensure

[mlir] provide a base class for transform interpreter passes

The transform dialect infrastructure does not provide a default
interpreter pass and instead expects users to create their own to ensure
all relevant extensions and dependent dialects are loaded. Provide a
base class for implementing such passes that includes the additional
facilities for debugging and is aware of the multithreaded nature of
pass execution.

Reviewed By: pifon2a, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D142729

show more ...

[mlir] multi-argument binding for top-level transform ops

`applyTransforms` now takes an optional mapping to be associated with
trailing block arguments of the top-level transform op, in addition to

[mlir] multi-argument binding for top-level transform ops

`applyTransforms` now takes an optional mapping to be associated with
trailing block arguments of the top-level transform op, in addition to
the payload root. This allows for more advanced forms of communication
between C++ code and the transform dialect interpreter, in particular
supplying operations without having to re-match them during
interpretation.

Reviewed By: shabalin

Differential Revision: https://reviews.llvm.org/D142559

show more ...

[mlir] add transform dialect entry point

Introduce `transform::applyTransforms` as a top-level entry point to the
Transform dialect-driven transformation infrastructure, by analogy with
`applyFull/P

[mlir] add transform dialect entry point

Introduce `transform::applyTransforms` as a top-level entry point to the
Transform dialect-driven transformation infrastructure, by analogy with
`applyFull/PartialConversion`. Clients are expected to use this function
and no longer need to maintain the transformation state. Make the
constructor of the TransformState private for that purpose.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D135681

show more ...

[mlir][Linalg] Retire LinalgStrategyTilePass and filter-based pattern.

Context: https://discourse.llvm.org/t/psa-retire-linalg-filter-based-patterns/63785

Uses of `LinalgTilingPattern::returningMat

[mlir][Linalg] Retire LinalgStrategyTilePass and filter-based pattern.

Context: https://discourse.llvm.org/t/psa-retire-linalg-filter-based-patterns/63785

Uses of `LinalgTilingPattern::returningMatchAndRewrite` are replaced by a top-level `tileWithLinalgTilingOptions` function that is marked obsolete and serves
as a temporary means to transition away from `LinalgTilingOptions`-based tiling.
LinalgTilingOptions supports too many options that have been orthogonalized with the use of the transform dialect.

Additionally, the revision introduces a `transform.structured.tile_to_scf_for` structured transform operation that is needed to properly tile `tensor.pad`
via the TilingInterface. Uses of `transform.structured.tile` will be deprecated and replaced by this new op.
This will achieve the deprecation of `linalg::tileLinalgOp`.
Context: https://discourse.llvm.org/t/psa-retire-tileandfuselinalgops-method/63850

In the process of transitioning, tests that were performing tile and distribute on tensors are retired: transformations should be orthogonalized better in the future.
In particular, tiling to specific loop types and tileAndDistribute behavior are not available via the transform ops.
The behavior is still available as part of the `tileWithLinalgTilingOptions` method to allow downstream clients to transition without breakages but is meant to be retired soon.

As more tests are ported to the transform dialect, it became necessary to introduce a test-transform-dialect-erase-schedule-pass to discard the transform specification
once applied so that e2e lowering and execution is possible.

Lastly, a number of redundant tests that were testing composition of patterns are retired as they are available with a better mechanism via the transform dialect.

Differential Revision: https://reviews.llvm.org/D135573

show more ...

[mlir] Introduce transform.alternatives op

Introduce a transform dialect op that allows one to attempt different
transformation sequences on the same piece of payload IR until one of them
succeeds.

[mlir] Introduce transform.alternatives op

Introduce a transform dialect op that allows one to attempt different
transformation sequences on the same piece of payload IR until one of them
succeeds. This op fundamentally expands the scope of possibilities in the
transform dialect that, until now, could only propagate transformation failure,
at least using in-tree operations. This requires a more detailed specification
of the execution model for the transform dialect that now indicates how failure
is handled and propagated.

Transformations described by transform operations now have tri-state results,
with some errors being fundamentally irrecoverable (e.g., generating malformed
IR) and some others being recoverable by containing ops. Existing transform ops
directly implementing the `apply` interface method are updated to produce this
directly. Transform ops with the `TransformEachTransformOpTrait` are currently
considered to produce only irrecoverable failures and will be updated
separately.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D127724

show more ...

[mlir] add a dynamic user-after-parent-freed transform dialect check

In the transform dialect, a transform IR handle may be pointing to a payload IR
operation that is an ancestor of another payload

[mlir] add a dynamic user-after-parent-freed transform dialect check

In the transform dialect, a transform IR handle may be pointing to a payload IR
operation that is an ancestor of another payload IR operation pointed to by
another handle. If such a "parent" handle is consumed by a transformation, this
indicates that the associated operation is likely rewritten, which in turn
means that the "child" handle may now be associated with a dangling pointer or
a pointer to a different operation than originally. Add a handle invalidation
mechanism to guard against such situations by reporting errors at runtime.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D127480

show more ...

[mlir][transform] Introduce transform.sequence op

Sequence is an important transform combination primitive that just indicates
transform ops being applied in a row. The simplest version requires fai

[mlir][transform] Introduce transform.sequence op

Sequence is an important transform combination primitive that just indicates
transform ops being applied in a row. The simplest version requires fails
immediately if any transformation in the sequence fails. Introducing this
operation allows one to start placing transform IR within other IR.

Depends On D123135

Reviewed By: Mogball, rriddle

Differential Revision: https://reviews.llvm.org/D123664

show more ...

[mlir] Introduce Transform dialect

This dialect provides operations that can be used to control transformation of
the IR using a different portion of the IR. It refers to the IR being
transformed as

[mlir] Introduce Transform dialect

This dialect provides operations that can be used to control transformation of
the IR using a different portion of the IR. It refers to the IR being
transformed as payload IR, and to the IR guiding the transformation as
transform IR.

The main use case for this dialect is orchestrating fine-grain transformations
on individual operations or sets thereof. For example, it may involve finding
loop-like operations with specific properties (e.g., large size) in the payload
IR, applying loop tiling to those and only those operations, and then applying
loop unrolling to the inner loops produced by the previous transformations. As
such, it is not intended as a replacement for the pass infrastructure, nor for
the pattern rewriting infrastructure. In the most common case, the transform IR
will be processed and applied to payload IR by a pass. Transformations
expressed by the transform dialect may be implemented using the pattern
infrastructure or any other relevant MLIR component.

This dialect is designed to be extensible, that is, clients of this dialect are
allowed to inject additional operations into this dialect using the newly
introduced in this patch `TransformDialectExtension` mechanism. This allows the
dialect to avoid a dependency on the implementation of the transformation as
well as to avoid introducing dialect-specific transform dialects.

See https://discourse.llvm.org/t/rfc-interfaces-and-dialects-for-precise-ir-transformation-control/60927.

Reviewed By: nicolasvasilache, Mogball, rriddle

Differential Revision: https://reviews.llvm.org/D123135

show more ...