[Verifier] Significantly speed up IsolatedFromAbove checking. NFC.

The implementation had a couple of problems, including checking
"isProperAncestor" in an inefficient way. It also recursed into
ot

[Verifier] Significantly speed up IsolatedFromAbove checking. NFC.

The implementation had a couple of problems, including checking
"isProperAncestor" in an inefficient way. It also recursed into
other "isolated from above" ops. In the case of CIRCT, we get
three levels of isolated ops:

mlir::ModuleOp
firrtl::CircuitOp
firrtl::FModuleOp

The verification for module would recurse into the circuits and
fmodules checking them. The verifier hook for circuit would
recurse into all the modules reverifying them, fmoduleop would
then reverify them. The same happens for mlir::ModuleOp and Func.

While here, fix an old design problem: IsolatedFromAbove checking
was implemented by a method on the Region class, which isn't
actually general and isn't used by anything else. Move it over
to be a trait impl verifier method like the others and specialize
it for its task.

Differential Revision: https://reviews.llvm.org/D103345

show more ...

Fix deletion of operations through the rewriter in a pattern matching a consumer operation

This allows for the conversion to match `A(B()) -> C()` with a pattern matching
`A` and marking `B` for del

Fix deletion of operations through the rewriter in a pattern matching a consumer operation

This allows for the conversion to match `A(B()) -> C()` with a pattern matching
`A` and marking `B` for deletion.

Also add better assertions when an operation is erased while still having uses.

Differential Revision: https://reviews.llvm.org/D99442

show more ...

[mlir] verify that operand/result_segment_sizes attributes have i32 element

This is an assumption that is made in numerous places in the code. In
particular, in the code generated by mlir-tblgen for

[mlir] verify that operand/result_segment_sizes attributes have i32 element

This is an assumption that is made in numerous places in the code. In
particular, in the code generated by mlir-tblgen for operand/result accessors
in ops with attr-sized operand or result lists. Make sure to verify this
assumption.

Note that the operation traits are verified before running the custom op
verifier, which can expect the trait verifier to have passed, but some traits
may be verified before the AttrSizedOperand/ResultTrait and should not make
such assumptions.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D99183

show more ...

[MLIR] Allow compatible shapes in `Elementwise` operations

Differential Revision: https://reviews.llvm.org/D98186

[mlir] Correct verifyCompatibleShapes

verifyCompatibleShapes is not transitive. Create an n-ary version and
update SameOperandShapes and SameOperandAndResultShapes traits to use
it.

Differential Re

[mlir] Correct verifyCompatibleShapes

verifyCompatibleShapes is not transitive. Create an n-ary version and
update SameOperandShapes and SameOperandAndResultShapes traits to use
it.

Differential Revision: https://reviews.llvm.org/D98331

show more ...

[mlir][IR] Refactor the internal implementation of Value

The current implementation of Value involves a pointer int pair with several different kinds of owners, i.e. BlockArgumentImpl*, Operation *,

[mlir][IR] Refactor the internal implementation of Value

The current implementation of Value involves a pointer int pair with several different kinds of owners, i.e. BlockArgumentImpl*, Operation *, TrailingOpResult*. This design arose from the desire to save memory overhead for operations that have a very small number of results (generally 0-2). There are, unfortunately, many problematic aspects of the current implementation that make Values difficult to work with or just inefficient.

Operation result types are stored as a separate array on the Operation. This is very inefficient for many reasons: we use TupleType for multiple results, which can lead to huge amounts of memory usage if multi-result operations change types frequently(they do). It also means that simple methods like Value::getType/Value::setType now require complex logic to get to the desired type.

Value only has one pointer bit free, severely limiting the ability to use it in things like PointerUnion/PointerIntPair. Given that we store the kind of a Value along with the "owner" pointer, we only leave one bit free for users of Value. This creates situations where we end up nesting PointerUnions to be able to use Value in one.

As noted above, most of the methods in Value need to branch on at least 3 different cases which is both inefficient, possibly error prone, and verbose. The current storage of results also creates problems for utilities like ValueRange/TypeRange, which want to efficiently store base pointers to ranges (of which Operation* isn't really useful as one).

This revision greatly simplifies the implementation of Value by the introduction of a new ValueImpl class. This class contains all of the state shared between all of the various derived value classes; i.e. the use list, the type, and the kind. This shared implementation class provides several large benefits:

* Most of the methods on value are now branchless, and often one-liners.

* The "kind" of the value is now stored in ValueImpl instead of Value
This frees up all of Value's pointer bits, allowing for users to take full advantage of PointerUnion/PointerIntPair/etc. It also allows for storing more operation results as "inline", 6 now instead of 2, freeing up 1 word per new inline result.

* Operation result types are now stored in the result, instead of a side array
This drops the size of zero-result operations by 1 word. It also removes the memory crushing use of TupleType for operations results (which could lead up to hundreds of megabytes of "dead" TupleTypes in the context). This also allowed restructured ValueRange, making it simpler and one word smaller.

This revision does come with two conceptual downsides:
* Operation::getResultTypes no longer returns an ArrayRef<Type>
This conceptually makes some usages slower, as the iterator increment is slightly more complex.
* OpResult::getOwner is slightly more expensive, as it now requires a little bit of arithmetic

From profiling, neither of the conceptual downsides have resulted in any perceivable hit to performance. Given the advantages of the new design, most compiles are slightly faster.

Differential Revision: https://reviews.llvm.org/D97804

show more ...

Split `ElementwiseMappable` trait into four more precise traits.

Some elementwise operations are not scalarizable, vectorizable, or tensorizable.
Split `ElementwiseMappable` trait into the following

Split `ElementwiseMappable` trait into four more precise traits.

Some elementwise operations are not scalarizable, vectorizable, or tensorizable.
Split `ElementwiseMappable` trait into the following, more precise traits.
- `Elementwise`
- `Scalarizable`
- `Vectorizable`
- `Tensorizable`
This allows for reuse of `Elementwise` in dialects like HLO.

Differential Revision: https://reviews.llvm.org/D97674

show more ...

Revert "Remove use of tuple for multiresult type storage"

This reverts commit 08f0764ff551c5aa2486c40871453e1ff40fb679.

Remove use of tuple for multiresult type storage

Move the results in line with the op instead. This results in each
operation having its own types recorded vs single tuple type, but comes
at benefit

Remove use of tuple for multiresult type storage

Move the results in line with the op instead. This results in each
operation having its own types recorded vs single tuple type, but comes
at benefit that every mutation doesn't incurs uniquing. Ran into cases
where updating result type of operation led to very large memory usage.

Differential Revision: https://reviews.llvm.org/D97652

show more ...

[mlir] Simplify various pieces of code now that Identifier has access to the Context/Dialect

This also exposed a bug in Dialect loading where it was not correctly identifying identifiers that had th

[mlir] Simplify various pieces of code now that Identifier has access to the Context/Dialect

This also exposed a bug in Dialect loading where it was not correctly identifying identifiers that had the dialect namespace as a prefix.

Differential Revision: https://reviews.llvm.org/D97431

show more ...

[mlir] Refactor InterfaceMap to use a sorted vector of interfaces, as opposed to a DenseMap

A majority of operations have a very small number of interfaces, which means that the cost of using a hash

[mlir] Refactor InterfaceMap to use a sorted vector of interfaces, as opposed to a DenseMap

A majority of operations have a very small number of interfaces, which means that the cost of using a hash map is generally larger for interface lookups than just a binary search. In the future when there are a number of operations with large amounts of interfaces, we can switch to a hybrid approach that optimizes lookups based on the number of interfaces. For now, however, a binary search is the best approach.

This dropped compile time on a largish TF MLIR module by 20%(half a second).

Differential Revision: https://reviews.llvm.org/D96085

show more ...

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days

[mlir][IR] Remove the concept of `OperationProperties`

These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days. Most of these properties are already checked via traits in most of the code. It is better to align the system around traits, and improve the performance/cost of traits in general.

Differential Revision: https://reviews.llvm.org/D96088

show more ...

Revert "Revert "Reorder MLIRContext location in BuiltinAttributes.h""

This reverts commit 511dd4f4383b1c2873beac4dbea2df302f1f9d0c along with
a couple fixes.

Original message:
Now the context is th

Revert "Revert "Reorder MLIRContext location in BuiltinAttributes.h""

This reverts commit 511dd4f4383b1c2873beac4dbea2df302f1f9d0c along with
a couple fixes.

Original message:
Now the context is the first, rather than the last input.

This better matches the rest of the infrastructure and makes
it easier to move these types to being declaratively specified.

Phabricator: https://reviews.llvm.org/D96111

show more ...

Revert "Reorder MLIRContext location in BuiltinAttributes.h"

This reverts commit 7827753f9810e846fb702f3e8dcff0bfb37344e1.

Reorder MLIRContext location in BuiltinAttributes.h

Now the context is the first, rather than the last input.

This better matches the rest of the infrastructure and makes
it easier to move these ty

Reorder MLIRContext location in BuiltinAttributes.h

Now the context is the first, rather than the last input.

This better matches the rest of the infrastructure and makes
it easier to move these types to being declaratively specified.

Differential Revision: https://reviews.llvm.org/D96111

show more ...

Remove deprecated methods from OpState.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D95123

[mlir] Add an interface for Cast-Like operations

A cast-like operation is one that converts from a set of input types to a set of output types. The arity of the inputs may be from 0-N, whereas the a

[mlir] Add an interface for Cast-Like operations

A cast-like operation is one that converts from a set of input types to a set of output types. The arity of the inputs may be from 0-N, whereas the arity of the outputs may be anything from 1-N. Cast-like operations are removable in cases where they produce a "no-op", i.e when the input types and output types match 1-1.

Differential Revision: https://reviews.llvm.org/D94831

show more ...

[mlir] Mark methods from mlir::OpState that just forward to mlir::Operation as deprecated.

The functions will be removed by January 20th.

All call sites within MLIR have been converted in previous

[mlir] Mark methods from mlir::OpState that just forward to mlir::Operation as deprecated.

The functions will be removed by January 20th.

All call sites within MLIR have been converted in previous changes.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D94191

show more ...

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

T

[mlir] Remove the MutableDictionaryAttr class

This class used to serve a few useful purposes:
* Allowed containing a null DictionaryAttr
* Provided some simple mutable API around a DictionaryAttr

The first of which is no longer an issue now that there is much better caching support for attributes in general, and a cache in the context for empty dictionaries. The second results in more trouble than it's worth because it mutates the internal dictionary on every action, leading to a potentially large number of dictionary copies. NamedAttrList is a much better alternative for the second use case, and should be modified as needed to better fit it's usage as a DictionaryAttrBuilder.

Differential Revision: https://reviews.llvm.org/D93442

show more ...

[mlir] Move `std.tensor_cast` -> `tensor.cast`.

This is almost entirely mechanical.

Differential Revision: https://reviews.llvm.org/D93357

[mlir][IR][NFC] Move context/location parameters of builtin Type::get methods to the start of the parameter list

This better matches the rest of the infrastructure, is much simpler, and makes it eas

[mlir][IR][NFC] Move context/location parameters of builtin Type::get methods to the start of the parameter list

This better matches the rest of the infrastructure, is much simpler, and makes it easier to move these types to being declaratively specified.

Differential Revision: https://reviews.llvm.org/D93432

show more ...

[mlir][IR] Move the storage for results to before the Operation instead of after.

Trailing objects are really nice for storing additional data inline with the main class, and is something that we he

[mlir][IR] Move the storage for results to before the Operation instead of after.

Trailing objects are really nice for storing additional data inline with the main class, and is something that we heavily take advantage of for Operation(and many other classes). To get the address of the inline data you need to compute the address by doing some pointer arithmetic taking into account any objects stored before the object you want to access. Most classes keep the count of the number of objects, so this is relatively cheap to compute. This is not the case for results though, which have two different types(inline and trailing) that are not necessarily as cheap to compute as the count for other objects. This revision moves the storage for results to before the operation and stores them in reverse order. This allows for getting results to still be very fast given that they are never iterated directly in order, and also greatly improves the speed when accessing the other trailing objects of an operation(operands/regions/blocks/etc).

This reduced compile time when compiling a decently sized mlir module by about ~400ms, or 2.17s -> 1.76s.

Differential Revision: https://reviews.llvm.org/D92687

show more ...

[mlir][Types][NFC] Move all of the builtin Type classes to BuiltinTypes.h

This is part of a larger refactoring the better congregates the builtin structures under the BuiltinDialect. This also remov

[mlir][Types][NFC] Move all of the builtin Type classes to BuiltinTypes.h

This is part of a larger refactoring the better congregates the builtin structures under the BuiltinDialect. This also removes the problematic "standard" naming that clashes with the "standard" dialect, which is not defined within IR/. A temporary forward is placed in StandardTypes.h to allow time for downstream users to replaced references.

Differential Revision: https://reviews.llvm.org/D92435

show more ...

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the

[mlir][PDL] Add support for PDL bytecode and expose PDL support to OwningRewritePatternList

PDL patterns are now supported via a new `PDLPatternModule` class. This class contains a ModuleOp with the pdl::PatternOp operations representing the patterns, as well as a collection of registered C++ functions for native constraints/creations/rewrites/etc. that may be invoked via the pdl patterns. Instances of this class are added to an OwningRewritePatternList in the same fashion as C++ RewritePatterns, i.e. via the `insert` method.

The PDL bytecode is an in-memory representation of the PDL interpreter dialect that can be efficiently interpreted/executed. The representation of the bytecode boils down to a code array(for opcodes/memory locations/etc) and a memory buffer(for storing attributes/operations/values/any other data necessary). The bytecode operations are effectively a 1-1 mapping to the PDLInterp dialect operations, with a few exceptions in cases where the in-memory representation of the bytecode can be more efficient than the MLIR representation. For example, a generic `AreEqual` bytecode op can be used to represent AreEqualOp, CheckAttributeOp, and CheckTypeOp.

The execution of the bytecode is split into two phases: matching and rewriting. When matching, all of the matched patterns are collected to avoid the overhead of re-running parts of the matcher. These matched patterns are then considered alongside the native C++ patterns, which rewrite immediately in-place via `RewritePattern::matchAndRewrite`, for the given root operation. When a PDL pattern is matched and has the highest benefit, it is passed back to the bytecode to execute its rewriter.

Differential Revision: https://reviews.llvm.org/D89107

show more ...

[mlir] Add an assertion on creating an Operation with null result types

Null types are commonly used as an error marker. Catch them in the constructor
of Operation if they are present in the result

[mlir] Add an assertion on creating an Operation with null result types

Null types are commonly used as an error marker. Catch them in the constructor
of Operation if they are present in the result type list, as otherwise this
could lead to further surprising behavior when querying op result types.

Fix AsyncToLLVM and StandardToLLVM that were using null types when constructing
operations.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D91770

show more ...