[mlir][Vector] Add LLVM lowering for masked reductions

This patch adds the conversion patterns to lower masked reduction
operations to the corresponding vp intrinsics in LLVM.

Reviewed By: ftynse

[mlir][Vector] Add LLVM lowering for masked reductions

This patch adds the conversion patterns to lower masked reduction
operations to the corresponding vp intrinsics in LLVM.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D142177

show more ...

[mlir][Vector] add pass option to `convert-vector-to-llvm` to emit opaque pointers

Part of https://discourse.llvm.org/t/rfc-switching-the-llvm-dialect-and-dialect-lowerings-to-opaque-pointers/68179

[mlir][Vector] add pass option to `convert-vector-to-llvm` to emit opaque pointers

Part of https://discourse.llvm.org/t/rfc-switching-the-llvm-dialect-and-dialect-lowerings-to-opaque-pointers/68179

This patch allows the use of `convert-vector-to-llvm` with LLVM opaque pointer types. The changes required were relatively minor:
* there were a few GEPs and loads that required specifying the result or base type
* a few bitcasts, especially when interacting with memref have to not be emitted when using opaque pointers

Differential Revision: https://reviews.llvm.org/D143853

show more ...

Add generic type attribute mapping infrastructure, use it in GpuToX

Remapping memory spaces is a function often needed in type
conversions, most often when going to LLVM or to/from SPIR-V (a future

Add generic type attribute mapping infrastructure, use it in GpuToX

Remapping memory spaces is a function often needed in type
conversions, most often when going to LLVM or to/from SPIR-V (a future
commit), and it is possible that such remappings may become more
common in the future as dialects take advantage of the more generic
memory space infrastructure.

Currently, memory space remappings are handled by running a
special-purpose conversion pass before the main conversion that
changes the address space attributes. In this commit, this approach is
replaced by adding a notion of type attribute conversions
TypeConverter, which is then used to convert memory space attributes.

Then, we use this infrastructure throughout the *ToLLVM conversions.
This has the advantage of loosing the requirements on the inputs to
those passes from "all address spaces must be integers" to "all
memory spaces must be convertible to integer spaces", a looser
requirement that reduces the coupling between portions of MLIR.

ON top of that, this change leads to the removal of most of the calls
to getMemorySpaceAsInt(), bringing us closer to removing it.

(A rework of the SPIR-V conversions to use this new system will be in
a folowup commit.)

As a note, one long-term motivation for this change is that I would
eventually like to add an allocaMemorySpace key to MLIR data layouts
and then call getMemRefAddressSpace(allocaMemorySpace) in the
relevant *ToLLVM in order to ensure all alloca()s, whether incoming or
produces during the LLVM lowering, have the correct address space for
a given target.

I expect that the type attribute conversion system may be useful in
other contexts.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D142159

show more ...

[VectorToLLVM] Use std::optional in ConvertVectorToLLVM.cpp (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x

[VectorToLLVM] Use std::optional in ConvertVectorToLLVM.cpp (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

[mlir] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of ma

[mlir] Use std::nullopt instead of None (NFC)

This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Merge kDynamicSize and kDynamicSentinel into one constant.

resolve conflicts

Differential Revision: https://reviews.llvm.org/D138282

[mlir][vector] Add vector.scalable.insert/extract ops

These new operations match the semantics of
llvm.experimental.vector.insert and llvm.experimental.vector.extract.

`vector.scalable.insert` and

[mlir][vector] Add vector.scalable.insert/extract ops

These new operations match the semantics of
llvm.experimental.vector.insert and llvm.experimental.vector.extract.

`vector.scalable.insert` and `vector.scalable.extract` allow,
respectively, insert vectors into scalable vectors, and extract vectors
from scalable vectors.

The discussion about the inclusion of these operations is here:
https://discourse.llvm.org/t/rfc-interfacing-between-fixed-length-and-scalable-vectors-for-vls-vector-code-on-scalable-vector-architectures

Differential Revision: https://reviews.llvm.org/D127875

show more ...

[mlir][arith] Change dialect name from Arithmetic to Arith

Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-ml

[mlir][arith] Change dialect name from Arithmetic to Arith

Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples`

and `bazel build --config=generic_clang @llvm-project//mlir:all`.

Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D134762

show more ...

[mlir][Vector] Fix unused variable warning in release builds. NFCI.

[mlir][Vector] Support 0-D vectors in ShuffleOp

Co-authored-by: Michal Terepeta <michalt@google.com>

Reviewed-by: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115744

[mlir][Vector] Support 0-D vectors in FMAOp

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115742

Reland "[MLIR]Extend vector.gather to support n-D result"

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D132507

Revert "[MLIR]Extend vector.gather to support n-D result"

This reverts commit 0cbfd6fd1633a075dcfd1bcd8a11e1c6d2785fa8.

A test is crashing with the shared_lib config.

[MLIR]Extend vector.gather to support n-D result

Currently vector.gather only supports reading memory into a 1-D result vector.
This patch extends it to support an n-D result vector with the indices

[MLIR]Extend vector.gather to support n-D result

Currently vector.gather only supports reading memory into a 1-D result vector.
This patch extends it to support an n-D result vector with the indices, masks,
and passthroughs in n-D vectors.

As we are trying to vectorize tensor.extract with vector.gather
(https://github.com/iree-org/iree/issues/9198), it will need to gather the
elements into an n-D vector. Having vector.gather with n-D results allows us
to avoid flatten and reshape at the vectorization stage. The backends can then
decide the optimal ways to lower the vector.gather op.

Note that this is different from n-D gathering, which is about reading n-D
memory with the n-D indices. The indices here are still only 1-D offsets on
the base.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D131905

show more ...

[mlir][LLVMIR] Change ShuffleVectorOp to use assembly format

This patch moves `LLVM::ShuffleVectorOp` to assembly format and in the
process drops the extra type that can be inferred (both operand ty

[mlir][LLVMIR] Change ShuffleVectorOp to use assembly format

This patch moves `LLVM::ShuffleVectorOp` to assembly format and in the
process drops the extra type that can be inferred (both operand types
are required to be the same) and switches to a dense integer array.

The syntax change:

```
// Before
%0 = llvm.shufflevector %0, %1 [0 : i32, 0 : i32, 0 : i32, 0 : i32] : vector<4xf32>, vector<4xf32>
// After
%0 = llvm.shufflevector %0, %1 [0, 0, 0, 0] : vector<4xf32>
```

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D132038

show more ...

[mlir][LLVMIR] "Modernize" Insert/ExtractValueOp

This patch "modernizes" the LLVM `insertvalue` and `extractvalue`
operations to use DenseI64ArrayAttr, since they only require an array of
indices an

[mlir][LLVMIR] "Modernize" Insert/ExtractValueOp

This patch "modernizes" the LLVM `insertvalue` and `extractvalue`
operations to use DenseI64ArrayAttr, since they only require an array of
indices and previously there was confusion about whether to use i32 or
i64 arrays, and to use assembly format.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131537

show more ...

[MLIR] Extend vector.gather to accept tensor as base

In addition to memref, accept ranked tensor as the base operand of vector.gather, similar to vector.trasnfer_read.

This will allow us to vectori

[MLIR] Extend vector.gather to accept tensor as base

In addition to memref, accept ranked tensor as the base operand of vector.gather, similar to vector.trasnfer_read.

This will allow us to vectorize noncontiguous tensor.extract into vector.gather. Full discussion can be found here: https://github.com/iree-org/iree/issues/9198

Reviewed By: hanchung, dcaballe

Differential Revision: https://reviews.llvm.org/D130097

show more ...

Use callables directly in any_of, count_if, etc (NFC)

[mlir][VectorToLLVM] Fix bug in lowering of vector.reduce fmax/fmin

The lowering of fmax/fmin reduce was ignoring the optional accumulator.

Differential Revision: https://reviews.llvm.org/D129597

[mlir][Vector] Fix reordering of floating point adds during lower of `vector.contract`.

Adding the accumulator value after the `vector.contract` changes the
precision of the operation. This makes su

[mlir][Vector] Fix reordering of floating point adds during lower of `vector.contract`.

Adding the accumulator value after the `vector.contract` changes the
precision of the operation. This makes sure the accumulator is carried
through to `vector.reduce` (and down to LLVM).

Differential Revision: https://reviews.llvm.org/D128674

show more ...

[mlir][vector][nfc] Rename index optimizations option

We are using "enable-index-optimizations" and "indexOptimizations" as
names for an optimization that consists of using i32 for indices within
a

[mlir][vector][nfc] Rename index optimizations option

We are using "enable-index-optimizations" and "indexOptimizations" as
names for an optimization that consists of using i32 for indices within
a vector. For instance, when building a vector comparison for mask
generation. The name is confusing and suggests a scope beyond these
vector indices. This change makes the function of the option explicit
in its name.

Differential Revision: https://reviews.llvm.org/D122415

show more ...

[mlir] Flip Vector dialect accessors used to prefixed form.

This has been on _Both for a couple of weeks. Flip usages in core with
intention to flip flag to _Prefixed in follow up. Needed to add a c

[mlir] Flip Vector dialect accessors used to prefixed form.

This has been on _Both for a couple of weeks. Flip usages in core with
intention to flip flag to _Prefixed in follow up. Needed to add a couple
of helper methods in AffineOps and Linalg to facilitate a pure flag flip
in follow up as some of these classes are used in templates and so
sensitive to Vector dialect changes.

Differential Revision: https://reviews.llvm.org/D122151

show more ...

[mlir][Vector] Enable create_mask for scalable vectors

The way vector.create_mask is currently lowered is
vector-length-dependent, and therefore incompatible with scalable vector
types. This patch a

[mlir][Vector] Enable create_mask for scalable vectors

The way vector.create_mask is currently lowered is
vector-length-dependent, and therefore incompatible with scalable vector
types. This patch adds an alternative lowering path for create_mask
operations that return a scalable vector mask.

Differential Revision: https://reviews.llvm.org/D118248

show more ...

[mlir] Trim a huge number of unnecessary dependencies on the Func dialect

The Func has a large number of legacy dependencies carried over from the old
Standard dialect, which was pervasive and conta

[mlir] Trim a huge number of unnecessary dependencies on the Func dialect

The Func has a large number of legacy dependencies carried over from the old
Standard dialect, which was pervasive and contained a large number of varied
operations. With the split of the standard dialect and its demise, a lot of lingering
dead dependencies have survived to the Func dialect. This commit removes a
large majority of then, greatly reducing the dependence surface area of the
Func dialect.

show more ...

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the init

[mlir] Rename the Standard dialect to the Func dialect

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the initial
renaming (which by itself is already huge), but there are a large number
of cleanups unlocked/necessary afterwards:

* Removing a bunch of unnecessary dependencies on Func
* Cleaning up the From/ToStandard conversion passes
* Preparing for the move of FuncOp to the Func dialect

See the discussion at https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Differential Revision: https://reviews.llvm.org/D120624

show more ...