[mlir][sparse] add parallelization options to mini pipeline (#104233)

[mlir][sparse] expose emit strategy option to mini pipeline (#96238)

[mlir][sparse] support sparsifying sparse kernels to sparse-iterator-based loop (#95858)

[mlir][sparse] expose optimization flags to mini pipeline (#95158)

Some of the options only fed into the full sparse pipeline. However,
some backends prefer to use the sparse minipipeline. This cha

[mlir][sparse] expose optimization flags to mini pipeline (#95158)

Some of the options only fed into the full sparse pipeline. However,
some backends prefer to use the sparse minipipeline. This change exposes
some important optimization flags to the pass as well. This prepares
some SIMDization of PyTorch sparsified code.

show more ...

[mlir][SparseTensor] Add missing dependent dialect to pass (#88870)

This commit fixes the following error when stopping the sparse compiler
pipeline after bufferization (e.g., with `test-analysis-o

[mlir][SparseTensor] Add missing dependent dialect to pass (#88870)

This commit fixes the following error when stopping the sparse compiler
pipeline after bufferization (e.g., with `test-analysis-only`):

```
LLVM ERROR: Building op `vector.print` but it isn't known in this MLIRContext: the dialect may not be loaded or this operation hasn't been added by the dialect. See also https://mlir.llvm.org/getting_started/Faq/#registered-loaded-dependent-whats-up-with-dialects-management
```

show more ...

[mlir][sparse] allow unknown ops in one-shot bufferization in mini-pipeline (#77688)

Rationale:
Since this mini-pipeline may be used in alternative pipelines (viz.
different from the default "spar

[mlir][sparse] allow unknown ops in one-shot bufferization in mini-pipeline (#77688)

Rationale:
Since this mini-pipeline may be used in alternative pipelines (viz.
different from the default "sparsifier" pipeline) where unknown ops are
handled by alternative bufferization methods that are downstream of this
mini-pipeline, we allow unknown ops by default (failure to bufferize is
eventually apparent by failing to convert to LLVM IR).

This is part of enabling e2e testing for TORCH-MLIR tests using a
sparsifier backend

show more ...

[mlir][sparse] refactoring: using util functions to query the index to load from position array for slice-driven loop. (#73986)

[mlir][sparse][gpu] re-enable all GPU libgen tests (#72185)

Previous change no longer properly used the GPU libgen pass (even though
most tests still passed falling back to CPU). This revision puts

[mlir][sparse][gpu] re-enable all GPU libgen tests (#72185)

Previous change no longer properly used the GPU libgen pass (even though
most tests still passed falling back to CPU). This revision puts the
proper pass order into place. Also bit of a cleanup of CPU codegen vs.
libgen setup.

show more ...

[mlir][sparse] end-to-end matmul between Dense and BSR tensors (#71448)

[mlir][sparse] first end-to-end linalg.generic op on BSR (#70880)

[mlir][sparse] implements tensor.insert on sparse tensors. (#70737)

[mlir][sparse] split post-sparsification-rewriting into two passes. (#70727)

[mlir][sparse] implement sparse_tensor.reorder_coo (#68916)

As a side effect of the change, it also unifies the convertOp
implementation between lib/codegen path.

[mlir][sparse] simplify ConvertOp rewriting rules (#68350)

Canonicalize complex convertOp into multiple stages, such that it can
either be done by a direct conversion or by sorting.

[mlir][sparse] Extract `StorageSpecifierToLLVMPass` from bufferization pipeline (#68635)

`StorageSpecifierToLLVMPass` does not have to be part of the
bufferization mini pipeline. It can run after t

[mlir][sparse] Extract `StorageSpecifierToLLVMPass` from bufferization pipeline (#68635)

`StorageSpecifierToLLVMPass` does not have to be part of the
bufferization mini pipeline. It can run after the bufferization
pipeline. This is desirable because it keeps the bufferization pipeline
smaller.

Also fix incorrect bufferization API usage: `bufferizeOp` instead of
`bufferizeModuleOp` was used, even though function boundaries were
bufferized.

show more ...

[mlir][bufferization][NFC] Simplify `bufferizeOp` function signature (#68625)

Remove the `opFilter` and `copyBeforeWrite` function arguments. These
options can already be configured in the `options

[mlir][bufferization][NFC] Simplify `bufferizeOp` function signature (#68625)

Remove the `opFilter` and `copyBeforeWrite` function arguments. These
options can already be configured in the `options` object.

show more ...

[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute (#66619)

This commit removes the deallocation capabilities of
one-shot-buffe

[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute (#66619)

This commit removes the deallocation capabilities of
one-shot-bufferization. One-shot-bufferization should never deallocate
any memrefs as this should be entirely handled by the
ownership-based-buffer-deallocation pass going forward. This means the
`allow-return-allocs` pass option will default to true now,
`create-deallocs` defaults to false and they, as well as the escape
attribute indicating whether a memref escapes the current region, will
be removed. A new `allow-return-allocs-from-loops` option is added as a
temporary workaround for some bufferization limitations.

show more ...

Revert "[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute"

This reverts commit 6a91dfedeb956dfa092a6a3f411e8b02f0d5d289.

This

Revert "[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute"

This reverts commit 6a91dfedeb956dfa092a6a3f411e8b02f0d5d289.

This caused problems in downstream projects. We are reverting to give
them more time for integration.

show more ...

[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute

This is the first commit in a series with the goal to rework the
BufferDeallo

[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute

This is the first commit in a series with the goal to rework the
BufferDeallocation pass. Currently, this pass heavily relies on copies
to perform correct deallocations, which leads to very slow code and
potentially high memory usage. Additionally, there are unsupported cases
such as returning memrefs which this series of commits aims to add
support for as well.

This first commit removes the deallocation capabilities of
one-shot-bufferization.One-shot-bufferization should never deallocate any
memrefs as this should be entirely handled by the buffer-deallocation pass
going forward. This means the allow-return-allocs pass option will
default to true now, create-deallocs defaults to false and they, as well
as the escape attribute indicating whether a memref escapes the current region,
will be removed.

The documentation should w.r.t. these pass option changes should also be
updated in this commit.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D156662

show more ...

[mlir][sparse] refactor sparsification and bufferization pass into proper TD pass

Registering the SparsificationAndBufferization into a proper TD pass
has the advantage that it can be invoked and te

[mlir][sparse] refactor sparsification and bufferization pass into proper TD pass

Registering the SparsificationAndBufferization into a proper TD pass
has the advantage that it can be invoked and tested in isolation. This
change also moves some bufferization specific set up from the pipeline
file into the pass file, keeping the logic more locally.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D158219

show more ...

[mlir][sparse][gpu] first implementation of the GPU libgen approach

The sparse compiler now has two prototype strategies for GPU acceleration:

* CUDA codegen: this converts sparsified code to CUDA

[mlir][sparse][gpu] first implementation of the GPU libgen approach

The sparse compiler now has two prototype strategies for GPU acceleration:

* CUDA codegen: this converts sparsified code to CUDA threads
* CUDA libgen: this converts pre-sparsified code to cuSPARSE library calls

This revision introduces the first steps required for the second approach.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D150170

show more ...

[mlir][sparse] add create-sparse-deallocs options to match the create-deallocs in BufferizationOption.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D147010

[mlir][sparse] add missing dependent dialect.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140595

[mlir][sparse] introduce sparse_tensor::StorageSpecifierToLLVM pass

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140122

[mlir][bufferize] Implement BufferizableOpInterface for tensor.empty

The op is not bufferizable but should be analyzable (for `EliminateEmptyTensors`, which uses the bufferization infrastructure).

[mlir][bufferize] Implement BufferizableOpInterface for tensor.empty

The op is not bufferizable but should be analyzable (for `EliminateEmptyTensors`, which uses the bufferization infrastructure).

Also improve debugging functionality and error messages.

Also adds a missing pass to the sparse pipeline. (tensor.empty should be replaced with bufferization.alloc_tensor, but it sometimes used to work without depending on how the tensor.empty is used. Now we always fail explicitly.)

show more ...