[mlir] Fix generating checks for multiple funcs in generate-test-checks

This regressed in D154458 due to the added tracking of used variable
names that now also has to be cleared alongside the count

[mlir] Fix generating checks for multiple funcs in generate-test-checks

This regressed in D154458 due to the added tracking of used variable
names that now also has to be cleared alongside the counter.

Reviewed By: rafaelubalmw, c-rhodes, awarzynski

Differential Revision: https://reviews.llvm.org/D156547

show more ...

New features and bug fix in MLIR test generation tool

- Option `--variable_names <names>` allows the user to pass names for FileCheck
regexps representing variables. Variable names are separated b

New features and bug fix in MLIR test generation tool

- Option `--variable_names <names>` allows the user to pass names for FileCheck
regexps representing variables. Variable names are separated by commas, and
empty names can be used to generate specific variable names automatically.
For example, `--variable-names arg_0,arg_1,,,result` will produce regexp names
`ARG_0`, `ARG_1`, `VAR_0`, `VAR_1`, `RESULT`, `VAR_2`, `VAR_3`, ...

- Option '--attribute_names <names>' can be used to generate global regexp names
to represent attributes. Useful for affine maps. Same behavior as
'--variable_names'.

- Bug fixed for scope detection of SSA variables in ops with nested regions that
return SSA values (e.g., 'linalg.generic'). Originally, returned SSA values were
inserted in the nested scope.

This version of the tool has been used to generate unit tests for the following
patch: https://reviews.llvm.org/D153291

For example, the main body of the test named 'test_select_2d_one_dynamic' was
generated using the following command:

```
$ mlir-opt -pass-pipeline='builtin.module(func.func(tosa-to-linalg))' test_select_2d_one_dynamic.tosa.mlir | generate-test-checks.py --attribute_names map0,map1,map2 --variable_names arg0,arg1,arg2,const1,arg0_dim1,arg1_dim1,,arg2_dim1,max_dim1,,,arg0_broadcast,,,,,,,arg1_broadcast,,,,,,,arg2_broadcast,,,,,,result
```

Reviewed By: eric-k256

Differential Revision: https://reviews.llvm.org/D154458

show more ...

[NFC][Py Reformat] Reformat python files in mlir subdir

This is an ongoing series of commits that are reformatting our
Python code.

Reformatting is done with `black`.

If you end up having problems

[NFC][Py Reformat] Reformat python files in mlir subdir

This is an ongoing series of commits that are reformatting our
Python code.

Reformatting is done with `black`.

If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.

If you run into any problems, post to discourse about it and
we will try to help.

RFC Thread below:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Differential Revision: https://reviews.llvm.org/D150782

show more ...

Update MLIR generate-test-checks.py to add the notice from the source into the generated file

Folks may not read the source of the tool and miss these instructions.

Differential Revision: https://r

Update MLIR generate-test-checks.py to add the notice from the source into the generated file

Folks may not read the source of the tool and miss these instructions.

Differential Revision: https://reviews.llvm.org/D110082

show more ...

[MLIR] Add documentation for generate-check-lines.py

[MLIR] Remove generated spaces at eof for generate-test-checks.py.

[MLIR] Rework generate-test-checks.py to attach CHECK lines to the source (test) file.

Summary:
This patch adds --source flag to indicate the source file. Then it tries to find insert
points in the

[MLIR] Rework generate-test-checks.py to attach CHECK lines to the source (test) file.

Summary:
This patch adds --source flag to indicate the source file. Then it tries to find insert
points in the source file and insert corresponding checks at those places.

Example output from Tensorflow XLA:

// -----

// CHECK-LABEL: func @main.3(
// CHECK-SAME: %[[VAL_0:.*]]: memref<2x2xf32> {xla_lhlo.params = 0 : index},
// CHECK-SAME: %[[VAL_1:.*]]: memref<16xi8> {xla_lhlo.alloc = 0 : index, xla_lhlo.liveout = true}) {
// CHECK: %[[VAL_2:.*]] = constant 0 : index
// CHECK: %[[VAL_3:.*]] = constant 0 : index
// CHECK: %[[VAL_4:.*]] = std.view %[[VAL_1]]{{\[}}%[[VAL_3]]][] : memref<16xi8> to memref<2x2xf32>
// CHECK: "xla_lhlo.tanh"(%[[VAL_0]], %[[VAL_4]]) : (memref<2x2xf32>, memref<2x2xf32>) -> ()
// CHECK: return
// CHECK: }
func @main(%value0: tensor<2x2xf32>) -> tensor<2x2xf32> {
%res = "xla_hlo.tanh"(%value0) : (tensor<2x2xf32>) -> tensor<2x2xf32>
return %res : tensor<2x2xf32>
}

Differential Revision: https://reviews.llvm.org/D81903

show more ...

Use python3 in generate-test-checks.py.

Summary: Upgrade to python3 in generate-test-checks.py .

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob,

Use python3 in generate-test-checks.py.

Summary: Upgrade to python3 in generate-test-checks.py .

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, msifontes

Tags: #mlir

Differential Revision: https://reviews.llvm.org/D81701

show more ...

[mlir] Change generate-test-checks.py to `%[[V:.*]]` style

Summary:
This style seems to be the preferred style lately as it plays better
with op syntaxes that include `[` and `]`.

Input:
```
module

[mlir] Change generate-test-checks.py to `%[[V:.*]]` style

Summary:
This style seems to be the preferred style lately as it plays better
with op syntaxes that include `[` and `]`.

Input:
```
module {
func @f(%arg0 : i32) -> i32 {
%0 = addi %arg0, %arg0 : i32
%1 = muli %arg0, %0 : i32
return %1 : i32
}
}
```

New output:

```
// NOTE: Assertions have been autogenerated by utils/generate-test-checks.py
// CHECK: module {

// CHECK-LABEL: func @f(
// CHECK-SAME: %[[VAL_0:.*]] : i32) -> i32 {
// CHECK: %[[VAL_1:.*]] = addi %[[VAL_0]], %[[VAL_0]] : i32
// CHECK: %[[VAL_2:.*]] = muli %[[VAL_0]], %[[VAL_1]] : i32
// CHECK: return %[[VAL_2]] : i32
// CHECK: }
// CHECK: }
```

Old output:
```
// NOTE: Assertions have been autogenerated by utils/generate-test-checks.py
// CHECK: module {

// CHECK-LABEL: func @f(
// CHECK-SAME: [[VAL_0:%.*]] : i32) -> i32 {
// CHECK: [[VAL_1:%.*]] = addi [[VAL_0]], [[VAL_0]] : i32
// CHECK: [[VAL_2:%.*]] = muli [[VAL_0]], [[VAL_1]] : i32
// CHECK: return [[VAL_2]] : i32
// CHECK: }
// CHECK: }
```

Differential Revision: https://reviews.llvm.org/D79836

show more ...

Import MLIR into the LLVM tree

Adjust License.txt file to use the LLVM license

PiperOrigin-RevId: 286906740

Fix minor spelling tweaks (NFC)

Closes tensorflow/mlir#175

PiperOrigin-RevId: 275726876

Remove unused import and two import forms

argparse was imported with both 'import' and 'import from' and string import was unused,

PiperOrigin-RevId: 270164488

Add a preprocess pass to remove sequences that are problematic with FileCheck

Add a preprocess phase to rewrite parts of the input line that may be problematic with filecheck. This change adds prepr

Add a preprocess pass to remove sequences that are problematic with FileCheck

Add a preprocess phase to rewrite parts of the input line that may be problematic with filecheck. This change adds preprocessing to escape '[[' bracket sequences, as these are used by FileCheck for variables.

PiperOrigin-RevId: 269648490

show more ...

Add a utility script to auto-generate CHECK commands for mlir test cases.

This script is a utility to add FileCheck patterns to an mlir file. The script will heuristically insert CHECK/CHECK-LABEL c

Add a utility script to auto-generate CHECK commands for mlir test cases.

This script is a utility to add FileCheck patterns to an mlir file. The script will heuristically insert CHECK/CHECK-LABEL commands for each line within the file. By default this script will also try to insert string substitution blocks for all SSA value names. The script is designed to make adding checks to a test case fast, it is *not* designed to be authoritative about what constitutes a good test!

Note: Some cases may not be handled well, e.g. operands to operations with regions, but this script is only intended to be a starting point.

Example usage:
$ generate-test-checks.py foo.mlir
$ mlir-opt foo.mlir -transformation | generate-test-checks.py

module {
func @fold_extract_element(%arg0: index) -> (f32, f16, f16, i32) {
%cst = constant 4.500000e+00 : f32
%cst_0 = constant -2.000000e+00 : f16
%cst_1 = constant 0.000000e+00 : f16
%c64_i32 = constant 64 : i32
return %cst, %cst_0, %cst_1, %c64_i32 : f32, f16, f16, i32
}
}

// CHECK-LABEL: func @fold_extract_element(
// CHECK-SAME: [[VAL_0:%.*]]: index) -> (f32, f16, f16, i32) {
// CHECK: [[VAL_1:%.*]] = constant 4.500000e+00 : f32
// CHECK: [[VAL_2:%.*]] = constant -2.000000e+00 : f16
// CHECK: [[VAL_3:%.*]] = constant 0.000000e+00 : f16
// CHECK: [[VAL_4:%.*]] = constant 64 : i32
// CHECK: return [[VAL_1]], [[VAL_2]], [[VAL_3]], [[VAL_4]] : f32, f16, f16, i32
// CHECK: }

PiperOrigin-RevId: 263242983

show more ...