[FileCheck] Extend -dump-input with substitutions

Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in trac

[FileCheck] Extend -dump-input with substitutions

Substitutions are already reported in the diagnostics appearing before
the input dump in the case of failed directives, and they're reported
in traces (produced by `-vv -dump-input=never`) in the case of
successful directives. However, those reports are not always
convenient to view while investigating the input dump, so this patch
adds the substitution report to the input dump too. For example:

```
$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: [[VERB]] [[WHAT]]

$ FileCheck -vv -DVERB=goodbye check < input |& tail -8
<<<<<<
1: hello world
check:1 ^~~~~~~~~~~
2: goodbye word
check:2'0 X~~~~~~~~~~~ error: no match found
check:2'1 with "VERB" equal to "goodbye"
check:2'2 with "WHAT" equal to "world"
>>>>>>
```

Without this patch, the location reported for a substitution for a
directive match is the directive's full match range. This location is
misleading as it implies the substitution itself matches that range.
This patch changes the reported location to just the match range start
to suggest the substitution is known at the start of the match. (As
in the above example, input dumps don't mark any range for
substitutions. The location info in that case simply identifies the
right line for the annotation.)

Reviewed By: mehdi_amini, thopre

Differential Revision: https://reviews.llvm.org/D83650

show more ...

[FileCheck] Fix up -dump-input* docs

In FileCheck.rst, add `-dump-input-context` and `-dump-input-filter`,
and fix some `-dump-input` documentation.

In `FileCheck -help`, `cl::value_desc("kind")` i

[FileCheck] Fix up -dump-input* docs

In FileCheck.rst, add `-dump-input-context` and `-dump-input-filter`,
and fix some `-dump-input` documentation.

In `FileCheck -help`, `cl::value_desc("kind")` is being ignored for
`-dump-input-filter`, so just drop it.

Extend `-dump-input=help` to mention FILECHECK_OPTS.

show more ...

[FileCheck] Implement -dump-input-filter

This makes the input dump filtering implemented by D82203 more
configurable. D82203 enables filtering out everything but the initial
input lines of error di

[FileCheck] Implement -dump-input-filter

This makes the input dump filtering implemented by D82203 more
configurable. D82203 enables filtering out everything but the initial
input lines of error diagnostics (plus some context). This patch
enables including any line with any kind of annotation.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83097

show more ...

[FileCheck] In input dump, elide only if ellipsis is shorter

For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leadi

[FileCheck] In input dump, elide only if ellipsis is shorter

For example, given `-dump-input-context=3 -vv`, the following now
shows more leading context for the error than requested because a
leading ellipsis would occupy the same number of lines as it would
elide:

```
<<<<<<
1: foo6
2: foo5
3: foo4
4: foo3
5: foo2
6: foo1
7: hello world
check:1 ^~~~~
check:2 X~~~~ error: no match found
8: foo1
check:2 ~~~~
9: foo2
check:2 ~~~~
10: foo3
check:2 ~~~~
.
.
.
>>>>>>
```

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D83526

show more ...

[FileCheck] Implement -dump-input-context

This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

[FileCheck] Implement -dump-input-context

This patch is motivated by discussions at each of:

* <https://reviews.llvm.org/D81422>
* <http://lists.llvm.org/pipermail/llvm-dev/2020-June/142369.html>

When input is dumped as specified by `-dump-input=fail`, this patch
filters the dump to show only input lines that are the starting lines
of error diagnostics plus the number of contextual lines specified
`-dump-input-context` (defaults to 5).

When `-dump-input=always`, there might be not be any errors, so all
input lines are printed, as without this patch.

Here's some sample output with `-dump-input-context=3 -vv`:

```
<<<<<<
.
.
.
13: foo
14: foo
15: hello world
check:1 ^~~~~~~~~~~
16: foo
check:2'0 X~~ error: no match found
17: foo
check:2'0 ~~~
18: foo
check:2'0 ~~~
19: foo
check:2'0 ~~~
.
.
.
27: foo
check:2'0 ~~~
28: foo
check:2'0 ~~~
29: foo
check:2'0 ~~~
30: goodbye word
check:2'0 ~~~~~~~~~~~~
check:2'1 ? possible intended match
31: foo
check:2'0 ~~~
32: foo
check:2'0 ~~~
33: foo
check:2'0 ~~~
.
.
.
>>>>>>
```

Reviewed By: mehdi_amini, arsenm, jhenderson, rsmith, SjoerdMeijer, Meinersbur, lattner

Differential Revision: https://reviews.llvm.org/D82203

show more ...

[FileCheck] Improve -dump-input documentation

Document the default of `fail` in `-help`. Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their ful

[FileCheck] Improve -dump-input documentation

Document the default of `fail` in `-help`. Extend `-dump-input=help`
to help users find related command-line options, but let `-help`
provide their full documentation.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D83091

show more ...

[FileCheck] Permit multiple -v or -vv

`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite ha

[FileCheck] Permit multiple -v or -vv

`FILECHECK_OPTS` was implemented so that a test runner, such as CI,
can specify FileCheck debugging options, such as `-v` and `-vv`.
However, if a test suite has a FileCheck call that already specifies
`-v` or `-vv`, then that call will fail if `FILECHECK_OPTS` also
specifies it.

For `-vv`, this problem already exists:

`clang/test/CodeGen/aarch64-v8.2a-fp16-intrinsics-constrained.c`

It's not yet clear if the `-vv` in that test was intentional, but this
usage shouldn't fail anyway. It's already true that FileCheck permits
`-vv` and `-v` together even though `-vv` implies `-v`.

Compare D70784, which fixed the same problem for `-dump-input`.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D82601

show more ...

[FileCheck][NFC] Remove redundant DumpInputDefault

Reviewed By: mehdi_amini, jhenderson

Differential Revision: https://reviews.llvm.org/D82480

Change filecheck default to dump input on failure

Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which real

Change filecheck default to dump input on failure

Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422

show more ...

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` d

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:

```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276

show more ...

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness v

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375

show more ...

Revert "[FileCheck] Make invalid prefix diagnostics more precise"

This reverts commit a78e13745d4ee4a42e41ebbe698159f651515fc5 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_

Revert "[FileCheck] Make invalid prefix diagnostics more precise"

This reverts commit a78e13745d4ee4a42e41ebbe698159f651515fc5 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

show more ...

Revert "[FileCheck] Support comment directives"

This reverts commit 9a9a5f9893c8db05cebc8818eb8485bff61f7c74 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-check

Revert "[FileCheck] Support comment directives"

This reverts commit 9a9a5f9893c8db05cebc8818eb8485bff61f7c74 to try to
fix a bot:

http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/23489

show more ...

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` d

[FileCheck] Support comment directives

Sometimes you want to disable a FileCheck directive without removing
it entirely, or you want to write comments that mention a directive by
name. The `COM:` directive makes it easy to do this. For example,
you might have:

```
; X32: pinsrd_1:
; X32: pinsrd $1, 4(%esp), %xmm0

; COM: FIXME: X64 isn't working correctly yet for this part of codegen, but
; COM: X64 will have something similar to X32:
; COM:
; COM: X64: pinsrd_1:
; COM: X64: pinsrd $1, %edi, %xmm0
```

Without this patch, you need to use some combination of rewording and
directive syntax mangling to prevent FileCheck from recognizing the
commented occurrences of `X32:` and `X64:` above as directives.
Moreover, FileCheck diagnostics have been proposed that might complain
about the occurrences of `X64` that don't have the trailing `:`
because they look like directive typos:

<http://lists.llvm.org/pipermail/llvm-dev/2020-April/140610.html>

I think dodging all these problems can prove tedious for test authors,
and directive syntax mangling already makes the purpose of existing
test code unclear. `COM:` can avoid all these problems.

This patch also updates the small set of existing tests that define
`COM` as a check prefix:

- clang/test/CodeGen/default-address-space.c
- clang/test/CodeGenOpenCL/addr-space-struct-arg.cl
- clang/test/Driver/hip-device-libs.hip
- llvm/test/Assembler/drop-debug-info-nonzero-alloca.ll

I think lit should support `COM:` as well. Perhaps `clang -verify`
should too.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79276

show more ...

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness v

[FileCheck] Make invalid prefix diagnostics more precise

This will prove especially helpful after D79276, which introduces
comment prefixes. Specifically, identifying whether there's a
uniqueness violation will be helpful as prefixes will be required to
be unique across both check prefixes and comment prefixes.

Also, remove a related comment about `cl::list` that no longer seems
relevant now that FileCheck is also a library.

Reviewed By: jhenderson, thopre

Differential Revision: https://reviews.llvm.org/D79375

show more ...

[FileCheck] - Refactor the code related to string arrays. NFCI.

There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
an

[FileCheck] - Refactor the code related to string arrays. NFCI.

There are few `std::vector<std::string>` members in
`FileCheckRequest`. This patch changes these arrays to `std::vector<StringRef>`
and refactors the code related to cleanup/improve/simplify it.

Differential revision: https://reviews.llvm.org/D78202

show more ...

[FileCheck] Fix --dump-input annotation sort per input line

Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line. That seem

[FileCheck] Fix --dump-input annotation sort per input line

Without this patch, `--dump-input` annotations on a single input line
are sorted by the associated directive's check-file line. That seemed
fine because that's often identical to the order in which FileCheck
looks for matches for those directives.

The first problem is that an `--implicit-check-not` pattern has no
check-file line. The logical equivalent is sorting in command-line
order, but that's not implemented.

The second problem is that, unlike a directive, an
`--implicit-check-not` pattern applies at many points, between many
different pairs of directives. However, sorting in command-line order
gathers all its associated diagnostics together at one point in an
input line's list of annotations.

In general, it seems to be easier to understand FileCheck's logic when
annotations on a single input line are sorted in the order FileCheck
produced the associated diagnostics, so this patch makes that change.
As documented in the patch, the annotation sort order is also
especially relevant to `CHECK-LABEL`, `CHECK-NOT`, and `CHECK-DAG`, so
this patch updates or extends tests to check the sort makes sense for
them. (However, the sort for `CHECK-DAG` annotations should not
actually be altered by this patch.)

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77607

show more ...

[FileCheck] Fix --dump-input implicit pattern location

Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
1: foo ba

[FileCheck] Fix --dump-input implicit pattern location

Currently, `--dump-input` implies that all `--implicit-check-not`
patterns appear on line 1 by printing annotations like:

```
1: foo bar baz
not:1 !~~ error: no match expected
```

This patch changes that to:

```
1: foo bar baz
not:imp1 !~~ error: no match expected
```

`imp1` indicates the first `--implicit-check-not` pattern.

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D77605

show more ...

Improve error message of FileCheck when stdin is empty

Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson,

Improve error message of FileCheck when stdin is empty

Summary: Replace '-' in the error message with <stdin>. This is also consistent with another error message in the code.

Reviewers: jhenderson, probinson, jdenny, grimar, arichardson

Reviewed By: jhenderson

Subscribers: thopre, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73793

show more ...

[FileCheck] Given multiple -dump-input, prefer most verbose

Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-in

[FileCheck] Given multiple -dump-input, prefer most verbose

Problem: `FILECHECK_OPTS` was implemented so that a test runner, such
as a bot, can specify FileCheck debugging options, such as
`-dump-input=fail`. However, some existing test suites have FileCheck
calls that already specify `-dump-input=fail` or `-dump-input=always`.
Without this patch, such tests fail under such a test runner because
FileCheck doesn't accept multiple occurrences of `-dump-input`.

Solution: This patch permits multiple occurrences of `-dump-input` by
assigning precedence to its values in the following descending order:
`help`, `always`, `fail`, and `never`. That is, any occurrence of
`help` always obtains help, and otherwise the behavior is similar to
`-v` vs. `-vv` in that the option specifying the greatest verbosity
has precedence.

Rationale: My justification for the new behavior is as follows. I
have not experienced use cases where, either as a test runner or as a
test author, I want to **limit** the permitted debugging verbosity
(except as a test author in FileCheck's or lit's test suites where the
FileCheck debugging output itself is under test, but the solution
there is `env FILECHECK_OPTS=`, and I imagine we should use the same
solution anywhere else this need might occur). Of course, as either a
test runner or test author, it is useful to **increase** debugging
verbosity.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D70784

show more ...

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case n

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374538

show more ...

Revert "[FileCheck] Implement --ignore-case option."

This reverts commit r374339. It broke tests:
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19066

llvm-svn: 374359

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case n

[FileCheck] Implement --ignore-case option.

The FileCheck utility is enhanced to support a `--ignore-case`
option. This is useful in cases where the output of Unix tools
differs in case (e.g. case not specified by Posix).

Reviewers: Bigcheese, jakehehrlich, rupprecht, espindola, alexshap, jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D68146

llvm-svn: 374339

show more ...

[FileCheck] Remove implementation types from API

Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation

[FileCheck] Remove implementation types from API

Summary:
Remove use of FileCheckPatternContext and FileCheckString concrete types
from FileCheck API to allow moving it and the other implementation only
only declarations into a private header file.

Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D68186

llvm-svn: 373211

show more ...

[FileCheck] Document FILECHECK_OPTS in -help

Reviewed By: thopre

Differential Revision: https://reviews.llvm.org/D65707

llvm-svn: 368787