[llvm][utils] Avoid 'raw_string_ostream::str()' (NFC) (#97160)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference

[llvm][utils] Avoid 'raw_string_ostream::str()' (NFC) (#97160)

Since `raw_string_ostream` doesn't own the string buffer, it is
desirable (in terms of memory safety) for users to directly reference
the string buffer rather than use `raw_string_ostream::str()`.

Work towards TODO comment to remove `raw_string_ostream::str()`.

show more ...

Add llvm::min/max_element and use it in llvm/ and mlir/ directories. (#84678)

For some reason this was missing from STLExtras.

[FileCheck] Don't use regex to find prefixes (#72237)

FileCheck currently compiles a regular expression of the form
`Prefix1|Prefix2|...` and uses it to find the next prefix in the input.

If we

[FileCheck] Don't use regex to find prefixes (#72237)

FileCheck currently compiles a regular expression of the form
`Prefix1|Prefix2|...` and uses it to find the next prefix in the input.

If we had a fast regex implementation, this would be a useful thing to
do, as the regex implementation would be able to match multiple prefixes
more efficiently than a naive approach. However, with our actual regex
implementation, finding the prefixes basically becomes O(InputLen *
RegexLen * LargeConstantFactor), which is a lot worse than a simple
string search.

Replace the regex with StringRef::find(), and keeping track of the next
position of each prefix. There are various ways this could be improved
on, but it's already significantly faster that the previous approach.

For me, this improves check-llvm time from 138.5s to 132.5s, so by
around 4-5%.

For vector-interleaved-load-i16-stride-7.ll in particular, test time
drops from 5s to 2.5s.

show more ...

[FileCheck] Fix performance-for-range-copy issues. NFC

[llvm] Remove unneeded cl::ZeroOrMore for cl::opt options. NFC

Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the err

[llvm] Remove unneeded cl::ZeroOrMore for cl::opt options. NFC

Some cl::ZeroOrMore were added to avoid the `may only occur zero or one times!`
error. More were added due to cargo cult. Since the error has been removed,
cl::ZeroOrMore is unneeded.

Also remove cl::init(false) while touching the lines.

show more ...

[FileCheck] GetCheckTypeAbbreviation() to handle the misspelled case.

Also fix directives not covered by D125604.

Cleanup includes: DebugInfo & CodeGen

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121332

[FileCheck] Fix initialized but never used static analyzer warning. NFC.

Remove superfluous variable initialization, the variable is assigned by both paths immediately afterward.

utils: Remove some no-op raw_string_ostream flush calls, NFC

Since 65b13610a5226b84889b923bae884ba395ad084d, raw_string_ostream has
been unbuffered by default. Based on an audit of llvm/utils/, this

utils: Remove some no-op raw_string_ostream flush calls, NFC

Since 65b13610a5226b84889b923bae884ba395ad084d, raw_string_ostream has
been unbuffered by default. Based on an audit of llvm/utils/, this
commit removes every call to `raw_string_ostream::flush()` and any call
to `raw_string_ostream::str()` whose result is ignored or that doesn't
help with clarity.

I left behind a few calls to `str()`. In these cases, the underlying
std::string was declared pretty far away and never used again, whereas
stream recently had its last write. The code is easier to read as-is;
the no-op call to `flush()` inside `str()` isn't harmful, and when
https://reviews.llvm.org/D115421 lands it'll be gone anyway.

show more ...

[FileCheck] Try to fix buildbot failures caused by c7c542e8f306

For example,

<https://lab.llvm.org/buildbot/#/builders/132/builds/3929>

has this diagnostic:

```
/opt/gcc/9.3.0/snos/include/g++/bi

[FileCheck] Try to fix buildbot failures caused by c7c542e8f306

For example,

<https://lab.llvm.org/buildbot/#/builders/132/builds/3929>

has this diagnostic:

```
/opt/gcc/9.3.0/snos/include/g++/bits/stl_tree.h:780:8: error: static assertion failed: comparison object must be invocable as const
780 | is_invocable_v<const _Compare&, const _Key&, const _Key&>,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

show more ...

[FileCheck] Fix -dump-input per-pattern diagnostic indexing

In input dump annotations, `check:2'1` indicates diagnostic 1 for the
`CHECK` directive on check file line 2. Without this patch,
`-dump-

[FileCheck] Fix -dump-input per-pattern diagnostic indexing

In input dump annotations, `check:2'1` indicates diagnostic 1 for the
`CHECK` directive on check file line 2. Without this patch,
`-dump-input` computes the diagnostic index with the assumption that
FileCheck *consecutively* produces all diagnostics for the same
pattern. Already, that can be a false assumption, as in the examples
below. Moreover, it seems like a brittle assumption as FileCheck
evolves. Finally, it actually complicates the implementation even if
it makes it slightly more efficient.

This patch avoids that assumption. Examples below show results after
applying this patch. Before applying this patch, `'N` is omitted
throughout these examples because the implementation doesn't notice
there's more than one diagnostic per pattern.

First, `CHECK-LABEL` violates the assumption because `CHECK-LABEL`
tries to match twice, and other directives can match in between:

```
$ cat check
CHECK: foobar
CHECK-LABEL: foobar

$ FileCheck -vv check < input |& tail -8
<<<<<<
1: text
2: foobar
label:2'0 ^~~~~~
check:1 ^~~~~~
label:2'1 X error: no match found
3: text
>>>>>>
```

Second, `--implicit-check-not` is obviously processed many times among
other directives:

```
$ cat check
CHECK: foo
CHECK: foo

$ FileCheck -vv -dump-input=always -implicit-check-not=foo \
check < input |& tail -16
<<<<<<
1: text
not:imp1'0 X~~~~
2: foo
check:1 ^~~
not:imp1'1 X
3: text
not:imp1'1 ~~~~~
4: foo
check:2 ^~~
not:imp1'2 X
5: text
not:imp1'2 ~~~~~
6:
eof:2 ^
>>>>>>
```

Reviewed By: thopre, jhenderson

Differential Revision: https://reviews.llvm.org/D97813

show more ...

[NFC] Reordering parameters in getFile and getFileOrSTDIN

In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed

[NFC] Reordering parameters in getFile and getFileOrSTDIN

In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.

```
static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFile(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true, bool IsVolatile = false);

static ErrorOr<std::unique_ptr<MemoryBuffer>>
getFileOrSTDIN(const Twine &Filename, bool IsText = false,
bool RequiresNullTerminator = true);

static ErrorOr<std::unique_ptr<MB>>
getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
bool IsText, bool RequiresNullTerminator, bool IsVolatile);

static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
getFile(const Twine &Filename, bool IsVolatile = false);
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D99182

show more ...

[NFC] Formatting changes

This patch addresses some formatting changes from the comments in https://reviews.llvm.org/D97785.

Reviewed By: anirudhp

Differential Revision: https://reviews.llvm.org/D9

[NFC] Formatting changes

This patch addresses some formatting changes from the comments in https://reviews.llvm.org/D97785.

Reviewed By: anirudhp

Differential Revision: https://reviews.llvm.org/D99072

show more ...

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to ge

[SystemZ][z/OS] Distinguish between text and binary files on z/OS

This patch consists of the initial changes to help distinguish between text and binary content correctly on z/OS. I would like to get feedback from Windows users on setting OF_None for all ToolOutputFiles. This seems to have been done as an optimization to prevent CRLF translation on Windows in the past.

Reviewed By: zibi

Differential Revision: https://reviews.llvm.org/D97785

show more ...

[FileCheck] Fix numeric error propagation

A more general name might be match-time error propagation. That is,
it's conceivable we'll one day have non-numeric errors that require
the handling fixed

[FileCheck] Fix numeric error propagation

A more general name might be match-time error propagation. That is,
it's conceivable we'll one day have non-numeric errors that require
the handling fixed by this patch.

Without this patch, FileCheck behaves as follows:

```
$ cat check
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]

$ FileCheck -vv -dump-input=never check < input
check:1:54: remark: implicit EOF: expected string found in input
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]
^
<stdin>:2:1: note: found here

^
check:1:15: error: unable to substitute variable or numeric expression: overflow error
CHECK-NOT: [[#0x8000000000000000+0x8000000000000000]]
^
$ echo $?
0
```

Notice that the exit status is 0 even though there's an error.
Moreover, FileCheck doesn't print the error diagnostic unless both
`-dump-input=never` and `-vv` are specified.

The same problem occurs when `CHECK-NOT` does have a match but a
capture fails due to overflow: exit status is 0, and no diagnostic is
printed unless both `-dump-input=never` and `-vv` are specified. The
usefulness of capturing from `CHECK-NOT` is questionable, but this
case should certainly produce an error.

With this patch, FileCheck always includes the error diagnostic and
has non-zero exit status for the above examples. It's conceivable
that this change will cause some existing tests to fail, but my
assumption is that they should fail. Moreover, with nearly every
project enabled, this patch didn't produce additional `check-all`
failures for me.

This patch also extends input dumps to include such numeric error
diagnostics for both expected and excluded patterns.

As noted in fixmes in some of the tests added by this patch, this
patch worsens an existing issue with redundant diagnostics. I'll fix
that bug in a subsequent patch.

Reviewed By: thopre, jhenderson

Differential Revision: https://reviews.llvm.org/D98086

show more ...

[FileCheck] Do not skip end of line in diagnostics

When commit da108b4ed4e6e7267701e76d5fd3b87609c9ab77 introduced
the CHECK-NEXT directive, it added logic to skip to the next line when
printing a d

[FileCheck] Do not skip end of line in diagnostics

When commit da108b4ed4e6e7267701e76d5fd3b87609c9ab77 introduced
the CHECK-NEXT directive, it added logic to skip to the next line when
printing a diagnostic if the current matching position is at the end of
a line. This was fine while FileCheck did not support regular expression
but since it does now it can be confusing when the pattern to match
starts with the expectation of a newline (e.g. CHECK-NEXT: {{\n}}foo).
It is also inconsistent with the column information in the diagnostic
which does point to the end of line.

This commit removes this logic altogether, such that failure to match
diagnostic for such cases would show the end of line and be consistent
with the column information. The commit also adapts all existing
testcases accordingly.

Note to reviewers: An alternative approach would be to restrict the code
to only skip to the next line if the first character of the pattern is
known not to match a whitespace-like character. This would respect the
original intent but keep the inconsistency in terms of column info and
requires more code. I've only chosen this current approach by laziness
and would be happy to restrict the logic instead.

Reviewed By: jdenny, jhenderson

Differential Revision: https://reviews.llvm.org/D93341

show more ...

[FileCheck] Default --allow-unused-prefixes to false

Link: https://lists.llvm.org/pipermail/llvm-dev/2020-October/146162.html "[RFC] FileCheck: (dis)allowing unused prefixes"

If a downstream projec

[FileCheck] Default --allow-unused-prefixes to false

Link: https://lists.llvm.org/pipermail/llvm-dev/2020-October/146162.html "[RFC] FileCheck: (dis)allowing unused prefixes"

If a downstream project using lit needs time for transition,
add the following to `lit.local.cfg`:

```
from lit.llvm.subst import ToolSubst

fc = ToolSubst('FileCheck', unresolved='fatal')
config.substitutions.insert(0, (fc.regex, 'FileCheck --allow-unused-prefixes'))
```

Differential Revision: https://reviews.llvm.org/D95849

show more ...

[FileCheck] Make --allow-unused-prefixes cl::ZeroOrMore

cl::ZeroOrMore allows the option to be specified multiple times, which makes
downstream projects possible to specify a default value in lit co

[FileCheck] Make --allow-unused-prefixes cl::ZeroOrMore

cl::ZeroOrMore allows the option to be specified multiple times, which makes
downstream projects possible to specify a default value in lit configuration
while some tests can override the value.

show more ...

[llvm] Use append_range (NFC)

[utils] Use llvm::sort (NFC)

[FileCheck] Report missing prefixes when more than one is provided.

If more than a prefix is provided - e.g. --check-prefixes=CHECK,FOO - we
don't report if (say) FOO is never used. This may lead to

[FileCheck] Report missing prefixes when more than one is provided.

If more than a prefix is provided - e.g. --check-prefixes=CHECK,FOO - we
don't report if (say) FOO is never used. This may lead to a gap in our
test coverage.

This patch introduces a new option, --allow-unused-prefixes. It
currently is set to true, keeping today's behavior. After we explicitly
set it in tests where this behavior was actually intentional, we will
switch it to false by default.

Differential Revision: https://reviews.llvm.org/D90281

show more ...

Internalize functions from various tools. NFC

And internalize some classes if I noticed them:)

Reland [FileCheck] Move FileCheck implementation out of LLVMSupport into its own library

This relands e9a3d1a401b07cbf7b11695637f1b549782a26cd which was originally
missing linking LLVMSupport into L

Reland [FileCheck] Move FileCheck implementation out of LLVMSupport into its own library

This relands e9a3d1a401b07cbf7b11695637f1b549782a26cd which was originally
missing linking LLVMSupport into LLMVFileCheck which broke the SHARED_LIBS build.

Original summary:

The actual FileCheck logic seems to be implemented in LLVMSupport. I don't see a
good reason for having FileCheck implemented there as it has a very specific use
while LLVMSupport is a dependency of pretty much every LLVM tool there is. In
fact, the only use of FileCheck I could find (outside the FileCheck tool and the
FileCheck unit test) is a single call in GISelMITest.h.

This moves the FileCheck logic to its own LLVMFileCheck library. This way only
FileCheck and the GlobalISelTests now have a dependency on this code.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86344

show more ...

Revert "[FileCheck] Move FileCheck implementation out of LLVMSupport into its own library"

This reverts commit e9a3d1a401b07cbf7b11695637f1b549782a26cd. Seems the new
FileCheck library doesn't link

Revert "[FileCheck] Move FileCheck implementation out of LLVMSupport into its own library"

This reverts commit e9a3d1a401b07cbf7b11695637f1b549782a26cd. Seems the new
FileCheck library doesn't link on some bots. Reverting for now.

show more ...

[FileCheck] Move FileCheck implementation out of LLVMSupport into its own library

The actual FileCheck logic seems to be implemented in LLVMSupport. I don't see a
good reason for having FileCheck im

[FileCheck] Move FileCheck implementation out of LLVMSupport into its own library

The actual FileCheck logic seems to be implemented in LLVMSupport. I don't see a
good reason for having FileCheck implemented there as it has a very specific use
while LLVMSupport is a dependency of pretty much every LLVM tool there is. In
fact, the only use of FileCheck I could find (outside the FileCheck tool and the
FileCheck unit test) is a single call in GISelMITest.h.

This moves the FileCheck logic to its own LLVMFileCheck library. This way only
FileCheck and the GlobalISelTests now have a dependency on this code.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D86344

show more ...