update_test_checks: recognize %if in RUN line (#108972)

Recognize %if for target-specific cases in RUN line and keep only tool
command with options

update_test_checks: match IR basic block labels (#88979)

Labels are matched using a regexp of the form '^(pattern):', which
requires the addition of a "suffix" concept to NamelessValue.

Aside fr

update_test_checks: match IR basic block labels (#88979)

Labels are matched using a regexp of the form '^(pattern):', which
requires the addition of a "suffix" concept to NamelessValue.

Aside from that, the key challenge is that block labels are values, and
we typically capture values including the prefix '%'. However, when
labels appear at the start of a basic block, the prefix '%' is not
included, so we must capture block label values *without* the prefix
'%'.

We don't know ahead of time whether an IR value is a label or not. In
most cases, they are prefixed by the word "label" (their type), but this
isn't the case in phi nodes. We solve this issue by leveraging the
two-phase nature of variable generalization: the first pass finds all
occurences of a variable and determines whether the '%' prefix can be
included or not. The second pass does the actual substitution.

This change also unifies the generalization path for assembly with that
for IR and analysis, in the hope that any future changes avoid diverging
those cases future.

I also considered the alternative of trying to detect the phi node case
using more regular expression special cases but ultimately decided
against that because it seemed more fragile, and perhaps the approach of
keeping a tentative prefix that may later be discarded could also be
eventually applied to some metadata and attribute cases.

Note that an early version of this change was reviewed as
https://reviews.llvm.org/D142452, before version numbers were
introduced. This is a substantially updated version of that change.

show more ...

update_test_checks: keep meta variables stable by default

Resubmitting this after previous revert with the following changes:

- Split table into table_rhs_idx and table_candidate_idx so that
bise

update_test_checks: keep meta variables stable by default

Resubmitting this after previous revert with the following changes:

- Split table into table_rhs_idx and table_candidate_idx so that
bisect.bisect_left can be used without the `key` argument, which
was introduced in Python 3.10
- Remove a re.Pattern type annotation

Original commit message:

Prior to this change, running UTC on larger tests, especially tests
with unnamed IR values, often resulted in a spuriously large diff
because e.g. TMPnn variables in the CHECK lines were renumbered. This
change attempts to reduce the diff by keeping those variable names the
same.

There are cases in which this "drift" of variable names can end up being
more confusing. The old behavior can be re-enabled with the
--reset-variable-names command line argument.

The improvement may not be immediately apparent in the diff of this change.
The point is that the diff of stable_ir_values.ll against
stable_ir_values.ll.expected after this change is smaller.

Ideally, we'd also keep meta variables for "global" objects stable, e.g.
for attributes (#nn) and metadata (!nn). However, that would require a
much more substantial refactoring of how we generate check lines, so I
left it for future work.

show more ...

Revert "update_test_checks: keep meta variables stable by default"

This reverts commit fb02f9ac84a6151e41aba8f7391edd132a9aaf14.

Looks like some Python version incompatibility, will investigate.

update_test_checks: keep meta variables stable by default

Prior to this change, running UTC on larger tests, especially tests
with unnamed IR values, often resulted in a spuriously large diff
becaus

update_test_checks: keep meta variables stable by default

Prior to this change, running UTC on larger tests, especially tests
with unnamed IR values, often resulted in a spuriously large diff
because e.g. TMPnn variables in the CHECK lines were renumbered. This
change attempts to reduce the diff by keeping those variable names the
same.

There are cases in which this "drift" of variable names can end up being
more confusing. The old behavior can be re-enabled with the
--reset-variable-names command line argument.

The improvement may not be immediately apparent in the diff of this change.
The point is that the diff of stable_ir_values.ll against
stable_ir_values.ll.expected after this change is smaller.

Ideally, we'd also keep meta variables for "global" objects stable, e.g.
for attributes (#nn) and metadata (!nn). However, that would require a
much more substantial refactoring of how we generate check lines, so I
left it for future work.

show more ...

[UTC] Don't leave dangling CHECK-SAME when removing CHECK lines (#82569)

When removing only lines that are global value CHECK lines, a related
CHECK-SAME line could be left dangling without a previ

[UTC] Don't leave dangling CHECK-SAME when removing CHECK lines (#82569)

When removing only lines that are global value CHECK lines, a related
CHECK-SAME line could be left dangling without a previous line to belong
to.

Resolves #78517

show more ...

Recommit changes to global checks (#71171)

Recommits the changes from https://reviews.llvm.org/D148216.
Explicitly named globals are now matched literally, instead of emitting
a capture group for

Recommit changes to global checks (#71171)

Recommits the changes from https://reviews.llvm.org/D148216.
Explicitly named globals are now matched literally, instead of emitting
a capture group for the name. This resolves #70047.
Metadata and annotations, on the other hand, are captured and matched
against by default, since their identifiers are not stable.

The reasons for revert (#63746) have been fixed:
The first issue, that of duplicated checkers, has already been resolved
in #70050.
This PR resolves the second issue listed in #63746, regarding the order
of named and unnamed globals. This is fixed by recording the index of
substrings containing global values, and sorting the checks according to
that index before emitting them. This results in global value checks
being emitted in the order they were seen instead of being grouped
separately.

show more ...

Revert "[UTC] Add fallback support for specific metadata, and check their defs"

This reverts commit 8a3fdf7b908978625e9a7e57fbb443e4e6f98976 as it is
broken. See https://github.com/llvm/llvm-project

Revert "[UTC] Add fallback support for specific metadata, and check their defs"

This reverts commit 8a3fdf7b908978625e9a7e57fbb443e4e6f98976 as it is
broken. See https://github.com/llvm/llvm-project/issues/63746.

Effectively fixes: https://github.com/llvm/llvm-project/issues/63746

show more ...

[UTC] Add fallback support for specific metadata, and check their defs

This prevents update_cc_tests.py from emitting hard-coded identifiers
for metadata (global variable checkers still check hard-c

[UTC] Add fallback support for specific metadata, and check their defs

This prevents update_cc_tests.py from emitting hard-coded identifiers
for metadata (global variable checkers still check hard-coded
identifiers). Instead it emits regex checkers that match even if the
identifiers change. Also adds a new mode for --check-globals: instead of
simply being on or off, it now has the options 'none', 'smart' and
'all', with 'none' and 'all' corresponding to the previous modes.

The 'smart' mode only emits checks for global definitions referenced
in the IR or other metadata that itself has a definition checker
emitted, making the rule transitive. It does not emit checks for
attribute sets, since that is better checked by --check-attributes. This
mode is made the new default. To make the change in default mode
backwards compatible a version bump is introduced (to v3), and the
default remains 'none' in v1 & v2.

This will result in metadata checks being emitted more often, so filters
are added to not check absolute file paths and compiler version git
hashes.

rdar://105239218

show more ...

[NFC][Py Reformat] Reformat python files in llvm

This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more in

[NFC][Py Reformat] Reformat python files in llvm

This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more information here:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: jhenderson, JDevlieghere, MatzeB

Differential Revision: https://reviews.llvm.org/D150545

show more ...

[UTC] Include return type/attributes under --version 2

If --function-signature is used with --version 2, then also include
the return type/attributes in the check lines. This is the
implementation o

[UTC] Include return type/attributes under --version 2

If --function-signature is used with --version 2, then also include
the return type/attributes in the check lines. This is the
implementation of D133943 rebased on the --version mechanism from
D142473.

This doesn't bump the default version yet, because I'd like to do
that together with D140212 (which enables --function-signature by
default), as these changes seem closely related. For now this
functionality can be accessed by explicitly passing --version 2
to UTC.

Fixes https://github.com/llvm/llvm-project/issues/61058.

Differential Revision: https://reviews.llvm.org/D144963

show more ...

[UpdateTestChecks][NFC] Share the code to get CHECK prefix between all scripts

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D143307

update_test_checks.py: pick up --tool from UTC_ARGS

It's not clear to me how to write a test for this. The tests run in an
environment where the tools may not be in PATH, and so the existing
custom-

update_test_checks.py: pick up --tool from UTC_ARGS

It's not clear to me how to write a test for this. The tests run in an
environment where the tools may not be in PATH, and so the existing
custom-tool.test needs to use --tool-binary when invoking
update_test_checks.py. But it can't do so without also using --tool.

This change does fix a problem though with using
update_any_test_checks.py in an environment where the tools *are*
available in PATH.

Differential Revision: https://reviews.llvm.org/D142450

show more ...

update_test_checks: fix typos

Found by our downstream CI.

[UpdateTestChecks] Fix `update_*_test_checks.py` to add "unused" prefixes

The support introduced in D124306 was only added to
update_llc_test_checks.py, but the motivating usecases (see
https://list

[UpdateTestChecks] Fix `update_*_test_checks.py` to add "unused" prefixes

The support introduced in D124306 was only added to
update_llc_test_checks.py, but the motivating usecases (see
https://lists.llvm.org/pipermail/llvm-dev/2021-February/148326.html)
cover update_test_checks.py, update_cc_test_checks.py, and
update_analyze_test_checks.py, too.

Issue #59220.

Differential Revision: https://reviews.llvm.org/D138836

show more ...

update_test_checks.py: allow use with custom tools

We have a downstream project with a command-line utility that operates
pretty much exactly like `opt`. So it would make sense for us to
maintain te

update_test_checks.py: allow use with custom tools

We have a downstream project with a command-line utility that operates
pretty much exactly like `opt`. So it would make sense for us to
maintain tests with update_test_checks.py with our custom tool
substituted for `opt`, as this change allows.

Differential Revision: https://reviews.llvm.org/D136329

show more ...

update-test-checks: safely handle tests with #if's

There is at least one Clang test (clang/test/CodeGen/arm_acle.c) which
has functions guarded by #if's that cause those functions to be compiled
onl

update-test-checks: safely handle tests with #if's

There is at least one Clang test (clang/test/CodeGen/arm_acle.c) which
has functions guarded by #if's that cause those functions to be compiled
only for a subset of RUN lines.

This results in a case where one RUN line has a body for the function
and another doesn't. Treat this case as a conflict for any prefixes that
the two RUN lines have in common.

This change exposed a bug where functions with '$' in the name weren't
properly recognized in ARM assembly (despite there being a test case
that was supposed to catch the problem!). This bug is fixed as well.

Differential Revision: https://reviews.llvm.org/D130089

show more ...

[UpdateTestChecks] Remove outdated help text

Manually modifying the result of update_test_checks.py is discouraged,
we prefer unmodified check lines where possible. The output is also
considered aut

[UpdateTestChecks] Remove outdated help text

Manually modifying the result of update_test_checks.py is discouraged,
we prefer unmodified check lines where possible. The output is also
considered authoritative nowadays, at least for tests targeting core
middle-end components, where not using it is an automatic review
rejection.

Differential Revision: https://reviews.llvm.org/D129259

show more ...

[UpdateTestChecks] Re-add --filter and --filter-out options

Re-add filtering options with fixes for failed tests. We were not passing the
is_filtered argument in all check generator calls in update

[UpdateTestChecks] Re-add --filter and --filter-out options

Re-add filtering options with fixes for failed tests. We were not passing the
is_filtered argument in all check generator calls in update_cc_test_checks.py

Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions. The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression. Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.

This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.

Differential Revision: https://reviews.llvm.org/D117694

show more ...

Revert "[UpdateTestChecks] Add --filter and --filter-out options"

Broke some update-test-checks tests. Reverting while developing a fix.

This reverts commit 030f71698d52f228929da5e3148602f4a3daff7

Revert "[UpdateTestChecks] Add --filter and --filter-out options"

Broke some update-test-checks tests. Reverting while developing a fix.

This reverts commit 030f71698d52f228929da5e3148602f4a3daff7d.

show more ...

[UpdateTestChecks] Add --filter and --filter-out options

Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions. The --filter option will emit

[UpdateTestChecks] Add --filter and --filter-out options

Enhance the various update_*_test_checks.py tools to allow filtering the tool
output with regular expressions. The --filter option will emit only tool output
lines matching the given regular expression while the --filter-out option will
emit only tools output lines not matching the given regular expression. Filters
are applied in order of appearance on the command line (or in UTC_ARGS) and the
first matching filter terminates the search.

This allows test authors to create more focused tests by removing irrelevant
tool output and checking only the pieces of output necessary to test the desired
functionality.

Differential Revision: https://reviews.llvm.org/D117694

show more ...

[Utils] Do not remove comments in llc test script

When checking if two prefixes can be merged for a function,
update_llc_test_checks.py removed IR comments before comparing
llc outputs of different

[Utils] Do not remove comments in llc test script

When checking if two prefixes can be merged for a function,
update_llc_test_checks.py removed IR comments before comparing
llc outputs of different RUN lines.
This means, if one RUN line emited lines starting with ';' and another
RUN line emited the same lines except the ones starting with ';', both
RUNs would be merged (if they share a prefix).

However, CHECK-NEXT lines check the comments, otherwise they fail, so
the script should not merge RUNs if they contain different comments.

Differential Revision: https://reviews.llvm.org/D101312

show more ...

[UpdateTestUtils] Print test filename when complaining about conflicting prefix

Now that FileCheck eagerly complains when prefixes are unused,
the update script does the same, and is becoming very

[UpdateTestUtils] Print test filename when complaining about conflicting prefix

Now that FileCheck eagerly complains when prefixes are unused,
the update script does the same, and is becoming very common
to need to drop some prefixes, yet figuring out the file
it complains about isn't obvious unless it actually tells us.

show more ...

[update_(llc_)test_checks.py] Support pre-processing commands

This has been rather useful in our downstream CHERI target where we want
to run tests both with addrspace(0) and addrspace(200) pointers

[update_(llc_)test_checks.py] Support pre-processing commands

This has been rather useful in our downstream CHERI target where we want
to run tests both with addrspace(0) and addrspace(200) pointers.
With this patch we can prefix the opt command with
`sed -e 's/addrspace(200)/addrspace(0)/g' -e 's/-A200-P200-G200//g'` to
test both cases using the same IR input.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95137

show more ...

[Utils] Check for more global information in update_test_checks

This allows to check for various globals (metadata/attributes/...) and
also resolves problems with globals (metadata/attributes/...) b

[Utils] Check for more global information in update_test_checks

This allows to check for various globals (metadata/attributes/...) and
also resolves problems with globals (metadata/attributes/...) being
reused across different prefixes.

Reviewed By: sstefan1

Differential Revision: https://reviews.llvm.org/D94741

show more ...