[UpdateTestChecks][llvm-mca] Use common.itertests in update_mca_test_checks (#67477)

common.itertests has useful functionality such as skipping test case if
it starts with UTC_AVOID comment string.

[UpdateTestChecks][llvm-mca] Use common.itertests in update_mca_test_checks (#67477)

common.itertests has useful functionality such as skipping test case if
it starts with UTC_AVOID comment string. It also does some of the
functionality, such as skipping test file pattern if not found, that was
duplicated by update_mca_test_checks.

This patch uses this function to take advantage of extra behavior and
remove duplication.

show more ...

[NFC][Py Reformat] Reformat python files in llvm

This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more in

[NFC][Py Reformat] Reformat python files in llvm

This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more information here:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: jhenderson, JDevlieghere, MatzeB

Differential Revision: https://reviews.llvm.org/D150545

show more ...

[UpdateTestChecks][NFC] Share the code to get CHECK prefix between all scripts

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D143307

[UpdateTestChecks] Change shebang from python to python3

'python' means Python 2 on some platforms while Python 3 on others.
'python3' is Python 3 only. Python 2.7 End of Life is set to January 1,
2

[UpdateTestChecks] Change shebang from python to python3

'python' means Python 2 on some platforms while Python 3 on others.
'python3' is Python 3 only. Python 2.7 End of Life is set to January 1,
2020. Getting rid of Python 2 support reduces maintenance burden.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D70730

show more ...

[UpdateTestChecks] Share the code to parse RUN: lines between all scripts

Summary:
This commit also introduces a common.debug() function to avoid many
`if args.verbose:` statements. Depends on D7042

[UpdateTestChecks] Share the code to parse RUN: lines between all scripts

Summary:
This commit also introduces a common.debug() function to avoid many
`if args.verbose:` statements. Depends on D70428.

Reviewers: xbolva00, MaskRay, jdoerfert

Reviewed By: MaskRay

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70432

show more ...

[UptestTestChecks][NFC] Share some common command line options code

Summary:
Add a function common.parse_commandline_args() that adds options common
to all tools (--verbose and --update-only) and re

[UptestTestChecks][NFC] Share some common command line options code

Summary:
Add a function common.parse_commandline_args() that adds options common
to all tools (--verbose and --update-only) and returns the parsed
commandline arguments. I plan to use the shared parsing of --verbose in a
follow-up commit to remove most of the `if args.verbose:` checks in the
scripts.

Reviewers: xbolva00, MaskRay

Reviewed By: MaskRay

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D70428

show more ...

[UpdateTestChecks] Emit warning when invalid value for -check-prefix(es) option

Summary:
The script is silent for the following issue:
FileCheck %s -check-prefix=CHECK,POPCOUNT

FileCheck will catch

[UpdateTestChecks] Emit warning when invalid value for -check-prefix(es) option

Summary:
The script is silent for the following issue:
FileCheck %s -check-prefix=CHECK,POPCOUNT

FileCheck will catch it later, but I think we can warn here too.

Now it warns:
./update_llc_test_checks.py file.ll
WARNING: Supplied prefix 'CHECK,POPCOUNT' is invalid. Prefix must contain only alphanumeric characters, hyphens and underscores. Did you mean --check-prefixes=CHECK,POPCOUNT?



Reviewers: lebedev.ri, spatel, RKSimon, craig.topper, nikic, gbedwell

Reviewed By: RKSimon

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64589

llvm-svn: 367244

show more ...

[utils] Fix update scripts output when run on python3.

This fixes a "bytes-like object is required, not 'str'" python3 error I hit on update_llc_test_checks.py (but present on the other scripts as w

[utils] Fix update scripts output when run on python3.

This fixes a "bytes-like object is required, not 'str'" python3 error I hit on update_llc_test_checks.py (but present on the other scripts as well) by matching what update_mca_test_checks.py already does, plus I've added an explicit 'utf-8' encoding.

llvm-svn: 352633

show more ...

Python compat - no explicit reference to Python version

Update documentation and shebang.

Differential Revision: https://reviews.llvm.org/D56252

llvm-svn: 350327

[llvm-mca][UpdateTestChecks] Don't try to align blocks that have already been subject to alignment in update_mca_test_checks.py

This fixes PR39466.

llvm-svn: 345499

[utils] Ensure that update_mca_test_checks.py writes prefixes in alphabetical order

llvm-svn: 343783

[utils] simple refactor in update_mca_test_checks.py to make intent more readable

llvm-svn: 343782

[utils] Cope with the binary having a .exe extension in update_mca_test_checks.py

llvm-svn: 343333

[utils] Stricter checking from update_mca_test_checks.py

If any prefixes have been specified on the RUN lines that do not end up
ever actually getting printed, raise an Error. This is either an
indi

[utils] Stricter checking from update_mca_test_checks.py

If any prefixes have been specified on the RUN lines that do not end up
ever actually getting printed, raise an Error. This is either an
indication that the run lines just need cleaning up, or that something
is more fundamentally wrong with the test.

Also raise an Error if there are any blocks which cannot be checked
because they are not uniquely covered by a prefix.

Fixed up a couple of tests where the extra checking flagged up issues.

Differential Revision: https://reviews.llvm.org/D48276

llvm-svn: 343332

show more ...

[utils] Allow better identification of matching blocks in update_mca_test_checks.py

Insert empty blocks to cause the positions of matching blocks to match
across lists where possible so that later s

[utils] Allow better identification of matching blocks in update_mca_test_checks.py

Insert empty blocks to cause the positions of matching blocks to match
across lists where possible so that later stages of the algorithm can
actually identify them as being identical.

Regenerated all tests with this change.

Differential Revision: https://reviews.llvm.org/D52560

llvm-svn: 343331

show more ...

[UpdateTestChecks] Error if --llvm-mca-binary gets an empty string

If the command line was mistyped like:
./update_mca_test_checks.py --llvm-mca-binary= /path/to/llvm-mca *.s

[UpdateTestChecks] Error if --llvm-mca-binary gets an empty string

If the command line was mistyped like:
./update_mca_test_checks.py --llvm-mca-binary= /path/to/llvm-mca *.s
^-- extra whitespace

then /path/to/llvm-mca would get treated by argparse as a test-path
pattern and could actually be opened in write mode and overwritten.

llvm-svn: 334029

show more ...

[llvm-mca][UpdateTestChecks] Prevent an IndexError being raised when given empty input

llvm-svn: 333894

[llvm-mca] Make sure not to end the test files with an empty line.

Summary:
It's super irritating.

[properly configured] git client then complains about that double-newline,
and you have to use `--

[llvm-mca] Make sure not to end the test files with an empty line.

Summary:
It's super irritating.

[properly configured] git client then complains about that double-newline,
and you have to use `--force` to ignore the warning, since even if you
fix it manually, it will be reintroduced the very next runtime :/

Reviewers: RKSimon, andreadb, courbet, craig.topper, javed.absar, gbedwell

Reviewed By: gbedwell

Subscribers: javed.absar, tschuett, gbedwell, llvm-commits

Differential Revision: https://reviews.llvm.org/D47697

llvm-svn: 333887

show more ...

[UpdateTestChecks] Improved update_mca_test_checks block analysis

Previously update_mca_test_checks worked entirely at "block" level where
a block is some sequence of lines delimited by at least one

[UpdateTestChecks] Improved update_mca_test_checks block analysis

Previously update_mca_test_checks worked entirely at "block" level where
a block is some sequence of lines delimited by at least one empty line.
This generally worked well, but could sometimes lead to excessive
repetition of check lines for various prefixes if some block was almost
identical between prefixes, but not quite (for example, due to a
different dispatch width in the otherwise identical summary views).

This new analyis attempts to split blocks further in the case where the
following conditions are met:
a) There is some prefix common to every RUN line (typically 'ALL').
b) The first line of the block is common to the output with every prefix.
c) The block has the same number of lines for the output with every prefix.

Also, regenerated all llvm-mca test files with the following command:
update_mca_test_checks.py "../test/tools/llvm-mca/*/*.s" "../test/tools/llvm-mca/*/*/*.s"

The new analysis showed a "multiple lines not disambiguated by prefixes" warning
for test "AArch64/Exynos/scheduler-queue-usage.s" so I've also added some
explicit prefixes to each of the RUN lines in that test.

Differential Revision: https://reviews.llvm.org/D47321

llvm-svn: 333204

show more ...

[UpdateTestChecks] Change update_mca_test_checks.py file mode to match the other scripts

llvm-svn: 330815

[UpdateTestChecks] Fix update_mca_test_checks.py slowness issue

The script was using Python's difflib module to calculate the number of
lines changed so that it could report it in its status output.

[UpdateTestChecks] Fix update_mca_test_checks.py slowness issue

The script was using Python's difflib module to calculate the number of
lines changed so that it could report it in its status output. It turns
out this can be very very slow on large sets of lines (Python bug 6931).
It's not worth the cost, so just remove the usage of difflib entirely.

llvm-svn: 330419

show more ...

[UpdateTestChecks] Add update_mca_test_checks.py script

This script can be used to regenerate tests in the
test/tools/llvm-mca directory (PR36904).

Regenerated a number of tests using the pattern:

[UpdateTestChecks] Add update_mca_test_checks.py script

This script can be used to regenerate tests in the
test/tools/llvm-mca directory (PR36904).

Regenerated a number of tests using the pattern: test/tools/llvm-mca/*/*/*.s

Differential Revision: https://reviews.llvm.org/D45369

llvm-svn: 330246

show more ...