|
Revision tags: llvmorg-21-init, llvmorg-19.1.7, llvmorg-19.1.6, llvmorg-19.1.5, llvmorg-19.1.4, llvmorg-19.1.3 |
|
| #
383df163 |
| 16-Oct-2024 |
David Spickett <david.spickett@linaro.org> |
[llvm][llvm-lit] Add total time for each testsuite in JUnit XML output (#112230)
Currently we write out a time taken to run all test suites:
```
<testsuites time="8.28">
```
And one for each tes
[llvm][llvm-lit] Add total time for each testsuite in JUnit XML output (#112230)
Currently we write out a time taken to run all test suites:
```
<testsuites time="8.28">
```
And one for each test:
```
<testcase classname="lldb-shell.Breakpoint" name="breakpoint-command.test" time="2.38"/>
```
However, the schema says there should be one for each suite and test,
but none for testsuites:
https://github.com/windyroad/JUnit-Schema/blob/cfa434d4b8e102a8f55b8727b552a0063ee9044e/JUnit.xsd#L216
I'm leaving the `testsuites` time in though because no one has
complained so far, and someone out there probably has a script relying
on it by now. Most XML tools handle unknown attributes quite well
anyway.
I'm adding a per testsuite time to comply with the schema and maybe be
more compatible with other JUnit tools.
```
<testsuite name="lldb-shell" ... time="12.34">
```
The test suite time is the sum of the time taken for all tests in the
suite. This will ignore some overhead in setting up the suite, and means
that the sum of the times for all individual suites may not equal the
`testsuites` time.
As we're usually focusing on the execution time of particular tests, not
lit's book keeping, I think this is a reasonable choice.
show more ...
|
|
Revision tags: llvmorg-19.1.2, llvmorg-19.1.1, llvmorg-19.1.0, llvmorg-19.1.0-rc4, llvmorg-19.1.0-rc3, llvmorg-19.1.0-rc2, llvmorg-19.1.0-rc1, llvmorg-20-init, llvmorg-18.1.8 |
|
| #
4b1fef6e |
| 10-Jun-2024 |
Jay Foad <jay.foad@amd.com> |
[lit] Skip xunit test on Windows only (#94980)
This test works on Linux with lit's internal shell. It fails on Windows
because sh is not available.
|
|
Revision tags: llvmorg-18.1.7, llvmorg-18.1.6, llvmorg-18.1.5, llvmorg-18.1.4, llvmorg-18.1.3, llvmorg-18.1.2, llvmorg-18.1.1, llvmorg-18.1.0, llvmorg-18.1.0-rc4, llvmorg-18.1.0-rc3, llvmorg-18.1.0-rc2, llvmorg-18.1.0-rc1, llvmorg-19-init, llvmorg-17.0.6, llvmorg-17.0.5, llvmorg-17.0.4, llvmorg-17.0.3, llvmorg-17.0.2, llvmorg-17.0.1, llvmorg-17.0.0, llvmorg-17.0.0-rc4, llvmorg-17.0.0-rc3, llvmorg-17.0.0-rc2, llvmorg-17.0.0-rc1, llvmorg-18-init, llvmorg-16.0.6, llvmorg-16.0.5, llvmorg-16.0.4, llvmorg-16.0.3, llvmorg-16.0.2, llvmorg-16.0.1, llvmorg-16.0.0, llvmorg-16.0.0-rc4, llvmorg-16.0.0-rc3, llvmorg-16.0.0-rc2, llvmorg-16.0.0-rc1, llvmorg-17-init, llvmorg-15.0.7, llvmorg-15.0.6, llvmorg-15.0.5, llvmorg-15.0.4, llvmorg-15.0.3, working, llvmorg-15.0.2, llvmorg-15.0.1, llvmorg-15.0.0, llvmorg-15.0.0-rc3, llvmorg-15.0.0-rc2, llvmorg-15.0.0-rc1, llvmorg-16-init, llvmorg-14.0.6, llvmorg-14.0.5, llvmorg-14.0.4, llvmorg-14.0.3, llvmorg-14.0.2, llvmorg-14.0.1, llvmorg-14.0.0, llvmorg-14.0.0-rc4, llvmorg-14.0.0-rc3, llvmorg-14.0.0-rc2, llvmorg-14.0.0-rc1, llvmorg-15-init, llvmorg-13.0.1, llvmorg-13.0.1-rc3, llvmorg-13.0.1-rc2, llvmorg-13.0.1-rc1, llvmorg-13.0.0, llvmorg-13.0.0-rc4, llvmorg-13.0.0-rc3, llvmorg-13.0.0-rc2, llvmorg-13.0.0-rc1, llvmorg-14-init, llvmorg-12.0.1, llvmorg-12.0.1-rc4, llvmorg-12.0.1-rc3, llvmorg-12.0.1-rc2, llvmorg-12.0.1-rc1, llvmorg-12.0.0, llvmorg-12.0.0-rc5, llvmorg-12.0.0-rc4, llvmorg-12.0.0-rc3, llvmorg-12.0.0-rc2, llvmorg-11.1.0, llvmorg-11.1.0-rc3, llvmorg-12.0.0-rc1, llvmorg-13-init, llvmorg-11.1.0-rc2, llvmorg-11.1.0-rc1, llvmorg-11.0.1, llvmorg-11.0.1-rc2, llvmorg-11.0.1-rc1, llvmorg-11.0.0, llvmorg-11.0.0-rc6, llvmorg-11.0.0-rc5, llvmorg-11.0.0-rc4, llvmorg-11.0.0-rc3, llvmorg-11.0.0-rc2, llvmorg-11.0.0-rc1 |
|
| #
18df607d |
| 26-Jul-2020 |
Alex Richardson <Alexander.Richardson@cl.cam.ac.uk> |
[lit] Don't include tests skipped due to sharding in reports
When running multiple shards, don't include skipped tests in the xunit
output since merging the files will result in duplicates.
In our C
[lit] Don't include tests skipped due to sharding in reports
When running multiple shards, don't include skipped tests in the xunit
output since merging the files will result in duplicates.
In our CHERI Jenkins CI, I configured the libc++ tests to run using sharding
(since we are testing using a single-CPU QEMU). We then merge the generated
XUnit xml files to produce a final result, but if the individual XMLs
report tests excluded due to sharding each test is included N times in the
final result. This also makes it difficult to find the tests that were
skipped due to missing REQUIRES: etc.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D84235
show more ...
|
| #
722e5d6a |
| 23-Jul-2020 |
Alex Richardson <Alexander.Richardson@cl.cam.ac.uk> |
[lit] Include total elapsed time in xunit output
The time= attribute can also be used for the testsuites tag. While this
attribute appears to be ignored by Jenkins (https://github.com/jenkinsci/juni
[lit] Include total elapsed time in xunit output
The time= attribute can also be used for the testsuites tag. While this
attribute appears to be ignored by Jenkins (https://github.com/jenkinsci/junit-plugin/blob/bab34bcc96154a494f8c371953efe06d45813f67/src/main/java/hudson/tasks/junit/SuiteResult.java#L202),
it is still useful if you manually inspect the xml file.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D84230
show more ...
|
|
Revision tags: llvmorg-12-init, llvmorg-10.0.1, llvmorg-10.0.1-rc4, llvmorg-10.0.1-rc3, llvmorg-10.0.1-rc2 |
|
| #
968f58c6 |
| 05-Jun-2020 |
Julian Lettner <julian.lettner@apple.com> |
[lit] Include unexecuted tests in xUnit report
Pass in all discovered tests to report generators.
The XunitReport generator now creates testcase items for unexecuted
tests and documents why they ha
[lit] Include unexecuted tests in xUnit report
Pass in all discovered tests to report generators.
The XunitReport generator now creates testcase items for unexecuted
tests and documents why they have been skipped. This makes it easier
to compare test runs with different filters or configurations, or across
platforms.
I don't know who is using the JsonReport generator and what the
expectations there are (it doesn't have tests), so decided to preserve
the old behavior by filtering out the unexecuted tests.
Reviewed By: jdenny
Differential Revision: https://reviews.llvm.org/D81316
show more ...
|
|
Revision tags: llvmorg-10.0.1-rc1 |
|
| #
201e73cb |
| 01-May-2020 |
Julian Lettner <julian.lettner@apple.com> |
[lit] Small refinements for xunit report output
|
|
Revision tags: llvmorg-10.0.0, llvmorg-10.0.0-rc6, llvmorg-10.0.0-rc5, llvmorg-10.0.0-rc4, llvmorg-10.0.0-rc3, llvmorg-10.0.0-rc2, llvmorg-10.0.0-rc1, llvmorg-11-init, llvmorg-9.0.1, llvmorg-9.0.1-rc3, llvmorg-9.0.1-rc2, llvmorg-9.0.1-rc1, llvmorg-9.0.0, llvmorg-9.0.0-rc6, llvmorg-9.0.0-rc5, llvmorg-9.0.0-rc4, llvmorg-9.0.0-rc3, llvmorg-9.0.0-rc2, llvmorg-9.0.0-rc1, llvmorg-10-init, llvmorg-8.0.1, llvmorg-8.0.1-rc4, llvmorg-8.0.1-rc3, llvmorg-8.0.1-rc2, llvmorg-8.0.1-rc1, llvmorg-8.0.0, llvmorg-8.0.0-rc5, llvmorg-8.0.0-rc4, llvmorg-8.0.0-rc3, llvmorg-7.1.0, llvmorg-7.1.0-rc1, llvmorg-8.0.0-rc2, llvmorg-8.0.0-rc1, llvmorg-7.0.1, llvmorg-7.0.1-rc3, llvmorg-7.0.1-rc2, llvmorg-7.0.1-rc1, llvmorg-7.0.0, llvmorg-7.0.0-rc3, llvmorg-7.0.0-rc2, llvmorg-7.0.0-rc1, llvmorg-6.0.1, llvmorg-6.0.1-rc3, llvmorg-6.0.1-rc2 |
|
| #
29770f7a |
| 25-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
Use quoteattr to ensure we make well formed attributes
We were making malformed XML on tests with ' in the name. Switch to
using saxutils to set all of our attributes, so it can handle quotes
etc c
Use quoteattr to ensure we make well formed attributes
We were making malformed XML on tests with ' in the name. Switch to
using saxutils to set all of our attributes, so it can handle quotes
etc correctly.
llvm-svn: 333249
show more ...
|
| #
2dd62a3d |
| 17-May-2018 |
Douglas Yung <douglas.yung@sony.com> |
Mark test with "REQUIRES: shell" since it directly invokes "sh" and was failing on Windows.
llvm-svn: 332563
|
| #
1bffd0f7 |
| 16-May-2018 |
Alexander Richardson <arichardson.kde@gmail.com> |
Escape ]]> in xunit xml output
Summary:
This sequence ends the CDATA block so any characters after that are no
longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>".
Reviewers
Escape ]]> in xunit xml output
Summary:
This sequence ends the CDATA block so any characters after that are no
longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>".
Reviewers: cmatthews
Reviewed By: cmatthews
Differential Revision: https://reviews.llvm.org/D46886
llvm-svn: 332440
show more ...
|
| #
5d1014bf |
| 16-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
Use not to catch unexpected pass as well as remove old test results
As per review feedback, make sure we rm temp files, and make the return
code checking for lit more specific.
llvm-svn: 332423
|
| #
d1386a88 |
| 11-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
[LIT] replace output escapes wit a cdata block
CDATA blocks don't need to have XML stuff escaped. Makes sense to wrap
output in them instead of escaping.
llvm-svn: 332116
|
| #
7d6224d2 |
| 11-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
Support Unsupported Tests in xunit output
We were reporting "Unsupported" tests in xunit as passes, however since
they are not run, it make more sense to mark them as skipped. The Junit
xml standar
Support Unsupported Tests in xunit output
We were reporting "Unsupported" tests in xunit as passes, however since
they are not run, it make more sense to mark them as skipped. The Junit
xml standard has support for that, so lets use it.
llvm-svn: 332065
show more ...
|
| #
4855c5f7 |
| 10-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
[LIT] Move xunit tests tests into their own location, and and add failures
Failures will increase coverage.
llvm-svn: 332056
|
| #
069a1eb3 |
| 09-May-2018 |
Chris Matthews <cmatthews5@apple.com> |
[LIT] Handle xml characters in test names
Lit creates malformed xml when the test case has an & in the name.
Escape those correctly.
This also adds a test case which I will add other nasty encodin
[LIT] Handle xml characters in test names
Lit creates malformed xml when the test case has an & in the name.
Escape those correctly.
This also adds a test case which I will add other nasty encoding issues to in some followup commits.
llvm-svn: 331942
show more ...
|
|
Revision tags: llvmorg-6.0.1-rc1, llvmorg-5.0.2, llvmorg-5.0.2-rc2, llvmorg-5.0.2-rc1, llvmorg-6.0.0 |
|
| #
9db7bd5c |
| 28-Feb-2018 |
Lei Huang <lei@ca.ibm.com> |
Losen time contraint to accommodate system loads
llvm-svn: 326359
|
|
Revision tags: llvmorg-6.0.0-rc3, llvmorg-6.0.0-rc2, llvmorg-6.0.0-rc1, llvmorg-5.0.1, llvmorg-5.0.1-rc3, llvmorg-5.0.1-rc2, llvmorg-5.0.1-rc1, llvmorg-5.0.0, llvmorg-5.0.0-rc5, llvmorg-5.0.0-rc4, llvmorg-5.0.0-rc3, llvmorg-5.0.0-rc2, llvmorg-5.0.0-rc1, llvmorg-4.0.1, llvmorg-4.0.1-rc3, llvmorg-4.0.1-rc2, llvmorg-4.0.1-rc1, llvmorg-4.0.0, llvmorg-4.0.0-rc4, llvmorg-4.0.0-rc3, llvmorg-4.0.0-rc2, llvmorg-4.0.0-rc1, llvmorg-3.9.1, llvmorg-3.9.1-rc3, llvmorg-3.9.1-rc2, llvmorg-3.9.1-rc1, llvmorg-3.9.0, llvmorg-3.9.0-rc3, llvmorg-3.9.0-rc2, llvmorg-3.9.0-rc1, llvmorg-3.8.1, llvmorg-3.8.1-rc1, llvmorg-3.8.0, llvmorg-3.8.0-rc3, llvmorg-3.8.0-rc2, llvmorg-3.8.0-rc1, llvmorg-3.7.1, llvmorg-3.7.1-rc2, llvmorg-3.7.1-rc1, llvmorg-3.7.0, llvmorg-3.7.0-rc4, llvmorg-3.7.0-rc3, studio-1.4, llvmorg-3.7.0-rc2, llvmorg-3.7.0-rc1, llvmorg-3.6.2, llvmorg-3.6.2-rc1 |
|
| #
8fcf5051 |
| 13-Jun-2015 |
Eric Fiselier <eric@efcs.ca> |
[LIT] Fix failing LIT tests
Summary:
I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.
I made the following changes for the follo
[LIT] Fix failing LIT tests
Summary:
I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.
I made the following changes for the following reasons.
1. google-test.py: The Google test format now checks for "[ PASSED ] 1 test." to check if a test passes.
2. discovery.py: The output appears in a different order on my machine than it did in the test.
3. unittest-adaptor.py: The output appears in a different order on my machine than it did in the test.
4. The classname is now formed differently in `getJUnitXML(...)`.
I'm not sure what is causing the output order to differ in discovery.py and unittest-adaptor.py. Does anybody have any thoughts?
Reviewers: ddunbar, danalbert, jroelofs
Reviewed By: jroelofs
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D9864
llvm-svn: 239663
show more ...
|
|
Revision tags: llvmorg-3.6.1, llvmorg-3.6.1-rc1, llvmorg-3.5.2, llvmorg-3.5.2-rc1, llvmorg-3.6.0, llvmorg-3.6.0-rc4, llvmorg-3.6.0-rc3, llvmorg-3.6.0-rc2, llvmorg-3.6.0-rc1, llvmorg-3.5.1, llvmorg-3.5.1-rc2, llvmorg-3.5.1-rc1 |
|
| #
be5f1689 |
| 03-Dec-2014 |
Chris Matthews <cmatthews5@apple.com> |
Add a test-case for lit xunit output
llvm-svn: 223307
|