[lit] Simplify test scheduling via multiprocessing.Pool

llvm-svn: 375458

[lit] Remove redundancy from names and comments

llvm-svn: 375456

[lit] Reduce value of synthesized timeouts

Large timeout values (one year, positive infinity) trip up Python on
Windows with "OverflowError: timeout value is too large". One week
seems to work and

[lit] Reduce value of synthesized timeouts

Large timeout values (one year, positive infinity) trip up Python on
Windows with "OverflowError: timeout value is too large". One week
seems to work and is still large enough in practice.

Thanks to Simon Pilgrim for helping me test this.
https://reviews.llvm.org/rL375171

llvm-svn: 375264

show more ...

[lit] Remove unnecessary tracking of test_index

llvm-svn: 375263

[lit] Only send back test result from worker process

Avoid sending back the whole run.Test object (which needs to be pickled)
from the worker process when we are only interested in the test result.

[lit] Only send back test result from worker process

Avoid sending back the whole run.Test object (which needs to be pickled)
from the worker process when we are only interested in the test result.

llvm-svn: 375262

show more ...

[lit] Move computation of deadline up into base class

llvm-svn: 375171

Reland "[lit] Synthesize artificial deadline"

We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (now plus one year) to simplify code
and do less

Reland "[lit] Synthesize artificial deadline"

We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (now plus one year) to simplify code
and do less busy waiting.

Thanks to Reid Kleckner for diagnosing that a deadline for of "positive
infinity" does not work with Python 3 anymore. See commit:
4ff1e34b606d9a9fcfd8b8b5449a558315af94e5

I tested this patch with Python 2 and Python 3.

llvm-svn: 375165

show more ...

Revert [lit] Synthesize artificial deadline

Python on Windows raises this OverflowError:
gotit = waiter.acquire(True, timeout)
OverflowError: timestamp too large to convert to C _PyTime_t

S

Revert [lit] Synthesize artificial deadline

Python on Windows raises this OverflowError:
gotit = waiter.acquire(True, timeout)
OverflowError: timestamp too large to convert to C _PyTime_t

So it seems this API behave the same way on every OS.

Also reverts the dependent commit a660dc590a5e8dafa1ba6ed56447ede151d17bd9.

llvm-svn: 375143

show more ...

[lit] Move computation of deadline up into base class

llvm-svn: 375130

[lit] Synthesize artificial deadline

We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (positive infinity) to simplify code
and do less busy wait

[lit] Synthesize artificial deadline

We always want to use a deadline when calling `result.await`. Let's
synthesize an artificial deadline (positive infinity) to simplify code
and do less busy waiting.

llvm-svn: 375129

show more ...

[lit] Create derived classes for serial/parallel test runs

The hope is that with a little OO we can nicely factor out the
differences.

llvm-svn: 375128

[lit] Improve lit.Run class

* Push timing of overall test time into run module
* Make lit.Run a proper class
* Add a few TODO comments

llvm-svn: 375065

[lit] Do not create semaphores when we do not need them

Parallelism groups and semaphores are only required for parallel
execution.

llvm-svn: 375055

[lit] Factor out separate methods for parallel and serial execution

llvm-svn: 375054

[lit] Refactor ProgressDisplay

Move progress display to separate file. Simplify some code paths.
Decouple from other components via progress callback. Remove unused
`_Display` class.

Reviewed By:

[lit] Refactor ProgressDisplay

Move progress display to separate file. Simplify some code paths.
Decouple from other components via progress callback. Remove unused
`_Display` class.

Reviewed By: serge-sans-paille

Differential Revision: https://reviews.llvm.org/D68525

llvm-svn: 374194

show more ...

[lit] Use better name for "test in parallel" concept

In the past, lit used threads to run tests in parallel. Today we use
`multiprocessing.Pool`, which uses processes. Let's stay more abstract
and u

[lit] Use better name for "test in parallel" concept

In the past, lit used threads to run tests in parallel. Today we use
`multiprocessing.Pool`, which uses processes. Let's stay more abstract
and use "worker" everywhere.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D68475

llvm-svn: 373794

show more ...

[lit] Allow setting parallelism groups to None

Check that we do not crash if a parallelism group is explicitly set to
None. Permits usage of the following pattern.

[lit.common.cfg]
lit_config.par

[lit] Allow setting parallelism groups to None

Check that we do not crash if a parallelism group is explicitly set to
None. Permits usage of the following pattern.

[lit.common.cfg]
lit_config.parallelism_groups['my_group'] = None
if <condition>:
lit_config.parallelism_groups['my_group'] = 3

[project/lit.cfg]
config.parallelism_group = 'my_group'

Reviewers: rnk

Differential Revision: https://reviews.llvm.org/D58305

llvm-svn: 354912

show more ...

[lit] Remove LitTestCase

From the docs: `class LitTestCase(unittest.TestCase)`
LitTestCase is an adaptor for providing a 'unittest' compatible
interface to 'lit' tests so that we can run lit tests w

[lit] Remove LitTestCase

From the docs: `class LitTestCase(unittest.TestCase)`
LitTestCase is an adaptor for providing a 'unittest' compatible
interface to 'lit' tests so that we can run lit tests with standard
python test runners.

It does not seem to be used anywhere.

Differential Revision: https://reviews.llvm.org/D58264

llvm-svn: 354188

show more ...

[lit][NFC] Cleanup lit worker process handling

Move code that is executed on worker process to separate file. This
makes the use of the pickled arguments stored in global variables in the
worker a b

[lit][NFC] Cleanup lit worker process handling

Move code that is executed on worker process to separate file. This
makes the use of the pickled arguments stored in global variables in the
worker a bit clearer. (Still not pretty though.)

Extract handling of parallelism groups to it's own function.

Use BoundedSemaphore instead of Semaphore. BoundedSemaphore raises for
unmatched release() calls.

Cleanup imports.

Differential Revision: https://reviews.llvm.org/D58196

llvm-svn: 354187

show more ...

[lit] Set --single-process for single tests and --threads=1

Summary:
Automatically upgrade debugging experience (single process, no thread
pool) when:
1) we only run a single test
2) user specif

[lit] Set --single-process for single tests and --threads=1

Summary:
Automatically upgrade debugging experience (single process, no thread
pool) when:
1) we only run a single test
2) user specifies `-j1`

Details:
Fix `--max-failures` in single process mode. Option did not have an
effect in single process mode.

Add display feedback for single process mode. Adapted test.

Improve argument checking (require positive integers).

`--single-process` is now essentially an alias for `-j1`. Should we
remove it?

Reviewers: rnk

Differential Revision: https://reviews.llvm.org/D58249

llvm-svn: 354068

show more ...

Revert 349677, it contained a whole bunch of stuff I did not mean to commit

llvm-svn: 349678

[gn build] Add build file for clang/lib/Basic and dependencies

Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.

Al

[gn build] Add build file for clang/lib/Basic and dependencies

Adds a build file for clang-tblgen and an action for running it, and uses that
to process all the .td files in include/clang/Basic.

Also adds an action to write include/clang/Config/config.h and
include/clang/Basic/Version.inc.

Differential Revision: https://reviews.llvm.org/D55847

llvm-svn: 349677

show more ...

[lit] Try to improve Ctrl-C behavior on Windows

This functionality was broken during a refactor a while back because
'pool' is no longer in scope.

llvm-svn: 318572

Revert "lit.py: Add new %{shared_output(LABEL)} substitution"

This reverts r315697 and my ill-fated attempts to fix it on Windows.
I'll try again when I get access to a Windows machine.

llvm-svn: 3

Revert "lit.py: Add new %{shared_output(LABEL)} substitution"

This reverts r315697 and my ill-fated attempts to fix it on Windows.
I'll try again when I get access to a Windows machine.

llvm-svn: 315793

show more ...

lit.py: Add new %{shared_output(LABEL)} substitution

This refers to a temporary path that can be shared across all tests,
identified by a particular label. This can be used for things like
caches.

lit.py: Add new %{shared_output(LABEL)} substitution

This refers to a temporary path that can be shared across all tests,
identified by a particular label. This can be used for things like
caches.

At the moment, the character set for the LABEL is limited to C
identifier characters, plus '-', '+', '=', and '.'. This is the same
set of characters currently allowed in REQUIRES clause identifiers.

llvm-svn: 315697

show more ...