[NFC] Add explicit #include llvm-config.h where its macros are used. (#106621)

Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial si

[NFC] Add explicit #include llvm-config.h where its macros are used. (#106621)

Without these explicit includes, removing other headers, who implicitly
include llvm-config.h, may have non-trivial side effects.

show more ...

Change type of DiagnosticHandlerTy (#86504)

Changing type of DiagnosticHandlerTy due to adding
-Wcast-function-type-mismatch to -Wextra
group(https://github.com/llvm/llvm-project/pull/86131#issuec

Change type of DiagnosticHandlerTy (#86504)

Changing type of DiagnosticHandlerTy due to adding
-Wcast-function-type-mismatch to -Wextra
group(https://github.com/llvm/llvm-project/pull/86131#issuecomment-2018014179).
Changed the reference argument DiagnosticInfo to a pointer and edited
the test cases failing due to this change. Added another small change
where Gtest api was throwing an warning due varargs argument not being
passed.

show more ...

Rename llvm::ThreadPool -> llvm::DefaultThreadPool (NFC) (#83702)

The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.

This is a breaking change: cli

Rename llvm::ThreadPool -> llvm::DefaultThreadPool (NFC) (#83702)

The base class llvm::ThreadPoolInterface will be renamed
llvm::ThreadPool in a subsequent commit.

This is a breaking change: clients who use to create a ThreadPool must
now create a DefaultThreadPool instead.

show more ...

Split the llvm::ThreadPool into an abstract base class and an implementation (#82094)

This decouples the public API used to enqueue tasks and wait for
completion from the actual implementation, and

Split the llvm::ThreadPool into an abstract base class and an implementation (#82094)

This decouples the public API used to enqueue tasks and wait for
completion from the actual implementation, and opens up the possibility
for clients to set their own thread pool implementation for the pool.

https://discourse.llvm.org/t/construct-threadpool-from-vector-of-existing-threads/76883

show more ...

[Support] Change SetVector's default template parameter to SmallVector<*, 0>

Similar to D156016 for MapVector.

This brings back commit fae7b98c221b5b28797f7b56b656b6b819d99f27 with a
fix to llvm/un

[Support] Change SetVector's default template parameter to SmallVector<*, 0>

Similar to D156016 for MapVector.

This brings back commit fae7b98c221b5b28797f7b56b656b6b819d99f27 with a
fix to llvm/unittests/Support/ThreadPool.cpp's `_WIN32` code path.

show more ...

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC w

[NFC][TargetParser] Replace uses of llvm/Support/Host.h

The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.

show more ...

[NFC][TargetParser] Remove llvm/ADT/Triple.h

I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.

[unittests] Use GTEST_SKIP() instead of return when appropriate

Basically NFC: A TEST/TEST_F/etc that bails out early (usually because
setup failed or some other runtime condition wasn't met) genera

[unittests] Use GTEST_SKIP() instead of return when appropriate

Basically NFC: A TEST/TEST_F/etc that bails out early (usually because
setup failed or some other runtime condition wasn't met) generally
should use GTEST_SKIP() to report its status correctly, unless it
takes steps to report another status (e.g., FAIL()).

I did see a handful of tests show up as SKIPPED after this change,
which is not unexpected. The status seemed appropriate in all the new
cases.

show more ...

[Support] On Windows 11 and Windows Server 2022, fix an affinity mask issue on large core count machines

Before Windows 11 and Windows Server 2022, only one 'processor group' is assigned by default

[Support] On Windows 11 and Windows Server 2022, fix an affinity mask issue on large core count machines

Before Windows 11 and Windows Server 2022, only one 'processor group' is assigned by default to a starting process, then the program is responsible for dispatching its own threads on more 'processor groups'. That is what 8404aeb56a73ab24f9b295111de3b37a37f0b841 was doing, allowing LLVM tools to automatically use all hardware threads in the machine.

After Windows 11 and Windows Server 2022, the OS takes care of that. This has an adverse effect reported in #56618 which is that using `GetProcessAffinityMask()` API in some edge cases seems buggy now. That API is used to detect if an affinity mask was set, and adjust accordingly the available threads for a ThreadPool.

With this patch, on one hand, we let the OS dispatch threads on all 'processor groups', but only for Windows 11 & Windows Server 2022 and after. We retain the old behavior for older OS versions. On the other hand, a workaround was added to mitigate the `GetProcessAffinityMask()` issue described above (see Threading.inc, L226).

Differential Revision: https://reviews.llvm.org/D138747

show more ...

[ThreadPool] add ability to group tasks into separate groups

This is needed for parallelizing of loading modules symbols in LLDB
(D122975). Currently LLDB can parallelize indexing symbols
when loadi

[ThreadPool] add ability to group tasks into separate groups

This is needed for parallelizing of loading modules symbols in LLDB
(D122975). Currently LLDB can parallelize indexing symbols
when loading a module, but modules are loaded sequentially. If LLDB
index cache is enabled, this means that the cache loading is not
parallelized, even though it could. However doing that creates
a threadpool-within-threadpool situation, so the number of threads
would not be properly limited.

This change adds ThreadPoolTaskGroup as a simple type that can be
used with ThreadPool calls to put tasks into groups that can be
independently waited for (even recursively from within a task)
but still run in the same thread pool.

Differential Revision: https://reviews.llvm.org/D123225

show more ...

[RGT] Use GTEST_SKIP() in more places where we skip a test

Simply returning will report the test as PASSED when it didn't
really do anything. SKIPPED is the correct result for these.

Found by the R

[RGT] Use GTEST_SKIP() in more places where we skip a test

Simply returning will report the test as PASSED when it didn't
really do anything. SKIPPED is the correct result for these.

Found by the Rotten Green Tests project.

show more ...

Recommit [ThreadPool] Support returning futures with results.

This reverts commit 71a7c55f0f021b04b9a7303d0cd391b9161cf303.

The revert broken building llvm-reduce and it is not clear it fixes an
is

Recommit [ThreadPool] Support returning futures with results.

This reverts commit 71a7c55f0f021b04b9a7303d0cd391b9161cf303.

The revert broken building llvm-reduce and it is not clear it fixes an
issue with LLVM_ENABLE_THREADS=OFF.

See discussion in https://reviews.llvm.org/D114183 for more details.

show more ...

Revert "[ThreadPool] Support returning futures with results."

This reverts commit 6149e57dc1313d32c85524f8009a1249e0b8f4d1.

The offending commit broke building with LLVM_ENABLE_THREADS=OFF.

[ThreadPool] Support returning futures with results.

This patch adjusts ThreadPool::async to return futures that wrap
the result type of the passed in callable.

To do so, ThreadPool::asyncImpl firs

[ThreadPool] Support returning futures with results.

This patch adjusts ThreadPool::async to return futures that wrap
the result type of the passed in callable.

To do so, ThreadPool::asyncImpl first creates a shared promise. The
result of the promise is set in a new callable that first executes the
task. The callable is added to the task queue.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D114183

show more ...

[Support] ThreadPool tests: silence warning unused variable 'It'

[RGT] Disable some tests on Windows at compile-time, not runtime

These show up as un-executed on non-Windows hosts.

Found by the Rotten Green Tests project.

[Support][test] Unconditionally use setenv macro when compiling on Windows

This test currently fails to compile when using a MinGW toolchain as setenv is not defined. This function is a POSIX functi

[Support][test] Unconditionally use setenv macro when compiling on Windows

This test currently fails to compile when using a MinGW toolchain as setenv is not defined. This function is a POSIX function Windows does not implement.

This patch enables the setenv macro used in the unit test for all of Windows, making the test compile and run successfully.

Differential Revision: https://reviews.llvm.org/D98271

show more ...

Re-land [Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /A

Re-land [Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF lld-link.exe ...

Would let LLD only use 4 hyper-threads.

Previously, there was an outstanding issue on Windows Server 2019 on dual-CPU machines, which was preventing from using both CPU sockets. In normal conditions, when no affinity mask was set, ProcessorGroup::AllThreads was different from ProcessorGroup::UsableThreads. The previous code in llvm/lib/Support/Windows/Threading.inc L201 was improperly assuming those two values to be equal, and consequently was limiting the execution to only one CPU socket.

Differential Revision: https://reviews.llvm.org/D92419

show more ...

Revert "[Support] On Windows, take the affinity mask into account"

This reverts commit 336ab2d51dfdd5ca09c2a9c506453db4fe653584.

[Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF

[Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF lld-link.exe ...

Would let LLD only use 4 hyper-threads.

Previously, there was an outstanding issue on Windows Server 2019 on dual-CPU machines, which was preventing from using both CPU sockets. In normal conditions, when no affinity mask was set, ProcessorGroup::AllThreads was different from ProcessorGroup::UsableThreads. The previous code in llvm/lib/Support/Windows/Threading.inc L201 was improperly assuming those two values to be equal, and consequently was limiting the execution to only one CPU socket.

Differential Revision: https://reviews.llvm.org/D92419

show more ...

Revert "[Support] Use unique_function rather than std::function for ThreadPool TaskTy."

This reverts commit d9c8b0256cfc673c2413b13993c9440be598818f.

Some MSVC std::packaged_task implementations ar

Revert "[Support] Use unique_function rather than std::function for ThreadPool TaskTy."

This reverts commit d9c8b0256cfc673c2413b13993c9440be598818f.

Some MSVC std::packaged_task implementations are not compatible with move-only types.
This caused failures on some of the Windows builders (e.g.
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/69412).

Reverting until I can come up with a workaround.

show more ...

[Support] Use unique_function rather than std::function for ThreadPool TaskTy.

This will allow non-copyable function objects (e.g. lambdas that capture
unique_ptrs) to be used with ThreadPool.

Diff

[Support] Use unique_function rather than std::function for ThreadPool TaskTy.

This will allow non-copyable function objects (e.g. lambdas that capture
unique_ptrs) to be used with ThreadPool.

Differential Revision: https://reviews.llvm.org/D87467

show more ...

Revert rGf0bab7875e78e01c149d12302dcc4b6d4c43e25c - "Triple.h - reduce Twine.h include to forward declarations. NFC."

This causes ICEs on the clang-ppc64be buildbots and I've limited ability to tria

Revert rGf0bab7875e78e01c149d12302dcc4b6d4c43e25c - "Triple.h - reduce Twine.h include to forward declarations. NFC."

This causes ICEs on the clang-ppc64be buildbots and I've limited ability to triage the problem.

show more ...

Triple.h - reduce Twine.h include to forward declarations. NFC.

Move include down to a number of other files that had an implicit dependency on the Twine class.

[Support] Fix fragile ThreadPool test

The test ThreadPoolTest.AllThreads_UseAllRessources occasionally fails on the bots: http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pr

[Support] Fix fragile ThreadPool test

The test ThreadPoolTest.AllThreads_UseAllRessources occasionally fails on the bots: http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/32015/steps/test-check-all/logs/FAIL%3A%20LLVM-Unit%3A%3AThreadPoolTest.AllThreads_UseAllRessources

This is because jobs were executed too fast on the first CPU socket, and never manage to reach the second CPU socket.

Differential Revision: https://reviews.llvm.org/D78832

show more ...