Reland "[llvm][Support] Add support for executing a detached process (#81708)" (#83367)

Relands #81708, which was reverted by
f410f74cd5b26319b5796e0404c6a0f3b5cc00a5, now with a corrected unit
te

Reland "[llvm][Support] Add support for executing a detached process (#81708)" (#83367)

Relands #81708, which was reverted by
f410f74cd5b26319b5796e0404c6a0f3b5cc00a5, now with a corrected unit
test. Origionally the test failed on Windows when run with lit as
`GetConsoleWindow` could not retrieve a window handle regardless of
whether `DetachProcess` was `true` or `false`. The test now uses
`GetStdHandle(STD_OUTPUT_HANDLE)` which does not rely on a console
window existing. Original commit message below.

Adds a new parameter, `bool DetachProcess` with a default option of
`false`, to `llvm::sys::ExecuteNoWait`, which, when set to `true`,
executes the specified program without a controlling terminal.

Functionality added so that the module build daemon can be run without a
controlling terminal.

show more ...

Revert "[llvm][Support] Add support for executing a detached process (#81708)"

This reverts commit 86f6caa562255f81b93e72a501a926b17f5ad244. Unit test
was failing on a few windows build bots

[llvm][Support] Add support for executing a detached process (#81708)

Adds a new parameter, `bool DetachProcess` with a default option of
`false`, to `llvm::sys::ExecuteNoWait`, which, when set to

[llvm][Support] Add support for executing a detached process (#81708)

Adds a new parameter, `bool DetachProcess` with a default option of
`false`, to `llvm::sys::ExecuteNoWait`, which, when set to `true`,
executes the specified program without a controlling terminal.

Functionality added so that the module build daemon can be run without a
controlling terminal.

show more ...

Support: Add polling option to sys::Wait

Currently the process is terminated after the timeout. Add an option
to let the process resume after the timeout instead.

https://reviews.llvm.org/D138952

Support: Make Wait's SecondsToWait be std::optional [NFC]

I found the interaction between SecondsToWait and
WaitUntilChildTerminates confusing. Rather than have a boolean to
ignore the value of Seco

Support: Make Wait's SecondsToWait be std::optional [NFC]

I found the interaction between SecondsToWait and
WaitUntilChildTerminates confusing. Rather than have a boolean to
ignore the value of SecondsToWait, combine these into one Optional
parameter.

show more ...

[llvm] Use std::nullopt instead of None in comments (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalu

[llvm] Use std::nullopt instead of None in comments (NFC)

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

show more ...

Support: Convert Program APIs to std::optional

Cleanup include: TableGen

This also includes a few cleanup from Support.

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.l

Cleanup include: TableGen

This also includes a few cleanup from Support.

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121331

show more ...

[SystemZ][z/OS][Windows] Add new functions that set Text/Binary mode for Stdin and Stdout based on OpenFlags

On Windows, we want to open a file in Binary mode if OF_CRLF bit is not set. On z/OS, we

[SystemZ][z/OS][Windows] Add new functions that set Text/Binary mode for Stdin and Stdout based on OpenFlags

On Windows, we want to open a file in Binary mode if OF_CRLF bit is not set. On z/OS, we want to open a file in Binary mode if the OF_Text bit is not set.

This patch creates two new functions called ChangeStdinMode and ChangeStdoutMode which will take OpenFlags as an arg to determine which mode to set stdin and stdout to. This will enable patches like https://reviews.llvm.org/D100056 to not affect Windows when setting the OF_Text flag for raw_fd_streams.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D100130

show more ...

Re-land [Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /A

Re-land [Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF lld-link.exe ...

Would let LLD only use 4 hyper-threads.

Previously, there was an outstanding issue on Windows Server 2019 on dual-CPU machines, which was preventing from using both CPU sockets. In normal conditions, when no affinity mask was set, ProcessorGroup::AllThreads was different from ProcessorGroup::UsableThreads. The previous code in llvm/lib/Support/Windows/Threading.inc L201 was improperly assuming those two values to be equal, and consequently was limiting the execution to only one CPU socket.

Differential Revision: https://reviews.llvm.org/D92419

show more ...

Revert "[Support] On Windows, take the affinity mask into account"

This reverts commit 336ab2d51dfdd5ca09c2a9c506453db4fe653584.

[Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF

[Support] On Windows, take the affinity mask into account

The number of hardware threads available to a ThreadPool can be limited if setting an affinity mask.
For example:

> start /B /AFFINITY 0xF lld-link.exe ...

Would let LLD only use 4 hyper-threads.

Previously, there was an outstanding issue on Windows Server 2019 on dual-CPU machines, which was preventing from using both CPU sockets. In normal conditions, when no affinity mask was set, ProcessorGroup::AllThreads was different from ProcessorGroup::UsableThreads. The previous code in llvm/lib/Support/Windows/Threading.inc L201 was improperly assuming those two values to be equal, and consequently was limiting the execution to only one CPU socket.

Differential Revision: https://reviews.llvm.org/D92419

show more ...

Reapply: [clang-cl] Always interpret the LIB env var as separated with semicolons

When cross compiling with clang-cl, clang splits the INCLUDE env
variable around semicolons (clang/lib/Driver/ToolCh

Reapply: [clang-cl] Always interpret the LIB env var as separated with semicolons

When cross compiling with clang-cl, clang splits the INCLUDE env
variable around semicolons (clang/lib/Driver/ToolChains/MSVC.cpp,
MSVCToolChain::AddClangSystemIncludeArgs) and lld splits the
LIB variable similarly (lld/COFF/Driver.cpp,
LinkerDriver::addLibSearchPaths). Therefore, the consensus for
cross compilation with clang-cl and lld-link seems to be to use
semicolons, despite path lists normally being separated by colons
on unix and EnvPathSeparator being set to that.

Therefore, handle the LIB variable similarly in Clang, when
handling lib file arguments when driving linking via Clang.

This fixes commands like "clang-cl test.c -Fetest.exe kernel32.lib" in
a cross compilation setting. Normally, most users call (lld-)link
directly, but meson happens to use this command syntax for
has_function() tests.

Reapply: Change Program.h to define procid_t as ::pid_t. When included
in lldb/unittests/Host/NativeProcessProtocolTest.cpp, it is included
after an lldb namespace containing an lldb::pid_t typedef, followed
later by a "using namespace lldb;". Previously, Program.h wasn't
included in this translation unit, but now it ends up included
transitively from Process.h.

Differential Revision: https://reviews.llvm.org/D88002

show more ...

[Windows] Fix limit on command line size

This reapplies commit d4020ef7c474, reverted in ac0edc55887b because it
broke build of LLDB. This commit contains appropriate changes for LLDB.
The original

[Windows] Fix limit on command line size

This reapplies commit d4020ef7c474, reverted in ac0edc55887b because it
broke build of LLDB. This commit contains appropriate changes for LLDB.
The original commit message is below.

Documentation on CreateProcessW states that maximal size of command line
is 32767 characters including ternimation null character. In the
function llvm::sys::commandLineFitsWithinSystemLimits this limit was set
to 32768. As a result if command line was exactly 32768 characters long,
a response file was not created and CreateProcessW was called with
too long command line.

Differential Revision: https://reviews.llvm.org/D83772

show more ...

Revert "[Windows] Fix limit on command line size"

This reverts commit d4020ef7c474b5e695d77aa100d7f68dc0c66b4e. It broke
LLDB buildbot: http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja/build

Revert "[Windows] Fix limit on command line size"

This reverts commit d4020ef7c474b5e695d77aa100d7f68dc0c66b4e. It broke
LLDB buildbot: http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja/builds/17702.

show more ...

[Windows] Fix limit on command line size

Documentation on CreateProcessW states that maximal size of command line
is 32767 characters including ternimation null character. In the
function llvm::sys:

[Windows] Fix limit on command line size

Documentation on CreateProcessW states that maximal size of command line
is 32767 characters including ternimation null character. In the
function llvm::sys::commandLineFitsWithinSystemLimits this limit was set
to 32768. As a result if command line was exactly 32768 characters long,
a response file was not created and CreateProcessW was called with
too long command line.

Differential Revision: https://reviews.llvm.org/D83772

show more ...

[NFC][Support] Add unit of measure to comment for PeakMemory

[Clang] Move clang::Job::printArg to llvm::sys::printArg. NFCI.

This patch is to support/simplify https://reviews.llvm.org/D80833

[Support] Get process statistics in ExecuteAndWait and Wait

The functions sys::ExcecuteAndWait and sys::Wait now have additional
argument of type pointer to structure, which is filled with process
e

[Support] Get process statistics in ExecuteAndWait and Wait

The functions sys::ExcecuteAndWait and sys::Wait now have additional
argument of type pointer to structure, which is filled with process
execution statistics upon process termination. These are total and user
execution times and peak memory consumption. By default this argument is
nullptr so existing users of these function must not change behavior.

Differential Revision: https://reviews.llvm.org/D78901

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

Refactor ExecuteAndWait to take StringRefs.

This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I th

Refactor ExecuteAndWait to take StringRefs.

This simplifies some code which had StringRefs to begin with, and
makes other code more complicated which had const char* to begin
with.

In the end, I think this makes for a more idiomatic and platform
agnostic API. Not all platforms launch process with null terminated
c-string arrays for the environment pointer and argv, but the api
was designed that way because it allowed easy pass-through for
posix-based platforms. There's a little additional overhead now
since on posix based platforms we'll be takign StringRefs which
were constructed from null terminated strings and then copying
them to null terminate them again, but from a readability and
usability standpoint of the API user, I think this API signature
is strictly better.

llvm-svn: 334518

show more ...

Attempt 3: Resubmit "[Support] Expose flattenWindowsCommandLine."

I took some liberties and quoted fewer characters than before,
based on an article from MSDN which says that only certain characters

Attempt 3: Resubmit "[Support] Expose flattenWindowsCommandLine."

I took some liberties and quoted fewer characters than before,
based on an article from MSDN which says that only certain characters
cause an arg to require quoting. This seems to be incorrect, though,
and worse it seems to be a difference in Windows version. The bot
that fails is Windows 7, and I can't reproduce the failure on Win
10. But it's definitely related to quoting and special characters,
because both tests that fail have a * in the argument, which is one
of the special characters that would cause an argument to be quoted
before but not any longer after the new patch.

Since I don't have Win 7, all I can do is just guess that I need to
restore the old quoting rules. So this patch does that in hopes that
it fixes the problem on Windows 7.

llvm-svn: 334375

show more ...

Revert "Resubmit "[Support] Expose flattenWindowsCommandLine.""

This reverts commit 65243b6d19143cb7a03f68df0169dcb63e8b4632.

Seems like it's not a flake. It might have something to do with
the '*

Revert "Resubmit "[Support] Expose flattenWindowsCommandLine.""

This reverts commit 65243b6d19143cb7a03f68df0169dcb63e8b4632.

Seems like it's not a flake. It might have something to do with
the '*' character being in a command line.

llvm-svn: 334356

show more ...

Resubmit "[Support] Expose flattenWindowsCommandLine."

There were a few linux compilation failures, but other than that
I think this was just a flake that caused the tests to fail. I'm
going to res

Resubmit "[Support] Expose flattenWindowsCommandLine."

There were a few linux compilation failures, but other than that
I think this was just a flake that caused the tests to fail. I'm
going to resubmit and see if the failures go away, if not I'll
revert again.

llvm-svn: 334355

show more ...

Revert "[Support] Expose flattenWindowsCommandLine."

This reverts commit 10d2e88e87150a35dc367ba30716189d2af26774.

This is causing some test failures for some reason, reverting
while I investigate.

Revert "[Support] Expose flattenWindowsCommandLine."

This reverts commit 10d2e88e87150a35dc367ba30716189d2af26774.

This is causing some test failures for some reason, reverting
while I investigate.

llvm-svn: 334354

show more ...