[libc++] Fix how we run codesign in the test suite when --codesign_identity is provided

[NFC][Py Reformat] Reformat python files in libcxx/libcxxabi

This is an ongoing series of commits that are reformatting our
Python code.

Reformatting is done with `black`.

If you end up having pro

[NFC][Py Reformat] Reformat python files in libcxx/libcxxabi

This is an ongoing series of commits that are reformatting our
Python code.

Reformatting is done with `black`.

If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.

If you run into any problems, post to discourse about it and
we will try to help.

RFC Thread below:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: #libc, kwk, Mordante

Differential Revision: https://reviews.llvm.org/D150763

show more ...

[libcxx] [test] Prepend to PATH instead of overriding it

On Windows, the PATH env variable is used for locating dynamically
linked librarys, akin to LD_LIBRARY_PATH on Linux.

The tests that run wit

[libcxx] [test] Prepend to PATH instead of overriding it

On Windows, the PATH env variable is used for locating dynamically
linked librarys, akin to LD_LIBRARY_PATH on Linux.

The tests that run with a dynamically linked libc++ used "--env
PATH=%{lib}" in the test config. This had the unfortunate side effect
of making other tools from PATH unavailable during the runtime of the
tests; in particular, it caused the "executor-has-no-bash" flag to be
set for all those Windows test configs (with the clang-cl static config
being the only one lacking it).

Thus, this increases the number of tests actually included in the
clang-cl dll and all mingw test configs by 9 tests.

The clang-cl static test configuration has been executing those tests
since the "--env PATH=%{lib}" was removed from that test config in
e78223e79efc886ef6f0ea5413deab3737d6d63b. (For mingw we haven't had a
need to split the test config between shared and static, which means
that the mingw static test config previously ran with --env PATH
needlessly.)

This increases the test coverage for patches like D146398 which
can't be executed in the executor-has-no-bash configs.

Change the default value of the arg.env to an empty array; when we do
pass values to the option, they get passed as an array of strings,
so make sure the variable behaves consistently when no arguments
have been passed.

Differential Revision: https://reviews.llvm.org/D148324

show more ...

[libc++][utils] Remove unused import in `run.py`

`import sys` is unused in `libcxx/utils/run.py`: remove it.

Differential Revision: https://reviews.llvm.org/D141331

[libc++] Fix codesigning in run.py

Without this patch, we'd always try to codesign the first argument in
the command line, which in some cases is not something we can codesign
(e.g. `bash` for some

[libc++] Fix codesigning in run.py

Without this patch, we'd always try to codesign the first argument in
the command line, which in some cases is not something we can codesign
(e.g. `bash` for some .sh.cpp tests).

Note that this "hack" is the same thing we do in `ssh.py` - we might need
to admit that it's not a hack after all in the future, but I'm not ready
for that yet.

Differential Revision: https://reviews.llvm.org/D99726

show more ...

[libcxx] [test] Pass some windows environment variables through to test processes

Normally, the run.py wrapper script runs the child processes in
a clean environment, with only the environment varia

[libcxx] [test] Pass some windows environment variables through to test processes

Normally, the run.py wrapper script runs the child processes in
a clean environment, with only the environment variables available
that are passed via the --env parameter.

However, the COMSPEC and TEMP variables are kind of necessary when
running some tests; COMSPEC is necessary for finding the interpreter
when executing commands via std::system().

Before f1a96de1bc8db527b5eb820c36c17e275900ca2b, tests were executed
via an intermediate shell which implicitly readded the COMSPEC variable.

The TEMP variable allows temp files to be placed in a sensible
location; if unset, they're placed in the default temp fallback of
C:\Windows instead.

Differential Revision: https://reviews.llvm.org/D97452

show more ...

[libc++] Don't run tests in a shell in the default executor

[libcxx][lit] Simplify parsing of trailing executor arguments

Adding a positional argparse.ONE_OR_MORE arguments will correctly remove
the "--" separator after --env and parse only the command. This

[libcxx][lit] Simplify parsing of trailing executor arguments

Adding a positional argparse.ONE_OR_MORE arguments will correctly remove
the "--" separator after --env and parse only the command. This also has
the advantage that misspelled flags raise an argparse error rather than
silently being added to the command to be executed.

I discovered this while adding a new commandline option to ssh.py to allow
passing additional arguments to the scp/ssh commands since this is required
for our CHERI CI where we need to pass `-F <custom_config_file>` to each
ssh/scp command to set various arguments such as the localhost port, usage
of controlmaster, etc. to speed up connections to our emulated QEMU systems.

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D84096

show more ...

[libc++] Make executor scripts executable

This allows running the scripts directly, without running them through
Python.

[libc++] Consider everything inside %T to be a dependency of each test

Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency.

[libc++] Consider everything inside %T to be a dependency of each test

Instead of passing file dependencies individually, assume that the
whole content of the unique test directory is a dependency. This
simplifies the test harness significantly, by making %T the directory
that contains everything required to run a test. This also removes the
need for the %{file_dependencies} substitution, which is removed by this
patch.

Furthermore, this patch also changes the harness to execute tests locally
inside %T, so as to avoid creating a separate directory for no purpose.

show more ...

[libc++] Use proper shell escaping in the executors

This was originally committed as f8452ddfcc33 and reverted in 7cb1aa9d9368.
The issue was that shell builtins were being escaped too, and apparent

[libc++] Use proper shell escaping in the executors

This was originally committed as f8452ddfcc33 and reverted in 7cb1aa9d9368.
The issue was that shell builtins were being escaped too, and apparently
Bash won't execute a builtin when it is quoted e.g. '!'. Instead, it
thinks it's a command and it can't find it.

Re-committing the change with that issue fixed.

show more ...

Revert "[libc++] Use proper shell escaping in the executors"

This reverts f8452ddfcc, which broke some bots. I'll figure out what's
wrong and commit it again.

[libc++] Use proper shell escaping in the executors

[libc++] run.py: Create directory hierarchy if it does not exist

This is useful when replaying tests after the fact and the directories
don't exist anymore.

[libc++] Run tests in a directory related to %t instead of /tmp

Instead of creating a temporary directory inside /tmp and running the
tests there, use a directory name based on LIT's %t substitution

[libc++] Run tests in a directory related to %t instead of /tmp

Instead of creating a temporary directory inside /tmp and running the
tests there, use a directory name based on LIT's %t substitution. This
has the benefit of not hitting /tmp so much (which is slow on some
filesystems). It also has the benefit that `ninja -C build clean` will
automatically remove the artifacts even if a test somehow failed to
remove its temporary directory (I've seen this happen when CTRL-C is
received).

show more ...

[libc++] Lit: Add default values for most arguments of test executors

[libc++] Execute tests from the Lit execution root instead of the test tree

Instead of executing tests from within the libc++ test suite, we execute
them from the Lit execution directory. However, s

[libc++] Execute tests from the Lit execution root instead of the test tree

Instead of executing tests from within the libc++ test suite, we execute
them from the Lit execution directory. However, since some tests have
file dependencies, we must copy those dependencies to the execution
directory where they are executed.

This has the major benefit that if a test modifies a file (whether it
is wanted or not), other tests will not see those modifications. This
is good because current tests assume that input data is never modified,
however this could be an incorrect assumption if some test does not
behave properly.

show more ...

[libc++] Execute tests using an external shell

This makes it closer to how one would run the tests by hand, and it is
also closer to how the SSHExecutor runs the tests remotely. It also
allows using

[libc++] Execute tests using an external shell

This makes it closer to how one would run the tests by hand, and it is
also closer to how the SSHExecutor runs the tests remotely. It also
allows using shell builtins in .sh.cpp tests when using %{exec}.

show more ...

[libc++] Properly handle environment variables with '=' in them

[libc++] Make the %run substitution closer to how .pass.cpp tests are executed

Before this patch, the %run substitution did not contain the same
environment variables as normal `pass.cpp` tests. It

[libc++] Make the %run substitution closer to how .pass.cpp tests are executed

Before this patch, the %run substitution did not contain the same
environment variables as normal `pass.cpp` tests. It also didn't
have the right working directory and the script wasn't aware of
potential file dependencies.

With this change, the combination of %build and %run in a .sh.cpp script
should match how pass.cpp tests are actually executed much more closely.

show more ...

[libcxx] Codesign test executables if necessary

If LLVM_CODESIGNING_IDENTITY is set, test executables need to be
codesigned.

Differential Revision: https://reviews.llvm.org/D66496

llvm-svn: 371126