Work around GCC test failure that is caused by enabling optimizations. (#73998)

While trying to work around MSAN/TSAN build timeouts, we enabled
optimizations on some tests. This caused GCC to star

Work around GCC test failure that is caused by enabling optimizations. (#73998)

While trying to work around MSAN/TSAN build timeouts, we enabled
optimizations on some tests. This caused GCC to start complaining that
some values may be uninitialized. I believe GCC is wrong, but more
investigation is needed.

The values are initialized when the variable `__g` is either < 0 or >=
0. Which only leaves out NaN I believe, which is likely well into UB
land anyway.

However, more investigation needed.

show more ...

[libc++] Revert "Compile MSAN/TSAN failing test with -O1 (#73555)"

This reverts commit 61aef978d6ab1553c48bbd9bf807a277b22451c1, which
broke the CI on GCC.

Compile MSAN/TSAN failing test with -O1 (#73555)

This attempts to fix flakes on the bots where the modified test times
out while running under sanitizers.

Turning on the optimizer for just this

Compile MSAN/TSAN failing test with -O1 (#73555)

This attempts to fix flakes on the bots where the modified test times
out while running under sanitizers.

Turning on the optimizer for just this test appears to mostly fix the
issue.

show more ...

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in `std::vector`, to `std::string` and `std::deque` collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for `std::deque`, or between the size and capacity bounds for `std::string`).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took `iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom comparison function). When object `iter1` was longer than `iter2`, read out-of-bounds on `iter2` could happen. Container sanitization would detect it.

This revision introduces annotations for `std::deque`. Each chunk of the container can now be annotated using the `__sanitizer_annotate_double_ended_contiguous_container` function, which was added in the rG1c5ad6d2c01294a0decde43a88e9c27d7437d157. Any attempt to access poisoned memory will trigger an ASan error. Although false negatives are rare, they are possible due to limitations in the ASan API, where a few (usually up to 7) bytes before the container may remain unpoisoned. There are no false positives in the same way as with `std::vector` annotations.

This patch only supports objects (deques) that use the standard allocator. However, it can be easily extended to support all allocators, as suggested in the D146815 revision.

Furthermore, the patch includes the addition of the `is_double_ended_contiguous_container_asan_correct` function to `libcxx/test/support/asan_testing.h`. This function can be used to verify whether a `std::deque` object has been correctly annotated.

Finally, the patch extends the unit tests to verify ASan annotations (added LIBCPP_ASSERTs).
If a program is compiled without ASan, all helper functions will be no-ops. In binaries with ASan, there is a negligible performance impact since the code from the change is only executed when the deque container changes in size and it’s proportional to the change. It is important to note that regardless of whether or not these changes are in use, every access to the container's memory is instrumented.

If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com

Reviewed By: #libc, philnik

Differential Revision: https://reviews.llvm.org/D132092

show more ...

[libc++] Mark slow tests as unsupported on GCC

Some tests in our test suite are unbelievably slow on GCC due to the
use of the always_inline attribute. See [1] for more details.

This patch introduc

[libc++] Mark slow tests as unsupported on GCC

Some tests in our test suite are unbelievably slow on GCC due to the
use of the always_inline attribute. See [1] for more details.

This patch introduces the GCC-ALWAYS_INLINE-FIXME lit feature to
disable tests that are plagued by that issue. At the same time, it
moves several existing tests from ad-hoc `UNSUPPORTED: gcc-12` markup
to the new GCC-ALWAYS_INLINE-FIXME feature, and marks the slowest tests
reported by the CI as `UNSUPPORTED: GCC-ALWAYS_INLINE-FIXME`.

[1]: https://discourse.llvm.org/t/rfc-stop-supporting-extern-instantiations-with-gcc/71277/1

Differential Revision: https://reviews.llvm.org/D152736

show more ...

Revert "[2a/3][ASan][libcxx] std::deque annotations"

This reverts commit 605b9c76e093f6ed713b3fea47cb9726b346edeb.

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and `std::deque` collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for `std::deque`, or between the size and capacity bounds for `std::string`).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took `iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom comparison function). When object `iter1` was longer than `iter2`, read out-of-bounds on `iter2` could happen. Container sanitization would detect it.

This revision introduces annotations for `std::deque`. Each chunk of the container can now be annotated using the `__sanitizer_annotate_double_ended_contiguous_container` function, which was added in the rG1c5ad6d2c01294a0decde43a88e9c27d7437d157. Any attempt to access poisoned memory will trigger an ASan error. Although false negatives are rare, they are possible due to limitations in the ASan API, where a few (usually up to 7) bytes before the container may remain unpoisoned. There are no false positives in the same way as with `std::vector` annotations.

This patch only supports objects (deques) that use the standard allocator. However, it can be easily extended to support all allocators, as suggested in the D146815 revision.

Furthermore, the patch includes the addition of the `is_double_ended_contiguous_container_asan_correct` function to libcxx/test/support/asan_testing.h. This function can be used to verify whether a `std::deque` object has been correctly annotated.

Finally, the patch extends the unit tests to verify ASan annotations (added LIBCPP_ASSERTs).
If a program is compiled without ASan, all helper functions will be no-ops. In binaries with ASan, there is a negligible performance impact since the code from the change is only executed when the deque container changes in size and it’s proportional to the change. It is important to note that regardless of whether or not these changes are in use, every access to the container's memory is instrumented.

Reviewed By: #libc, philnik

Spies: vitalybuka, hans, mikhail.ramalho, Enna1, #sanitizers, philnik, libcxx-commits

Differential Revision: https://reviews.llvm.org/D132092

show more ...

Revert "[2a/3][ASan][libcxx] std::deque annotations"

https://lab.llvm.org/buildbot/#/builders/168/builds/13310
https://lab.llvm.org/buildbot/#/builders/239/builds/2107

This reverts commit 9a5f28313

Revert "[2a/3][ASan][libcxx] std::deque annotations"

https://lab.llvm.org/buildbot/#/builders/168/builds/13310
https://lab.llvm.org/buildbot/#/builders/239/builds/2107

This reverts commit 9a5f283139201ba5878780c06c97e4ad1f5eac39.

show more ...

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to

[2a/3][ASan][libcxx] std::deque annotations

This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and `std::deque` collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for `std::deque`, or between the size and capacity bounds for `std::string`).

The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took `iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom comparison function). When object `iter1` was longer than `iter2`, read out-of-bounds on `iter2` could happen. Container sanitization would detect it.

This revision introduces annotations for `std::deque`. Each chunk of the container can now be annotated using the `__sanitizer_annotate_double_ended_contiguous_container` function, which was added in the rG1c5ad6d2c01294a0decde43a88e9c27d7437d157. Any attempt to access poisoned memory will trigger an ASan error. Although false negatives are rare, they are possible due to limitations in the ASan API, where a few (usually up to 7) bytes before the container may remain unpoisoned. There are no false positives in the same way as with `std::vector` annotations.

This patch only supports objects (deques) that use the standard allocator. However, it can be easily extended to support all allocators, as suggested in the D146815 revision.

Furthermore, the patch includes the addition of the `is_double_ended_contiguous_container_asan_correct` function to libcxx/test/support/asan_testing.h. This function can be used to verify whether a `std::deque` object has been correctly annotated.

Finally, the patch extends the unit tests to verify ASan annotations (added LIBCPP_ASSERTs).
If a program is compiled without ASan, all helper functions will be no-ops. In binaries with ASan, there is a negligible performance impact since the code from the change is only executed when the deque container changes in size and it’s proportional to the change. It is important to note that regardless of whether or not these changes are in use, every access to the container's memory is instrumented.

Reviewed By: #libc, philnik

Spies: mikhail.ramalho, Enna1, #sanitizers, philnik, libcxx-commits

Differential Revision: https://reviews.llvm.org/D132092

show more ...

[libc++] [test] Qualify `distance` as `std::distance` in a lot of tests. NFCI.

We shouldn't be calling `distance` via ADL -- and neither should anybody
in the wild be calling it via ADL, so it's not

[libc++] [test] Qualify `distance` as `std::distance` in a lot of tests. NFCI.

We shouldn't be calling `distance` via ADL -- and neither should anybody
in the wild be calling it via ADL, so it's not like we need to test
this ADL ability of `distance` in particular.

Differential Revision: https://reviews.llvm.org/D119685

show more ...

[libc++] Comma-operator-proof a lot of algorithm/container code.

Detected by evil-izing the widely used `MoveOnly` testing type.
I had to patch some tests that were themselves using its comma operat

[libc++] Comma-operator-proof a lot of algorithm/container code.

Detected by evil-izing the widely used `MoveOnly` testing type.
I had to patch some tests that were themselves using its comma operator,
but I think that's a worthwhile cost in order to catch more places
in our headers that needed comma-proofing.

The trick here is that even `++ptr, SomeClass()` can find a comma operator
by ADL, if `ptr` is of type `Evil*`. (A comma between two operands
of non-class-or-enum type is always treated as the built-in
comma, without ADL. But if either operand is class-or-enum, then
ADL happens for _both_ operands' types.)

Differential Revision: https://reviews.llvm.org/D109414

show more ...

[libcxx][nfc] prefixes test type `input_iterator` with `cpp17_`

C++20 revised the definition of what it means to be an iterator. While
all _Cpp17InputIterators_ satisfy `std::input_iterator`, the re

[libcxx][nfc] prefixes test type `input_iterator` with `cpp17_`

C++20 revised the definition of what it means to be an iterator. While
all _Cpp17InputIterators_ satisfy `std::input_iterator`, the reverse
isn't true. D100271 introduces a new test adaptor to accommodate this
new definition (`cpp20_input_iterator`).

In order to help readers immediately distinguish which input iterator
adaptor is _Cpp17InputIterator_, the current `input_iterator` adaptor
has been prefixed with `cpp17_`.

Differential Revision: https://reviews.llvm.org/D101242

show more ...

Support tests in freestanding

Summary:
Freestanding is *weird*. The standard allows it to differ in a bunch of odd
manners from regular C++, and the committee would like to improve that
situation. I

Support tests in freestanding

Summary:
Freestanding is *weird*. The standard allows it to differ in a bunch of odd
manners from regular C++, and the committee would like to improve that
situation. I'd like to make libc++ behave better with what freestanding should
be, so that it can be a tool we use in improving the standard. To do that we
need to try stuff out, both with "freestanding the language mode" and
"freestanding the library subset".

Let's start with the super basic: run the libc++ tests in freestanding, using
clang as the compiler, and see what works. The easiest hack to do this:

In utils/libcxx/test/config.py add:

self.cxx.compile_flags += ['-ffreestanding']

Run the tests and they all fail.

Why? Because in freestanding `main` isn't special. This "not special" property
has two effects: main doesn't get mangled, and main isn't allowed to omit its
`return` statement. The first means main gets mangled and the linker can't
create a valid executable for us to test. The second means we spew out warnings
(ew) and the compiler doesn't insert the `return` we omitted, and main just
falls of the end and does whatever undefined behavior (if you're luck, ud2
leading to non-zero return code).

Let's start my work with the basics. This patch changes all libc++ tests to
declare `main` as `int main(int, char**` so it mangles consistently (enabling us
to declare another `extern "C"` main for freestanding which calls the mangled
one), and adds `return 0;` to all places where it was missing. This touches 6124
files, and I apologize.

The former was done with The Magic Of Sed.

The later was done with a (not quite correct but decent) clang tool:

https://gist.github.com/jfbastien/793819ff360baa845483dde81170feed

This works for most tests, though I did have to adjust a few places when e.g.
the test runs with `-x c`, macros are used for main (such as for the filesystem
tests), etc.

Once this is in we can create a freestanding bot which will prevent further
regressions. After that, we can start the real work of supporting C++
freestanding fairly well in libc++.

<rdar://problem/47754795>

Reviewers: ldionne, mclow.lists, EricWF

Subscribers: christof, jkorous, dexonsmith, arphaman, miyuki, libcxx-commits

Differential Revision: https://reviews.llvm.org/D57624

llvm-svn: 353086

show more ...

Update more file headers across all of the LLVM projects in the monorepo
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.

We understand that

Update more file headers across all of the LLVM projects in the monorepo
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351648

show more ...

[libcxx] [test] D27021: Fix MSVC warning C4389 "signed/unsigned mismatch", part 8/12.

Add static_cast<std::size_t> when comparing distance() to size().

These replacements were performed programmati

[libcxx] [test] D27021: Fix MSVC warning C4389 "signed/unsigned mismatch", part 8/12.

Add static_cast<std::size_t> when comparing distance() to size().

These replacements were performed programmatically with regex_replace():

const vector<pair<regex, string>> reg_fmt = {
{ regex(R"(assert\((\w+)\.size\(\) == std::distance\((\w+, \w+)\)\))"),
"assert($1.size() == static_cast<std::size_t>(std::distance($2)))" },
{ regex(R"(assert\(distance\((\w+\.begin\(\), \w+\.end\(\))\) == (\w+)\.size\(\)\))"),
"assert(static_cast<std::size_t>(distance($1)) == $2.size())" },
{ regex(R"(assert\(std::distance\((\w+\.\w*begin\(\), \w+\.\w*end\(\))\) == (\w+)\.size\(\)\))"),
"assert(static_cast<std::size_t>(std::distance($1)) == $2.size())" },
};

Also, include <cstddef> when it wasn't already being included.

llvm-svn: 288745

show more ...

[libcxx] [test] D27014: Fix MSVC warning C4018 "signed/unsigned mismatch", part 2/12.

Add static_cast<std::size_t> when comparing int to std::size_t.

Also, include <cstddef> when it wasn't already

[libcxx] [test] D27014: Fix MSVC warning C4018 "signed/unsigned mismatch", part 2/12.

Add static_cast<std::size_t> when comparing int to std::size_t.

Also, include <cstddef> when it wasn't already being included.

llvm-svn: 287822

show more ...

[libc++] Fix stack_allocator

Summary:
To quote STL the problems with stack allocator are"

>"stack_allocator<T, N> is seriously nonconformant to N4582 17.6.3.5 [allocator.requirements].
> First, it

[libc++] Fix stack_allocator

Summary:
To quote STL the problems with stack allocator are"

>"stack_allocator<T, N> is seriously nonconformant to N4582 17.6.3.5 [allocator.requirements].
> First, it lacks a rebinding constructor. (The nested "struct rebind" isn't sufficient.)
> Second, it lacks templated equality/inequality.
> Third, it completely ignores alignment.
> Finally, and most severely, the Standard forbids its existence. Allocators are forbidden from returning memory "inside themselves". This requirement is implied by the Standard's requirements for rebinding and equality. It's permitted to return memory from a separate buffer object on the stack, though."

This patch attempts to address all of those issues.

First, instead of storing the buffer inside the allocator I've change `stack_allocator` to accept the buffer as an argument.

Second, in order to fix rebinding I changed the parameter list from `<class T, size_t NumElements>` to `<class T, size_t NumBytes>`. This allows allocator rebinding
between types that have different sizes.

Third, I added copy and rebinding constructors and assignment operators.

And finally I fixed the allocation logic to always return properly aligned storage.



Reviewers: mclow.lists, howard.hinnant, STL_MSFT

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D25154

llvm-svn: 283631

show more ...

Fix warnings in deque tests

llvm-svn: 242632

Fix more issues exposed by -pedantic-errors in c++03 mode

llvm-svn: 228711

We had two identical files named 'MoveOnly.h' in the test suite. Move one to support/, remove the other, and update all the tests that included them. No functionality change.

llvm-svn: 227370

Fix PR#22284. Add a new overload to deque::insert to handle forward iterators. Update tests to exercise this case.

llvm-svn: 226847

Move test into test/std subdirectory.

llvm-svn: 224658