[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack ar

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).

In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.

To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).

In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.

Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:

1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.

2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.

The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").

v4:
- Add interface attribute to __sanitizer_internal_mem* declarations as
well, as otherwise some compilers (MSVC) will complain.
- Add SANITIZER_COMMON_NO_REDEFINE_BUILTINS to source files using
C++STL, since this could lead to ODR violations (see added comment).

v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.

v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".

Reviewed By: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D151152

show more ...

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit fc011a72881cdddc95bfa61f3f38916c29b7e362.
This reverts commit 4ad6a0c9a409b19b950a6a2a90d5405cea2e9b89.
T

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit fc011a72881cdddc95bfa61f3f38916c29b7e362.
This reverts commit 4ad6a0c9a409b19b950a6a2a90d5405cea2e9b89.
This reverts commit 4b1eb4cf0e8eff5f68410720167b4986da597010.

Still causes Windows build bots to fail.

show more ...

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack ar

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).

In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.

To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).

In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.

Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:

1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.

2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.

The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").

v3:
- Don't use ALIAS() to alias internal_mem*() functions to
__sanitizer_internal_mem*() functions, but just define them as
ALWAYS_INLINE functions instead. This will work on darwin and windows.

v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".

Reviewed By: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D151152

show more ...

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit 4369de7af46605522bf7dbe3bc31d00b0eb4bee6.

Fails on Mac OS with "sanitizer_libc.cpp:109:5: error: aliases

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit 4369de7af46605522bf7dbe3bc31d00b0eb4bee6.

Fails on Mac OS with "sanitizer_libc.cpp:109:5: error: aliases are not
supported on darwin".

show more ...

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack ar

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).

In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.

To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).

In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.

Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:

1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.

2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.

The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").

v2:
- Fix ubsan_minimal build where compiler decides to insert
memset/memcpy: ubsan_minimal has work without RTSanitizerCommonLibc,
therefore do not redefine the builtins.
- Fix definition of internal_mem* functions with compilers that want the
aliased function to already be defined before.
- Fix definition of __sanitizer_internal_mem* functions with compilers
more pedantic about attribute placement around extern "C".

Reviewed By: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D151152

show more ...

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit e614d5667f6c6fc6c645587cb9aee1a058285454.

Build bot failures:

| FAILED: lib/clang/17/lib/linux/libclang

Revert "[compiler-rt] Avoid memintrinsic calls inserted by the compiler"

This reverts commit e614d5667f6c6fc6c645587cb9aee1a058285454.

Build bot failures:

| FAILED: lib/clang/17/lib/linux/libclang_rt.ubsan_minimal-i386.so
| : && /usr/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -Wno-unused-parameter -O3 -DNDEBUG -Wl,-z,defs -Wl,-z,nodelete -m32 -nodefaultlibs -Wl,-z,text -nostdlib++ -shared -Wl,-soname,libclang_rt.ubsan_minimal-i386.so -o lib/clang/17/lib/linux/libclang_rt.ubsan_minimal-i386.so projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o -lgcc_s -lc && :
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o: in function `__ubsan_handle_type_mismatch_minimal':
| /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:120: undefined reference to `__sanitizer_internal_memcpy'
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o: in function `__ubsan_handle_type_mismatch_minimal_abort':
| /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:120: undefined reference to `__sanitizer_internal_memcpy'
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o: in function `__ubsan_handle_alignment_assumption_minimal':
| /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:121: undefined reference to `__sanitizer_internal_memcpy'
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o: in function `__ubsan_handle_alignment_assumption_minimal_abort':
| /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:121: undefined reference to `__sanitizer_internal_memcpy'
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o: in function `__ubsan_handle_add_overflow_minimal':
| /b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:122: undefined reference to `__sanitizer_internal_memcpy'
| /usr/bin/ld: projects/compiler-rt/lib/ubsan_minimal/CMakeFiles/RTUbsan_minimal.i386.dir/ubsan_minimal_handlers.cpp.o:/b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm-project/compiler-rt/lib/ubsan_minimal/ubsan_minimal_handlers.cpp:122: more undefined references to `__sanitizer_internal_memcpy' follow

Link: https://lab.llvm.org/buildbot#builders/74/builds/19569

show more ...

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack ar

[compiler-rt] Avoid memintrinsic calls inserted by the compiler

D135716 introduced -ftrivial-auto-var-init=pattern where supported.
Unfortunately this introduces unwanted memset() for large stack arrays,
as shown by the new tests added for asan and msan (tsan already had this
test).

In general, the problem of compiler-inserted memintrinsic calls
(memset/memcpy/memmove) is not new to compiler-rt, and has been a
problem before.

To avoid introducing unwanted memintrinsic calls, we redefine
memintrinsics as __sanitizer_internal_mem* at the assembly level for
most source files automatically (where sanitizer_common_internal_defs.h
is included).

In few cases, redefining a symbol in this way causes issues for
interceptors, namely the memintrinsic interceptor themselves. For such
source files we have to selectively disable the redefinition.

Other alternatives have been considered, but simply do not work well in
the context of compiler-rt:

1. Linker --wrap: this does not work because --wrap only
applies to the final link, and would not apply when building
sanitizer static libraries.

2. Changing references to memset() via objcopy: this may work,
but due to the complexities of the build system, introducing
such a post-processing step for the right object files (in
particular object files defining memset cannot be touched)
seems infeasible.

The chosen solution works well (as shown by the tests). Other libraries
have chosen the same solution where nothing else works (see e.g. glibc's
"symbol-hacks.h").

Reviewed By: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D151152

show more ...

Revert "Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests""

This reverts commit 255c3e3dcb06299aa2365f70817322a8a381c351, which
breaks Msan-x86_64-Test.

Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests"

When trying to debug some `compiler-rt` unittests, I initially had a hard
time because

- even in a `Debug` build one nee

Reland "[compiler-rt][test] Heed COMPILER_RT_DEBUG when compiling unittests"

When trying to debug some `compiler-rt` unittests, I initially had a hard
time because

- even in a `Debug` build one needs to set `COMPILER_RT_DEBUG` to get
debugging info for some of the code and
- even so the unittests used a hardcoded `-O2` which often makes debugging
impossible.

This patch addresses this by instead using `-O0` if `COMPILER_RT_DEBUG`.

Changes relative to the previous commit:

- Use `string(APPEND)` for `COMPILER_RT_TEST_COMPILER_CFLAGS`.
- Omit `-O3` from `COMPILER_RT_TEST_COMPILER_CFLAGS` in non-debug builds for now.
- Provide `__sanitizer::integral_constant<bool, true>::value` instantiation
for `sanitizer_type_traits_test.cpp` in debug builds.
- Disable subtests of `tsan/tests/unit/tsan_trace_test.cpp` that deadlock
in debug builds.
- `XFAIL` `tsan/Linux/check_memcpy.c` in debug builds.

Tested on `sparcv9-sun-solaris2.11`, `amd64-pc-solaris2.11`, and
`x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D91620

show more ...

[compiler-rt][test] Add shared_unwind requirement

When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though i

[compiler-rt][test] Add shared_unwind requirement

When using a static libunwind, the check_memcpy.c can fail because it checks
that tsan intercepted all memcpy/memmoves in the final binary. Though if the
static libunwind is not instrumented, then this will fail because it may contain
regular memcpy/memmoves.

This adds a new REQUIRES check for ensuring that this test won't run unless a
dynamic libunwind.so is provided.

Differential Revision: https://reviews.llvm.org/D111194

show more ...

sanitizers: compile with -O1 under debug

Tsan's check_memcpy.c test was disabled under debug because it failed.
But it points to real issues and does not help to just disable it.
I tried to enable i

sanitizers: compile with -O1 under debug

Tsan's check_memcpy.c test was disabled under debug because it failed.
But it points to real issues and does not help to just disable it.
I tried to enable it and see what fail and the first hit was default ctor for:

struct ChainedOriginDepotDesc {
u32 here_id;
u32 prev_id;
};

initializing these fields to 0's help partially,
but compiler still emits memset before calling ctor.
I did not try to see what's the next failure, because if it fails
on such small structs, it won't be realistic to fix everything
and keep working.

Compile runtimes with -O1 under debug instead.
It seems to fix all current failures. At least I run check-tsan
under clang/gcc x debug/non-debug and all combinations passed.
-O1 does not usually use too aggressive optimizations
and sometimes even makes debugging easier because machine code
is not exceedingly verbose.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D107962

show more ...

[compiler-rt] Build custom libcxx with libcxxabi

This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The r

[compiler-rt] Build custom libcxx with libcxxabi

This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The resulting libFuzzer doesn't expose C++ internals and looks
like a plain C library.
2) We don't have to manually link in libstdc++ to provide cxxabi.
3) The sanitizer tests cannot interfere with an installed version
of libc++.so in LD_LIBRARY_PATH.

Differential Revision: https://reviews.llvm.org/D58013

llvm-svn: 354212

show more ...