[libFuzzer] replace Vector/Set with std::vector/std::set. The custom names are not required any more since we now build with a private version of libc++. Fix some of the 81+ character lines. Mechanic

[libFuzzer] replace Vector/Set with std::vector/std::set. The custom names are not required any more since we now build with a private version of libc++. Fix some of the 81+ character lines. Mechanical change, NFC expected.

[libFuzzer] replace Vector/Set with std::vector/std::set.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D107374

show more ...

Revert "Refactor mutation strategies into a standalone library"

This reverts commit 361f742f168de0f0f256802a329c19d081615d0d.

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing s

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
* UseCmp
* UseMemmem
* SetCustomMutator
* SetCustomCrossOver
* LateInitialize (similar to the MutationDispatcher's original constructor)
* Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
* StartMutationSequence
* MutationSequence
* DictionaryEntrySequence
* RecommendDictionary
* RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447

show more ...

Revert "Refactor mutation strategies into a standalone library"

This reverts commit c4a41cd77c15c2905ac74beeec09f8343a65a549 due to
buildbot failure.

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing s

Refactor mutation strategies into a standalone library

This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
* UseCmp
* UseMemmem
* SetCustomMutator
* SetCustomCrossOver
* LateInitialize (similar to the MutationDispatcher's original constructor)
* Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
* StartMutationSequence
* MutationSequence
* DictionaryEntrySequence
* RecommendDictionary
* RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447

show more ...

Revert "[libFuzzer] Expose fuzzer::FuzzerDriver()"

This reverts commit 22a376e73bc218137738ee4ac8af44e223576475.

[libFuzzer] Expose fuzzer::FuzzerDriver()

Summary:
Allows other-language libFuzzer derivatives to invoke the driver.

Patch By: Ian Eldred Pudney <ipudney@google.com>

Reviewers: morehouse, kcc

Rev

[libFuzzer] Expose fuzzer::FuzzerDriver()

Summary:
Allows other-language libFuzzer derivatives to invoke the driver.

Patch By: Ian Eldred Pudney <ipudney@google.com>

Reviewers: morehouse, kcc

Reviewed By: morehouse

Subscribers: #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D84425

show more ...

[libFuzzer] Separate platform related macros out from FuzzerDefs.h into FuzzerPlatform.h, and adjust includes in other files.

Summary: This patch separates platform related macros in lib/fuzzer/Fuzz

[libFuzzer] Separate platform related macros out from FuzzerDefs.h into FuzzerPlatform.h, and adjust includes in other files.

Summary: This patch separates platform related macros in lib/fuzzer/FuzzerDefs.h into lib/fuzzer/FuzzerPlatform.h, and use FuzzerPlatform.h where necessary. This separation helps when compiling libFuzzer's interceptor module (under review); an unnecessary include of standard headers (such as string.h) may produce conflicts/ambiguation with the interceptor's declarations/definitions of library functions, which complicates interceptor implementation.

Reviewers: morehouse, hctim

Reviewed By: morehouse

Subscribers: krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D83805

show more ...

[fuzzer] Add basic support for emscripten.

Summary:
Add basic support for emscripten.

This enables libFuzzer to build (using build.sh) for emscripten and fuzz
a target compiled with
-fsanitize-cove

[fuzzer] Add basic support for emscripten.

Summary:
Add basic support for emscripten.

This enables libFuzzer to build (using build.sh) for emscripten and fuzz
a target compiled with
-fsanitize-coverage=inline-8bit-counters.

Basic fuzzing and bug finding work with this commit.
RSS limit and timeouts will not work because they depend on system
functions that are not implemented/widely supported in emscripten.

Reviewers: kcc, vitalybuka, hctim

Reviewed By: hctim

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71285

show more ...

[libFuzzer] Make -merge=1 to reuse coverage information from the control file.

Summary:
This change allows to perform corpus merging in two steps. This is useful when
the user wants to address the f

[libFuzzer] Make -merge=1 to reuse coverage information from the control file.

Summary:
This change allows to perform corpus merging in two steps. This is useful when
the user wants to address the following two points simultaneously:

1) Get trustworthy incremental stats for the coverage and corpus size changes
when adding new corpus units.
2) Make sure the shorter units will be preferred when two or more units give the
same unique signal (equivalent to the `REDUCE` logic).

This solution was brainstormed together with @kcc, hopefully it looks good to
the other people too. The proposed use case scenario:

1) We have a `fuzz_target` binary and `existing_corpus` directory.
2) We do fuzzing and write new units into the `new_corpus` directory.
3) We want to merge the new corpus into the existing corpus and satisfy the
points mentioned above.
4) We create an empty directory `merged_corpus` and run the first merge step:

`
./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus
`

this provides the initial stats for `existing_corpus`, e.g. from the output:

`
MERGE-OUTER: 3 new files with 11 new features added; 11 new coverage edges
`

5) We recreate `merged_corpus` directory and run the second merge step:

`
./fuzz_target -merge=1 -merge_control_file=MCF ./merged_corpus ./existing_corpus ./new_corpus
`

this provides the final stats for the merged corpus, e.g. from the output:

`
MERGE-OUTER: 6 new files with 14 new features added; 14 new coverage edges
`

Alternative solutions to this approach are:

A) Store precise coverage information for every unit (not only unique signal).
B) Execute the same two steps without reusing the control file.

Either of these would be suboptimal as it would impose an extra disk or CPU load
respectively, which is bad given the quadratic complexity in the worst case.

Tested on Linux, Mac, Windows.

Reviewers: morehouse, metzman, hctim, kcc

Reviewed By: morehouse

Subscribers: JDevlieghere, delcypher, mgrang, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66107

llvm-svn: 371620

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[libFuzzer][MSVC] Make attribute-use compatible with MSVC

Summary:
Replace attributes with macros that use equivalent declspecs
for MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differentia

[libFuzzer][MSVC] Make attribute-use compatible with MSVC

Summary:
Replace attributes with macros that use equivalent declspecs
for MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D56512

llvm-svn: 351456

show more ...

[libfuzzer][MSVC] Make calls to builtin functions work with MSVC

Summary:
Replace calls to builtin functions with macros or functions that call the
Windows-equivalents when targeting windows and cal

[libfuzzer][MSVC] Make calls to builtin functions work with MSVC

Summary:
Replace calls to builtin functions with macros or functions that call the
Windows-equivalents when targeting windows and call the original
builtin functions everywhere else.
This change makes more parts of libFuzzer buildable with MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: mgorny, rnk, thakis

Differential Revision: https://reviews.llvm.org/D56439

llvm-svn: 350766

show more ...

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It a

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Patch By: metzman

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 341082

show more ...

Revert "[libFuzzer] Port to Windows"

This reverts r340949 due to bot breakage again.

llvm-svn: 340954

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It a

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: #sanitizers, delcypher, morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 340949

show more ...

Revert "[libFuzzer] Port to Windows"

This reverts commit r340860 due to failing tests.

llvm-svn: 340867

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It a

[libFuzzer] Port to Windows

Summary:
Port libFuzzer to windows-msvc.
This patch allows libFuzzer targets to be built and run on Windows, using -fsanitize=fuzzer and/or fsanitize=fuzzer-no-link. It allows these forms of coverage instrumentation to work on Windows as well.
It does not fix all issues, such as those with -fsanitize-coverage=stack-depth, which is not usable on Windows as of this patch.
It also does not fix any libFuzzer integration tests. Nearly all of them fail to compile, fixing them will come in a later patch, so libFuzzer tests are disabled on Windows until them.

Patch By: metzman

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: morehouse, kcc, eraman

Differential Revision: https://reviews.llvm.org/D51022

llvm-svn: 340860

show more ...

libFuzzer: prevent irrelevant strings from leaking into auto-dictionary

This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previ

libFuzzer: prevent irrelevant strings from leaking into auto-dictionary

This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.

Patch By: pdknsk

Differential Revision: https://reviews.llvm.org/D48800

llvm-svn: 337296

show more ...

Fix compile error with libstdc++.

By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of

Fix compile error with libstdc++.

By adding a ctor to create fuzzer_allocator<T> from fuzzer_allocator<U>.
This mimics construcotrs of std::allocator<T>.

Without the constructors, some versions of libstdc++ can't compile
`vector<bool, fuzzer_allocator<bool>>`.

llvm-svn: 334077

show more ...

[libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal

[libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage

llvm-svn: 332034

show more ...

libFuzzer, OpenBSD support

Summary:
- Enabling libfuzzer on OpenBSD
- OpenBSD can t support asan, msan ... the tests can t be run.

Patch by David CARLIER

Reviewers: eugenis, phosek, vitalybuka

Re

libFuzzer, OpenBSD support

Summary:
- Enabling libfuzzer on OpenBSD
- OpenBSD can t support asan, msan ... the tests can t be run.

Patch by David CARLIER

Reviewers: eugenis, phosek, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44877

llvm-svn: 329631

show more ...

lib Fuzzer FreeBSD support

Summary: Patch by David CARLIER

Reviewers: vitalybuka, kcc, dim, emaste, davide, morehouse, george.karpenkov

Reviewed By: morehouse

Subscribers: george.karpenkov, kubam

lib Fuzzer FreeBSD support

Summary: Patch by David CARLIER

Reviewers: vitalybuka, kcc, dim, emaste, davide, morehouse, george.karpenkov

Reviewed By: morehouse

Subscribers: george.karpenkov, kubamracek, srhines, mgorny, emaste, krytarowski

Differential Revision: https://reviews.llvm.org/D41642

llvm-svn: 322380

show more ...

[libFuzzer] Add support for Fuchsia OS.

Summary:
This patch adds the initial support for Fuchsia.

- LIBFUZZER_FUCHSIA is added as an OS type in FuzzerDefs.h
- Fuchsia is, by design, not POSIX compl

[libFuzzer] Add support for Fuchsia OS.

Summary:
This patch adds the initial support for Fuchsia.

- LIBFUZZER_FUCHSIA is added as an OS type in FuzzerDefs.h
- Fuchsia is, by design, not POSIX compliant. However, it does use ELF and
supports common POSIX I/O functions. Thus, FuzzerExtFunctions.h and
FuzzerIO.h are implemented by extending the header guards in
FuzzerExtFunctionsWeak.cpp and FuzzerIOPosix.cpp to include
LIBFUZZER_FUCHSIA.
- The platform-specific portions of FuzzerUtil.h are implemented by
FuzzerUtilFuchsia.cpp, which makes use of exception ports, syscalls, and
the launchpad library.
- The experimental equivalence server is not currently supported, so
FuzzerShmem.h is implemented by stub methods in FuzzerShmemFuchsia.cpp.
Any future implementation will likely involve VMOs.

Tested with ASAN/SanCov on Fuchsia/x86-64 with the canonical toy fuzzer.

Patch By: aarongreen

Reviewers: kcc, morehouse, flowerhack, phosek

Reviewed By: kcc, phosek, Eugene.Zelenko

Subscribers: srhines, mgorny, Eugene.Zelenko

Differential Revision: https://reviews.llvm.org/D40974

llvm-svn: 320210

show more ...

Add preliminary NetBSD support in libfuzzer

Summary:
This code already works and passes some number of tests.

There is need to finish remaining sanitizers to get better coverage.

Many tests fail d

Add preliminary NetBSD support in libfuzzer

Summary:
This code already works and passes some number of tests.

There is need to finish remaining sanitizers to get better coverage.

Many tests fail due to overly long file names of executables (>31).
This is a current shortcoming of the NetBSD 8(beta) kernel, as
certain functions can fail (like retrieving file name of executable).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, george.karpenkov

Reviewed By: kcc

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37304

llvm-svn: 312183

show more ...