1*ece814e9SGreg Tucker# ISA-L Testing 2*ece814e9SGreg Tucker 3*ece814e9SGreg TuckerTests are divided into check tests, unit tests and fuzz tests. Check tests, 4*ece814e9SGreg Tuckerbuilt with `make check`, should have no additional dependencies. Other unit 5*ece814e9SGreg Tuckertests built with `make test` may have additional dependencies in order to make 6*ece814e9SGreg Tuckercomparisons of the output of ISA-L to other standard libraries and ensure 7*ece814e9SGreg Tuckercompatibility. Fuzz tests are meant to be run with a fuzzing tool such as [AFL] 8*ece814e9SGreg Tuckeror [llvm libFuzzer] fuzzing to direct the input data based on coverage. There 9*ece814e9SGreg Tuckerare a number of scripts in the /tools directory to help with automating the 10*ece814e9SGreg Tuckerrunning of tests. 11*ece814e9SGreg Tucker 12*ece814e9SGreg Tucker## Test check 13*ece814e9SGreg Tucker 14*ece814e9SGreg Tucker`./tools/test_autorun.sh` is a helper script for kicking off check tests, that 15*ece814e9SGreg Tuckertypically run for a few minutes, or extended tests that could run much 16*ece814e9SGreg Tuckerlonger. The command `test_autorun.sh check` build and runs all check tests with 17*ece814e9SGreg Tuckerautotools and runs other short tests to ensure check tests, unit tests, 18*ece814e9SGreg Tuckerexamples, install, exe stack, format are correct. Each run of `test_autorun.sh` 19*ece814e9SGreg Tuckerbuilds tests with a new random test seed that ensures that each run is unique to 20*ece814e9SGreg Tuckerthe seed but deterministic for debugging. Tests are also built with sanitizers 21*ece814e9SGreg Tuckerand Electric Fence if available. 22*ece814e9SGreg Tucker 23*ece814e9SGreg Tucker## Extended tests 24*ece814e9SGreg Tucker 25*ece814e9SGreg TuckerExtended tests are initiated with the command `./tools/test_autorun.sh 26*ece814e9SGreg Tuckerext`. These build and run check tests, unit tests, and other utilities that can 27*ece814e9SGreg Tuckertake much longer than check tests alone. This includes special compression tools 28*ece814e9SGreg Tuckerand some cross targets such as the no-arch build of base functions only and 29*ece814e9SGreg Tuckermingw build if tools are available. 30*ece814e9SGreg Tucker 31*ece814e9SGreg Tucker## Fuzz testing 32*ece814e9SGreg Tucker 33*ece814e9SGreg Tucker`./tools/test_fuzz.sh` is a helper script for fuzzing to setup, build and run 34*ece814e9SGreg Tuckerthe ISA-L inflate fuzz tests on multiple fuzz tools. Fuzzing with 35*ece814e9SGreg Tucker[llvm libFuzzer] requires clang compiler tools with `-fsanitize=fuzzer` or 36*ece814e9SGreg Tucker`libFuzzer` installed. You can invoke the default fuzz tests under llvm with 37*ece814e9SGreg Tucker 38*ece814e9SGreg Tucker ./tools/test_fuzz.sh -e checked 39*ece814e9SGreg Tucker 40*ece814e9SGreg TuckerTo use [AFL], install tools and system setup for `afl-fuzz` and run 41*ece814e9SGreg Tucker 42*ece814e9SGreg Tucker ./tools/test_fuzz.sh -e checked --afl 1 --llvm -1 -d 1 43*ece814e9SGreg Tucker 44*ece814e9SGreg TuckerThis uses internal vectors as a seed. You can also specify a sample file to use 45*ece814e9SGreg Tuckeras a seed instead with `-f <file>`. One of three fuzz tests can be invoked: 46*ece814e9SGreg Tuckerchecked, simple, and round_trip. 47*ece814e9SGreg Tucker 48*ece814e9SGreg Tucker[llvm libFuzzer]: https://llvm.org/docs/LibFuzzer.html 49*ece814e9SGreg Tucker[AFL]: https://github.com/google/AFL 50