diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/Changelog.md | 39 | ||||
| -rw-r--r-- | docs/INSTALL.md | 126 | ||||
| -rw-r--r-- | docs/env_variables.md | 3 | ||||
| -rw-r--r-- | docs/fuzzing_in_depth.md | 2 | ||||
| -rw-r--r-- | docs/third_party_tools.md | 4 | ||||
| -rw-r--r-- | docs/tutorials.md | 3 |
6 files changed, 102 insertions, 75 deletions
diff --git a/docs/Changelog.md b/docs/Changelog.md index 48003f4b..2428d63f 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -3,6 +3,45 @@ This is the list of all noteworthy changes made in every public release of the tool. See README.md for the general instruction manual. +### Version ++4.20c (release) + ! A new forkserver communication model is now introduced. afl-fuzz is + backward compatible to old compiled targets if they are not built + for CMPLOG/Redqueen, but new compiled targets will not work with + old afl-fuzz versions! + ! Recompile all targets that are instrumented for CMPLOG/Redqueen! + - AFL++ now supports up to 4 billion coverage edges, up from 6 million. + - New compile option: `make PERFORMANCE=1` - this will enable special + CPU dependent optimizations that make everything more performant - but + the binaries will likely won't work on different platforms. Also + enables a faster hasher if the CPU requirements are met. + - The persistent record feature (see config.h) was expanded to also + support replay, thanks to @quarta-qti ! + - afl-fuzz: + - the new deterministic fuzzing feature is now activated by default, + deactivate with -z. Parameters -d and -D are ignored. + - small improvements to CMPLOG/redqueen + - workround for a bug with MOpt -L when used with -M - in the future + we will either remove or rewrite MOpt. + - fix for `-t xxx+` feature + - -e extension option now saves the queue items, crashes, etc. with the + extension too + - fixes for trimmming, correct -V time and reading stats on resume by eqv + thanks a lot! + - afl-cc: + - added collision free caller instrumentation to LTO mode. activate with + `AFL_LLVM_LTO_CALLER=1`. You can set a max depth to go through single + block functions with `AFL_LLVM_LTO_CALLER_DEPTH` (default 0) + - fixes for COMPCOV/LAF and most other modules + - fix for GCC_PLUGIN cmplog that broke on std::strings + - afl-whatsup: + - now also displays current average speed + - small bugfixes + - Fixes for aflpp custom mutator and standalone tool + - Minor edits to afl-persistent-config + - Prevent temporary files being left behind on aborted afl-whatsup + - More CPU benchmarks added to benchmark/ + + ### Version ++4.10c (release) - afl-fuzz: - default power schedule is now EXPLORE, due a fix in fast schedules diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 84bbe3ea..3089aab2 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -21,7 +21,7 @@ If you want to build AFL++ yourself, you have many options. The easiest choice is to build and install everything: NOTE: depending on your Debian/Ubuntu/Kali/... release, replace `-14` with -whatever llvm version is available. We recommend llvm 13, 14, 15 or 16. +whatever llvm version is available. We recommend llvm 13 or newer. ```shell sudo apt-get update @@ -67,19 +67,20 @@ These build targets exist: * unit: perform unit tests (based on cmocka) * help: shows these build options -[Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html), +[Unless you are on macOS](https://developer.apple.com/library/archive/qa/qa1118/_index.html), you can also build statically linked versions of the AFL++ binaries by passing -the `STATIC=1` argument to make: +the `PERFORMANCE=1` argument to make: ```shell -make STATIC=1 +make PERFORMANCE=1 ``` These build options exist: -* STATIC - compile AFL++ static -* CODE_COVERAGE - compile the target for code coverage (see docs/instrumentation/README.llvm.md) -* ASAN_BUILD - compiles AFL++ with memory sanitizer for debug purposes +* PERFORMANCE - compile with performance options that make the binary not transferable to other systems. Recommended (except on macOS)! +* STATIC - compile AFL++ static (does not work on macOS) +* CODE_COVERAGE - compile the target for code coverage (see [README.llvm.md](../instrumentation/README.llvm.md)) +* ASAN_BUILD - compiles AFL++ with address sanitizer for debug purposes * UBSAN_BUILD - compiles AFL++ tools with undefined behaviour sanitizer for debug purposes * DEBUG - no optimization, -ggdb3, all warnings and -Werror * LLVM_DEBUG - shows llvm deprecation warnings @@ -91,101 +92,78 @@ These build options exist: * NO_NYX - disable building nyx mode dependencies * NO_CORESIGHT - disable building coresight (arm64 only) * NO_UNICORN_ARM64 - disable building unicorn on arm64 -* AFL_NO_X86 - if compiling on non-intel/amd platforms +* AFL_NO_X86 - if compiling on non-Intel/AMD platforms * LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config (e.g., Debian) e.g.: `make LLVM_CONFIG=llvm-config-14` -## MacOS X on x86 and arm64 (M1) +## macOS on x86_64 and arm64 -MacOS has some gotchas due to the idiosyncrasies of the platform. +macOS has some gotchas due to the idiosyncrasies of the platform. -To build AFL, install llvm (and perhaps gcc) from brew and follow the general -instructions for Linux. If possible, avoid Xcode at all cost. +macOS supports SYSV shared memory used by AFL++'s instrumentation, but the +default settings aren't sufficient. Before even building, increase +them by running the provided script: + +```shell +sudo afl-system-config +``` + +See +[https://www.spy-hill.com/help/apple/SharedMemory.html](https://www.spy-hill.com/help/apple/SharedMemory.html) +for documentation for the shared memory settings and how to make them permanent. + +Next, to build AFL++, install the following packages from brew: ```shell brew install wget git make cmake llvm gdb coreutils ``` -Be sure to setup `PATH` to point to the correct clang binaries and use the -freshly installed clang, clang++, llvm-config, gmake and coreutils, e.g.: +Depending on your macOS system + brew version, brew may be installed in different places. +You can check with `brew info llvm` to know where, then create a variable for it: ```shell -# Depending on your MacOS system + brew version it is either -export PATH="/opt/homebrew/opt/llvm/bin:$PATH" -# or -export PATH="/usr/local/opt/llvm/bin:/usr/local/opt/coreutils/libexec/gnubin:$PATH" -# you can check with "brew info llvm" +export HOMEBREW_BASE="/opt/homebrew/opt" +``` -export PATH="/usr/local/bin:$PATH" +or + +```shell +export HOMEBREW_BASE="/usr/local/opt" +``` + +Set `PATH` to point to the brew clang, clang++, llvm-config, gmake and coreutils. +Also use the brew clang compiler; the Xcode clang compiler must not be used. + +```shell +export PATH="$HOMEBREW_BASE/coreutils/libexec/gnubin:/usr/local/bin:$HOMEBREW_BASE/llvm/bin:$PATH" export CC=clang export CXX=clang++ -gmake -cd frida_mode -gmake -cd .. -sudo gmake install ``` -`afl-gcc` will fail unless you have GCC installed, but that is using outdated -instrumentation anyway. `afl-clang` might fail too depending on your PATH setup. -But you don't want neither, you want `afl-clang-fast` anyway :) Note that -`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on MacOS. +Then build following the general Linux instructions. -The crash reporting daemon that comes by default with MacOS X will cause -problems with fuzzing. You need to turn it off: +If everything worked, you should then have `afl-clang-fast` installed, which you can check with: +```shell +which afl-clang-fast ``` -launchctl unload -w /System/Library/LaunchAgents/com.apple.ReportCrash.plist -sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.ReportCrash.Root.plist -``` -The `fork()` semantics on OS X are a bit unusual compared to other unix systems +Note that `afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on macOS. + +The crash reporting daemon that comes by default with macOS will cause +problems with fuzzing. You need to turn it off, which you can do with `afl-system-config`. + +The `fork()` semantics on macOS are a bit unusual compared to other unix systems and definitely don't look POSIX-compliant. This means two things: - Fuzzing will be probably slower than on Linux. In fact, some folks report considerable performance gains by running the jobs inside a Linux VM on - MacOS X. + macOS. - Some non-portable, platform-specific code may be incompatible with the AFL++ forkserver. If you run into any problems, set `AFL_NO_FORKSRV=1` in the environment before starting afl-fuzz. -User emulation mode of QEMU does not appear to be supported on MacOS X, so +User emulation mode of QEMU does not appear to be supported on macOS, so black-box instrumentation mode (`-Q`) will not work. However, FRIDA mode (`-O`) -works on both x86 and arm64 MacOS boxes. - -MacOS X supports SYSV shared memory used by AFL's instrumentation, but the -default settings aren't usable with AFL++. The default settings on 10.14 seem to -be: - -```bash -$ ipcs -M -IPC status from <running system> as of XXX -shminfo: - shmmax: 4194304 (max shared memory segment size) - shmmin: 1 (min shared memory segment size) - shmmni: 32 (max number of shared memory identifiers) - shmseg: 8 (max shared memory segments per process) - shmall: 1024 (max amount of shared memory in pages) -``` - -To temporarily change your settings to something minimally usable with AFL++, -run these commands as root: - -```bash -sysctl kern.sysv.shmmax=8388608 -sysctl kern.sysv.shmall=4096 -``` - -If you're running more than one instance of AFL, you likely want to make -`shmall` bigger and increase `shmseg` as well: - -```bash -sysctl kern.sysv.shmmax=8388608 -sysctl kern.sysv.shmseg=48 -sysctl kern.sysv.shmall=98304 -``` - -See -[http://www.spy-hill.com/help/apple/SharedMemory.html](http://www.spy-hill.com/help/apple/SharedMemory.html) -for documentation for these settings and how to make them permanent. +works on both x86 and arm64 macOS boxes. diff --git a/docs/env_variables.md b/docs/env_variables.md index a972b6da..1e4fc7ba 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -248,6 +248,9 @@ use (which only ever the author of this LTO implementation will use). These are used if several separated instrumentations are performed which are then later combined. + - `AFL_LLVM_LTO_CALLER` activates collision free CALLER instrumentation + - `AFL_LLVM_LTO_CALLER` sets the maximum mumber of single block functions + to dig deeper into a real function. Default 0. - `AFL_LLVM_DOCUMENT_IDS=file` will document to a file which edge ID was given to which function. This helps to identify functions with variable bytes or which functions were touched by an input. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 6a217641..82437807 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -958,7 +958,7 @@ too long for your overall available fuzz run time. campaign but not good for short CI runs. How this can look like can, e.g., be seen at AFL++'s setup in Google's -[oss-fuzz](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/compile_afl) +[previous oss-fuzz version](https://github.com/google/oss-fuzz/blob/3e2c5312417d1a6f9564472f3df1fd27759b289d/infra/base-images/base-builder/compile_afl) and [clusterfuzz](https://github.com/google/clusterfuzz/blob/master/src/clusterfuzz/_internal/bot/fuzzers/afl/launcher.py). diff --git a/docs/third_party_tools.md b/docs/third_party_tools.md index 02a40ce5..64a37c83 100644 --- a/docs/third_party_tools.md +++ b/docs/third_party_tools.md @@ -5,6 +5,10 @@ * [afl-rs](https://github.com/rust-fuzz/afl.rs) - AFL++ for RUST * [WASM](https://github.com/fgsect/WAFL) - AFL++ for WASM +## Starting multiple AFL++ instances in parallel with recommended settings: +* [https://github.com/0xricksanchez/AFL_Runner](https://github.com/0xricksanchez/AFL_Runner) +* [https://github.com/MegaManSec/AFLplusplus-Parallel-Gen](https://github.com/MegaManSec/AFLplusplus-Parallel-Gen) + ## Speeding up fuzzing * [libfiowrapper](https://github.com/marekzmyslowski/libfiowrapper) - if the diff --git a/docs/tutorials.md b/docs/tutorials.md index 0a09f6dc..f9378232 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -21,6 +21,9 @@ training, then we can highly recommend the following: * [https://github.com/antonio-morales/Fuzzing101](https://github.com/antonio-morales/Fuzzing101) +A good workflow overview (like our [fuzzing_in_depth.md](fuzzing_in_depth.md)): +* [https://appsec.guide/docs/fuzzing/c-cpp/aflpp/](https://appsec.guide/docs/fuzzing/c-cpp/aflpp/) + Here is a good workflow description (and tutorial) for qemu_mode: * [https://airbus-seclab.github.io/AFLplusplus-blogpost/](https://airbus-seclab.github.io/AFLplusplus-blogpost/) |