diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/Changelog.md | 233 | ||||
| -rw-r--r-- | docs/FAQ.md | 170 | ||||
| -rw-r--r-- | docs/INSTALL.md | 28 | ||||
| l--------- | docs/README.md | 1 | ||||
| -rw-r--r-- | docs/binaryonly_fuzzing.md | 25 | ||||
| -rw-r--r-- | docs/custom_mutators.md | 86 | ||||
| -rw-r--r-- | docs/docs.md | 122 | ||||
| -rw-r--r-- | docs/env_variables.md | 522 | ||||
| -rw-r--r-- | docs/ideas.md | 91 | ||||
| -rw-r--r-- | docs/life_pro_tips.md | 13 | ||||
| -rw-r--r-- | docs/notes_for_asan.md | 6 | ||||
| -rw-r--r-- | docs/parallel_fuzzing.md | 113 | ||||
| -rw-r--r-- | docs/perf_tips.md | 8 | ||||
| -rw-r--r-- | docs/power_schedules.md | 4 | ||||
| -rw-r--r-- | docs/rpc_statsd.md | 143 | ||||
| -rw-r--r-- | docs/sister_projects.md | 12 | ||||
| -rw-r--r-- | docs/statsd/grafana-afl++.json | 1816 | ||||
| -rw-r--r-- | docs/status_screen.md | 41 | ||||
| -rw-r--r-- | docs/visualization/statsd-grafana.png | bin | 0 -> 163646 bytes |
19 files changed, 3010 insertions, 424 deletions
diff --git a/docs/Changelog.md b/docs/Changelog.md index 1e7a1c1d..8dc218af 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -8,25 +8,250 @@ Want to stay in the loop on major new features? Join our mailing list by sending a mail to <afl-users+subscribe@googlegroups.com>. +### Version ++3.12a (dev) + - afl-cc: + - fix cmplog rtn (rare crash and not being able to gather ptr data) + - link runtime not to shared libs + - ensure shared libraries are properly built and instrumented + - qemu_mode (thanks @realmadsci): + - move AFL_PRELOAD and AFL_USE_QASAN logic inside afl-qemu-trace + - add AFL_QEMU_CUSTOM_BIN + +### Version ++3.11c (release) + - afl-fuzz: + - better auto detection of map size + - fix sanitizer settings (bug since 3.10c) + - fix an off-by-one overwrite in cmplog + - add non-unicode variants from unicode-looking dictionary entries + - Rust custom mutator API improvements + - Imported crash stats painted yellow on resume (only new ones are red) + - afl-cc: + - added AFL_NOOPT that will just pass everything to the normal + gcc/clang compiler without any changes - to pass weird configure + scripts + - fixed a crash that can occur with ASAN + CMPLOG together plus + better support for unicode (thanks to @stbergmann for reporting!) + - fixed a crash in LAF transform for empty strings + - handle erroneous setups in which multiple afl-compiler-rt are + compiled into the target. This now also supports dlopen() + instrumented libs loaded before the forkserver and even after the + forkserver is started (then with collisions though) + - the compiler rt was added also in object building (-c) which + should have been fixed years ago but somewhere got lost :( + - Renamed CTX to CALLER, added correct/real CTX implementation to + CLASSIC + - qemu_mode: + - added AFL_QEMU_EXCLUDE_RANGES env by @realmadsci, thanks! + - if no new/updated checkout is wanted, build with: + NO_CHECKOUT=1 ./build_qemu_support.sh + - we no longer perform a "git drop" + - afl-cmin: support filenames with spaces + +### Version ++3.10c (release) + - Mac OS ARM64 support + - Android support fixed and updated by Joey Jiaojg - thanks! + - New selective instrumentation option with __AFL_COVERAGE_* commands + to be placed in the source code. + Check out instrumentation/README.instrument_list.md + - afl-fuzz + - Making AFL_MAP_SIZE (mostly) obsolete - afl-fuzz now learns on + start the target map size + - upgraded cmplog/redqueen: solving for floating point, solving + transformations (e.g. toupper, tolower, to/from hex, xor, + arithmetics, etc.). This is costly hence new command line option + `-l` that sets the intensity (values 1 to 3). Recommended is 2. + - added `AFL_CMPLOG_ONLY_NEW` to not use cmplog on initial seeds + from `-i` or resumes (these have most likely already been done) + - fix crash for very, very fast targets+systems (thanks to mhlakhani + for reporting) + - on restarts (`-i`)/autoresume (AFL_AUTORESUME) the stats are now + reloaded and used, thanks to Vimal Joseph for this patch! + - changed the meaning of '+' of the '-t' option, it now means to + auto-calculate the timeout with the value given being the max + timeout. The original meaning of skipping timeouts instead of + abort is now inherent to the -t option. + - if deterministic mode is active (`-D`, or `-M` without `-d`) then + we sync after every queue entry as this can take very long time + otherwise + - added minimum SYNC_TIME to include/config.h (30 minutes default) + - better detection if a target needs a large shared map + - fix for `-Z` + - fixed a few crashes + - switched to an even faster RNG + - added hghwng's patch for faster trace map analysis + - printing suggestions for mistyped `AFL_` env variables + - added Rust bindings for custom mutators (thanks @julihoh) + - afl-cc + - allow instrumenting LLVMFuzzerTestOneInput + - fixed endless loop for allow/blocklist lines starting with a + comment (thanks to Zherya for reporting) + - cmplog/redqueen now also tracks floating point, _ExtInt() + 128bit + - cmplog/redqueen can now process basic libc++ and libstdc++ + std::string comparisons (no position or length type variants) + - added support for __afl_coverage_interesting() for LTO and our + own PCGUARD (llvm 10.0.1+), read more about this function and + selective coverage in instrumentation/README.instrument_list.md + - added AFL_LLVM_INSTRUMENT option NATIVE for native clang pc-guard + support (less performant than our own), GCC for old afl-gcc and + CLANG for old afl-clang + - fixed a potential crash in the LAF feature + - workaround for llvm bitcast lto bug + - workaround for llvm 13 + - qemuafl + - QASan (address sanitizer for Qemu) ported to qemuafl! + See qemu_mode/libqasan/README.md + - solved some persistent mode bugs (thanks Dil4rd) + - solved an issue when dumping the memory maps (thanks wizche) + - Android support for QASan + - unicornafl + - Substantial speed gains in python bindings for certain use cases + - Improved rust bindings + - Added a new example harness to compare python, c and rust bindings + - afl-cmin and afl-showmap now support the -f option + - afl_plot now also generates a graph on the discovered edges + - changed default: no memory limit for afl-cmin and afl-cmin.bash + - warn on any _AFL and __AFL env vars. + - set AFL_IGNORE_UNKNOWN_ENVS to not warn on unknown AFL_... env vars + - added dummy Makefile to instrumentation/ + - Updated utils/afl_frida to be 5% faster, 7% on x86_x64 + - Added `AFL_KILL_SIGNAL` env variable (thanks @v-p-b) + - @Edznux added a nice documentation on how to use rpc.statsd with + afl++ in docs/rpc_statsd.md, thanks! + +### Version ++3.00c (release) + - llvm_mode/ and gcc_plugin/ moved to instrumentation/ + - examples/ renamed to utils/ + - moved libdislocator, libtokencap and qdbi_mode to utils/ + - all compilers combined to afl-cc which emulates the previous ones + - afl-llvm/gcc-rt.o merged into afl-compiler-rt.o + - afl-fuzz + - not specifying -M or -S will now auto-set "-S default" + - deterministic fuzzing is now disabled by default and can be enabled with + -D. It is still enabled by default for -M. + - a new seed selection was implemented that uses weighted randoms based on + a schedule performance score, which is much better that the previous + walk the whole queue approach. Select the old mode with -Z (auto enabled + with -M) + - Marcel Boehme submitted a patch that improves all AFFast schedules :) + - the default schedule is now FAST + - memory limits are now disabled by default, set them with -m if required + - rpc.statsd support, for stats and charts, by Edznux, thanks a lot! + - reading testcases from -i now descends into subdirectories + - allow the -x command line option up to 4 times + - loaded extras now have a duplication protection + - If test cases are too large we do a partial read on the maximum + supported size + - longer seeds with the same trace information will now be ignored + for fuzzing but still be used for splicing + - crashing seeds are now not prohibiting a run anymore but are + skipped - they are used for splicing, though + - update MOpt for expanded havoc modes + - setting the env var AFL_NO_AUTODICT will not load an LTO autodictionary + - added NO_SPLICING compile option and makefile define + - added INTROSPECTION make target that writes all mutations to + out/NAME/introspection.txt + - print special compile time options used in help output + - when using -c cmplog, one of the childs was not killed, fixed + - somewhere we broke -n dumb fuzzing, fixed + - added afl_custom_describe to the custom mutator API to allow for easy + mutation reproduction on crashing inputs + - new env. var. AFL_NO_COLOR (or AFL_NO_COLOUR) to suppress colored + console output (when configured with USE_COLOR and not ALWAYS_COLORED) + - instrumentation + - We received an enhanced gcc_plugin module from AdaCore, thank you + very much!! + - not overriding -Ox or -fno-unroll-loops anymore + - we now have our own trace-pc-guard implementation. It is the same as + -fsanitize-coverage=trace-pc-guard from llvm 12, but: it is a) inline + and b) works from llvm 10.0.1 + onwards :) + - new llvm pass: dict2file via AFL_LLVM_DICT2FILE, create afl-fuzz + -x dictionary of string comparisons found during compilation + - LTO autodict now also collects interesting cmp comparisons, + std::string compare + find + ==, bcmp + - fix crash in dict2file for integers > 64 bit + - custom mutators + - added a new custom mutator: symcc -> https://github.com/eurecom-s3/symcc/ + - added a new custom mutator: libfuzzer that integrates libfuzzer mutations + - Our afl++ Grammar-Mutator is now better integrated into custom_mutators/ + - added INTROSPECTION support for custom modules + - python fuzz function was not optional, fixed + - some python mutator speed improvements + - afl-cmin/afl-cmin.bash now search first in PATH and last in AFL_PATH + - unicornafl synced with upstream version 1.02 (fixes, better rust bindings) + - renamed AFL_DEBUG_CHILD_OUTPUT to AFL_DEBUG_CHILD + - added AFL_CRASH_EXITCODE env variable to treat a child exitcode as crash + + +### Version ++2.68c (release) + - added the GSoC excellent afl++ grammar mutator by Shengtuo to our + custom_mutators/ (see custom_mutators/README.md) - or get it here: + https://github.com/AFLplusplus/Grammar-Mutator + - a few QOL changes for Apple and its outdated gmake + - afl-fuzz: + - fix for auto dictionary entries found during fuzzing to not throw out + a -x dictionary + - added total execs done to plot file + - AFL_MAX_DET_EXTRAS env variable added to control the amount of + deterministic dict entries without recompiling. + - AFL_FORKSRV_INIT_TMOUT env variable added to control the time to wait + for the forkserver to come up without the need to increase the overall + timeout. + - bugfix for cmplog that results in a heap overflow based on target data + (thanks to the magma team for reporting!) + - write fuzzing setup into out/fuzzer_setup (environment variables and + command line) + - custom mutators: + - added afl_custom_fuzz_count/fuzz_count function to allow specifying + the number of fuzz attempts for custom_fuzz + - llvm_mode: + - ported SanCov to LTO, and made it the default for LTO. better + instrumentation locations + - Further llvm 12 support (fast moving target like afl++ :-) ) + - deprecated LLVM SKIPSINGLEBLOCK env environment + -### Version ++2.66d (devel) +### Version ++2.67c (release) - Support for improved afl++ snapshot module: https://github.com/AFLplusplus/AFL-Snapshot-LKM + - Due to the instrumentation needing more memory, the initial memory sizes + for -m have been increased - afl-fuzz: - added -F option to allow -M main fuzzers to sync to foreign fuzzers, e.g. honggfuzz or libfuzzer + - added -b option to bind to a specific CPU - eliminated CPU affinity race condition for -S/-M runs - expanded havoc mode added, on no cycle finds add extra splicing and MOpt into the mix + - fixed a bug in redqueen for strings and made deterministic with -s + - Compiletime autodictionary fixes - llvm_mode: - - now supports llvm 12! + - now supports llvm 12 + - support for AFL_LLVM_ALLOWLIST/AFL_LLVM_DENYLIST (previous + AFL_LLVM_WHITELIST and AFL_LLVM_INSTRUMENT_FILE are deprecated and + are matched to AFL_LLVM_ALLOWLIST). The format is compatible to llvm + sancov, and also supports function matching :) + - added neverzero counting to trace-pc/pcgard - fixes for laf-intel float splitting (thanks to mark-griffin for reporting) - - LTO: autodictionary mode is a default + - fixes for llvm 4.0 + - skipping ctors and ifuncs for instrumentation + - LTO: switch default to the dynamic memory map, set AFL_LLVM_MAP_ADDR + for a fixed map address (eg. 0x10000) + - LTO: improved stability for persistent mode, no other instrumentation + has that advantage + - LTO: fixed autodict for long strings + - LTO: laf-intel and redqueen/cmplog are now applied at link time + to prevent llvm optimizing away the splits + - LTO: autodictionary mode is a fixed default now - LTO: instrim instrumentation disabled, only classic support used as it is always better + - LTO: env var AFL_LLVM_DOCUMENT_IDS=file will document which edge ID + was given to which function during compilation + - LTO: single block functions were not implemented by default, fixed + - LTO: AFL_LLVM_SKIP_NEVERZERO behaviour was inversed, fixed - setting AFL_LLVM_LAF_SPLIT_FLOATS now activates AFL_LLVM_LAF_SPLIT_COMPARES + - support for -E and -shared compilation runs - added honggfuzz mangle as a custom mutator in custom_mutators/honggfuzz - added afl-frida gum solution to examples/afl_frida (mostly imported from https://github.com/meme/hotwax/) @@ -373,7 +598,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - big code refactoring: * all includes are now in include/ * all afl sources are now in src/ - see src/README.md - * afl-fuzz was splitted up in various individual files for including + * afl-fuzz was split up in various individual files for including functionality in other programs (e.g. forkserver, memory map, etc.) for better readability. * new code indention everywhere diff --git a/docs/FAQ.md b/docs/FAQ.md index e09385a8..714d50eb 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -2,31 +2,94 @@ ## Contents - 1. [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - 2. [What is an edge?](#what-is-an-edge) - 3. [Why is my stability below 100%?](#why-is-my-stability-below-100) - 4. [How can I improve the stability value](#how-can-i-improve-the-stability-value) + * [What is the difference between afl and afl++?](#what-is-the-difference-between-afl-and-afl) + * [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) + * [How do I fuzz a network service?](#how-do-i-fuzz-a-network-service) + * [How do I fuzz a GUI program?](#how-do-i-fuzz-a-gui-program) + * [What is an edge?](#what-is-an-edge) + * [Why is my stability below 100%?](#why-is-my-stability-below-100) + * [How can I improve the stability value?](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) -## How to improve the fuzzing speed +## What is the difference between afl and afl++? - 1. use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) +American Fuzzy Lop (AFL) was developed by Michał "lcamtuf" Zalewski starting in +2013/2014, and when he left Google end of 2017 he stopped developing it. + +At the end of 2019 the Google fuzzing team took over maintenance of AFL, however +it is only accepting PRs from the community and is not developing enhancements +anymore. + +In the second quarter of 2019, 1 1/2 year later when no further development of +AFL had happened and it became clear there would none be coming, afl++ +was born, where initially community patches were collected and applied +for bug fixes and enhancements. Then from various AFL spin-offs - mostly academic +research - features were integrated. This already resulted in a much advanced +AFL. + +Until the end of 2019 the afl++ team had grown to four active developers which +then implemented their own research and features, making it now by far the most +flexible and feature rich guided fuzzer available as open source. +And in independent fuzzing benchmarks it is one of the best fuzzers available, +e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) + +## How to improve the fuzzing speed? + + 1. Use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) 2. Use [persistent mode](llvm_mode/README.persistent_mode.md) (x2-x20 speed increase) 3. Use the [afl++ snapshot module](https://github.com/AFLplusplus/AFL-Snapshot-LKM) (x2 speed increase) - 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to point the input file on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) - 5. Improve kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) + 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input file directory on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) + 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system less secure) 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) -## What is an "edge" +## How do I fuzz a network service? + +The short answer is - you cannot, at least not "out of the box". + +Using a network channel is inadequate for several reasons: +- it has a slow-down of x10-20 on the fuzzing speed +- it does not scale to fuzzing multiple instances easily, +- instead of one initial data packet often a back-and-forth interplay of packets is needed for stateful protocols (which is totally unsupported by most coverage aware fuzzers). + +The established method to fuzz network services is to modify the source code +to read from a file or stdin (fd 0) (or even faster via shared memory, combine +this with persistent mode [llvm_mode/README.persistent_mode.md](llvm_mode/README.persistent_mode.md) +and you have a performance gain of x10 instead of a performance loss of over +x10 - that is a x100 difference!). + +If modifying the source is not an option (e.g. because you only have a binary +and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD +to emulate the network. This is also much faster than the real network would be. +See [utils/socket_fuzzing/](../utils/socket_fuzzing/). + +There is an outdated afl++ branch that implements networking if you are +desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - +however a better option is AFLnet ([https://github.com/aflnet/aflnet](https://github.com/aflnet/aflnet)) +which allows you to define network state with different type of data packets. + +## How do I fuzz a GUI program? + +If the GUI program can read the fuzz data from a file (via the command line, +a fixed location or via an environment variable) without needing any user +interaction then it would be suitable for fuzzing. + +Otherwise it is not possible without modifying the source code - which is a +very good idea anyway as the GUI functionality is a huge CPU/time overhead +for the fuzzing. + +So create a new `main()` that just reads the test case and calls the +functionality for processing the input that the GUI program is using. + +## What is an "edge"? A program contains `functions`, `functions` contain the compiled machine code. The compiled machine code in a `function` can be in a single or many `basic blocks`. A `basic block` is the largest possible number of subsequent machine code -instructions that runs independent, meaning it does not split up to different -locations nor is it jumped into it from a different location: +instructions that has exactly one entrypoint (which can be be entered by multiple other basic blocks) +and runs linearly without branching or jumping to other addresses (except at the end). ``` function() { A: @@ -36,7 +99,7 @@ function() { if (x) goto C; else goto D; C: some code - goto D + goto E D: some code goto B @@ -46,7 +109,7 @@ function() { ``` Every code block between two jump locations is a `basic block`. -An `edge` is then the unique relationship between two `basic blocks` (from the +An `edge` is then the unique relationship between two directly connected `basic blocks` (from the code example above): ``` Block A @@ -61,31 +124,48 @@ code example above): Block E ``` Every line between two blocks is an `edge`. +Note that a few basic block loop to itself, this too would be an edge. -## Why is my stability below 100 +## Why is my stability below 100%? Stability is measured by how many percent of the edges in the target are "stable". Sending the same input again and again should take the exact same path through the target every time. If that is the case, the stability is 100%. -If however randomness happens, e.g. a thread reading from shared memory, +If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc. then in some of the re-executions with the same data -will result in the edge information being different accross runs. +the edge coverage result will be different accross runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for afl++ to identify valid new paths. A value above 90% is usually fine and a value above 80% is also still ok, and -even above 20% can still result in successful finds of bugs. -However, it is recommended that below 90% or 80% you should take measures to -improve the stability. +even a value above 20% can still result in successful finds of bugs. +However, it is recommended that for values below 90% or 80% you should take +countermeasures to improve stability. + +## How can I improve the stability value? -## How can I improve the stability value +For fuzzing a 100% stable target that covers all edges is the best case. +A 90% stable target that covers all edges is however better than a 100% stable +target that ignores 10% of the edges. -Four steps are required to do this and requires quite some knowledge of -coding and/or disassembly and it is only effectively possible with -afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! +With instability you basically have a partial coverage loss on an edge, with +ignored functions you have a full loss on that edges. + +There are functions that are unstable, but also provide value to coverage, eg +init functions that use fuzz data as input for example. +If however a function that has nothing to do with the input data is the +source of instability, e.g. checking jitter, or is a hash map function etc. +then it should not be instrumented. + +To be able to exclude these functions (based on AFL++'s measured stability) +the following process will allow to identify functions with variable edges. + +Four steps are required to do this and it also requires quite some knowledge +of coding and/or disassembly and is effectively possible only with +afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. 1. First step: Identify which edge ID numbers are unstable @@ -93,32 +173,48 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! The out/fuzzer_stats file will then show the edge IDs that were identified as unstable. - 2. Second step: Find the responsible function. + 2. Second step: Find the responsible function(s). - a) For LTO instrumented binaries just disassemble or decompile the target - and look which edge is writing to that edge ID. Ghidra is a good tool - for this: [https://ghidra-sre.org/](https://ghidra-sre.org/) + a) For LTO instrumented binaries this can be documented during compile + time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. + This file will have one assigned edge ID and the corresponding + function per line. - b) For PCGUARD instrumented binaries it is more difficult. Here you can - either modify the __sanitizer_cov_trace_pc_guard function in + b) For PCGUARD instrumented binaries it is much more difficult. Here you + can either modify the __sanitizer_cov_trace_pc_guard function in llvm_mode/afl-llvm-rt.o.c to write a backtrace to a file if the ID in - __afl_area_ptr[*guard] is one of the unstable edge IDs. Then recompile - and reinstall llvm_mode and rebuild your target. Run the recompiled - target with afl-fuzz for a while and then check the file that you - wrote with the backtrace information. + __afl_area_ptr[*guard] is one of the unstable edge IDs. + (Example code is already there). + Then recompile and reinstall llvm_mode and rebuild your target. + Run the recompiled target with afl-fuzz for a while and then check the + file that you wrote with the backtrace information. Alternatively you can use `gdb` to hook __sanitizer_cov_trace_pc_guard_init on start, check to which memory address the edge ID value is written and set a write breakpoint to that address (`watch 0x.....`). - 3. Third step: create a text file with the filenames + c) in all other instrumentation types this is not possible. So just + recompile with the two mentioned above. This is just for + identifying the functions that have unstable edges. + + 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to - remove from instrumentation. + remove from instrumentation, or just specify the functions you want to + skip for instrumentation. Note that optimization might inline functions! + + Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) + If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): + [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) - Simply follow this document on how to do this: [llvm_mode/README.instrument_file.md](llvm_mode/README.instrument_file.md) - If PCGUARD is used, then you need to follow this guide: [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) + Only exclude those functions from instrumentation that provide no value + for coverage - that is if it does not process any fuzz data directly + or indirectly (e.g. hash maps, thread management etc.). + If however a function directly or indirectly handles fuzz data then you + should not put the function in a deny instrumentation list and rather + live with the instability it comes with. 4. Fourth step: recompile the target Recompile, fuzz it, be happy :) + This link explains this process for [Fuzzbench](https://github.com/google/fuzzbench/issues/677) diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 766f24d7..e3c06c9d 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -4,7 +4,7 @@ issues for a variety of platforms. See README.md for the general instruction manual. -## 1) Linux on x86 +## 1. Linux on x86 --------------- This platform is expected to work well. Compile the program with: @@ -24,15 +24,17 @@ There are no special dependencies to speak of; you will need GNU make and a working compiler (gcc or clang). Some of the optional scripts bundled with the program may depend on bash, gdb, and similar basic tools. -If you are using clang, please review llvm_mode/README.md; the LLVM +If you are using clang, please review README.llvm.md; the LLVM integration mode can offer substantial performance gains compared to the traditional approach. +Likewise, if you are using GCC, please review instrumentation/README.gcc_plugin.md. + You may have to change several settings to get optimal results (most notably, disable crash reporting utilities and switch to a different CPU governor), but afl-fuzz will guide you through that if necessary. -## OpenBSD, FreeBSD, NetBSD on x86 +## 2. OpenBSD, FreeBSD, NetBSD on x86 Similarly to Linux, these platforms are expected to work well and are regularly tested. Compile everything with GNU make: @@ -52,10 +54,10 @@ sudo gmake install Keep in mind that if you are using csh as your shell, the syntax of some of the shell commands given in the README.md and other docs will be different. -The `llvm_mode` requires a dynamically linked, fully-operational installation of +The `llvm` requires a dynamically linked, fully-operational installation of clang. At least on FreeBSD, the clang binaries are static and do not include some of the essential tools, so if you want to make it work, you may need to -follow the instructions in llvm_mode/README.md. +follow the instructions in README.llvm.md. Beyond that, everything should work as advertised. @@ -97,27 +99,24 @@ and definitely don't look POSIX-compliant. This means two things: User emulation mode of QEMU does not appear to be supported on MacOS X, so black-box instrumentation mode (`-Q`) will not work. -The llvm_mode requires a fully-operational installation of clang. The one that +The llvm instrumentation requires a fully-operational installation of clang. The one that comes with Xcode is missing some of the essential headers and helper tools. -See llvm_mode/README.md for advice on how to build the compiler from scratch. +See README.llvm.md for advice on how to build the compiler from scratch. ## 4. Linux or *BSD on non-x86 systems Standard build will fail on non-x86 systems, but you should be able to leverage two other options: - - The LLVM mode (see llvm_mode/README.md), which does not rely on + - The LLVM mode (see README.llvm.md), which does not rely on x86-specific assembly shims. It's fast and robust, but requires a complete installation of clang. - The QEMU mode (see qemu_mode/README.md), which can be also used for fuzzing cross-platform binaries. It's slower and more fragile, but can be used even when you don't have the source for the tested app. -If you're not sure what you need, you need the LLVM mode. To get it, try: - -```bash -AFL_NO_X86=1 gmake && gmake -C llvm_mode -``` +If you're not sure what you need, you need the LLVM mode, which is built by +default. ...and compile your target program with afl-clang-fast or afl-clang-fast++ instead of the traditional afl-gcc or afl-clang wrappers. @@ -160,7 +159,8 @@ instrumentation mode (`-Q`) will not work. ## 6. Everything else You're on your own. On POSIX-compliant systems, you may be able to compile and -run the fuzzer; and the LLVM mode may offer a way to instrument non-x86 code. +run the fuzzer; and the LLVM and GCC plugin modes may offer a way to instrument +non-x86 code. The fuzzer will run on Windows in WSL only. It will not work under Cygwin on in the normal Windows world. It could be ported to the latter platform fairly easily, but it's a pretty bad diff --git a/docs/README.md b/docs/README.md deleted file mode 120000 index 32d46ee8..00000000 --- a/docs/README.md +++ /dev/null @@ -1 +0,0 @@ -../README.md \ No newline at end of file diff --git a/docs/binaryonly_fuzzing.md b/docs/binaryonly_fuzzing.md index 111147e2..787d970d 100644 --- a/docs/binaryonly_fuzzing.md +++ b/docs/binaryonly_fuzzing.md @@ -6,18 +6,18 @@ However, if there is only the binary program and no source code available, then standard `afl-fuzz -n` (non-instrumented mode) is not effective. - The following is a description of how these binaries can be fuzzed with afl++ + The following is a description of how these binaries can be fuzzed with afl++. ## TL;DR: qemu_mode in persistent mode is the fastest - if the stability is high enough. Otherwise try retrowrite, afl-dyninst and if these - fail too then standard qemu_mode with AFL_ENTRYPOINT to where you need it. + fail too then try standard qemu_mode with AFL_ENTRYPOINT to where you need it. - If your a target is library use examples/afl_frida/. + If your target is a library use utils/afl_frida/. - If your target is non-linux then use unicorn_mode/ + If your target is non-linux then use unicorn_mode/. ## QEMU @@ -29,10 +29,10 @@ The speed decrease is at about 50%. However various options exist to increase the speed: - - using AFL_ENTRYPOINT to move the forkserver to a later basic block in + - using AFL_ENTRYPOINT to move the forkserver entry to a later basic block in the binary (+5-10% speed) - using persistent mode [qemu_mode/README.persistent.md](../qemu_mode/README.persistent.md) - this will result in 150-300% overall speed - so 3-8x the original + this will result in 150-300% overall speed increase - so 3-8x the original qemu_mode speed! - using AFL_CODE_START/AFL_CODE_END to only instrument specific parts @@ -65,14 +65,14 @@ ## AFL FRIDA If you want to fuzz a binary-only shared library then you can fuzz it with - frida-gum via examples/afl_frida/, you will have to write a harness to + frida-gum via utils/afl_frida/, you will have to write a harness to call the target function in the library, use afl-frida.c as a template. ## AFL UNTRACER If you want to fuzz a binary-only shared library then you can fuzz it with - examples/afl_untracer/, use afl-untracer.c as a template. + utils/afl_untracer/, use afl-untracer.c as a template. It is slower than AFL FRIDA (see above). @@ -104,7 +104,7 @@ ## RETROWRITE - If you have an x86/x86_64 binary that still has it's symbols, is compiled + If you have an x86/x86_64 binary that still has its symbols, is compiled with position independant code (PIC/PIE) and does not use most of the C++ features then the retrowrite solution might be for you. It decompiles to ASM files which can then be instrumented with afl-gcc. @@ -148,7 +148,7 @@ ## CORESIGHT Coresight is ARM's answer to Intel's PT. - There is no implementation so far which handle coresight and getting + There is no implementation so far which handles coresight and getting it working on an ARM Linux is very difficult due to custom kernel building on embedded systems is difficult. And finding one that has coresight in the ARM chip is difficult too. @@ -174,7 +174,7 @@ Pintool and Dynamorio are dynamic instrumentation engines, and they can be used for getting basic block information at runtime. - Pintool is only available for Intel x32/x64 on Linux, Mac OS and Windows + Pintool is only available for Intel x32/x64 on Linux, Mac OS and Windows, whereas Dynamorio is additionally available for ARM and AARCH64. Dynamorio is also 10x faster than Pintool. @@ -182,7 +182,7 @@ Dynamorio has a speed decrease of 98-99% Pintool has a speed decrease of 99.5% - Hence Dynamorio is the option to go for if everything fails, and Pintool + Hence Dynamorio is the option to go for if everything else fails, and Pintool only if Dynamorio fails too. Dynamorio solutions: @@ -205,6 +205,7 @@ * QSYM: [https://github.com/sslab-gatech/qsym](https://github.com/sslab-gatech/qsym) * Manticore: [https://github.com/trailofbits/manticore](https://github.com/trailofbits/manticore) * S2E: [https://github.com/S2E](https://github.com/S2E) + * Tinyinst [https://github.com/googleprojectzero/TinyInst](https://github.com/googleprojectzero/TinyInst) (Mac/Windows only) * ... please send me any missing that are good diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index a22c809b..61d711e4 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -4,6 +4,11 @@ This file describes how you can implement custom mutations to be used in AFL. For now, we support C/C++ library and Python module, collectivelly named as the custom mutator. +There is also experimental support for Rust in `custom_mutators/rust`. +Please refer to that directory for documentation. +Run ```cargo doc -p custom_mutator --open``` in that directory to view the +documentation in your web browser. + Implemented by - C/C++ library (`*.so`): Khaled Yakdan from Code Intelligence (<yakdan@code-intelligence.de>) - Python module: Christian Holler from Mozilla (<choller@mozilla.com>) @@ -31,16 +36,19 @@ performed with the custom mutator. C/C++: ```c -void *afl_custom_init(afl_t *afl, unsigned int seed); -size_t afl_custom_fuzz(void *data, uint8_t *buf, size_t buf_size, u8 **out_buf, uint8_t *add_buf, size_t add_buf_size, size_t max_size); -size_t afl_custom_post_process(void *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf); -int32_t afl_custom_init_trim(void *data, uint8_t *buf, size_t buf_size); -size_t afl_custom_trim(void *data, uint8_t **out_buf); -int32_t afl_custom_post_trim(void *data, int success); -size_t afl_custom_havoc_mutation(void *data, u8 *buf, size_t buf_size, u8 **out_buf, size_t max_size); -uint8_t afl_custom_havoc_mutation_probability(void *data); -uint8_t afl_custom_queue_get(void *data, const uint8_t *filename); -void afl_custom_queue_new_entry(void *data, const uint8_t *filename_new_queue, const uint8_t *filename_orig_queue); +void *afl_custom_init(afl_state_t *afl, unsigned int seed); +unsigned int afl_custom_fuzz_count(void *data, const unsigned char *buf, size_t buf_size); +size_t afl_custom_fuzz(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf, unsigned char *add_buf, size_t add_buf_size, size_t max_size); +const char *afl_custom_describe(void *data, size_t max_description_len); +size_t afl_custom_post_process(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf); +int afl_custom_init_trim(void *data, unsigned char *buf, size_t buf_size); +size_t afl_custom_trim(void *data, unsigned char **out_buf); +int afl_custom_post_trim(void *data, unsigned char success); +size_t afl_custom_havoc_mutation(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf, size_t max_size); +unsigned char afl_custom_havoc_mutation_probability(void *data); +unsigned char afl_custom_queue_get(void *data, const unsigned char *filename); +void afl_custom_queue_new_entry(void *data, const unsigned char *filename_new_queue, const unsigned int *filename_orig_queue); +const char* afl_custom_introspection(my_mutator_t *data); void afl_custom_deinit(void *data); ``` @@ -49,9 +57,15 @@ Python: def init(seed): pass +def fuzz_count(buf, add_buf, max_size): + return cnt + def fuzz(buf, add_buf, max_size): return mutated_out +def describe(max_description_length): + return "description_of_current_mutation" + def post_process(buf): return out_buf @@ -77,6 +91,9 @@ def queue_new_entry(filename_new_queue, filename_orig_queue): pass ``` +def introspection(): + return string + ### Custom Mutation - `init`: @@ -88,18 +105,33 @@ def queue_new_entry(filename_new_queue, filename_orig_queue): This method determines whether the custom fuzzer should fuzz the current queue entry or not +- `fuzz_count` (optional): + + When a queue entry is selected to be fuzzed, afl-fuzz selects the number + of fuzzing attempts with this input based on a few factors. + If however the custom mutator wants to set this number instead on how often + it is called for a specific queue entry, use this function. + This function is most useful if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. + - `fuzz` (optional): This method performs custom mutations on a given input. It also accepts an additional test case. Note that this function is optional - but it makes sense to use it. You would only skip this if `post_process` is used to fix checksums etc. - so you are using it e.g. as a post processing library. + so if you are using it e.g. as a post processing library. + +- `describe` (optional): + + When this function is called, it shall describe the current testcase, + generated by the last mutation. This will be called, for example, + to name the written testcase file after a crash occurred. + Using it can help to reproduce crashing mutations. - `havoc_mutation` and `havoc_mutation_probability` (optional): `havoc_mutation` performs a single custom mutation on a given input. This - mutation is stacked with the other mutations in havoc. The other method, + mutation is stacked with other mutations in havoc. The other method, `havoc_mutation_probability`, returns the probability that `havoc_mutation` is called in havoc. By default, it is 6%. @@ -114,10 +146,19 @@ def queue_new_entry(filename_new_queue, filename_orig_queue): `post_process` function. This function is then transforming the data into the format expected by the API before executing the target. + This can return any python object that implements the buffer protocol and + supports PyBUF_SIMPLE. These include bytes, bytearray, etc. + - `queue_new_entry` (optional): This methods is called after adding a new test case to the queue. +- `introspection` (optional): + + This method is called after a new queue entry, crash or timeout is + discovered if compiled with INTROSPECTION. The custom mutator can then + return a string (const char *) that reports the exact mutations used. + - `deinit`: The last method to be called, deinitializing the state. @@ -146,7 +187,7 @@ trimmed input. Here's a quick API description: on this input (e.g. if your input has n elements and you want to remove them one by one, return n, if you do a binary search, return log(n), and so on). - If your trimming algorithm doesn't allow you to determine the amount of + If your trimming algorithm doesn't allow to determine the amount of (remaining) steps easily (esp. while running), then you can alternatively return 1 here and always return 0 in `post_trim` until you are finished and no steps remain. In that case, returning 1 in `post_trim` will end the @@ -188,19 +229,20 @@ Optionally, the following environment variables are supported: - `AFL_PYTHON_ONLY` - Deprecated and removed, use `AFL_CUSTOM_MUTATOR_ONLY` instead - trimming can cause the same test breakage like havoc and splice. + Deprecated and removed, use `AFL_CUSTOM_MUTATOR_ONLY` instead. - `AFL_DEBUG` - When combined with `AFL_NO_UI`, this causes the C trimming code to emit additional messages about the performance and actions of your custom trimmer. Use this to see if it works :) + When combined with `AFL_NO_UI`, this causes the C trimming code to emit + additional messages about the performance and actions of your custom + trimmer. Use this to see if it works :) ## 3) Usage ### Prerequisite -For Python mutator, the python 3 or 2 development package is required. On -Debian/Ubuntu/Kali this can be done: +For Python mutators, the python 3 or 2 development package is required. On +Debian/Ubuntu/Kali it can be installed like this: ```bash sudo apt install python3-dev @@ -218,13 +260,13 @@ In case your setup is different, set the necessary variables like this: ### Custom Mutator Preparation -For C/C++ mutator, the source code must be compiled as a shared object: +For C/C++ mutators, the source code must be compiled as a shared object: ```bash gcc -shared -Wall -O3 example.c -o example.so ``` Note that if you specify multiple custom mutators, the corresponding functions will be called in the order in which they are specified. e.g first `post_process` function of -`example_first.so` will be called and then that of `example_second.so` +`example_first.so` will be called and then that of `example_second.so`. ### Run @@ -243,8 +285,8 @@ afl-fuzz /path/to/program ## 4) Example -Please see [example.c](../examples/custom_mutators/example.c) and -[example.py](../examples/custom_mutators/example.py) +Please see [example.c](../utils/custom_mutators/example.c) and +[example.py](../utils/custom_mutators/example.py) ## 5) Other Resources diff --git a/docs/docs.md b/docs/docs.md new file mode 100644 index 00000000..ed6ec85e --- /dev/null +++ b/docs/docs.md @@ -0,0 +1,122 @@ +# Restructure afl++'s documentation + +## About us + +We are dedicated to everything around fuzzing, our main and most well known +contribution is the fuzzer `afl++` which is part of all major Unix +distributions (e.g. Debian, Arch, FreeBSD, etc.) and is deployed on Google's +oss-fuzz and clusterfuzz. It is rated the top fuzzer on Google's fuzzbench. + +We are four individuals from Europe supported by a large community. + +All our tools are open source. + +## About the afl++ fuzzer project + +afl++ inherited it's documentation from the original Google afl project. +Since then it has been massively improved - feature and performance wise - +and although the documenation has likewise been continued it has grown out +of proportion. +The documentation is done by non-natives to the English language, plus +none of us has a writer background. + +We see questions on afl++ usage on mailing lists (e.g. afl-users), discord +channels, web forums and as issues in our repository. + +This only increases as afl++ has been on the top of Google's fuzzbench +statistics (which measures the performance of fuzzers) and is now being +integrated in Google's oss-fuzz and clusterfuzz - and is in many Unix +packaging repositories, e.g. Debian, FreeBSD, etc. + +afl++ now has 44 (!) documentation files with 13k total lines of content. +This is way too much. + +Hence afl++ needs a complete overhaul of it's documentation, both on a +organisation/structural level as well as the content. + +Overall the following actions have to be performed: + * Create a better structure of documentation so it is easier to find the + information that is being looked for, combining and/or splitting up the + existing documents as needed. + * Rewrite some documentation to remove duplication. Several information is + present several times in the documentation. These should be removed to + where needed so that we have as little bloat as possible. + * The documents have been written and modified by a lot of different people, + most of them non-native English speaker. Hence an overall review where + parts should be rewritten has to be performed and then the rewrite done. + * Create a cheat-sheet for a very short best-setup build and run of afl++ + * Pictures explain more than 1000 words. We need at least 4 images that + explain the workflow with afl++: + - the build workflow + - the fuzzing workflow + - the fuzzing campaign management workflow + - the overall workflow that is an overview of the above + - maybe more? where the technical writes seems it necessary for + understanding. + +Requirements: + * Documentation has to be in Markdown format + * Images have to be either in SVG or PNG format. + * All documentation should be (moved) in(to) docs/ + +The project does not require writing new documentation or tutorials beside the +cheat sheet. The technical information for the cheat sheet will be provided by +us. + +## Metrics + +afl++ is a the highest performant fuzzer publicly available - but is also the +most feature rich and complex. With the publicity of afl++' success and +deployment in Google projects internally and externally and availability as +a package on most Linux distributions we see more and more issues being +created and help requests on our Discord channel that would not be +necessary if people would have read through all our documentation - which +is unrealistic. + +We expect the the new documenation after this project to be cleaner, easier +accessible and lighter to digest by our users, resulting in much less +help requests. On the other hand the amount of users using afl++ should +increase as well as it will be more accessible which would also increase +questions again - but overall resulting in a reduction of help requests. + +In numbers: we currently have per week on average 5 issues on Github, +10 questions on discord and 1 on mailing lists that would not be necessary +with perfect documentation and perfect people. + +We would consider this project a success if afterwards we only have +2 issues on Github and 3 questions on discord anymore that would be answered +by reading the documentation. The mailing list is usually used by the most +novice users and we don't expect any less questions there. + +## Project Budget + +We have zero experience with technical writers, so this is very hard for us +to calculate. We expect it to be a lot of work though because of the amount +of documentation we have that needs to be restructured and partially rewritten +(44 documents with 13k total lines of content). + +We assume the daily rate of a very good and experienced technical writer in +times of a pandemic to be ~500$ (according to web research), and calculate +the overall amout of work to be around 20 days for everything incl. the +graphics (but again - this is basically just guessing). + +Technical Writer 10000$ +Volunteer stipends 0$ (waved) +T-Shirts for the top 10 contributors and helpers to this documentation project: + 10 afl++ logo t-shirts 20$ each 200$ + 10 shipping cost of t-shirts 10$ each 100$ + +Total: 10.300$ +(in the submission form 10.280$ was entered) + +## Additional Information + +We have participated in Google Summer of Code in 2020 and hope to be selected +again in 2021. + +We have no experience with a technical writer, but we will support that person +with video calls, chats, emails and messaging, provide all necessary information +and write technical contents that is required for the success of this project. +It is clear to us that a technical writer knows how to write, but cannot know +the technical details in a complex tooling like in afl++. This guidance, input, +etc. has to come from us. diff --git a/docs/env_variables.md b/docs/env_variables.md index 87344331..c6ad0aa4 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -2,49 +2,78 @@ This document discusses the environment variables used by American Fuzzy Lop++ to expose various exotic functions that may be (rarely) useful for power - users or for some types of custom fuzzing setups. See README.md for the general + users or for some types of custom fuzzing setups. See [README.md](README.md) for the general instruction manual. -## 1) Settings for afl-gcc, afl-clang, and afl-as - and gcc_plugin afl-gcc-fast - -Because they can't directly accept command-line options, the compile-time -tools make fairly broad use of environmental variables: - - - Most afl tools do not print any ouput if stout/stderr are redirected. - If you want to have the output into a file then set the AFL_DEBUG + Note that most tools will warn on any unknown AFL environment variables. + This is for warning on typos that can happen. If you want to disable this + check then set the `AFL_IGNORE_UNKNOWN_ENVS` environment variable. + +## 1) Settings for all compilers + +Starting with afl++ 3.0 there is only one compiler: afl-cc +To select the different instrumentation modes this can be done by + 1. passing the --afl-MODE command line option to the compiler + 2. or using a symlink to afl-cc: afl-gcc, afl-g++, afl-clang, afl-clang++, + afl-clang-fast, afl-clang-fast++, afl-clang-lto, afl-clang-lto++, + afl-gcc-fast, afl-g++-fast + 3. or using the environment variable `AFL_CC_COMPILER` with `MODE` + +`MODE` can be one of `LTO` (afl-clang-lto*), `LLVM` (afl-clang-fast*), `GCC_PLUGIN` +(afl-g*-fast) or `GCC` (afl-gcc/afl-g++). + +Because (with the exception of the --afl-MODE command line option) the +compile-time tools do not accept afl specific command-line options, they +make fairly broad use of environmental variables instead: + + - Some build/configure scripts break with afl++ compilers. To be able to + pass them, do: +``` + export CC=afl-cc + export CXX=afl-c++ + export AFL_NOOPT=1 + ./configure --disable-shared --disabler-werror + unset AFL_NOOPT + make +``` + + - Most afl tools do not print any output if stdout/stderr are redirected. + If you want to get the output into a file then set the `AFL_DEBUG` environment variable. This is sadly necessary for various build processes which fail otherwise. - - Setting AFL_HARDEN automatically adds code hardening options when invoking - the downstream compiler. This currently includes -D_FORTIFY_SOURCE=2 and - -fstack-protector-all. The setting is useful for catching non-crashing + - Setting `AFL_HARDEN` automatically adds code hardening options when invoking + the downstream compiler. This currently includes `-D_FORTIFY_SOURCE=2` and + `-fstack-protector-all`. The setting is useful for catching non-crashing memory bugs at the expense of a very slight (sub-5%) performance loss. - - By default, the wrapper appends -O3 to optimize builds. Very rarely, this - will cause problems in programs built with -Werror, simply because -O3 + - By default, the wrapper appends `-O3` to optimize builds. Very rarely, this + will cause problems in programs built with -Werror, simply because `-O3` enables more thorough code analysis and can spew out additional warnings. - To disable optimizations, set AFL_DONT_OPTIMIZE. + To disable optimizations, set `AFL_DONT_OPTIMIZE`. + However if `-O...` and/or `-fno-unroll-loops` are set, these are not + overridden. - - Setting AFL_USE_ASAN automatically enables ASAN, provided that your + - Setting `AFL_USE_ASAN` automatically enables ASAN, provided that your compiler supports that. Note that fuzzing with ASAN is mildly challenging - see [notes_for_asan.md](notes_for_asan.md). - (You can also enable MSAN via AFL_USE_MSAN; ASAN and MSAN come with the + (You can also enable MSAN via `AFL_USE_MSAN`; ASAN and MSAN come with the same gotchas; the modes are mutually exclusive. UBSAN can be enabled - similarly by setting the environment variable AFL_USE_UBSAN=1. Finally + similarly by setting the environment variable `AFL_USE_UBSAN=1`. Finally there is the Control Flow Integrity sanitizer that can be activated by - AFL_USE_CFISAN=1) + `AFL_USE_CFISAN=1`) - - Setting AFL_CC, AFL_CXX, and AFL_AS lets you use alternate downstream + - Setting `AFL_CC`, `AFL_CXX`, and `AFL_AS` lets you use alternate downstream compilation tools, rather than the default 'clang', 'gcc', or 'as' binaries - in your $PATH. + in your `$PATH`. - - AFL_PATH can be used to point afl-gcc to an alternate location of afl-as. - One possible use of this is examples/clang_asm_normalize/, which lets + - `AFL_PATH` can be used to point afl-gcc to an alternate location of afl-as. + One possible use of this is utils/clang_asm_normalize/, which lets you instrument hand-written assembly when compiling clang code by plugging a normalizer into the chain. (There is no equivalent feature for GCC.) - - Setting AFL_INST_RATIO to a percentage between 0 and 100% controls the + - Setting `AFL_INST_RATIO` to a percentage between 0 and 100 controls the probability of instrumenting every branch. This is (very rarely) useful when dealing with exceptionally complex programs that saturate the output bitmap. Examples include v8, ffmpeg, and perl. @@ -52,467 +81,554 @@ tools make fairly broad use of environmental variables: (If this ever happens, afl-fuzz will warn you ahead of the time by displaying the "bitmap density" field in fiery red.) - Setting AFL_INST_RATIO to 0 is a valid choice. This will instrument only + Setting `AFL_INST_RATIO` to 0 is a valid choice. This will instrument only the transitions between function entry points, but not individual branches. - - AFL_NO_BUILTIN causes the compiler to generate code suitable for use with + Note that this is an outdated variable. A few instances (e.g. afl-gcc) + still support these, but state-of-the-art (e.g. LLVM LTO and LLVM PCGUARD) + do not need this. + + - `AFL_NO_BUILTIN` causes the compiler to generate code suitable for use with libtokencap.so (but perhaps running a bit slower than without the flag). - - TMPDIR is used by afl-as for temporary files; if this variable is not set, + - `TMPDIR` is used by afl-as for temporary files; if this variable is not set, the tool defaults to /tmp. - - Setting AFL_KEEP_ASSEMBLY prevents afl-as from deleting instrumented - assembly files. Useful for troubleshooting problems or understanding how - the tool works. To get them in a predictable place, try something like: - - mkdir assembly_here - TMPDIR=$PWD/assembly_here AFL_KEEP_ASSEMBLY=1 make clean all - - If you are a weird person that wants to compile and instrument asm - text files then use the AFL_AS_FORCE_INSTRUMENT variable: - AFL_AS_FORCE_INSTRUMENT=1 afl-gcc foo.s -o foo + text files then use the `AFL_AS_FORCE_INSTRUMENT` variable: + `AFL_AS_FORCE_INSTRUMENT=1 afl-gcc foo.s -o foo` - - Setting AFL_QUIET will prevent afl-cc and afl-as banners from being + - Setting `AFL_QUIET` will prevent afl-cc and afl-as banners from being displayed during compilation, in case you find them distracting. - - Setting AFL_CAL_FAST will speed up the initial calibration, if the - application is very slow + - Setting `AFL_CAL_FAST` will speed up the initial calibration, if the + application is very slow. -## 2) Settings for afl-clang-fast / afl-clang-fast++ / afl-gcc-fast / afl-g++-fast +## 2) Settings for LLVM and LTO: afl-clang-fast / afl-clang-fast++ / afl-clang-lto / afl-clang-lto++ -The native instrumentation helpers (llvm_mode and gcc_plugin) accept a subset -of the settings discussed in section #1, with the exception of: +The native instrumentation helpers (instrumentation and gcc_plugin) accept a subset +of the settings discussed in section 1, with the exception of: - - Setting AFL_LLVM_SKIPSINGLEBLOCK=1 will skip instrumenting - functions with a single basic block. This is useful for most C and - some C++ targets. This works for all instrumentation modes. + - LLVM modes support `AFL_LLVM_DICT2FILE=/absolute/path/file.txt` which will + write all constant string comparisons to this file to be used later with + afl-fuzz' `-x` option. - - AFL_AS, since this toolchain does not directly invoke GNU as. + - `AFL_AS`, since this toolchain does not directly invoke GNU as. - - TMPDIR and AFL_KEEP_ASSEMBLY, since no temporary assembly files are + - `TMPDIR` and `AFL_KEEP_ASSEMBLY`, since no temporary assembly files are created. - - AFL_INST_RATIO, as we switched for instrim instrumentation which - is more effective but makes not much sense together with this option. + - `AFL_INST_RATIO`, as we by default use collision free instrumentation. + Not all passes support this option though as it is an outdated feature. -Then there are a few specific features that are only available in llvm_mode: +Then there are a few specific features that are only available in instrumentation mode: ### Select the instrumentation mode - - AFL_LLVM_INSTRUMENT - this configures the instrumentation mode. + - `AFL_LLVM_INSTRUMENT` - this configures the instrumentation mode. Available options: + PCGUARD - our own pcgard based instrumentation (default) + NATIVE - clang's original pcguard based instrumentation CLASSIC - classic AFL (map[cur_loc ^ prev_loc >> 1]++) (default) CFG - InsTrim instrumentation (see below) LTO - LTO instrumentation (see below) CTX - context sensitive instrumentation (see below) NGRAM-x - deeper previous location coverage (from NGRAM-2 up to NGRAM-16) + GCC - outdated gcc instrumentation + CLANG - outdated clang instrumentation In CLASSIC (default) and CFG/INSTRIM you can also specify CTX and/or NGRAM, seperate the options with a comma "," then, e.g.: - AFL_LLVM_INSTRUMENT=CFG,CTX,NGRAM-4 + `AFL_LLVM_INSTRUMENT=CFG,CTX,NGRAM-4` Not that this is a good idea to use both CTX and NGRAM :) ### LTO - This is a different kind way of instrumentation: first it compiles all + This is a different kind way of instrumentation: first it compiles all code in LTO (link time optimization) and then performs an edge inserting instrumentation which is 100% collision free (collisions are a big issue in afl and afl-like instrumentations). This is performed by using afl-clang-lto/afl-clang-lto++ instead of afl-clang-fast, but is only built if LLVM 11 or newer is used. - - AFL_LLVM_INSTRUMENT=CFG will use Control Flow Graph instrumentation. - (recommended) - - - AFL_LLVM_LTO_AUTODICTIONARY will generate a dictionary in the target - binary based on string compare and memory compare functions. - afl-fuzz will automatically get these transmitted when starting to - fuzz. + - `AFL_LLVM_INSTRUMENT=CFG` will use Control Flow Graph instrumentation. + (not recommended for afl-clang-fast, default for afl-clang-lto as there + it is a different and better kind of instrumentation.) - None of the following options are necessary to be used and are rather for + None of the following options are necessary to be used and are rather for manual use (which only ever the author of this LTO implementation will use). - These are used if several seperated instrumentation are performed which + These are used if several separated instrumentations are performed which are then later combined. - - AFL_LLVM_MAP_ADDR sets the fixed map address to a different address than - the default 0x10000. A value of 0 or empty sets the map address to be + - `AFL_LLVM_DOCUMENT_IDS=file` will document to a file which edge ID was given + to which function. This helps to identify functions with variable bytes + or which functions were touched by an input. + - `AFL_LLVM_MAP_ADDR` sets the fixed map address to a different address than + the default `0x10000`. A value of 0 or empty sets the map address to be dynamic (the original afl way, which is slower) - - AFL_LLVM_MAP_DYNAMIC sets the shared memory address to be dynamic - - AFL_LLVM_LTO_STARTID sets the starting location ID for the instrumentation. + - `AFL_LLVM_MAP_DYNAMIC` sets the shared memory address to be dynamic + - `AFL_LLVM_LTO_STARTID` sets the starting location ID for the instrumentation. This defaults to 1 - - AFL_LLVM_LTO_DONTWRITEID prevents that the highest location ID written + - `AFL_LLVM_LTO_DONTWRITEID` prevents that the highest location ID written into the instrumentation is set in a global variable - See llvm_mode/README.LTO.md for more information. + See [instrumentation/README.lto.md](../instrumentation/README.lto.md) for more information. ### INSTRIM - This feature increases the speed by ~15% without any disadvantages to the + This feature increases the speed by ~15% without any disadvantages to the classic instrumentation. - Note that there is also an LTO version (if you have llvm 11 or higher) - + Note that there is also an LTO version (if you have llvm 11 or higher) - that is the best instrumentation we have. Use `afl-clang-lto` to activate. The InsTrim LTO version additionally has all the options and features of LTO (see above). - - Setting AFL_LLVM_INSTRIM or AFL_LLVM_INSTRUMENT=CFG to activates this mode + - Setting `AFL_LLVM_INSTRIM` or `AFL_LLVM_INSTRUMENT=CFG` activates this mode - - Setting AFL_LLVM_INSTRIM_LOOPHEAD=1 expands on INSTRIM to optimize loops. + - Setting `AFL_LLVM_INSTRIM_LOOPHEAD=1` expands on INSTRIM to optimize loops. afl-fuzz will only be able to see the path the loop took, but not how many times it was called (unless it is a complex loop). - See llvm_mode/README.instrim.md + See [instrumentation/README.instrim.md](../instrumentation/README.instrim.md) ### NGRAM - - Setting AFL_LLVM_NGRAM_SIZE or AFL_LLVM_INSTRUMENT=NGRAM-{value} + - Setting `AFL_LLVM_NGRAM_SIZE` or `AFL_LLVM_INSTRUMENT=NGRAM-{value}` activates ngram prev_loc coverage, good values are 2, 4 or 8 (any value between 2 and 16 is valid). - It is highly recommended to increase the MAP_SIZE_POW2 definition in + It is highly recommended to increase the `MAP_SIZE_POW2` definition in config.h to at least 18 and maybe up to 20 for this as otherwise too many map collisions occur. - See llvm_mode/README.ctx.md + See [instrumentation/README.ngram.md](../instrumentation/README.ngram.md) ### CTX - - Setting AFL_LLVM_CTX or AFL_LLVM_INSTRUMENT=CTX + - Setting `AFL_LLVM_CTX` or `AFL_LLVM_INSTRUMENT=CTX` activates context sensitive branch coverage - meaning that each edge is additionally combined with its caller. - It is highly recommended to increase the MAP_SIZE_POW2 definition in + It is highly recommended to increase the `MAP_SIZE_POW2` definition in config.h to at least 18 and maybe up to 20 for this as otherwise too many map collisions occur. - See llvm_mode/README.ngram.md + See [instrumentation/README.ctx.md](../instrumentation/README.ctx.md) ### LAF-INTEL - This great feature will split compares to series of single byte comparisons + This great feature will split compares into series of single byte comparisons to allow afl-fuzz to find otherwise rather impossible paths. It is not restricted to Intel CPUs ;-) - - Setting AFL_LLVM_LAF_TRANSFORM_COMPARES will split string compare functions + - Setting `AFL_LLVM_LAF_TRANSFORM_COMPARES` will split string compare functions - - Setting AFL_LLVM_LAF_SPLIT_SWITCHES will split switch()es + - Setting `AFL_LLVM_LAF_SPLIT_SWITCHES` will split all `switch` constructs - - Setting AFL_LLVM_LAF_SPLIT_COMPARES will split all floating point and + - Setting `AFL_LLVM_LAF_SPLIT_COMPARES` will split all floating point and 64, 32 and 16 bit integer CMP instructions - - Setting AFL_LLVM_LAF_SPLIT_FLOATS will split floating points, needs + - Setting `AFL_LLVM_LAF_SPLIT_FLOATS` will split floating points, needs AFL_LLVM_LAF_SPLIT_COMPARES to be set - - Setting AFL_LLVM_LAF_ALL sets all of the above + - Setting `AFL_LLVM_LAF_ALL` sets all of the above - See llvm_mode/README.laf-intel.md for more information. + See [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) for more information. -### INSTRUMENT_FILE +### INSTRUMENT LIST (selectively instrument files and functions) - This feature allows selectively instrumentation of the source + This feature allows selective instrumentation of the source - - Setting AFL_LLVM_INSTRUMENT_FILE with a filename will only instrument those - files that match the names listed in this file. + - Setting `AFL_LLVM_ALLOWLIST` or `AFL_LLVM_DENYLIST` with a filenames and/or + function will only instrument (or skip) those files that match the names + listed in the specified file. - See llvm_mode/README.instrument_file.md for more information. + See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information. ### NOT_ZERO - - Setting AFL_LLVM_NOT_ZERO=1 during compilation will use counters + - Setting `AFL_LLVM_NOT_ZERO=1` during compilation will use counters that skip zero on overflow. This is the default for llvm >= 9, however for llvm versions below that this will increase an unnecessary slowdown due a performance issue that is only fixed in llvm 9+. This feature increases path discovery by a little bit. - - Setting AFL_LLVM_SKIP_NEVERZERO=1 will not implement the skip zero + - Setting `AFL_LLVM_SKIP_NEVERZERO=1` will not implement the skip zero test. If the target performs only few loops then this will give a small performance boost. - See llvm_mode/README.neverzero.md + See [instrumentation/README.neverzero.md](../instrumentation/README.neverzero.md) ### CMPLOG - - Setting AFL_LLVM_CMPLOG=1 during compilation will tell afl-clang-fast to - produce a CmpLog binary. See llvm_mode/README.cmplog.md + - Setting `AFL_LLVM_CMPLOG=1` during compilation will tell afl-clang-fast to + produce a CmpLog binary. - See llvm_mode/README.neverzero.md + See [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md) -Then there are a few specific features that are only available in the gcc_plugin: +## 3) Settings for GCC / GCC_PLUGIN modes -### INSTRUMENT_FILE +Then there are a few specific features that are only available in GCC and +GCC_PLUGIN mode. - This feature allows selective instrumentation of the source - - - Setting AFL_GCC_INSTRUMENT_FILE with a filename will only instrument those - files that match the names listed in this file (one filename per line). - - See gcc_plugin/README.instrument_file.md for more information. + - Setting `AFL_KEEP_ASSEMBLY` prevents afl-as from deleting instrumented + assembly files. Useful for troubleshooting problems or understanding how + the tool works. (GCC mode only) + To get them in a predictable place, try something like: +``` + mkdir assembly_here + TMPDIR=$PWD/assembly_here AFL_KEEP_ASSEMBLY=1 make clean all +``` + - Setting `AFL_GCC_INSTRUMENT_FILE` with a filename will only instrument those + files that match the names listed in this file (one filename per line). + See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information. + (GCC_PLUGIN mode only) -## 3) Settings for afl-fuzz +## 4) Settings for afl-fuzz The main fuzzer binary accepts several options that disable a couple of sanity checks or alter some of the more exotic semantics of the tool: - - Setting AFL_SKIP_CPUFREQ skips the check for CPU scaling policy. This is + - Setting `AFL_SKIP_CPUFREQ` skips the check for CPU scaling policy. This is useful if you can't change the defaults (e.g., no root access to the system) and are OK with some performance loss. - - Setting AFL_NO_FORKSRV disables the forkserver optimization, reverting to - fork + execve() call for every tested input. This is useful mostly when - working with unruly libraries that create threads or do other crazy - things when initializing (before the instrumentation has a chance to run). - - Note that this setting inhibits some of the user-friendly diagnostics - normally done when starting up the forkserver and causes a pretty - significant performance drop. - - - AFL_EXIT_WHEN_DONE causes afl-fuzz to terminate when all existing paths + - `AFL_EXIT_WHEN_DONE` causes afl-fuzz to terminate when all existing paths have been fuzzed and there were no new finds for a while. This would be normally indicated by the cycle counter in the UI turning green. May be convenient for some types of automated jobs. - - AFL_MAP_SIZE sets the size of the shared map that afl-fuzz, afl-showmap, + - `AFL_MAP_SIZE` sets the size of the shared map that afl-fuzz, afl-showmap, afl-tmin and afl-analyze create to gather instrumentation data from the target. This must be equal or larger than the size the target was compiled with. - - Setting AFL_NO_AFFINITY disables attempts to bind to a specific CPU core + - `AFL_CMPLOG_ONLY_NEW` will only perform the expensive cmplog feature for + newly found testcases and not for testcases that are loaded on startup + (`-i in`). This is an important feature to set when resuming a fuzzing + session. + + - `AFL_TESTCACHE_SIZE` allows you to override the size of `#define TESTCASE_CACHE` + in config.h. Recommended values are 50-250MB - or more if your fuzzing + finds a huge amount of paths for large inputs. + + - Setting `AFL_DISABLE_TRIM` tells afl-fuzz not to trim test cases. This is + usually a bad idea! + + - Setting `AFL_NO_AFFINITY` disables attempts to bind to a specific CPU core on Linux systems. This slows things down, but lets you run more instances of afl-fuzz than would be prudent (if you really want to). - - AFL_SKIP_CRASHES causes AFL to tolerate crashing files in the input + - Setting `AFL_NO_AUTODICT` will not load an LTO generated auto dictionary + that is compiled into the target. + + - `AFL_SKIP_CRASHES` causes AFL++ to tolerate crashing files in the input queue. This can help with rare situations where a program crashes only intermittently, but it's not really recommended under normal operating conditions. - - Setting AFL_HANG_TMOUT allows you to specify a different timeout for + - Setting `AFL_HANG_TMOUT` allows you to specify a different timeout for deciding if a particular test case is a "hang". The default is 1 second - or the value of the -t parameter, whichever is larger. Dialing the value + or the value of the `-t` parameter, whichever is larger. Dialing the value down can be useful if you are very concerned about slow inputs, or if you - don't want AFL to spend too much time classifying that stuff and just + don't want AFL++ to spend too much time classifying that stuff and just rapidly put all timeouts in that bin. - - AFL_NO_ARITH causes AFL to skip most of the deterministic arithmetics. + - Setting `AFL_FORKSRV_INIT_TMOUT` allows you to specify a different timeout + to wait for the forkserver to spin up. The default is the `-t` value times + `FORK_WAIT_MULT` from `config.h` (usually 10), so for a `-t 100`, the + default would wait for `1000` milliseconds. Setting a different time here is useful + if the target has a very slow startup time, for example when doing + full-system fuzzing or emulation, but you don't want the actual runs + to wait too long for timeouts. + + - `AFL_NO_ARITH` causes AFL++ to skip most of the deterministic arithmetics. This can be useful to speed up the fuzzing of text-based file formats. - - AFL_NO_SNAPSHOT will advice afl-fuzz not to use the snapshot feature + - `AFL_NO_SNAPSHOT` will advice afl-fuzz not to use the snapshot feature if the snapshot lkm is loaded - - AFL_SHUFFLE_QUEUE randomly reorders the input queue on startup. Requested + - `AFL_SHUFFLE_QUEUE` randomly reorders the input queue on startup. Requested by some users for unorthodox parallelized fuzzing setups, but not advisable otherwise. - - AFL_TMPDIR is used to write the .cur_input file to if exists, and in + - `AFL_TMPDIR` is used to write the `.cur_input` file to if exists, and in the normal output directory otherwise. You would use this to point to a ramdisk/tmpfs. This increases the speed by a small value but also reduces the stress on SSDs. - When developing custom instrumentation on top of afl-fuzz, you can use - AFL_SKIP_BIN_CHECK to inhibit the checks for non-instrumented binaries - and shell scripts; and AFL_DUMB_FORKSRV in conjunction with the -n + `AFL_SKIP_BIN_CHECK` to inhibit the checks for non-instrumented binaries + and shell scripts; and `AFL_DUMB_FORKSRV` in conjunction with the `-n` setting to instruct afl-fuzz to still follow the fork server protocol without expecting any instrumentation data in return. - - When running in the -M or -S mode, setting AFL_IMPORT_FIRST causes the + - When running in the `-M` or `-S` mode, setting `AFL_IMPORT_FIRST` causes the fuzzer to import test cases from other instances before doing anything else. This makes the "own finds" counter in the UI more accurate. Beyond counter aesthetics, not much else should change. - - Note that AFL_POST_LIBRARY is deprecated, use AFL_CUSTOM_MUTATOR_LIBRARY + - Note that `AFL_POST_LIBRARY` is deprecated, use `AFL_CUSTOM_MUTATOR_LIBRARY` instead (see below). - - Setting AFL_CUSTOM_MUTATOR_LIBRARY to a shared library with + - `AFL_KILL_SIGNAL`: Set the signal ID to be delivered to child processes on timeout. + Unless you implement your own targets or instrumentation, you likely don't have to set it. + By default, on timeout and on exit, `SIGKILL` (`AFL_KILL_SIGNAL=9`) will be delivered to the child. + + - Setting `AFL_CUSTOM_MUTATOR_LIBRARY` to a shared library with afl_custom_fuzz() creates additional mutations through this library. If afl-fuzz is compiled with Python (which is autodetected during builing - afl-fuzz), setting AFL_PYTHON_MODULE to a Python module can also provide + afl-fuzz), setting `AFL_PYTHON_MODULE` to a Python module can also provide additional mutations. - If AFL_CUSTOM_MUTATOR_ONLY is also set, all mutations will solely be + If `AFL_CUSTOM_MUTATOR_ONLY` is also set, all mutations will solely be performed with the custom mutator. This feature allows to configure custom mutators which can be very helpful, e.g. fuzzing XML or other highly flexible structured input. Please see [custom_mutators.md](custom_mutators.md). - - AFL_FAST_CAL keeps the calibration stage about 2.5x faster (albeit less + - `AFL_FAST_CAL` keeps the calibration stage about 2.5x faster (albeit less precise), which can help when starting a session against a slow target. - The CPU widget shown at the bottom of the screen is fairly simplistic and may complain of high load prematurely, especially on systems with low core - counts. To avoid the alarming red color, you can set AFL_NO_CPU_RED. + counts. To avoid the alarming red color, you can set `AFL_NO_CPU_RED`. + + - In QEMU mode (-Q), `AFL_PATH` will be searched for afl-qemu-trace. - - In QEMU mode (-Q), AFL_PATH will be searched for afl-qemu-trace. + - In QEMU mode (-Q), setting `AFL_QEMU_CUSTOM_BIN` cause afl-fuzz to skip + prepending `afl-qemu-trace` to your command line. Use this if you wish to use a + custom afl-qemu-trace or if you need to modify the afl-qemu-trace arguments. - - Setting AFL_PRELOAD causes AFL to set LD_PRELOAD for the target binary + - Setting `AFL_CYCLE_SCHEDULES` will switch to a different schedule everytime + a cycle is finished. + + - Setting `AFL_EXPAND_HAVOC_NOW` will start in the extended havoc mode that + includes costly mutations. afl-fuzz automatically enables this mode when + deemed useful otherwise. + + - Setting `AFL_PRELOAD` causes AFL++ to set `LD_PRELOAD` for the target binary without disrupting the afl-fuzz process itself. This is useful, among other things, for bootstrapping libdislocator.so. - - Setting AFL_NO_UI inhibits the UI altogether, and just periodically prints + - Setting `AFL_NO_UI` inhibits the UI altogether, and just periodically prints some basic stats. This behavior is also automatically triggered when the output from afl-fuzz is redirected to a file or to a pipe. - - Setting AFL_FORCE_UI will force painting the UI on the screen even if + - Setting `AFL_NO_COLOR` or `AFL_NO_COLOUR` will omit control sequences for + coloring console output when configured with USE_COLOR and not ALWAYS_COLORED. + + - Setting `AFL_FORCE_UI` will force painting the UI on the screen even if no valid terminal was detected (for virtual consoles) - - If you are Jakub, you may need AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES. + - If you are Jakub, you may need `AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES`. Others need not apply. - - Benchmarking only: AFL_BENCH_JUST_ONE causes the fuzzer to exit after - processing the first queue entry; and AFL_BENCH_UNTIL_CRASH causes it to + - Benchmarking only: `AFL_BENCH_JUST_ONE` causes the fuzzer to exit after + processing the first queue entry; and `AFL_BENCH_UNTIL_CRASH` causes it to exit soon after the first crash is found. - - Setting AFL_DEBUG_CHILD_OUTPUT will not suppress the child output. + - Setting `AFL_DEBUG_CHILD` will not suppress the child output. + This lets you see all output of the child, making setup issues obvious. + For example, in an unicornafl harness, you might see python stacktraces. + You may also see other logs that way, indicating why the forkserver won't start. Not pretty but good for debugging purposes. + Note that `AFL_DEBUG_CHILD_OUTPUT` is deprecated. - - Setting AFL_NO_CPU_RED will not display very high cpu usages in red color. + - Setting `AFL_NO_CPU_RED` will not display very high cpu usages in red color. - - Setting AFL_AUTORESUME will resume a fuzz run (same as providing `-i -`) + - Setting `AFL_AUTORESUME` will resume a fuzz run (same as providing `-i -`) for an existing out folder, even if a different `-i` was provided. Without this setting, afl-fuzz will refuse execution for a long-fuzzed out dir. - - Outdated environment variables that are that not supported anymore: - AFL_DEFER_FORKSRV - AFL_PERSISTENT + - Setting `AFL_MAX_DET_EXRAS` will change the threshold at what number of elements + in the `-x` dictionary and LTO autodict (combined) the probabilistic mode will + kick off. In probabilistic mode not all dictionary entires will be used all + of the times for fuzzing mutations to not slow down fuzzing. + The default count is `200` elements. So for the 200 + 1st element, there is a + 1 in 201 chance, that one of the dictionary entries will not be used directly. -## 4) Settings for afl-qemu-trace + - Setting `AFL_NO_FORKSRV` disables the forkserver optimization, reverting to + fork + execve() call for every tested input. This is useful mostly when + working with unruly libraries that create threads or do other crazy + things when initializing (before the instrumentation has a chance to run). + + Note that this setting inhibits some of the user-friendly diagnostics + normally done when starting up the forkserver and causes a pretty + significant performance drop. + + - Setting `AFL_STATSD` enables StatsD metrics collection. + By default AFL++ will send these metrics over UDP to 127.0.0.1:8125. + The host and port are configurable with `AFL_STATSD_HOST` and `AFL_STATSD_PORT` respectively. + To enable tags (banner and afl_version) you should provide `AFL_STATSD_TAGS_FLAVOR` that matches + your StatsD server (see `AFL_STATSD_TAGS_FLAVOR`) + + - Setting `AFL_STATSD_TAGS_FLAVOR` to one of `dogstatsd`, `librato`, `signalfx` or `influxdb` + allows you to add tags to your fuzzing instances. This is especially useful when running + multiple instances (`-M/-S` for example). Applied tags are `banner` and `afl_version`. + `banner` corresponds to the name of the fuzzer provided through `-M/-S`. + `afl_version` corresponds to the currently running afl version (e.g `++3.0c`). + Default (empty/non present) will add no tags to the metrics. + See [rpc_statsd.md](rpc_statsd.md) for more information. + + - Setting `AFL_CRASH_EXITCODE` sets the exit code afl treats as crash. + For example, if `AFL_CRASH_EXITCODE='-1'` is set, each input resulting + in an `-1` return code (i.e. `exit(-1)` got called), will be treated + as if a crash had ocurred. + This may be beneficial if you look for higher-level faulty conditions in which your + target still exits gracefully. + + - Outdated environment variables that are not supported anymore: + `AFL_DEFER_FORKSRV` + `AFL_PERSISTENT` + +## 5) Settings for afl-qemu-trace The QEMU wrapper used to instrument binary-only code supports several settings: - - It is possible to set AFL_INST_RATIO to skip the instrumentation on some + - It is possible to set `AFL_INST_RATIO` to skip the instrumentation on some of the basic blocks, which can be useful when dealing with very complex binaries. - - Setting AFL_INST_LIBS causes the translator to also instrument the code + - Setting `AFL_INST_LIBS` causes the translator to also instrument the code inside any dynamically linked libraries (notably including glibc). - - Setting AFL_COMPCOV_LEVEL enables the CompareCoverage tracing of all cmp + - Setting `AFL_COMPCOV_LEVEL` enables the CompareCoverage tracing of all cmp and sub in x86 and x86_64 and memory comparions functions (e.g. strcmp, - memcmp, ...) when libcompcov is preloaded using AFL_PRELOAD. + memcmp, ...) when libcompcov is preloaded using `AFL_PRELOAD`. More info at qemu_mode/libcompcov/README.md. - There are two levels at the moment, AFL_COMPCOV_LEVEL=1 that instruments + There are two levels at the moment, `AFL_COMPCOV_LEVEL=1` that instruments only comparisons with immediate values / read-only memory and - AFL_COMPCOV_LEVEL=2 that instruments all the comparions. Level 2 is more + `AFL_COMPCOV_LEVEL=2` that instruments all the comparions. Level 2 is more accurate but may need a larger shared memory. - - Setting AFL_QEMU_COMPCOV enables the CompareCoverage tracing of all + - Setting `AFL_QEMU_COMPCOV` enables the CompareCoverage tracing of all cmp and sub in x86 and x86_64. - This is an alias of AFL_COMPCOV_LEVEL=1 when AFL_COMPCOV_LEVEL is + This is an alias of `AFL_COMPCOV_LEVEL=1` when `AFL_COMPCOV_LEVEL` is not specified. - The underlying QEMU binary will recognize any standard "user space - emulation" variables (e.g., QEMU_STACK_SIZE), but there should be no + emulation" variables (e.g., `QEMU_STACK_SIZE`), but there should be no reason to touch them. - - AFL_DEBUG will print the found entrypoint for the binary to stderr. + - `AFL_DEBUG` will print the found entrypoint for the binary to stderr. Use this if you are unsure if the entrypoint might be wrong - but - use it directly, e.g. afl-qemu-trace ./program + use it directly, e.g. `afl-qemu-trace ./program` - - AFL_ENTRYPOINT allows you to specify a specific entrypoint into the + - `AFL_ENTRYPOINT` allows you to specify a specific entrypoint into the binary (this can be very good for the performance!). - The entrypoint is specified as hex address, e.g. 0x4004110 + The entrypoint is specified as hex address, e.g. `0x4004110` Note that the address must be the address of a basic block. - When the target is i386/x86_64 you can specify the address of the function that has to be the body of the persistent loop using - AFL_QEMU_PERSISTENT_ADDR=`start addr`. + `AFL_QEMU_PERSISTENT_ADDR=start addr`. - Another modality to execute the persistent loop is to specify also the - AFL_QEMU_PERSISTENT_RET=`end addr` env variable. + `AFL_QEMU_PERSISTENT_RET=end addr` env variable. With this variable assigned, instead of patching the return address, the specified instruction is transformed to a jump towards `start addr`. - - AFL_QEMU_PERSISTENT_GPR=1 QEMU will save the original value of general + - `AFL_QEMU_PERSISTENT_GPR=1` QEMU will save the original value of general purpose registers and restore them in each persistent cycle. - - With AFL_QEMU_PERSISTENT_RETADDR_OFFSET you can specify the offset from the + - With `AFL_QEMU_PERSISTENT_RETADDR_OFFSET` you can specify the offset from the stack pointer in which QEMU can find the return address when `start addr` is - hitted. + hit. + + - With `AFL_USE_QASAN` you can enable QEMU AddressSanitizer for dynamically + linked binaries. -## 5) Settings for afl-cmin + - With `AFL_QEMU_FORCE_DFL` you force QEMU to ignore the registered signal + handlers of the target. + +## 6) Settings for afl-cmin The corpus minimization script offers very little customization: - - Setting AFL_PATH offers a way to specify the location of afl-showmap - and afl-qemu-trace (the latter only in -Q mode). + - Setting `AFL_PATH` offers a way to specify the location of afl-showmap + and afl-qemu-trace (the latter only in `-Q` mode). - - AFL_KEEP_TRACES makes the tool keep traces and other metadata used for + - `AFL_KEEP_TRACES` makes the tool keep traces and other metadata used for minimization and normally deleted at exit. The files can be found in the - <out_dir>/.traces/*. + `<out_dir>/.traces/` directory. - - AFL_ALLOW_TMP permits this and some other scripts to run in /tmp. This is + - `AFL_ALLOW_TMP` permits this and some other scripts to run in /tmp. This is a modest security risk on multi-user systems with rogue users, but should be safe on dedicated fuzzing boxes. -# #6) Settings for afl-tmin +## 7) Settings for afl-tmin -Virtually nothing to play with. Well, in QEMU mode (-Q), AFL_PATH will be -searched for afl-qemu-trace. In addition to this, TMPDIR may be used if a +Virtually nothing to play with. Well, in QEMU mode (`-Q`), `AFL_PATH` will be +searched for afl-qemu-trace. In addition to this, `TMPDIR` may be used if a temporary file can't be created in the current working directory. -You can specify AFL_TMIN_EXACT if you want afl-tmin to require execution paths +You can specify `AFL_TMIN_EXACT` if you want afl-tmin to require execution paths to match when minimizing crashes. This will make minimization less useful, but may prevent the tool from "jumping" from one crashing condition to another in -very buggy software. You probably want to combine it with the -e flag. +very buggy software. You probably want to combine it with the `-e` flag. -## 7) Settings for afl-analyze +## 8) Settings for afl-analyze -You can set AFL_ANALYZE_HEX to get file offsets printed as hexadecimal instead +You can set `AFL_ANALYZE_HEX` to get file offsets printed as hexadecimal instead of decimal. -## 8) Settings for libdislocator +## 9) Settings for libdislocator The library honors these environmental variables: - - AFL_LD_LIMIT_MB caps the size of the maximum heap usage permitted by the + - `AFL_LD_LIMIT_MB` caps the size of the maximum heap usage permitted by the library, in megabytes. The default value is 1 GB. Once this is exceeded, allocations will return NULL. - - AFL_LD_HARD_FAIL alters the behavior by calling abort() on excessive - allocations, thus causing what AFL would perceive as a crash. Useful for + - `AFL_LD_HARD_FAIL` alters the behavior by calling `abort()` on excessive + allocations, thus causing what AFL++ would perceive as a crash. Useful for programs that are supposed to maintain a specific memory footprint. - - AFL_LD_VERBOSE causes the library to output some diagnostic messages + - `AFL_LD_VERBOSE` causes the library to output some diagnostic messages that may be useful for pinpointing the cause of any observed issues. - - AFL_LD_NO_CALLOC_OVER inhibits abort() on calloc() overflows. Most + - `AFL_LD_NO_CALLOC_OVER` inhibits `abort()` on `calloc()` overflows. Most of the common allocators check for that internally and return NULL, so it's a security risk only in more exotic setups. - - AFL_ALIGNED_ALLOC=1 will force the alignment of the allocation size to - max_align_t to be compliant with the C standard. + - `AFL_ALIGNED_ALLOC=1` will force the alignment of the allocation size to + `max_align_t` to be compliant with the C standard. -## 9) Settings for libtokencap +## 10) Settings for libtokencap -This library accepts AFL_TOKEN_FILE to indicate the location to which the +This library accepts `AFL_TOKEN_FILE` to indicate the location to which the discovered tokens should be written. -## 10) Third-party variables set by afl-fuzz & other tools +## 11) Third-party variables set by afl-fuzz & other tools Several variables are not directly interpreted by afl-fuzz, but are set to optimal values if not already present in the environment: - - By default, LD_BIND_NOW is set to speed up fuzzing by forcing the + - By default, `LD_BIND_NOW` is set to speed up fuzzing by forcing the linker to do all the work before the fork server kicks in. You can - override this by setting LD_BIND_LAZY beforehand, but it is almost + override this by setting `LD_BIND_LAZY` beforehand, but it is almost certainly pointless. - - By default, ASAN_OPTIONS are set to: - + - By default, `ASAN_OPTIONS` are set to: +``` abort_on_error=1 detect_leaks=0 malloc_context_size=0 symbolize=0 allocator_may_return_null=1 - - If you want to set your own options, be sure to include abort_on_error=1 - +``` + If you want to set your own options, be sure to include `abort_on_error=1` - otherwise, the fuzzer will not be able to detect crashes in the tested - app. Similarly, include symbolize=0, since without it, AFL may have + app. Similarly, include `symbolize=0`, since without it, AFL++ may have difficulty telling crashes and hangs apart. - - In the same vein, by default, MSAN_OPTIONS are set to: - + - In the same vein, by default, `MSAN_OPTIONS` are set to: +``` exit_code=86 (required for legacy reasons) abort_on_error=1 symbolize=0 msan_track_origins=0 allocator_may_return_null=1 - - Be sure to include the first one when customizing anything, since some - MSAN versions don't call abort() on error, and we need a way to detect +``` + Be sure to include the first one when customizing anything, since some + MSAN versions don't call `abort()` on error, and we need a way to detect faults. diff --git a/docs/ideas.md b/docs/ideas.md index 65e2e8e6..0130cf61 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,48 +3,56 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. -For GSOC2020 interested students please see -[https://github.com/AFLplusplus/AFLplusplus/issues/208](https://github.com/AFLplusplus/AFLplusplus/issues/208) +# GSoC 2021 -## Flexible Grammar Mutator (currently in development) +All GSoC 2021 projects will be in the Rust development language! -Currently, AFL++'s mutation does not have deeper knowledge about the fuzzed -binary, apart from feedback, even though the developer may have insights -about the target. +## UI for libaflrs -A developer may choose to provide dictionaries and implement own mutations -in python or C, but an easy mutator that behaves according to a given grammar, -does not exist. +Write a user interface to libaflrs, the upcoming backend of afl++. +This might look like the afl-fuzz UI, but you can improve on it - and should! -State-of-the-art research on grammar fuzzing has some problems in their -implementations like code quality, scalability, or ease of use and other -common issues of the academic code. +## Schedulers for libaflrs -We aim to develop a pluggable grammar mutator for afl++ that combines -various results. +Schedulers is a mechanism that selects items from the fuzzing corpus based +on strategy and randomness. One scheduler might focus on long paths, +another on rarity of edges disocvered, still another on a combination on +things. Some of the schedulers in afl++ have to be ported, but you are free +to come up with your own if you want to - and see how it performs. -Mentor: andreafioraldi +## Forkserver support for libaflrs -## perf-fuzz Linux Kernel Module +The current libaflrs implementation fuzzes in-memory, however obviously we +want to support afl instrumented binaries as well. +Hence a forkserver support needs to be implemented - forking off the target +and talking to the target via a socketpair and the communication protocol +within. -Expand on [snapshot LKM](https://github.com/AFLplusplus/AFL-Snapshot-LKM) -To make it thread safe, can snapshot several processes at once and increase -overall performance. +## More Observers for libaflrs -Mentor: any +An observer is measuring functionality that looks at the target being fuzzed +and documents something about it. In traditional fuzzing this is the coverage +in the target, however we want to add various more observers, e.g. stack depth, +heap usage, etc. - this is a topic for an experienced Rust developer. -## QEMU 5-based Instrumentation +# Generic ideas and wishlist - NOT PART OF GSoC 2021 ! -First tests to use QEMU 4 for binary-only AFL++ showed that caching behavior -changed, which vastly decreases fuzzing speeds. +The below list is not part of GSoC 2021. -In this task test if QEMU 5 performs better and port the afl++ QEMU 3.1 -patches to QEMU 5. +## Analysis software -Understanding the current instrumentation and fixing the current caching -issues will be needed. +Currently analysis is done by using afl-plot, which is rather outdated. +A GTK or browser tool to create run-time analysis based on fuzzer_stats, +queue/id* information and plot_data that allows for zooming in and out, +changing min/max display values etc. and doing that for a single run, +different runs and campaigns vs campaigns. +Interesting values are execs, and execs/s, edges discovered (total, when +each edge was discovered and which other fuzzer share finding that edge), +test cases executed. +It should be clickable which value is X and Y axis, zoom factor, log scaling +on-off, etc. -Mentor: andreafioraldi +Mentor: vanhauser-thc ## WASM Instrumentation @@ -66,33 +74,6 @@ Either improve a single mutator thorugh learning of many different bugs Mentor: domenukk -## Reengineer `afl-fuzz` as Thread Safe, Embeddable Library (currently in development) - -Right now, afl-fuzz is single threaded, cannot safely be embedded in tools, -and not multi-threaded. It makes use of a large number of globals, must always -be the parent process and exec child processes. -Instead, afl-fuzz could be refactored to contain no global state and globals. -This allows for different use cases that could be implemented during this -project. -Note that in the mean time a lot has happened here already, but e.g. making -it all work and implement multithreading in afl-fuzz ... there is still quite -some work to do. - -Mentor: hexcoder- or vanhauser-thc - -## Collision-free Binary-Only Maps - -AFL++ supports collison-free maps using an LTO (link-time-optimization) pass. -This should be possible to implement for QEMU and Unicorn instrumentations. -As the forkserver parent caches just in time translated translation blocks, -adding a simple counter between jumps should be doable. - -Note: this is already in development for qemu by Andrea, so for people who -want to contribute it might make more sense to port his solution to unicorn. - -Mentor: andreafioraldi or domenukk -Issue/idea tracker: [https://github.com/AFLplusplus/AFLplusplus/issues/237](https://github.com/AFLplusplus/AFLplusplus/issues/237) - ## Your idea! Finally, we are open to proposals! diff --git a/docs/life_pro_tips.md b/docs/life_pro_tips.md index a5bd7286..50ad75d4 100644 --- a/docs/life_pro_tips.md +++ b/docs/life_pro_tips.md @@ -13,7 +13,7 @@ See [parallel_fuzzing.md](parallel_fuzzing.md) for step-by-step tips. ## Improve the odds of spotting memory corruption bugs with libdislocator.so! -It's easy. Consult [libdislocator/README.md](../libdislocator/README.md) for usage tips. +It's easy. Consult [utils/libdislocator/README.md](../utils/libdislocator/README.md) for usage tips. ## Want to understand how your target parses a particular input file? @@ -30,10 +30,10 @@ Check out the `fuzzer_stats` file in the AFL output dir or try `afl-whatsup`. It could be important - consult docs/status_screen.md right away! ## Know your target? Convert it to persistent mode for a huge performance gain! -Consult section #5 in llvm_mode/README.md for tips. +Consult section #5 in README.llvm.md for tips. ## Using clang? -Check out llvm_mode/ for a faster alternative to afl-gcc! +Check out instrumentation/ for a faster alternative to afl-gcc! ## Did you know that AFL can fuzz closed-source or cross-platform binaries? Check out qemu_mode/README.md and unicorn_mode/README.md for more. @@ -78,13 +78,10 @@ Be sure to check out docs/sister_projects.md before writing your own. ## Need to fuzz the command-line arguments of a particular program? -You can find a simple solution in examples/argv_fuzzing. +You can find a simple solution in utils/argv_fuzzing. ## Attacking a format that uses checksums? Remove the checksum-checking code or use a postprocessor! -See examples/custom_mutators/ for more. +See utils/custom_mutators/ for more. -## Dealing with a very slow target or hoping for instant results? - -Specify `-d` when calling afl-fuzz! diff --git a/docs/notes_for_asan.md b/docs/notes_for_asan.md index 2e18c15f..2b3bc028 100644 --- a/docs/notes_for_asan.md +++ b/docs/notes_for_asan.md @@ -20,7 +20,7 @@ Because of this, fuzzing with ASAN is recommended only in four scenarios: - Precisely gauge memory needs using http://jwilk.net/software/recidivm . - Limit the memory available to process using cgroups on Linux (see - examples/asan_cgroups). + utils/asan_cgroups). To compile with ASAN, set AFL_USE_ASAN=1 before calling 'make clean all'. The afl-gcc / afl-clang wrappers will pick that up and add the appropriate flags. @@ -35,7 +35,7 @@ no sanitizers compiled in. There is also the option of generating a corpus using a non-ASAN binary, and then feeding it to an ASAN-instrumented one to check for bugs. This is faster, and can give you somewhat comparable results. You can also try using -libdislocator (see libdislocator/README.dislocator.md in the parent directory) as a +libdislocator (see [utils/libdislocator/README.dislocator.md](../utils/libdislocator/README.dislocator.md) in the parent directory) as a lightweight and hassle-free (but less thorough) alternative. ## 2) Long version @@ -74,7 +74,7 @@ There are also cgroups, but they are Linux-specific, not universally available even on Linux systems, and they require root permissions to set up; I'm a bit hesitant to make afl-fuzz require root permissions just for that. That said, if you are on Linux and want to use cgroups, check out the contributed script -that ships in examples/asan_cgroups/. +that ships in utils/asan_cgroups/. In settings where cgroups aren't available, we have no nice, portable way to avoid counting the ASAN allocation toward the limit. On 32-bit systems, or for diff --git a/docs/parallel_fuzzing.md b/docs/parallel_fuzzing.md index 2ab1466c..8f2afe1b 100644 --- a/docs/parallel_fuzzing.md +++ b/docs/parallel_fuzzing.md @@ -10,8 +10,8 @@ n-core system, you can almost always run around n concurrent fuzzing jobs with virtually no performance hit (you can use the afl-gotcpu tool to make sure). In fact, if you rely on just a single job on a multi-core system, you will -be underutilizing the hardware. So, parallelization is usually the right -way to go. +be underutilizing the hardware. So, parallelization is always the right way to +go. When targeting multiple unrelated binaries or using the tool in "non-instrumented" (-n) mode, it is perfectly fine to just start up several @@ -65,22 +65,7 @@ still perform deterministic checks; while the secondary instances will proceed straight to random tweaks. Note that you must always have one -M main instance! - -Note that running multiple -M instances is wasteful, although there is an -experimental support for parallelizing the deterministic checks. To leverage -that, you need to create -M instances like so: - -``` -./afl-fuzz -i testcase_dir -o sync_dir -M mainA:1/3 [...] -./afl-fuzz -i testcase_dir -o sync_dir -M mainB:2/3 [...] -./afl-fuzz -i testcase_dir -o sync_dir -M mainC:3/3 [...] -``` - -...where the first value after ':' is the sequential ID of a particular main -instance (starting at 1), and the second value is the total number of fuzzers to -distribute the deterministic fuzzing across. Note that if you boot up fewer -fuzzers than indicated by the second number passed to -M, you may end up with -poor coverage. +Running multiple -M instances is wasteful! You can also monitor the progress of your jobs from the command line with the provided afl-whatsup tool. When the instances are no longer finding new paths, @@ -99,61 +84,88 @@ example may be: This is not a concern if you use @@ without -f and let afl-fuzz come up with the file name. -## 3) Syncing with non-afl fuzzers or independant instances +## 3) Multiple -M mains + + +There is support for parallelizing the deterministic checks. +This is only needed where + + 1. many new paths are found fast over a long time and it looks unlikely that + main node will ever catch up, and + 2. deterministic fuzzing is actively helping path discovery (you can see this + in the main node for the first for lines in the "fuzzing strategy yields" + section. If the ration `found/attemps` is high, then it is effective. It + most commonly isn't.) + +Only if both are true it is beneficial to have more than one main. +You can leverage this by creating -M instances like so: + +``` +./afl-fuzz -i testcase_dir -o sync_dir -M mainA:1/3 [...] +./afl-fuzz -i testcase_dir -o sync_dir -M mainB:2/3 [...] +./afl-fuzz -i testcase_dir -o sync_dir -M mainC:3/3 [...] +``` + +... where the first value after ':' is the sequential ID of a particular main +instance (starting at 1), and the second value is the total number of fuzzers to +distribute the deterministic fuzzing across. Note that if you boot up fewer +fuzzers than indicated by the second number passed to -M, you may end up with +poor coverage. + +## 4) Syncing with non-afl fuzzers or independant instances A -M main node can be told with the `-F other_fuzzer_queue_directory` option to sync results from other fuzzers, e.g. libfuzzer or honggfuzz. Only the specified directory will by synced into afl, not subdirectories. -The specified directories do not need to exist yet at the start of afl. +The specified directory does not need to exist yet at the start of afl. -## 4) Multi-system parallelization +The `-F` option can be passed to the main node several times. + +## 5) Multi-system parallelization The basic operating principle for multi-system parallelization is similar to the mechanism explained in section 2. The key difference is that you need to write a simple script that performs two actions: - Uses SSH with authorized_keys to connect to every machine and retrieve - a tar archive of the /path/to/sync_dir/<fuzzer_id>/queue/ directories for - every <fuzzer_id> local to the machine. It's best to use a naming scheme - that includes host name in the fuzzer ID, so that you can do something - like: + a tar archive of the /path/to/sync_dir/<main_node(s)> directory local to + the machine. + It is best to use a naming scheme that includes host name and it's being + a main node (e.g. main1, main2) in the fuzzer ID, so that you can do + something like: ```sh - for s in {1..10}; do - ssh user@host${s} "tar -czf - sync/host${s}_fuzzid*/[qf]*" >host${s}.tgz + for host in `cat HOSTLIST`; do + ssh user@$host "tar -czf - sync/$host_main*/" > $host.tgz done ``` - Distributes and unpacks these files on all the remaining machines, e.g.: ```sh - for s in {1..10}; do - for d in {1..10}; do - test "$s" = "$d" && continue - ssh user@host${d} 'tar -kxzf -' <host${s}.tgz + for srchost in `cat HOSTLIST`; do + for dsthost in `cat HOSTLIST`; do + test "$srchost" = "$dsthost" && continue + ssh user@$srchost 'tar -kxzf -' < $dsthost.tgz done done ``` -There is an example of such a script in examples/distributed_fuzzing/; -you can also find a more featured, experimental tool developed by -Martijn Bogaard at: - - https://github.com/MartijnB/disfuzz-afl - -Another client-server implementation from Richo Healey is: +There is an example of such a script in utils/distributed_fuzzing/. - https://github.com/richo/roving +There are other (older) more featured, experimental tools: + * https://github.com/richo/roving + * https://github.com/MartijnB/disfuzz-afl -Note that these third-party tools are unsafe to run on systems exposed to the -Internet or to untrusted users. +However these do not support syncing just main nodes (yet). When developing custom test case sync code, there are several optimizations to keep in mind: - The synchronization does not have to happen very often; running the - task every 30 minutes or so may be perfectly fine. + task every 60 minutes or even less often at later fuzzing stages is + fine - There is no need to synchronize crashes/ or hangs/; you only need to copy over queue/* (and ideally, also fuzzer_stats). @@ -179,19 +191,24 @@ to keep in mind: - You do not want a "main" instance of afl-fuzz on every system; you should run them all with -S, and just designate a single process somewhere within the fleet to run with -M. + + - Syncing is only necessary for the main nodes on a system. It is possible + to run main-less with only secondaries. However then you need to find out + which secondary took over the temporary role to be the main node. Look for + the `is_main_node` file in the fuzzer directories, eg. `sync-dir/hostname-*/is_main_node` It is *not* advisable to skip the synchronization script and run the fuzzers directly on a network filesystem; unexpected latency and unkillable processes in I/O wait state can mess things up. -## 5) Remote monitoring and data collection +## 6) Remote monitoring and data collection You can use screen, nohup, tmux, or something equivalent to run remote instances of afl-fuzz. If you redirect the program's output to a file, it will automatically switch from a fancy UI to more limited status reports. There is -also basic machine-readable information always written to the fuzzer_stats file -in the output directory. Locally, that information can be interpreted with -afl-whatsup. +also basic machine-readable information which is always written to the +fuzzer_stats file in the output directory. Locally, that information can be +interpreted with afl-whatsup. In principle, you can use the status screen of the main (-M) instance to monitor the overall fuzzing progress and decide when to stop. In this @@ -208,7 +225,7 @@ Keep in mind that crashing inputs are *not* automatically propagated to the main instance, so you may still want to monitor for crashes fleet-wide from within your synchronization or health checking scripts (see afl-whatsup). -## 6) Asymmetric setups +## 7) Asymmetric setups It is perhaps worth noting that all of the following is permitted: @@ -224,7 +241,7 @@ It is perhaps worth noting that all of the following is permitted: the discovered test cases can have synergistic effects and improve the overall coverage. - (In this case, running one -M instance per each binary is a good plan.) + (In this case, running one -M instance per target is necessary.) - Having some of the fuzzers invoke the binary in different ways. For example, 'djpeg' supports several DCT modes, configurable with diff --git a/docs/perf_tips.md b/docs/perf_tips.md index 7a690b77..fbcb4d8d 100644 --- a/docs/perf_tips.md +++ b/docs/perf_tips.md @@ -51,7 +51,7 @@ a file. ## 3. Use LLVM instrumentation When fuzzing slow targets, you can gain 20-100% performance improvement by -using the LLVM-based instrumentation mode described in [the llvm_mode README](../llvm_mode/README.md). +using the LLVM-based instrumentation mode described in [the instrumentation README](../instrumentation/README.llvm.md). Note that this mode requires the use of clang and will not work with GCC. The LLVM mode also offers a "persistent", in-process fuzzing mode that can @@ -62,12 +62,12 @@ modes require you to edit the source code of the fuzzed program, but the changes often amount to just strategically placing a single line or two. If there are important data comparisons performed (e.g. `strcmp(ptr, MAGIC_HDR)`) -then using laf-intel (see llvm_mode/README.laf-intel.md) will help `afl-fuzz` a lot +then using laf-intel (see instrumentation/README.laf-intel.md) will help `afl-fuzz` a lot to get to the important parts in the code. If you are only interested in specific parts of the code being fuzzed, you can instrument_files the files that are actually relevant. This improves the speed and -accuracy of afl. See llvm_mode/README.instrument_file.md +accuracy of afl. See instrumentation/README.instrument_list.md Also use the InsTrim mode on larger binaries, this improves performance and coverage a lot. @@ -110,7 +110,7 @@ e.g.: https://launchpad.net/libeatmydata In programs that are slow due to unavoidable initialization overhead, you may -want to try the LLVM deferred forkserver mode (see llvm_mode/README.md), +want to try the LLVM deferred forkserver mode (see README.llvm.md), which can give you speed gains up to 10x, as mentioned above. Last but not least, if you are using ASAN and the performance is unacceptable, diff --git a/docs/power_schedules.md b/docs/power_schedules.md index 06fefa12..493f9609 100644 --- a/docs/power_schedules.md +++ b/docs/power_schedules.md @@ -13,8 +13,8 @@ We find that AFL's exploitation-based constant schedule assigns **too much energ | AFL flag | Power Schedule | | ------------- | -------------------------- | -| `-p explore` (default)|  | -| `-p fast` | =\\min\\left(\\frac{\\alpha(i)}{\\beta}\\cdot\\frac{2^{s(i)}}{f(i)},M\\right)) | +| `-p explore` |  | +| `-p fast` (default)| =\\min\\left(\\frac{\\alpha(i)}{\\beta}\\cdot\\frac{2^{s(i)}}{f(i)},M\\right)) | | `-p coe` |  | | `-p quad` |  | | `-p lin` |  | diff --git a/docs/rpc_statsd.md b/docs/rpc_statsd.md new file mode 100644 index 00000000..fb97aa09 --- /dev/null +++ b/docs/rpc_statsd.md @@ -0,0 +1,143 @@ +# Remote monitoring with StatsD + +StatsD allows you to receive and aggregate metrics from a wide range of applications and retransmit them to the backend of your choice. +This enables you to create nice and readable dashboards containing all the information you need on your fuzzer instances. +No need to write your own statistics parsing system, deploy and maintain it to all your instances, sync with your graph rendering system... + +The available metrics are : +- cycle_done +- cycles_wo_finds +- execs_done +- execs_per_sec +- paths_total +- paths_favored +- paths_found +- paths_imported +- max_depth +- cur_path +- pending_favs +- pending_total +- variable_paths +- unique_crashes +- unique_hangs +- total_crashes +- slowest_exec_ms +- edges_found +- var_byte_count +- havoc_expansion + +Compared to the default integrated UI, these metrics give you the opportunity to visualize trends and fuzzing state over time. +By doing so, you might be able to see when the fuzzing process has reached a state of no progress, visualize what are the "best strategies" +(according to your own criteria) for your targets, etc. And doing so without requiring to log into each instance manually. + +An example visualisation may look like the following: + + +*Notes: The exact same dashboard can be imported with [this JSON template](statsd/grafana-afl++.json).* + +## How to use + +To enable the StatsD reporting on your fuzzer instances, you need to set the environment variable `AFL_STATSD=1`. + +Setting `AFL_STATSD_TAGS_FLAVOR` to the provider of your choice will assign tags / labels to each metric based on their format. +The possible values are `dogstatsd`, `librato`, `signalfx` or `influxdb`. +For more information on these env vars, check out `docs/env_variables.md`. + +The simplest way of using this feature is to use any metric provider and change the host/port of your StatsD daemon, +with `AFL_STATSD_HOST` and `AFL_STATSD_PORT`, if required (defaults are `localhost` and port `8125`). +To get started, here are some instructions with free and open source tools. +The following setup is based on Prometheus, statsd_exporter and Grafana. +Grafana here is not mandatory, but gives you some nice graphs and features. + +Depending on your setup and infrastructure, you may want to run these applications not on your fuzzer instances. +Only one instance of these 3 application is required for all your fuzzers. + +To simplify everything, we will use Docker and docker-compose. +Make sure you have them both installed. On most common Linux distributions, it's as simple as: + +```sh +curl -fsSL https://get.docker.com -o get-docker.sh +sh get-docker.sh +``` + +Once that's done, we can create the infrastructure. +Create and move into the directory of your choice. This will store all the configurations files required. + +First, create a `docker-compose.yml` containing the following: +```yml +version: '3' + +networks: + statsd-net: + driver: bridge + +services: + prometheus: + image: prom/prometheus + container_name: prometheus + volumes: + - ./prometheus.yml:/prometheus.yml + command: + - '--config.file=/prometheus.yml' + restart: unless-stopped + ports: + - "9090:9090" + networks: + - statsd-net + + statsd_exporter: + image: prom/statsd-exporter + container_name: statsd_exporter + volumes: + - ./statsd_mapping.yml:/statsd_mapping.yml + command: + - "--statsd.mapping-config=/statsd_mapping.yml" + ports: + - "9102:9102/tcp" + - "8125:9125/udp" + networks: + - statsd-net + + grafana: + image: grafana/grafana + container_name: grafana + restart: unless-stopped + ports: + - "3000:3000" + networks: + - statsd-net +``` + +Then `prometheus.yml` +```yml +global: + scrape_interval: 15s + evaluation_interval: 15s + +scrape_configs: + - job_name: 'fuzzing_metrics' + static_configs: + - targets: ['statsd_exporter:9102'] +``` + +And finally `statsd_mapping.yml` +```yml +mappings: +- match: "fuzzing.*" + name: "fuzzing" + labels: + type: "$1" +``` + +Run `docker-compose up -d`. + +Everything should now be setup, you are now able to run your fuzzers with + +``` +AFL_STATSD_TAGS_FLAVOR=dogstatsd AFL_STATSD=1 afl-fuzz -M test-fuzzer-1 -i i -o o ./bin/my-application @@ +AFL_STATSD_TAGS_FLAVOR=dogstatsd AFL_STATSD=1 afl-fuzz -S test-fuzzer-2 -i i -o o ./bin/my-application @@ +... +``` + +This setup may be modified before use in a production environment. Depending on your needs: adding passwords, creating volumes for storage, +tweaking the metrics gathering to get host metrics (CPU, RAM ...). diff --git a/docs/sister_projects.md b/docs/sister_projects.md index a501ecbd..5cb3a102 100644 --- a/docs/sister_projects.md +++ b/docs/sister_projects.md @@ -52,7 +52,7 @@ options. Provides an evolutionary instrumentation-guided fuzzing harness that allows some programs to be fuzzed without the fork / execve overhead. (Similar functionality is now available as the "persistent" feature described in -[the llvm_mode readme](../llvm_mode/README.md)) +[the llvm_mode readme](../instrumentation/README.llvm.md)) http://llvm.org/docs/LibFuzzer.html @@ -119,10 +119,18 @@ Simplifies the triage of discovered crashes, start parallel instances, etc. https://github.com/rc0r/afl-utils +### AFL crash analyzer (floyd) + Another crash triage tool: https://github.com/floyd-fuh/afl-crash-analyzer +### afl-extras (fekir) + +Collect data, parallel afl-tmin, startup scripts. + +https://github.com/fekir/afl-extras + ### afl-fuzzing-scripts (Tobias Ospelt) Simplifies starting up multiple parallel AFL jobs. @@ -245,7 +253,7 @@ https://code.google.com/p/address-sanitizer/wiki/AsanCoverage#Coverage_counters ### AFL JS (Han Choongwoo) One-off optimizations to speed up the fuzzing of JavaScriptCore (now likely -superseded by LLVM deferred forkserver init - see llvm_mode/README.md). +superseded by LLVM deferred forkserver init - see README.llvm.md). https://github.com/tunz/afl-fuzz-js diff --git a/docs/statsd/grafana-afl++.json b/docs/statsd/grafana-afl++.json new file mode 100644 index 00000000..96e824de --- /dev/null +++ b/docs/statsd/grafana-afl++.json @@ -0,0 +1,1816 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": "-- Grafana --", + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "gnetId": null, + "graphTooltip": 0, + "id": 1, + "links": [], + "panels": [ + { + "datasource": null, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 16, + "title": "Row title", + "type": "row" + }, + { + "alert": { + "alertRuleTags": {}, + "conditions": [ + { + "evaluator": { + "params": [ + 500 + ], + "type": "lt" + }, + "operator": { + "type": "and" + }, + "query": { + "params": [ + "A", + "5m", + "now" + ] + }, + "reducer": { + "params": [], + "type": "avg" + }, + "type": "query" + } + ], + "executionErrorState": "alerting", + "for": "5m", + "frequency": "1m", + "handler": 1, + "name": "Slow exec per sec", + "noDataState": "no_data", + "notifications": [] + }, + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 6, + "w": 10, + "x": 0, + "y": 1 + }, + "hiddenSeries": false, + "id": 12, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"execs_per_sec\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [ + { + "colorMode": "critical", + "fill": true, + "line": true, + "op": "lt", + "value": 500 + } + ], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Exec/s", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 6, + "w": 10, + "x": 10, + "y": 1 + }, + "hiddenSeries": false, + "id": 8, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"total_crashes\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Total Crashes", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 4, + "x": 20, + "y": 1 + }, + "hiddenSeries": false, + "id": 19, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"var_byte_count\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Var Byte Count", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 6, + "w": 10, + "x": 0, + "y": 7 + }, + "hiddenSeries": false, + "id": 10, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"unique_crashes\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Unique Crashes", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 6, + "w": 10, + "x": 10, + "y": 7 + }, + "hiddenSeries": false, + "id": 14, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"unique_hangs\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Unique Hangs", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 0, + "y": 13 + }, + "hiddenSeries": false, + "id": 23, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"slowest_exec_ms\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Slowest Exec Ms", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 5, + "y": 13 + }, + "hiddenSeries": false, + "id": 4, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"cycle_done\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Cycles dones", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 10, + "y": 13 + }, + "hiddenSeries": false, + "id": 13, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"execs_done\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Total Execs", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 15, + "y": 13 + }, + "hiddenSeries": false, + "id": 2, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"cur_path\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Curent path", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 0, + "y": 18 + }, + "hiddenSeries": false, + "id": 6, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"cycles_wo_finds\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [], + "timeShift": null, + "title": "Cycles done without find", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 5, + "y": 18 + }, + "hiddenSeries": false, + "id": 25, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"paths_favored\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Path Favored", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 10, + "y": 18 + }, + "hiddenSeries": false, + "id": 22, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"havoc_expansion\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Havoc Expansion", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 15, + "y": 18 + }, + "hiddenSeries": false, + "id": 17, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"edges_found\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Edges Found", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 0, + "y": 23 + }, + "hiddenSeries": false, + "id": 24, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"paths_imported\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Path Imported", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 5, + "y": 23 + }, + "hiddenSeries": false, + "id": 21, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"pending_total\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Pending Total", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 10, + "y": 23 + }, + "hiddenSeries": false, + "id": 20, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"pending_favs\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Pending favs", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + }, + { + "aliasColors": {}, + "bars": false, + "dashLength": 10, + "dashes": false, + "datasource": null, + "fieldConfig": { + "defaults": { + "custom": {} + }, + "overrides": [] + }, + "fill": 1, + "fillGradient": 0, + "gridPos": { + "h": 5, + "w": 5, + "x": 15, + "y": 23 + }, + "hiddenSeries": false, + "id": 18, + "legend": { + "avg": false, + "current": false, + "max": false, + "min": false, + "show": false, + "total": false, + "values": false + }, + "lines": true, + "linewidth": 1, + "nullPointMode": "null", + "options": { + "alertThreshold": true + }, + "percentage": false, + "pluginVersion": "7.3.7", + "pointradius": 2, + "points": false, + "renderer": "flot", + "seriesOverrides": [], + "spaceLength": 10, + "stack": false, + "steppedLine": false, + "targets": [ + { + "expr": "fuzzing{type=\"max_depth\"}", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "thresholds": [], + "timeFrom": null, + "timeRegions": [ + { + "colorMode": "background6", + "fill": true, + "fillColor": "rgba(234, 112, 112, 0.12)", + "line": false, + "lineColor": "rgba(237, 46, 24, 0.60)", + "op": "time" + } + ], + "timeShift": null, + "title": "Max Depth", + "tooltip": { + "shared": true, + "sort": 0, + "value_type": "individual" + }, + "type": "graph", + "xaxis": { + "buckets": null, + "mode": "time", + "name": null, + "show": true, + "values": [] + }, + "yaxes": [ + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": "0", + "show": true + }, + { + "format": "short", + "label": null, + "logBase": 1, + "max": null, + "min": null, + "show": true + } + ], + "yaxis": { + "align": false, + "alignLevel": null + } + } + ], + "refresh": false, + "schemaVersion": 26, + "style": "dark", + "tags": [], + "templating": { + "list": [] + }, + "time": { + "from": "now-30m", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "Fuzzing", + "uid": "sRI6PCfGz", + "version": 2 +} \ No newline at end of file diff --git a/docs/status_screen.md b/docs/status_screen.md index b89468ce..0329d960 100644 --- a/docs/status_screen.md +++ b/docs/status_screen.md @@ -29,13 +29,18 @@ With that out of the way, let's talk about what's actually on the screen... ### The status bar +``` +american fuzzy lop ++3.01a (default) [fast] {0} +``` + The top line shows you which mode afl-fuzz is running in (normal: "american fuzy lop", crash exploration mode: "peruvian rabbit mode") and the version of afl++. Next to the version is the banner, which, if not set with -T by hand, will either show the binary name being fuzzed, or the -M/-S main/secondary name for parallel fuzzing. -Finally, the last item is the power schedule mode being run (default: explore). +Second to last is the power schedule mode being run (default: fast). +Finally, the last item is the CPU id. ### Process timing @@ -86,10 +91,7 @@ Every fuzzing session should be allowed to complete at least one cycle; and ideally, should run much longer than that. As noted earlier, the first pass can take a day or longer, so sit back and -relax. If you want to get broader but more shallow coverage right away, try -the `-d` option - it gives you a more familiar experience by skipping the -deterministic fuzzing steps. It is, however, inferior to the standard mode in -a couple of subtle ways. +relax. To help make the call on when to hit `Ctrl-C`, the cycle counter is color-coded. It is shown in magenta during the first pass, progresses to yellow if new finds @@ -118,9 +120,6 @@ inputs it decided to ditch because they were persistently timing out. The "*" suffix sometimes shown in the first line means that the currently processed path is not "favored" (a property discussed later on). -If you feel that the fuzzer is progressing too slowly, see the note about the -`-d` option in this doc. - ### Map coverage ``` @@ -324,7 +323,7 @@ there are several things to look at: - Multiple threads executing at once in semi-random order. This is harmless when the 'stability' metric stays over 90% or so, but can become an issue if not. Here's what to try: - * Use afl-clang-fast from [llvm_mode](../llvm_mode/) - it uses a thread-local tracking + * Use afl-clang-fast from [instrumentation](../instrumentation/) - it uses a thread-local tracking model that is less prone to concurrency issues, * See if the target can be compiled or run without threads. Common `./configure` options include `--without-threads`, `--disable-pthreads`, or @@ -412,3 +411,27 @@ Most of these map directly to the UI elements discussed earlier on. On top of that, you can also find an entry called `plot_data`, containing a plottable history for most of these fields. If you have gnuplot installed, you can turn this into a nice progress report with the included `afl-plot` tool. + + +### Addendum: Automatically send metrics with StatsD + +In a CI environment or when running multiple fuzzers, it can be tedious to +log into each of them or deploy scripts to read the fuzzer statistics. +Using `AFL_STATSD` (and the other related environment variables `AFL_STATSD_HOST`, +`AFL_STATSD_PORT`, `AFL_STATSD_TAGS_FLAVOR`) you can automatically send metrics +to your favorite StatsD server. Depending on your StatsD server you will be able +to monitor, trigger alerts or perform actions based on these metrics (e.g: alert on +slow exec/s for a new build, threshold of crashes, time since last crash > X, etc). + +The selected metrics are a subset of all the metrics found in the status and in +the plot file. The list is the following: `cycle_done`, `cycles_wo_finds`, +`execs_done`,`execs_per_sec`, `paths_total`, `paths_favored`, `paths_found`, +`paths_imported`, `max_depth`, `cur_path`, `pending_favs`, `pending_total`, +`variable_paths`, `unique_crashes`, `unique_hangs`, `total_crashes`, +`slowest_exec_ms`, `edges_found`, `var_byte_count`, `havoc_expansion`. +Their definitions can be found in the addendum above. + +When using multiple fuzzer instances with StatsD it is *strongly* recommended to setup +the flavor (AFL_STATSD_TAGS_FLAVOR) to match your StatsD server. This will allow you +to see individual fuzzer performance, detect bad ones, see the progress of each +strategy... diff --git a/docs/visualization/statsd-grafana.png b/docs/visualization/statsd-grafana.png new file mode 100644 index 00000000..1bdc1722 --- /dev/null +++ b/docs/visualization/statsd-grafana.png Binary files differ |
