Merge pull request #594 from AFLplusplus/dev

push to stable

9 files changed, 344 insertions, 300 deletions

diff --git a/docs/Changelog.md b/docs/Changelog.md
index 6321aee4..50c1d48a 100644
--- a/docs/Changelog.md
+++ b/docs/Changelog.md
@@ -9,6 +9,53 @@ Want to stay in the loop on major new features? Join our mailing list by
 sending a mail to <afl-users+subscribe@googlegroups.com>.
 
 
+### Version ++3.00a (develop)
+  - llvm_mode/ and gcc_plugin/ moved to instrumentation/
+  - all compilers combined to afl-cc which emulates the previous ones
+  - afl-llvm/gcc-rt.o merged into afl-compiler-rt.o
+  - afl-fuzz
+    - memory limits are now disabled by default, set them with -m if required
+    - deterministic fuzzing is now disabled by default and can be enabled with
+      -D. It is still enabled by default for -M.
+    - a new seed selection was implemented that uses weighted randoms based on
+      a schedule performance score, which is much better that the previous
+      walk the whole queue approach. Select the old mode with -Z (auto enabled
+      with -M)
+    - rpc.statsd support by Edznux, thanks a lot!
+    - Marcel Boehme submitted a patch that improves all AFFast schedules :)
+    - not specifying -M or -S will now auto-set "-S default"
+    - reading testcases from -i now descends into subdirectories
+    - allow up to 4 times the -x command line option
+    - loaded extras now have a duplicate protection
+    - If test cases are too large we do a partial read on the maximum
+      supported size
+    - longer seeds with the same trace information will now be ignored
+      for fuzzing but still be used for splicing
+    - crashing seeds are now not prohibiting a run anymore but are
+      skipped. They are used for splicing though.
+    - update MOpt for expanded havoc modes
+    - added NO_SPLICING compile option and makefile define
+    - added INTROSPECTION make target that writes all mutations to
+      out/NAME/introspection.txt
+    - print special compile time options used in help output
+  - instrumentation
+    - We received an enhanced gcc_plugin module from AdaCore, thank you
+      very much!!
+    - not overriding -Ox or -fno-unroll-loops anymore
+    - we now have our own trace-pc-guard implementation. It is the same as
+      -fsanitize-coverage=trace-pc-guard from llvm 12, but: it is a) inline
+      and b) works from llvm 10+ on :)
+    - new llvm pass: dict2file via AFL_LLVM_DICT2FILE, create afl-fuzz
+      -x dictionary of string comparisons found during compilation
+    - LTO autodict now also collects interesting cmp comparisons,
+      std::string compare + find + ==, bcmp
+    - fix crash in dict2file for integers > 64 bit
+  - unicornafl synced with upstream (arm64 fix, better rust bindings)
+  - added a new custom mutator: symcc -> https://github.com/eurecom-s3/symcc/
+  - added a new custom mutator: libfuzzer that integrates libfuzzer mutations
+  - Our afl++ Grammar-Mutator is now better integrated into custom_mutators/
+
+
 ### Version ++2.68c (release)
   - added the GSoC excellent afl++ grammar mutator by Shengtuo to our
     custom_mutators/ (see custom_mutators/README.md) - or get it here:
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
index 766f24d7..8e1e266f 100644
--- a/docs/INSTALL.md
+++ b/docs/INSTALL.md
@@ -24,10 +24,12 @@ There are no special dependencies to speak of; you will need GNU make and a
 working compiler (gcc or clang). Some of the optional scripts bundled with the
 program may depend on bash, gdb, and similar basic tools.
 
-If you are using clang, please review llvm_mode/README.md; the LLVM
+If you are using clang, please review README.llvm.md; the LLVM
 integration mode can offer substantial performance gains compared to the
 traditional approach.
 
+Likewise, if you are using GCC, please review instrumentation/README.gcc_plugin.md.
+
 You may have to change several settings to get optimal results (most notably,
 disable crash reporting utilities and switch to a different CPU governor), but
 afl-fuzz will guide you through that if necessary.
@@ -52,10 +54,10 @@ sudo gmake install
 Keep in mind that if you are using csh as your shell, the syntax of some of the
 shell commands given in the README.md and other docs will be different.
 
-The `llvm_mode` requires a dynamically linked, fully-operational installation of
+The `llvm` requires a dynamically linked, fully-operational installation of
 clang. At least on FreeBSD, the clang binaries are static and do not include
 some of the essential tools, so if you want to make it work, you may need to
-follow the instructions in llvm_mode/README.md.
+follow the instructions in README.llvm.md.
 
 Beyond that, everything should work as advertised.
 
@@ -97,27 +99,24 @@ and definitely don't look POSIX-compliant. This means two things:
 User emulation mode of QEMU does not appear to be supported on MacOS X, so
 black-box instrumentation mode (`-Q`) will not work.
 
-The llvm_mode requires a fully-operational installation of clang. The one that
+The llvm instrumentation requires a fully-operational installation of clang. The one that
 comes with Xcode is missing some of the essential headers and helper tools.
-See llvm_mode/README.md for advice on how to build the compiler from scratch.
+See README.llvm.md for advice on how to build the compiler from scratch.
 
 ## 4. Linux or *BSD on non-x86 systems
 
 Standard build will fail on non-x86 systems, but you should be able to
 leverage two other options:
 
-  - The LLVM mode (see llvm_mode/README.md), which does not rely on
+  - The LLVM mode (see README.llvm.md), which does not rely on
     x86-specific assembly shims. It's fast and robust, but requires a
     complete installation of clang.
   - The QEMU mode (see qemu_mode/README.md), which can be also used for
     fuzzing cross-platform binaries. It's slower and more fragile, but
     can be used even when you don't have the source for the tested app.
 
-If you're not sure what you need, you need the LLVM mode. To get it, try:
-
-```bash
-AFL_NO_X86=1 gmake && gmake -C llvm_mode
-```
+If you're not sure what you need, you need the LLVM mode, which is built by
+default.
 
 ...and compile your target program with afl-clang-fast or afl-clang-fast++
 instead of the traditional afl-gcc or afl-clang wrappers.
@@ -160,7 +159,8 @@ instrumentation mode (`-Q`) will not work.
 ## 6. Everything else
 
 You're on your own. On POSIX-compliant systems, you may be able to compile and
-run the fuzzer; and the LLVM mode may offer a way to instrument non-x86 code.
+run the fuzzer; and the LLVM and GCC plugin modes may offer a way to instrument
+non-x86 code.
 
 The fuzzer will run on Windows in WSL only. It will not work under Cygwin on in the normal Windows world. It
 could be ported to the latter platform fairly easily, but it's a pretty bad
diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md
index a128f587..81ee9de4 100644
--- a/docs/custom_mutators.md
+++ b/docs/custom_mutators.md
@@ -31,17 +31,17 @@ performed with the custom mutator.
 
 C/C++:
 ```c
-void *afl_custom_init(afl_t *afl, unsigned int seed);
-uint32_t afl_custom_fuzz_count(void *data, const u8 *buf, size_t buf_size);
-size_t afl_custom_fuzz(void *data, uint8_t *buf, size_t buf_size, u8 **out_buf, uint8_t *add_buf, size_t add_buf_size, size_t max_size);
-size_t afl_custom_post_process(void *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf);
-int32_t afl_custom_init_trim(void *data, uint8_t *buf, size_t buf_size);
-size_t afl_custom_trim(void *data, uint8_t **out_buf);
-int32_t afl_custom_post_trim(void *data, int success);
-size_t afl_custom_havoc_mutation(void *data, u8 *buf, size_t buf_size, u8 **out_buf, size_t max_size);
-uint8_t afl_custom_havoc_mutation_probability(void *data);
-uint8_t afl_custom_queue_get(void *data, const uint8_t *filename);
-void afl_custom_queue_new_entry(void *data, const uint8_t *filename_new_queue, const uint8_t *filename_orig_queue);
+void *afl_custom_init(afl_state_t *afl, unsigned int seed);
+unsigned int afl_custom_fuzz_count(void *data, const unsigned char *buf, size_t buf_size);
+size_t afl_custom_fuzz(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf, unsigned char *add_buf, size_t add_buf_size, size_t max_size);
+size_t afl_custom_post_process(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf);
+int afl_custom_init_trim(void *data, unsigned char *buf, size_t buf_size);
+size_t afl_custom_trim(void *data, unsigned char **out_buf);
+int afl_custom_post_trim(void *data, unsigned char success);
+size_t afl_custom_havoc_mutation(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf, size_t max_size);
+unsigned char afl_custom_havoc_mutation_probability(void *data);
+unsigned char afl_custom_queue_get(void *data, const unsigned char *filename);
+void afl_custom_queue_new_entry(void *data, const unsigned char *filename_new_queue, const unsigned int *filename_orig_queue);
 void afl_custom_deinit(void *data);
 ```
 
diff --git a/docs/env_variables.md b/docs/env_variables.md
index c47d10e8..a36f2b4e 100644
--- a/docs/env_variables.md
+++ b/docs/env_variables.md
@@ -2,49 +2,64 @@
 
   This document discusses the environment variables used by American Fuzzy Lop++
   to expose various exotic functions that may be (rarely) useful for power
-  users or for some types of custom fuzzing setups. See README.md for the general
+  users or for some types of custom fuzzing setups. See [README.md](README.md) for the general
   instruction manual.
 
-## 1) Settings for afl-gcc, afl-clang, and afl-as - and gcc_plugin afl-gcc-fast
+## 1) Settings for all compilers
 
-Because they can't directly accept command-line options, the compile-time
-tools make fairly broad use of environmental variables:
+Starting with afl++ 3.0 there is only one compiler: afl-cc
+To select the different instrumentation modes this can be done by
+  1. passing the --afl-MODE command line option to the compiler
+  2. or using a symlink to afl-cc: afl-gcc, afl-g++, afl-clang, afl-clang++,
+     afl-clang-fast, afl-clang-fast++, afl-clang-lto, afl-clang-lto++,
+     afl-gcc-fast, afl-g++-fast
+  3. or using the environment variable `AFL_CC_COMPILER` with `MODE`
+
+`MODE` can be one of `LTO` (afl-clang-lto*), `LLVM` (afl-clang-fast*), `GCC_PLUGIN`
+(afl-g*-fast) or `GCC` (afl-gcc/afl-g++).
+
+
+Because (with the exception of the --afl-MODE command line option) the
+compile-time tools do not accept afl specific command-line options, they
+make fairly broad use of environmental variables instead:
 
   - Most afl tools do not print any output if stdout/stderr are redirected.
-    If you want to save the output in a file then set the AFL_DEBUG
+    If you want to get the output into a file then set the `AFL_DEBUG`
     environment variable.
     This is sadly necessary for various build processes which fail otherwise.
 
-  - Setting AFL_HARDEN automatically adds code hardening options when invoking
-    the downstream compiler. This currently includes -D_FORTIFY_SOURCE=2 and
-    -fstack-protector-all. The setting is useful for catching non-crashing
+  - Setting `AFL_HARDEN` automatically adds code hardening options when invoking
+    the downstream compiler. This currently includes `-D_FORTIFY_SOURCE=2` and
+    `-fstack-protector-all`. The setting is useful for catching non-crashing
     memory bugs at the expense of a very slight (sub-5%) performance loss.
 
-  - By default, the wrapper appends -O3 to optimize builds. Very rarely, this
-    will cause problems in programs built with -Werror, simply because -O3
+  - By default, the wrapper appends `-O3` to optimize builds. Very rarely, this
+    will cause problems in programs built with -Werror, simply because `-O3`
     enables more thorough code analysis and can spew out additional warnings.
-    To disable optimizations, set AFL_DONT_OPTIMIZE.
+    To disable optimizations, set `AFL_DONT_OPTIMIZE`.
+    However if `-O...` and/or `-fno-unroll-loops` are set, these are not
+    overridden.
 
-  - Setting AFL_USE_ASAN automatically enables ASAN, provided that your
+  - Setting `AFL_USE_ASAN` automatically enables ASAN, provided that your
     compiler supports that. Note that fuzzing with ASAN is mildly challenging
     - see [notes_for_asan.md](notes_for_asan.md).
 
-    (You can also enable MSAN via AFL_USE_MSAN; ASAN and MSAN come with the
+    (You can also enable MSAN via `AFL_USE_MSAN`; ASAN and MSAN come with the
     same gotchas; the modes are mutually exclusive. UBSAN can be enabled
-    similarly by setting the environment variable AFL_USE_UBSAN=1. Finally
+    similarly by setting the environment variable `AFL_USE_UBSAN=1`. Finally
     there is the Control Flow Integrity sanitizer that can be activated by
-    AFL_USE_CFISAN=1)
+    `AFL_USE_CFISAN=1`)
 
-  - Setting AFL_CC, AFL_CXX, and AFL_AS lets you use alternate downstream
+  - Setting `AFL_CC`, `AFL_CXX`, and `AFL_AS` lets you use alternate downstream
     compilation tools, rather than the default 'clang', 'gcc', or 'as' binaries
-    in your $PATH.
+    in your `$PATH`.
 
-  - AFL_PATH can be used to point afl-gcc to an alternate location of afl-as.
+  - `AFL_PATH` can be used to point afl-gcc to an alternate location of afl-as.
     One possible use of this is examples/clang_asm_normalize/, which lets
     you instrument hand-written assembly when compiling clang code by plugging
     a normalizer into the chain. (There is no equivalent feature for GCC.)
 
-  - Setting AFL_INST_RATIO to a percentage between 0% and 100% controls the
+  - Setting `AFL_INST_RATIO` to a percentage between 0 and 100 controls the
     probability of instrumenting every branch. This is (very rarely) useful
     when dealing with exceptionally complex programs that saturate the output
     bitmap. Examples include v8, ffmpeg, and perl.
@@ -52,49 +67,51 @@ tools make fairly broad use of environmental variables:
     (If this ever happens, afl-fuzz will warn you ahead of the time by
     displaying the "bitmap density" field in fiery red.)
 
-    Setting AFL_INST_RATIO to 0 is a valid choice. This will instrument only
+    Setting `AFL_INST_RATIO` to 0 is a valid choice. This will instrument only
     the transitions between function entry points, but not individual branches.
 
-  - AFL_NO_BUILTIN causes the compiler to generate code suitable for use with
+    Note that this is an outdated variable. A few instances (e.g. afl-gcc)
+    still support these, but state-of-the-art (e.g. LLVM LTO and LLVM PCGUARD)
+    do not need this.
+
+  - `AFL_NO_BUILTIN` causes the compiler to generate code suitable for use with
     libtokencap.so (but perhaps running a bit slower than without the flag).
 
-  - TMPDIR is used by afl-as for temporary files; if this variable is not set,
+  - `TMPDIR` is used by afl-as for temporary files; if this variable is not set,
     the tool defaults to /tmp.
 
-  - Setting AFL_KEEP_ASSEMBLY prevents afl-as from deleting instrumented
-    assembly files. Useful for troubleshooting problems or understanding how
-    the tool works. To get them in a predictable place, try something like:
-
-    mkdir assembly_here
-    TMPDIR=$PWD/assembly_here AFL_KEEP_ASSEMBLY=1 make clean all
-
   - If you are a weird person that wants to compile and instrument asm
-    text files then use the AFL_AS_FORCE_INSTRUMENT variable:
-      AFL_AS_FORCE_INSTRUMENT=1 afl-gcc foo.s -o foo
+    text files then use the `AFL_AS_FORCE_INSTRUMENT` variable:
+      `AFL_AS_FORCE_INSTRUMENT=1 afl-gcc foo.s -o foo`
 
-  - Setting AFL_QUIET will prevent afl-cc and afl-as banners from being
+  - Setting `AFL_QUIET` will prevent afl-cc and afl-as banners from being
     displayed during compilation, in case you find them distracting.
 
-  - Setting AFL_CAL_FAST will speed up the initial calibration, if the
-    application is very slow
+  - Setting `AFL_CAL_FAST` will speed up the initial calibration, if the
+    application is very slow.
 
-## 2) Settings for afl-clang-fast / afl-clang-fast++ / afl-gcc-fast / afl-g++-fast
+## 2) Settings for LLVM and LTO: afl-clang-fast / afl-clang-fast++ / afl-clang-lto / afl-clang-lto++
 
-The native instrumentation helpers (llvm_mode and gcc_plugin) accept a subset
-of the settings discussed in section #1, with the exception of:
+The native instrumentation helpers (instrumentation and gcc_plugin) accept a subset
+of the settings discussed in section 1, with the exception of:
 
-  - AFL_AS, since this toolchain does not directly invoke GNU as.
+  - LLVM modes support `AFL_LLVM_DICT2FILE=/absolute/path/file.txt` which will
+    write all constant string comparisons  to this file to be used later with
+    afl-fuzz' `-x` option.
 
-  - TMPDIR and AFL_KEEP_ASSEMBLY, since no temporary assembly files are
+  - `AFL_AS`, since this toolchain does not directly invoke GNU as.
+
+  - `TMPDIR` and `AFL_KEEP_ASSEMBLY`, since no temporary assembly files are
     created.
 
-  - AFL_INST_RATIO, as we by default use collision free instrumentation.
+  - `AFL_INST_RATIO`, as we by default use collision free instrumentation.
+    Not all passes support this option though as it is an outdated feature.
 
-Then there are a few specific features that are only available in llvm_mode:
+Then there are a few specific features that are only available in instrumentation mode:
 
 ### Select the instrumentation mode
 
-    - AFL_LLVM_INSTRUMENT - this configures the instrumentation mode. 
+    - `AFL_LLVM_INSTRUMENT` - this configures the instrumentation mode. 
       Available options:
         CLASSIC - classic AFL (map[cur_loc ^ prev_loc >> 1]++) (default)
         CFG - InsTrim instrumentation (see below)
@@ -103,273 +120,276 @@ Then there are a few specific features that are only available in llvm_mode:
         NGRAM-x - deeper previous location coverage (from NGRAM-2 up to NGRAM-16)
       In CLASSIC (default) and CFG/INSTRIM you can also specify CTX and/or
       NGRAM, seperate the options with a comma "," then, e.g.:
-        AFL_LLVM_INSTRUMENT=CFG,CTX,NGRAM-4
+        `AFL_LLVM_INSTRUMENT=CFG,CTX,NGRAM-4`
       Not that this is a good idea to use both CTX and NGRAM :)
 
 ### LTO
 
-    This is a different kind way of instrumentation: first it compiles all
+  This is a different kind way of instrumentation: first it compiles all
     code in LTO (link time optimization) and then performs an edge inserting
     instrumentation which is 100% collision free (collisions are a big issue
     in afl and afl-like instrumentations). This is performed by using
     afl-clang-lto/afl-clang-lto++ instead of afl-clang-fast, but is only
     built if LLVM 11 or newer is used.
 
-   - AFL_LLVM_INSTRUMENT=CFG will use Control Flow Graph instrumentation.
+   - `AFL_LLVM_INSTRUMENT=CFG` will use Control Flow Graph instrumentation.
      (not recommended for afl-clang-fast, default for afl-clang-lto as there
       it is a different and better kind of instrumentation.)
 
-    None of the following options are necessary to be used and are rather for
+  None of the following options are necessary to be used and are rather for
     manual use (which only ever the author of this LTO implementation will use).
-    These are used if several seperated instrumentations are performed which
+    These are used if several separated instrumentations are performed which
     are then later combined.
 
-   - AFL_LLVM_DOCUMENT_IDS=file will document to a file which edge ID was given
+   - `AFL_LLVM_DOCUMENT_IDS=file` will document to a file which edge ID was given
      to which function. This helps to identify functions with variable bytes
      or which functions were touched by an input.
-   - AFL_LLVM_MAP_ADDR sets the fixed map address to a different address than
-     the default 0x10000. A value of 0 or empty sets the map address to be
+   - `AFL_LLVM_MAP_ADDR` sets the fixed map address to a different address than
+     the default `0x10000`. A value of 0 or empty sets the map address to be
      dynamic (the original afl way, which is slower)
-   - AFL_LLVM_MAP_DYNAMIC sets the shared memory address to be dynamic
-   - AFL_LLVM_LTO_STARTID sets the starting location ID for the instrumentation.
+   - `AFL_LLVM_MAP_DYNAMIC` sets the shared memory address to be dynamic
+   - `AFL_LLVM_LTO_STARTID` sets the starting location ID for the instrumentation.
      This defaults to 1
-   - AFL_LLVM_LTO_DONTWRITEID prevents that the highest location ID written
+   - `AFL_LLVM_LTO_DONTWRITEID` prevents that the highest location ID written
      into the instrumentation is set in a global variable
 
-    See llvm_mode/README.LTO.md for more information.
+  See [instrumentation/README.lto.md](../instrumentation/README.lto.md) for more information.
 
 ### INSTRIM
 
-    This feature increases the speed by ~15% without any disadvantages to the
+  This feature increases the speed by ~15% without any disadvantages to the
     classic instrumentation.
 
-    Note that there is also an LTO version (if you have llvm 11 or higher) -
+  Note that there is also an LTO version (if you have llvm 11 or higher) -
     that is the best instrumentation we have. Use `afl-clang-lto` to activate.
     The InsTrim LTO version additionally has all the options and features of
     LTO (see above).
 
-    - Setting AFL_LLVM_INSTRIM or AFL_LLVM_INSTRUMENT=CFG to activates this mode
+   - Setting `AFL_LLVM_INSTRIM` or `AFL_LLVM_INSTRUMENT=CFG` activates this mode
 
-    - Setting AFL_LLVM_INSTRIM_LOOPHEAD=1 expands on INSTRIM to optimize loops.
+   - Setting `AFL_LLVM_INSTRIM_LOOPHEAD=1` expands on INSTRIM to optimize loops.
       afl-fuzz will only be able to see the path the loop took, but not how
       many times it was called (unless it is a complex loop).
 
-    See llvm_mode/README.instrim.md
+  See [instrumentation/README.instrim.md](../instrumentation/README.instrim.md)
 
 ### NGRAM
 
-    - Setting AFL_LLVM_NGRAM_SIZE or AFL_LLVM_INSTRUMENT=NGRAM-{value}
+   - Setting `AFL_LLVM_NGRAM_SIZE` or `AFL_LLVM_INSTRUMENT=NGRAM-{value}`
       activates ngram prev_loc coverage, good values are 2, 4 or 8
       (any value between 2 and 16 is valid).
-      It is highly recommended to increase the MAP_SIZE_POW2 definition in
+      It is highly recommended to increase the `MAP_SIZE_POW2` definition in
       config.h to at least 18 and maybe up to 20 for this as otherwise too
       many map collisions occur.
 
-    See llvm_mode/README.ctx.md
+  See [instrumentation/README.ngram.md](../instrumentation/README.ngram.md)
 
 ### CTX
 
-    - Setting AFL_LLVM_CTX or AFL_LLVM_INSTRUMENT=CTX
+   - Setting `AFL_LLVM_CTX` or `AFL_LLVM_INSTRUMENT=CTX`
       activates context sensitive branch coverage - meaning that each edge
       is additionally combined with its caller.
-      It is highly recommended to increase the MAP_SIZE_POW2 definition in
+      It is highly recommended to increase the `MAP_SIZE_POW2` definition in
       config.h to at least 18 and maybe up to 20 for this as otherwise too
       many map collisions occur.
 
-    See llvm_mode/README.ngram.md
+  See [instrumentation/README.ctx.md](../instrumentation/README.ctx.md)
 
 ### LAF-INTEL
 
-    This great feature will split compares to series of single byte comparisons
+  This great feature will split compares into series of single byte comparisons
     to allow afl-fuzz to find otherwise rather impossible paths. It is not
     restricted to Intel CPUs ;-)
 
-    - Setting AFL_LLVM_LAF_TRANSFORM_COMPARES will split string compare functions
+   - Setting `AFL_LLVM_LAF_TRANSFORM_COMPARES` will split string compare functions
 
-    - Setting AFL_LLVM_LAF_SPLIT_SWITCHES will split switch()es
+   - Setting `AFL_LLVM_LAF_SPLIT_SWITCHES` will split all `switch` constructs
 
-    - Setting AFL_LLVM_LAF_SPLIT_COMPARES will split all floating point and
+   - Setting `AFL_LLVM_LAF_SPLIT_COMPARES` will split all floating point and
       64, 32 and 16 bit integer CMP instructions
 
-    - Setting AFL_LLVM_LAF_SPLIT_FLOATS will split floating points, needs
+   - Setting `AFL_LLVM_LAF_SPLIT_FLOATS` will split floating points, needs
       AFL_LLVM_LAF_SPLIT_COMPARES to be set
 
-    - Setting AFL_LLVM_LAF_ALL sets all of the above
+   - Setting `AFL_LLVM_LAF_ALL` sets all of the above
 
-    See llvm_mode/README.laf-intel.md for more information.
+  See [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) for more information.
 
 ### INSTRUMENT LIST (selectively instrument files and functions)
 
-    This feature allows selective instrumentation of the source
+  This feature allows selective instrumentation of the source
 
-    - Setting AFL_LLVM_ALLOWLIST or AFL_LLVM_DENYLIST with a filenames and/or
+   - Setting `AFL_LLVM_ALLOWLIST` or `AFL_LLVM_DENYLIST` with a filenames and/or
       function will only instrument (or skip) those files that match the names
       listed in the specified file.
 
-    See llvm_mode/README.instrument_list.md for more information.
+  See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information.
 
 ### NOT_ZERO
 
-    - Setting AFL_LLVM_NOT_ZERO=1 during compilation will use counters
+   - Setting `AFL_LLVM_NOT_ZERO=1` during compilation will use counters
       that skip zero on overflow. This is the default for llvm >= 9,
       however for llvm versions below that this will increase an unnecessary
       slowdown due a performance issue that is only fixed in llvm 9+.
       This feature increases path discovery by a little bit.
 
-    - Setting AFL_LLVM_SKIP_NEVERZERO=1 will not implement the skip zero
+   - Setting `AFL_LLVM_SKIP_NEVERZERO=1` will not implement the skip zero
       test. If the target performs only few loops then this will give a
       small performance boost.
 
-    See llvm_mode/README.neverzero.md
+  See [instrumentation/README.neverzero.md](../instrumentation/README.neverzero.md)
 
 ### CMPLOG
 
-    - Setting AFL_LLVM_CMPLOG=1 during compilation will tell afl-clang-fast to
-      produce a CmpLog binary. See llvm_mode/README.cmplog.md
-
-    See llvm_mode/README.neverzero.md
-
-Then there are a few specific features that are only available in the gcc_plugin:
+   - Setting `AFL_LLVM_CMPLOG=1` during compilation will tell afl-clang-fast to
+      produce a CmpLog binary.
 
-### INSTRUMENT_FILE
+  See [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md)
 
-    This feature allows selective instrumentation of the source
+## 3) Settings for GCC / GCC_PLUGIN modes
 
-    - Setting AFL_GCC_INSTRUMENT_FILE with a filename will only instrument those
-      files that match the names listed in this file (one filename per line).
+Then there are a few specific features that are only available in GCC and
+GCC_PLUGIN mode.
 
-    See gcc_plugin/README.instrument_list.md for more information.
+  - Setting `AFL_KEEP_ASSEMBLY` prevents afl-as from deleting instrumented
+    assembly files. Useful for troubleshooting problems or understanding how
+    the tool works. (GCC mode only)
+    To get them in a predictable place, try something like:
+```
+    mkdir assembly_here
+    TMPDIR=$PWD/assembly_here AFL_KEEP_ASSEMBLY=1 make clean all
+```
+  - Setting `AFL_GCC_INSTRUMENT_FILE` with a filename will only instrument those
+    files that match the names listed in this file (one filename per line).
+    See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information.
+    (GCC_PLUGIN mode only)
 
-## 3) Settings for afl-fuzz
+## 4) Settings for afl-fuzz
 
 The main fuzzer binary accepts several options that disable a couple of sanity
 checks or alter some of the more exotic semantics of the tool:
 
-  - Setting AFL_SKIP_CPUFREQ skips the check for CPU scaling policy. This is
+  - Setting `AFL_SKIP_CPUFREQ` skips the check for CPU scaling policy. This is
     useful if you can't change the defaults (e.g., no root access to the
     system) and are OK with some performance loss.
 
-  - AFL_EXIT_WHEN_DONE causes afl-fuzz to terminate when all existing paths
+  - `AFL_EXIT_WHEN_DONE` causes afl-fuzz to terminate when all existing paths
     have been fuzzed and there were no new finds for a while. This would be
     normally indicated by the cycle counter in the UI turning green. May be
     convenient for some types of automated jobs.
 
-  - AFL_MAP_SIZE sets the size of the shared map that afl-fuzz, afl-showmap,
+  - `AFL_MAP_SIZE` sets the size of the shared map that afl-fuzz, afl-showmap,
     afl-tmin and afl-analyze create to gather instrumentation data from
     the target. This must be equal or larger than the size the target was
     compiled with.
 
-  - Setting AFL_DISABLE_TRIM tells afl-fuzz to no trim test cases. This is
+  - `AFL_TESTCACHE_SIZE` allows you to override the size of `#define TESTCASE_CACHE`
+    in config.h. Recommended values are 50-250MB - or more if your fuzzing
+    finds a huge amount of paths for large inputs.
+
+  - Setting `AFL_DISABLE_TRIM` tells afl-fuzz not to trim test cases. This is
     usually a bad idea!
 
-  - Setting AFL_NO_AFFINITY disables attempts to bind to a specific CPU core
+  - Setting `AFL_NO_AFFINITY` disables attempts to bind to a specific CPU core
     on Linux systems. This slows things down, but lets you run more instances
     of afl-fuzz than would be prudent (if you really want to).
 
-  - AFL_SKIP_CRASHES causes AFL to tolerate crashing files in the input
+  - `AFL_SKIP_CRASHES` causes AFL++ to tolerate crashing files in the input
     queue. This can help with rare situations where a program crashes only
     intermittently, but it's not really recommended under normal operating
     conditions.
 
-  - Setting AFL_HANG_TMOUT allows you to specify a different timeout for
+  - Setting `AFL_HANG_TMOUT` allows you to specify a different timeout for
     deciding if a particular test case is a "hang". The default is 1 second
-    or the value of the -t parameter, whichever is larger. Dialing the value
+    or the value of the `-t` parameter, whichever is larger. Dialing the value
     down can be useful if you are very concerned about slow inputs, or if you
-    don't want AFL to spend too much time classifying that stuff and just
+    don't want AFL++ to spend too much time classifying that stuff and just
     rapidly put all timeouts in that bin.
 
-  - Setting AFL_FORKSRV_INIT_TMOUT allows yout to specify a different timeout
-    to wait for the forkserver to spin up. The default is the `-t` value times
-    `FORK_WAIT_MULT` from `config.h` (usually 10), so for a `-t 100`, the
-    default would wait `1000` milis. Setting a different time here is useful
-    if the target has a very slow startup time, for example when doing
-    full-system fuzzing or emulation, but you don't want the actual runs
-    to wait too long for timeouts.
-
-  - AFL_NO_ARITH causes AFL to skip most of the deterministic arithmetics.
+  - `AFL_NO_ARITH` causes AFL++ to skip most of the deterministic arithmetics.
     This can be useful to speed up the fuzzing of text-based file formats.
 
-  - AFL_NO_SNAPSHOT will advice afl-fuzz not to use the snapshot feature
+  - `AFL_NO_SNAPSHOT` will advice afl-fuzz not to use the snapshot feature
     if the snapshot lkm is loaded
 
-  - AFL_SHUFFLE_QUEUE randomly reorders the input queue on startup. Requested
+  - `AFL_SHUFFLE_QUEUE` randomly reorders the input queue on startup. Requested
     by some users for unorthodox parallelized fuzzing setups, but not
     advisable otherwise.
 
-  - AFL_TMPDIR is used to write the .cur_input file to if exists, and in
+  - `AFL_TMPDIR` is used to write the `.cur_input` file to if exists, and in
     the normal output directory otherwise. You would use this to point to
     a ramdisk/tmpfs. This increases the speed by a small value but also
     reduces the stress on SSDs.
 
   - When developing custom instrumentation on top of afl-fuzz, you can use
-    AFL_SKIP_BIN_CHECK to inhibit the checks for non-instrumented binaries
-    and shell scripts; and AFL_DUMB_FORKSRV in conjunction with the -n
+    `AFL_SKIP_BIN_CHECK` to inhibit the checks for non-instrumented binaries
+    and shell scripts; and `AFL_DUMB_FORKSRV` in conjunction with the `-n`
     setting to instruct afl-fuzz to still follow the fork server protocol
     without expecting any instrumentation data in return.
 
-  - When running in the -M or -S mode, setting AFL_IMPORT_FIRST causes the
+  - When running in the `-M` or `-S` mode, setting `AFL_IMPORT_FIRST` causes the
     fuzzer to import test cases from other instances before doing anything
     else. This makes the "own finds" counter in the UI more accurate.
     Beyond counter aesthetics, not much else should change.
 
-  - Note that AFL_POST_LIBRARY is deprecated, use AFL_CUSTOM_MUTATOR_LIBRARY
+  - Note that `AFL_POST_LIBRARY` is deprecated, use `AFL_CUSTOM_MUTATOR_LIBRARY`
     instead (see below).
 
-  - Setting AFL_CUSTOM_MUTATOR_LIBRARY to a shared library with
+  - Setting `AFL_CUSTOM_MUTATOR_LIBRARY` to a shared library with
     afl_custom_fuzz() creates additional mutations through this library.
     If afl-fuzz is compiled with Python (which is autodetected during builing
-    afl-fuzz), setting AFL_PYTHON_MODULE to a Python module can also provide
+    afl-fuzz), setting `AFL_PYTHON_MODULE` to a Python module can also provide
     additional mutations.
-    If AFL_CUSTOM_MUTATOR_ONLY is also set, all mutations will solely be
+    If `AFL_CUSTOM_MUTATOR_ONLY` is also set, all mutations will solely be
     performed with the custom mutator.
     This feature allows to configure custom mutators which can be very helpful,
     e.g. fuzzing XML or other highly flexible structured input.
     Please see [custom_mutators.md](custom_mutators.md).
 
-  - AFL_FAST_CAL keeps the calibration stage about 2.5x faster (albeit less
+  - `AFL_FAST_CAL` keeps the calibration stage about 2.5x faster (albeit less
     precise), which can help when starting a session against a slow target.
 
   - The CPU widget shown at the bottom of the screen is fairly simplistic and
     may complain of high load prematurely, especially on systems with low core
-    counts. To avoid the alarming red color, you can set AFL_NO_CPU_RED.
+    counts. To avoid the alarming red color, you can set `AFL_NO_CPU_RED`.
 
-  - In QEMU mode (-Q), AFL_PATH will be searched for afl-qemu-trace.
+  - In QEMU mode (-Q), `AFL_PATH` will be searched for afl-qemu-trace.
 
-  - Setting AFL_CYCLE_SCHEDULES will switch to a different schedule everytime
+  - Setting `AFL_CYCLE_SCHEDULES` will switch to a different schedule everytime
     a cycle is finished.
 
-  - Setting AFL_EXPAND_HAVOC_NOW will start in the extended havoc mode that
+  - Setting `AFL_EXPAND_HAVOC_NOW` will start in the extended havoc mode that
     includes costly mutations. afl-fuzz automatically enables this mode when
     deemed useful otherwise.
 
-  - Setting AFL_PRELOAD causes AFL to set LD_PRELOAD for the target binary
+  - Setting `AFL_PRELOAD` causes AFL++ to set `LD_PRELOAD` for the target binary
     without disrupting the afl-fuzz process itself. This is useful, among other
     things, for bootstrapping libdislocator.so.
 
-  - Setting AFL_NO_UI inhibits the UI altogether, and just periodically prints
+  - Setting `AFL_NO_UI` inhibits the UI altogether, and just periodically prints
     some basic stats. This behavior is also automatically triggered when the
     output from afl-fuzz is redirected to a file or to a pipe.
 
-  - Setting AFL_FORCE_UI will force painting the UI on the screen even if
+  - Setting `AFL_FORCE_UI` will force painting the UI on the screen even if
     no valid terminal was detected (for virtual consoles)
 
-  - If you are Jakub, you may need AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES.
+  - If you are Jakub, you may need `AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES`.
     Others need not apply.
 
-  - Benchmarking only: AFL_BENCH_JUST_ONE causes the fuzzer to exit after
-    processing the first queue entry; and AFL_BENCH_UNTIL_CRASH causes it to
+  - Benchmarking only: `AFL_BENCH_JUST_ONE` causes the fuzzer to exit after
+    processing the first queue entry; and `AFL_BENCH_UNTIL_CRASH` causes it to
     exit soon after the first crash is found.
 
-  - Setting AFL_DEBUG_CHILD_OUTPUT will not suppress the child output.
+  - Setting `AFL_DEBUG_CHILD_OUTPUT` will not suppress the child output.
     Not pretty but good for debugging purposes.
 
-  - Setting AFL_NO_CPU_RED will not display very high cpu usages in red color.
+  - Setting `AFL_NO_CPU_RED` will not display very high cpu usages in red color.
 
-  - Setting AFL_AUTORESUME will resume a fuzz run (same as providing `-i -`)
+  - Setting `AFL_AUTORESUME` will resume a fuzz run (same as providing `-i -`)
     for an existing out folder, even if a different `-i` was provided.
     Without this setting, afl-fuzz will refuse execution for a long-fuzzed out dir.
 
-  - Setting AFL_NO_FORKSRV disables the forkserver optimization, reverting to
+  - Setting `AFL_NO_FORKSRV` disables the forkserver optimization, reverting to
     fork + execve() call for every tested input. This is useful mostly when
     working with unruly libraries that create threads or do other crazy
     things when initializing (before the instrumentation has a chance to run).
@@ -377,165 +397,163 @@ checks or alter some of the more exotic semantics of the tool:
     Note that this setting inhibits some of the user-friendly diagnostics
     normally done when starting up the forkserver and causes a pretty
     significant performance drop.
-  
-  - Setting AFL_MAX_DET_EXTRAS changes the count of dictionary entries/extras
-    (default 200), after which the entries will be used probabilistically.
-    So, if the dict/extras file (`-x`) contains more tokens than this threshold,
-    not all of the tokens will be used in each fuzzing step, every time.
-    Instead, there is a chance that the entry will be skipped during fuzzing.
-    This makes sure that the fuzzer doesn't spend all its time only inserting
-    the extras, but will still do other mutations. However, it decreases the
-    likelihood for each token to be inserted, before the next queue entry is fuzzed.
-    Either way, all tokens will be used eventually, in a longer fuzzing campaign.
-
-  - Outdated environment variables that are that not supported anymore:
-    AFL_DEFER_FORKSRV
-    AFL_PERSISTENT
-
-## 4) Settings for afl-qemu-trace
+
+  - Setting `AFL_STATSD` enable StatsD metrics collection.
+    By default AFL++ will send these metrics over UDP to 127.0.0.1:8125.
+    The host and port are configurable with `AFL_STATSD_HOST` and `AFL_STATSD_PORT`
+    respectively.
+    To get the most out of this, you should provide `AFL_STATSD_TAGS_FLAVOR` that
+    matches your StatsD server.
+    Available flavors are `dogstatsd`, `librato`, `signalfx` and `influxdb`.
+
+  - Outdated environment variables that are not supported anymore:
+    `AFL_DEFER_FORKSRV`
+    `AFL_PERSISTENT`
+
+## 5) Settings for afl-qemu-trace
 
 The QEMU wrapper used to instrument binary-only code supports several settings:
 
-  - It is possible to set AFL_INST_RATIO to skip the instrumentation on some
+  - It is possible to set `AFL_INST_RATIO` to skip the instrumentation on some
     of the basic blocks, which can be useful when dealing with very complex
     binaries.
 
-  - Setting AFL_INST_LIBS causes the translator to also instrument the code
+  - Setting `AFL_INST_LIBS` causes the translator to also instrument the code
     inside any dynamically linked libraries (notably including glibc).
 
-  - Setting AFL_COMPCOV_LEVEL enables the CompareCoverage tracing of all cmp
+  - Setting `AFL_COMPCOV_LEVEL` enables the CompareCoverage tracing of all cmp
     and sub in x86 and x86_64 and memory comparions functions (e.g. strcmp,
-    memcmp, ...) when libcompcov is preloaded using AFL_PRELOAD.
+    memcmp, ...) when libcompcov is preloaded using `AFL_PRELOAD`.
     More info at qemu_mode/libcompcov/README.md.
-    There are two levels at the moment, AFL_COMPCOV_LEVEL=1 that instruments
+    There are two levels at the moment, `AFL_COMPCOV_LEVEL=1` that instruments
     only comparisons with immediate values / read-only memory and
-    AFL_COMPCOV_LEVEL=2 that instruments all the comparions. Level 2 is more
+    `AFL_COMPCOV_LEVEL=2` that instruments all the comparions. Level 2 is more
     accurate but may need a larger shared memory.
 
-  - Setting AFL_QEMU_COMPCOV enables the CompareCoverage tracing of all
+  - Setting `AFL_QEMU_COMPCOV` enables the CompareCoverage tracing of all
     cmp and sub in x86 and x86_64.
-    This is an alias of AFL_COMPCOV_LEVEL=1 when AFL_COMPCOV_LEVEL is
+    This is an alias of `AFL_COMPCOV_LEVEL=1` when `AFL_COMPCOV_LEVEL` is
     not specified.
 
   - The underlying QEMU binary will recognize any standard "user space
-    emulation" variables (e.g., QEMU_STACK_SIZE), but there should be no
+    emulation" variables (e.g., `QEMU_STACK_SIZE`), but there should be no
     reason to touch them.
 
-  - AFL_DEBUG will print the found entrypoint for the binary to stderr.
+  - `AFL_DEBUG` will print the found entrypoint for the binary to stderr.
     Use this if you are unsure if the entrypoint might be wrong - but
-    use it directly, e.g. afl-qemu-trace ./program
+    use it directly, e.g. `afl-qemu-trace ./program`
 
-  - AFL_ENTRYPOINT allows you to specify a specific entrypoint into the
+  - `AFL_ENTRYPOINT` allows you to specify a specific entrypoint into the
     binary (this can be very good for the performance!).
-    The entrypoint is specified as hex address, e.g. 0x4004110
+    The entrypoint is specified as hex address, e.g. `0x4004110`
     Note that the address must be the address of a basic block.
 
   - When the target is i386/x86_64 you can specify the address of the function
     that has to be the body of the persistent loop using
-    AFL_QEMU_PERSISTENT_ADDR=`start addr`.
+    `AFL_QEMU_PERSISTENT_ADDR=start addr`.
 
   - Another modality to execute the persistent loop is to specify also the
-    AFL_QEMU_PERSISTENT_RET=`end addr` env variable.
+    `AFL_QEMU_PERSISTENT_RET=end addr` env variable.
     With this variable assigned, instead of patching the return address, the
     specified instruction is transformed to a jump towards `start addr`.
 
-  - AFL_QEMU_PERSISTENT_GPR=1 QEMU will save the original value of general
+  - `AFL_QEMU_PERSISTENT_GPR=1` QEMU will save the original value of general
     purpose registers and restore them in each persistent cycle.
 
-  - With AFL_QEMU_PERSISTENT_RETADDR_OFFSET you can specify the offset from the
+  - With `AFL_QEMU_PERSISTENT_RETADDR_OFFSET` you can specify the offset from the
     stack pointer in which QEMU can find the return address when `start addr` is
-    hitted.
+    hit.
 
-## 5) Settings for afl-cmin
+## 6) Settings for afl-cmin
 
 The corpus minimization script offers very little customization:
 
-  - Setting AFL_PATH offers a way to specify the location of afl-showmap
-    and afl-qemu-trace (the latter only in -Q mode).
+  - Setting `AFL_PATH` offers a way to specify the location of afl-showmap
+    and afl-qemu-trace (the latter only in `-Q` mode).
 
-  - AFL_KEEP_TRACES makes the tool keep traces and other metadata used for
+  - `AFL_KEEP_TRACES` makes the tool keep traces and other metadata used for
     minimization and normally deleted at exit. The files can be found in the
-    <out_dir>/.traces/*.
+    `<out_dir>/.traces/` directory.
 
-  - AFL_ALLOW_TMP permits this and some other scripts to run in /tmp. This is
+  - `AFL_ALLOW_TMP` permits this and some other scripts to run in /tmp. This is
     a modest security risk on multi-user systems with rogue users, but should
     be safe on dedicated fuzzing boxes.
 
 # #6) Settings for afl-tmin
 
-Virtually nothing to play with. Well, in QEMU mode (-Q), AFL_PATH will be
-searched for afl-qemu-trace. In addition to this, TMPDIR may be used if a
+Virtually nothing to play with. Well, in QEMU mode (`-Q`), `AFL_PATH` will be
+searched for afl-qemu-trace. In addition to this, `TMPDIR` may be used if a
 temporary file can't be created in the current working directory.
 
-You can specify AFL_TMIN_EXACT if you want afl-tmin to require execution paths
+You can specify `AFL_TMIN_EXACT` if you want afl-tmin to require execution paths
 to match when minimizing crashes. This will make minimization less useful, but
 may prevent the tool from "jumping" from one crashing condition to another in
-very buggy software. You probably want to combine it with the -e flag.
+very buggy software. You probably want to combine it with the `-e` flag.
 
-## 7) Settings for afl-analyze
+## 8) Settings for afl-analyze
 
-You can set AFL_ANALYZE_HEX to get file offsets printed as hexadecimal instead
+You can set `AFL_ANALYZE_HEX` to get file offsets printed as hexadecimal instead
 of decimal.
 
-## 8) Settings for libdislocator
+## 9) Settings for libdislocator
 
 The library honors these environmental variables:
 
-  - AFL_LD_LIMIT_MB caps the size of the maximum heap usage permitted by the
+  - `AFL_LD_LIMIT_MB` caps the size of the maximum heap usage permitted by the
     library, in megabytes. The default value is 1 GB. Once this is exceeded,
     allocations will return NULL.
 
-  - AFL_LD_HARD_FAIL alters the behavior by calling abort() on excessive
-    allocations, thus causing what AFL would perceive as a crash. Useful for
+  - `AFL_LD_HARD_FAIL` alters the behavior by calling `abort()` on excessive
+    allocations, thus causing what AFL++ would perceive as a crash. Useful for
     programs that are supposed to maintain a specific memory footprint.
 
-  - AFL_LD_VERBOSE causes the library to output some diagnostic messages
+  - `AFL_LD_VERBOSE` causes the library to output some diagnostic messages
     that may be useful for pinpointing the cause of any observed issues.
 
-  - AFL_LD_NO_CALLOC_OVER inhibits abort() on calloc() overflows. Most
+  - `AFL_LD_NO_CALLOC_OVER` inhibits `abort()` on `calloc()` overflows. Most
     of the common allocators check for that internally and return NULL, so
     it's a security risk only in more exotic setups.
 
-  - AFL_ALIGNED_ALLOC=1 will force the alignment of the allocation size to
-    max_align_t to be compliant with the C standard.
+  - `AFL_ALIGNED_ALLOC=1` will force the alignment of the allocation size to
+    `max_align_t` to be compliant with the C standard.
 
-## 9) Settings for libtokencap
+## 10) Settings for libtokencap
 
-This library accepts AFL_TOKEN_FILE to indicate the location to which the
+This library accepts `AFL_TOKEN_FILE` to indicate the location to which the
 discovered tokens should be written.
 
-## 10) Third-party variables set by afl-fuzz & other tools
+## 11) Third-party variables set by afl-fuzz & other tools
 
 Several variables are not directly interpreted by afl-fuzz, but are set to
 optimal values if not already present in the environment:
 
-  - By default, LD_BIND_NOW is set to speed up fuzzing by forcing the
+  - By default, `LD_BIND_NOW` is set to speed up fuzzing by forcing the
     linker to do all the work before the fork server kicks in. You can
-    override this by setting LD_BIND_LAZY beforehand, but it is almost
+    override this by setting `LD_BIND_LAZY` beforehand, but it is almost
     certainly pointless.
 
-  - By default, ASAN_OPTIONS are set to:
-
+  - By default, `ASAN_OPTIONS` are set to:
+```
     abort_on_error=1
     detect_leaks=0
     malloc_context_size=0
     symbolize=0
     allocator_may_return_null=1
-
-    If you want to set your own options, be sure to include abort_on_error=1 -
+```
+  If you want to set your own options, be sure to include `abort_on_error=1` -
     otherwise, the fuzzer will not be able to detect crashes in the tested
-    app. Similarly, include symbolize=0, since without it, AFL may have
+    app. Similarly, include `symbolize=0`, since without it, AFL++ may have
     difficulty telling crashes and hangs apart.
 
-  - In the same vein, by default, MSAN_OPTIONS are set to:
-
+  - In the same vein, by default, `MSAN_OPTIONS` are set to:
+```
     exit_code=86 (required for legacy reasons)
     abort_on_error=1
     symbolize=0
     msan_track_origins=0
     allocator_may_return_null=1
-
-    Be sure to include the first one when customizing anything, since some
-    MSAN versions don't call abort() on error, and we need a way to detect
+```
+  Be sure to include the first one when customizing anything, since some
+    MSAN versions don't call `abort()` on error, and we need a way to detect
     faults.
 
diff --git a/docs/ideas.md b/docs/ideas.md
index 65e2e8e6..aaa3eed1 100644
--- a/docs/ideas.md
+++ b/docs/ideas.md
@@ -3,48 +3,18 @@
 In the following, we describe a variety of ideas that could be implemented
 for future AFL++ versions.
 
-For GSOC2020 interested students please see
-[https://github.com/AFLplusplus/AFLplusplus/issues/208](https://github.com/AFLplusplus/AFLplusplus/issues/208)
-
-## Flexible Grammar Mutator (currently in development)
-
-Currently, AFL++'s mutation does not have deeper knowledge about the fuzzed
-binary, apart from feedback, even though the developer may have insights
-about the target.
-
-A developer may choose to provide dictionaries and implement own mutations
-in python or C, but an easy mutator that behaves according to a given grammar,
-does not exist.
-
-State-of-the-art research on grammar fuzzing has some problems in their
-implementations like code quality, scalability, or ease of use and other
-common issues of the academic code.
-
-We aim to develop a pluggable grammar mutator for afl++ that combines
-various results.
-
-Mentor: andreafioraldi 
-
-## perf-fuzz Linux Kernel Module
-
-Expand on [snapshot LKM](https://github.com/AFLplusplus/AFL-Snapshot-LKM)
-To make it thread safe, can snapshot several processes at once and increase
-overall performance.
-
-Mentor: any
-
-## QEMU 5-based Instrumentation
-
-First tests to use QEMU 4 for binary-only AFL++ showed that caching behavior
-changed, which vastly decreases fuzzing speeds.
-
-In this task test if QEMU 5 performs better and port the afl++ QEMU 3.1
-patches to QEMU 5.
-
-Understanding the current instrumentation and fixing the current caching
-issues will be needed.
-
-Mentor: andreafioraldi
+## Analysis software
+
+Currently analysis is done by using afl-plot, which is rather outdated.
+A GTK or browser tool to create run-time analysis based on fuzzer_stats,
+queue/id* information and plot_data that allows for zooming in and out,
+changing min/max display values etc. and doing that for a single run,
+different runs and campaigns vs campaigns.
+Interesting values are execs, and execs/s, edges discovered (total, when
+each edge was discovered and which other fuzzer share finding that edge),
+test cases executed.
+It should be clickable which value is X and Y axis, zoom factor, log scaling
+on-off, etc.
 
 ## WASM Instrumentation
 
@@ -66,20 +36,6 @@ Either improve a single mutator thorugh learning of many different bugs
 
 Mentor: domenukk
 
-## Reengineer `afl-fuzz` as Thread Safe, Embeddable Library (currently in development)
-
-Right now, afl-fuzz is single threaded, cannot safely be embedded in tools,
-and not multi-threaded. It makes use of a large number of globals, must always
-be the parent process and exec child processes. 
-Instead, afl-fuzz could be refactored to contain no global state and globals.
-This allows for different use cases that could be implemented during this
-project.
-Note that in the mean time a lot has happened here already, but e.g. making
-it all work and implement multithreading in afl-fuzz ... there is still quite
-some work to do.
-
-Mentor: hexcoder- or vanhauser-thc
-
 ## Collision-free Binary-Only Maps
 
 AFL++ supports collison-free maps using an LTO (link-time-optimization) pass.
diff --git a/docs/life_pro_tips.md b/docs/life_pro_tips.md
index a5bd7286..323f16f1 100644
--- a/docs/life_pro_tips.md
+++ b/docs/life_pro_tips.md
@@ -30,10 +30,10 @@ Check out the `fuzzer_stats` file in the AFL output dir or try `afl-whatsup`.
 It could be important - consult docs/status_screen.md right away!
 
 ## Know your target? Convert it to persistent mode for a huge performance gain!
-Consult section #5 in llvm_mode/README.md for tips.
+Consult section #5 in README.llvm.md for tips.
 
 ## Using clang? 
-Check out llvm_mode/ for a faster alternative to afl-gcc!
+Check out instrumentation/ for a faster alternative to afl-gcc!
 
 ## Did you know that AFL can fuzz closed-source or cross-platform binaries?
 Check out qemu_mode/README.md and unicorn_mode/README.md for more.
@@ -85,6 +85,3 @@ You can find a simple solution in examples/argv_fuzzing.
 Remove the checksum-checking code or use a postprocessor!
 See examples/custom_mutators/ for more.
 
-## Dealing with a very slow target or hoping for instant results? 
-
-Specify `-d` when calling afl-fuzz!
diff --git a/docs/perf_tips.md b/docs/perf_tips.md
index 731dc238..fbcb4d8d 100644
--- a/docs/perf_tips.md
+++ b/docs/perf_tips.md
@@ -51,7 +51,7 @@ a file.
 ## 3. Use LLVM instrumentation
 
 When fuzzing slow targets, you can gain 20-100% performance improvement by
-using the LLVM-based instrumentation mode described in [the llvm_mode README](../llvm_mode/README.md).
+using the LLVM-based instrumentation mode described in [the instrumentation README](../instrumentation/README.llvm.md).
 Note that this mode requires the use of clang and will not work with GCC.
 
 The LLVM mode also offers a "persistent", in-process fuzzing mode that can
@@ -62,12 +62,12 @@ modes require you to edit the source code of the fuzzed program, but the
 changes often amount to just strategically placing a single line or two.
 
 If there are important data comparisons performed (e.g. `strcmp(ptr, MAGIC_HDR)`)
-then using laf-intel (see llvm_mode/README.laf-intel.md) will help `afl-fuzz` a lot
+then using laf-intel (see instrumentation/README.laf-intel.md) will help `afl-fuzz` a lot
 to get to the important parts in the code.
 
 If you are only interested in specific parts of the code being fuzzed, you can
 instrument_files the files that are actually relevant. This improves the speed and
-accuracy of afl. See llvm_mode/README.instrument_list.md
+accuracy of afl. See instrumentation/README.instrument_list.md
 
 Also use the InsTrim mode on larger binaries, this improves performance and
 coverage a lot.
@@ -110,7 +110,7 @@ e.g.:
   https://launchpad.net/libeatmydata
 
 In programs that are slow due to unavoidable initialization overhead, you may
-want to try the LLVM deferred forkserver mode (see llvm_mode/README.md),
+want to try the LLVM deferred forkserver mode (see README.llvm.md),
 which can give you speed gains up to 10x, as mentioned above.
 
 Last but not least, if you are using ASAN and the performance is unacceptable,
diff --git a/docs/sister_projects.md b/docs/sister_projects.md
index a501ecbd..5cb3a102 100644
--- a/docs/sister_projects.md
+++ b/docs/sister_projects.md
@@ -52,7 +52,7 @@ options.
 Provides an evolutionary instrumentation-guided fuzzing harness that allows
 some programs to be fuzzed without the fork / execve overhead. (Similar
 functionality is now available as the "persistent" feature described in
-[the llvm_mode readme](../llvm_mode/README.md))
+[the llvm_mode readme](../instrumentation/README.llvm.md))
 
 http://llvm.org/docs/LibFuzzer.html
 
@@ -119,10 +119,18 @@ Simplifies the triage of discovered crashes, start parallel instances, etc.
 
 https://github.com/rc0r/afl-utils
 
+### AFL crash analyzer (floyd)
+
 Another crash triage tool:
 
 https://github.com/floyd-fuh/afl-crash-analyzer
 
+###  afl-extras (fekir)
+
+Collect data, parallel afl-tmin, startup scripts.
+
+https://github.com/fekir/afl-extras
+
 ### afl-fuzzing-scripts (Tobias Ospelt)
 
 Simplifies starting up multiple parallel AFL jobs.
@@ -245,7 +253,7 @@ https://code.google.com/p/address-sanitizer/wiki/AsanCoverage#Coverage_counters
 ### AFL JS (Han Choongwoo)
 
 One-off optimizations to speed up the fuzzing of JavaScriptCore (now likely
-superseded by LLVM deferred forkserver init - see llvm_mode/README.md).
+superseded by LLVM deferred forkserver init - see README.llvm.md).
 
 https://github.com/tunz/afl-fuzz-js
 
diff --git a/docs/status_screen.md b/docs/status_screen.md
index b89468ce..0cede6ff 100644
--- a/docs/status_screen.md
+++ b/docs/status_screen.md
@@ -86,10 +86,7 @@ Every fuzzing session should be allowed to complete at least one cycle; and
 ideally, should run much longer than that.
 
 As noted earlier, the first pass can take a day or longer, so sit back and
-relax. If you want to get broader but more shallow coverage right away, try
-the `-d` option - it gives you a more familiar experience by skipping the
-deterministic fuzzing steps. It is, however, inferior to the standard mode in
-a couple of subtle ways.
+relax. 
 
 To help make the call on when to hit `Ctrl-C`, the cycle counter is color-coded.
 It is shown in magenta during the first pass, progresses to yellow if new finds
@@ -118,9 +115,6 @@ inputs it decided to ditch because they were persistently timing out.
 The "*" suffix sometimes shown in the first line means that the currently
 processed path is not "favored" (a property discussed later on).
 
-If you feel that the fuzzer is progressing too slowly, see the note about the
-`-d` option in this doc.
-
 ### Map coverage
 
 ```
@@ -324,7 +318,7 @@ there are several things to look at:
   - Multiple threads executing at once in semi-random order. This is harmless
     when the 'stability' metric stays over 90% or so, but can become an issue
     if not. Here's what to try:
-    * Use afl-clang-fast from [llvm_mode](../llvm_mode/) - it uses a thread-local tracking
+    * Use afl-clang-fast from [instrumentation](../instrumentation/) - it uses a thread-local tracking
       model that is less prone to concurrency issues,
     * See if the target can be compiled or run without threads. Common
       `./configure` options include `--without-threads`, `--disable-pthreads`, or
@@ -412,3 +406,27 @@ Most of these map directly to the UI elements discussed earlier on.
 On top of that, you can also find an entry called `plot_data`, containing a
 plottable history for most of these fields. If you have gnuplot installed, you
 can turn this into a nice progress report with the included `afl-plot` tool.
+
+
+### Addendum: Automatically send metrics with StatsD
+
+In a CI environment or when running multiple fuzzers, it can be tedious to
+log into each of them or deploy scripts to read the fuzzer statistics.
+Using `AFL_STATSD` (and the other related environment variables `AFL_STATSD_HOST`,
+`AFL_STATSD_PORT`, `AFL_STATSD_TAGS_FLAVOR`) you can automatically send metrics
+to your favorite StatsD server. Depending on your StatsD server you will be able
+to monitor, trigger alerts or perform actions based on these metrics (e.g: alert on
+slow exec/s for a new build, threshold of crashes, time since last crash > X, etc). 
+
+The selected metrics are a subset of all the metrics found in the status and in
+the plot file. The list is the following: `cycle_done`, `cycles_wo_finds`,
+`execs_done`,`execs_per_sec`, `paths_total`, `paths_favored`, `paths_found`,
+`paths_imported`, `max_depth`, `cur_path`, `pending_favs`, `pending_total`,
+`variable_paths`, `unique_crashes`, `unique_hangs`, `total_crashes`,
+`slowest_exec_ms`, `edges_found`, `var_byte_count`, `havoc_expansion`.
+Their definitions can be found in the addendum above.
+
+When using multiple fuzzer instances with StatsD it is *strongly* recommended to setup
+the flavor (AFL_STATSD_TAGS_FLAVOR) to match your StatsD server. This will allow you
+to see individual fuzzer performance, detect bad ones, see the progress of each
+strategy...