Merge branch 'dev' of https://github.com/AFLplusplus/AFLplusplus into dev

1 files changed, 229 insertions, 4 deletions

diff --git a/docs/Changelog.md b/docs/Changelog.md
index 1e7a1c1d..8dc218af 100644
--- a/docs/Changelog.md
+++ b/docs/Changelog.md
@@ -8,25 +8,250 @@
 Want to stay in the loop on major new features? Join our mailing list by
 sending a mail to <afl-users+subscribe@googlegroups.com>.
 
+### Version ++3.12a (dev)
+  - afl-cc:
+    - fix cmplog rtn (rare crash and not being able to gather ptr data)
+    - link runtime not to shared libs
+    - ensure shared libraries are properly built and instrumented
+  - qemu_mode (thanks @realmadsci):
+    - move AFL_PRELOAD and AFL_USE_QASAN logic inside afl-qemu-trace
+    - add AFL_QEMU_CUSTOM_BIN
+
+### Version ++3.11c (release)
+  - afl-fuzz:
+    - better auto detection of map size
+    - fix sanitizer settings (bug since 3.10c)
+    - fix an off-by-one overwrite in cmplog
+    - add non-unicode variants from unicode-looking dictionary entries
+    - Rust custom mutator API improvements
+    - Imported crash stats painted yellow on resume (only new ones are red)
+  - afl-cc:
+    - added AFL_NOOPT that will just pass everything to the normal
+      gcc/clang compiler without any changes - to pass weird configure
+      scripts
+    - fixed a crash that can occur with ASAN + CMPLOG together plus
+      better support for unicode (thanks to @stbergmann for reporting!)
+    - fixed a crash in LAF transform for empty strings
+    - handle erroneous setups in which multiple afl-compiler-rt are
+      compiled into the target. This now also supports dlopen()
+      instrumented libs loaded before the forkserver and even after the
+      forkserver is started (then with collisions though)
+    - the compiler rt was added also in object building (-c) which
+      should have been fixed years ago but somewhere got lost :(
+    - Renamed CTX to CALLER, added correct/real CTX implementation to
+      CLASSIC
+  - qemu_mode:
+    - added AFL_QEMU_EXCLUDE_RANGES env by @realmadsci, thanks!
+    - if no new/updated checkout is wanted, build with:
+      NO_CHECKOUT=1 ./build_qemu_support.sh
+    - we no longer perform a "git drop"
+  - afl-cmin: support filenames with spaces
+
+### Version ++3.10c (release)
+  - Mac OS ARM64 support
+  - Android support fixed and updated by Joey Jiaojg - thanks!
+  - New selective instrumentation option with __AFL_COVERAGE_* commands
+    to be placed in the source code.
+    Check out instrumentation/README.instrument_list.md
+  - afl-fuzz
+    - Making AFL_MAP_SIZE (mostly) obsolete - afl-fuzz now learns on
+      start the target map size
+    - upgraded cmplog/redqueen: solving for floating point, solving
+      transformations (e.g. toupper, tolower, to/from hex, xor,
+      arithmetics, etc.). This is costly hence new command line option
+      `-l` that sets the intensity (values 1 to 3). Recommended is 2.
+    - added `AFL_CMPLOG_ONLY_NEW` to not use cmplog on initial seeds
+      from `-i` or resumes (these have most likely already been done)
+    - fix crash for very, very fast targets+systems (thanks to mhlakhani
+      for reporting)
+    - on restarts (`-i`)/autoresume (AFL_AUTORESUME) the stats are now
+      reloaded and used, thanks to Vimal Joseph for this patch! 
+    - changed the meaning of '+' of the '-t' option, it now means to
+      auto-calculate the timeout with the value given being the max
+      timeout. The original meaning of skipping timeouts instead of
+      abort is now inherent to the -t option.
+    - if deterministic mode is active (`-D`, or `-M` without `-d`) then
+      we sync after every queue entry as this can take very long time
+      otherwise
+    - added minimum SYNC_TIME to include/config.h (30 minutes default)
+    - better detection if a target needs a large shared map
+    - fix for `-Z`
+    - fixed a few crashes
+    - switched to an even faster RNG
+    - added hghwng's patch for faster trace map analysis
+    - printing suggestions for mistyped `AFL_` env variables
+    - added Rust bindings for custom mutators (thanks @julihoh)
+  - afl-cc
+    - allow instrumenting LLVMFuzzerTestOneInput
+    - fixed endless loop for allow/blocklist lines starting with a
+      comment (thanks to Zherya for reporting)
+    - cmplog/redqueen now also tracks floating point, _ExtInt() + 128bit
+    - cmplog/redqueen can now process basic libc++ and libstdc++
+      std::string comparisons (no position or length type variants)
+    - added support for __afl_coverage_interesting() for LTO and our
+      own PCGUARD (llvm 10.0.1+), read more about this function and
+      selective coverage in instrumentation/README.instrument_list.md
+    - added AFL_LLVM_INSTRUMENT option NATIVE for native clang pc-guard
+      support (less performant than our own), GCC for old afl-gcc and
+      CLANG for old afl-clang
+    - fixed a potential crash in the LAF feature
+    - workaround for llvm bitcast lto bug
+    - workaround for llvm 13
+  - qemuafl
+    - QASan (address sanitizer for Qemu) ported to qemuafl!
+      See qemu_mode/libqasan/README.md
+    - solved some persistent mode bugs (thanks Dil4rd)
+    - solved an issue when dumping the memory maps (thanks wizche)
+    - Android support for QASan
+  - unicornafl
+    - Substantial speed gains in python bindings for certain use cases
+    - Improved rust bindings
+    - Added a new example harness to compare python, c and rust bindings
+  - afl-cmin and afl-showmap now support the -f option
+  - afl_plot now also generates a graph on the discovered edges
+  - changed default: no memory limit for afl-cmin and afl-cmin.bash
+  - warn on any _AFL and __AFL env vars.
+  - set AFL_IGNORE_UNKNOWN_ENVS to not warn on unknown AFL_... env vars
+  - added dummy Makefile to instrumentation/
+  - Updated utils/afl_frida to be 5% faster, 7% on x86_x64
+  - Added `AFL_KILL_SIGNAL` env variable (thanks @v-p-b)
+  - @Edznux added a nice documentation on how to use rpc.statsd with
+    afl++ in docs/rpc_statsd.md, thanks!
+
+### Version ++3.00c (release)
+  - llvm_mode/ and gcc_plugin/ moved to instrumentation/
+  - examples/ renamed to utils/
+  - moved libdislocator, libtokencap and qdbi_mode to utils/
+  - all compilers combined to afl-cc which emulates the previous ones
+  - afl-llvm/gcc-rt.o merged into afl-compiler-rt.o
+  - afl-fuzz
+    - not specifying -M or -S will now auto-set "-S default"
+    - deterministic fuzzing is now disabled by default and can be enabled with
+      -D. It is still enabled by default for -M.
+    - a new seed selection was implemented that uses weighted randoms based on
+      a schedule performance score, which is much better that the previous
+      walk the whole queue approach. Select the old mode with -Z (auto enabled
+      with -M)
+    - Marcel Boehme submitted a patch that improves all AFFast schedules :)
+    - the default schedule is now FAST
+    - memory limits are now disabled by default, set them with -m if required
+    - rpc.statsd support, for stats and charts, by Edznux, thanks a lot!
+    - reading testcases from -i now descends into subdirectories
+    - allow the -x command line option up to 4 times
+    - loaded extras now have a duplication protection
+    - If test cases are too large we do a partial read on the maximum
+      supported size
+    - longer seeds with the same trace information will now be ignored
+      for fuzzing but still be used for splicing
+    - crashing seeds are now not prohibiting a run anymore but are
+      skipped - they are used for splicing, though
+    - update MOpt for expanded havoc modes
+    - setting the env var AFL_NO_AUTODICT will not load an LTO autodictionary
+    - added NO_SPLICING compile option and makefile define
+    - added INTROSPECTION make target that writes all mutations to
+      out/NAME/introspection.txt
+    - print special compile time options used in help output
+    - when using -c cmplog, one of the childs was not killed, fixed
+    - somewhere we broke -n dumb fuzzing, fixed
+    - added afl_custom_describe to the custom mutator API to allow for easy
+      mutation reproduction on crashing inputs
+    - new env. var. AFL_NO_COLOR (or AFL_NO_COLOUR) to suppress colored
+      console output (when configured with USE_COLOR and not ALWAYS_COLORED)
+  - instrumentation
+    - We received an enhanced gcc_plugin module from AdaCore, thank you
+      very much!!
+    - not overriding -Ox or -fno-unroll-loops anymore
+    - we now have our own trace-pc-guard implementation. It is the same as
+      -fsanitize-coverage=trace-pc-guard from llvm 12, but: it is a) inline
+      and b) works from llvm 10.0.1 + onwards :)
+    - new llvm pass: dict2file via AFL_LLVM_DICT2FILE, create afl-fuzz
+      -x dictionary of string comparisons found during compilation
+    - LTO autodict now also collects interesting cmp comparisons,
+      std::string compare + find + ==, bcmp
+    - fix crash in dict2file for integers > 64 bit
+  - custom mutators
+    - added a new custom mutator: symcc -> https://github.com/eurecom-s3/symcc/
+    - added a new custom mutator: libfuzzer that integrates libfuzzer mutations
+    - Our afl++ Grammar-Mutator is now better integrated into custom_mutators/
+    - added INTROSPECTION support for custom modules
+    - python fuzz function was not optional, fixed
+    - some python mutator speed improvements
+  - afl-cmin/afl-cmin.bash now search first in PATH and last in AFL_PATH
+  - unicornafl synced with upstream version 1.02 (fixes, better rust bindings)
+  - renamed AFL_DEBUG_CHILD_OUTPUT to AFL_DEBUG_CHILD
+  - added AFL_CRASH_EXITCODE env variable to treat a child exitcode as crash
+
+
+### Version ++2.68c (release)
+  - added the GSoC excellent afl++ grammar mutator by Shengtuo to our
+    custom_mutators/ (see custom_mutators/README.md) - or get it here:
+    https://github.com/AFLplusplus/Grammar-Mutator
+  - a few QOL changes for Apple and its outdated gmake
+  - afl-fuzz:
+    - fix for auto dictionary entries found during fuzzing to not throw out
+      a -x dictionary
+    - added total execs done to plot file
+    - AFL_MAX_DET_EXTRAS env variable added to control the amount of
+      deterministic dict entries without recompiling.
+    - AFL_FORKSRV_INIT_TMOUT env variable added to control the time to wait
+      for the forkserver to come up without the need to increase the overall
+      timeout.
+    - bugfix for cmplog that results in a heap overflow based on target data
+      (thanks to the magma team for reporting!)
+    - write fuzzing setup into out/fuzzer_setup (environment variables and
+      command line)
+  - custom mutators:
+    - added afl_custom_fuzz_count/fuzz_count function to allow specifying
+      the number of fuzz attempts for custom_fuzz
+  - llvm_mode:
+    - ported SanCov to LTO, and made it the default for LTO. better
+      instrumentation locations
+    - Further llvm 12 support (fast moving target like afl++ :-) )
+    - deprecated LLVM SKIPSINGLEBLOCK env environment
+
 
-### Version ++2.66d (devel)
+### Version ++2.67c (release)
   - Support for improved afl++ snapshot module:
     https://github.com/AFLplusplus/AFL-Snapshot-LKM
+  - Due to the instrumentation needing more memory, the initial memory sizes
+    for -m have been increased
   - afl-fuzz:
      - added -F option to allow -M main fuzzers to sync to foreign fuzzers,
        e.g. honggfuzz or libfuzzer
+     - added -b option to bind to a specific CPU
      - eliminated CPU affinity race condition for -S/-M runs
      - expanded havoc mode added, on no cycle finds add extra splicing and
        MOpt into the mix
+     - fixed a bug in redqueen for strings and made deterministic with -s
+     - Compiletime autodictionary fixes
   - llvm_mode:
-     - now supports llvm 12!
+     - now supports llvm 12
+     - support for AFL_LLVM_ALLOWLIST/AFL_LLVM_DENYLIST (previous
+       AFL_LLVM_WHITELIST and AFL_LLVM_INSTRUMENT_FILE are deprecated and
+       are matched to AFL_LLVM_ALLOWLIST). The format is compatible to llvm
+       sancov, and also supports function matching :)
+     - added neverzero counting to trace-pc/pcgard
      - fixes for laf-intel float splitting (thanks to mark-griffin for
        reporting)
-     - LTO: autodictionary mode is a default
+     - fixes for llvm 4.0
+     - skipping ctors and ifuncs for instrumentation
+     - LTO: switch default to the dynamic memory map, set AFL_LLVM_MAP_ADDR
+            for a fixed map address (eg. 0x10000)
+     - LTO: improved stability for persistent mode, no other instrumentation
+            has that advantage
+     - LTO: fixed autodict for long strings
+     - LTO: laf-intel and redqueen/cmplog are now applied at link time
+            to prevent llvm optimizing away the splits
+     - LTO: autodictionary mode is a fixed default now
      - LTO: instrim instrumentation disabled, only classic support used
             as it is always better
+     - LTO: env var AFL_LLVM_DOCUMENT_IDS=file will document which edge ID
+            was given to which function during compilation
+     - LTO: single block functions were not implemented by default, fixed
+     - LTO: AFL_LLVM_SKIP_NEVERZERO behaviour was inversed, fixed
      - setting AFL_LLVM_LAF_SPLIT_FLOATS now activates
        AFL_LLVM_LAF_SPLIT_COMPARES
+     - support for -E and -shared compilation runs
   - added honggfuzz mangle as a custom mutator in custom_mutators/honggfuzz
   - added afl-frida gum solution to examples/afl_frida (mostly imported
     from https://github.com/meme/hotwax/)
@@ -373,7 +598,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
   - big code refactoring:
     * all includes are now in include/
     * all afl sources are now in src/ - see src/README.md
-    * afl-fuzz was splitted up in various individual files for including
+    * afl-fuzz was split up in various individual files for including
       functionality in other programs (e.g. forkserver, memory map, etc.)
       for better readability.
     * new code indention everywhere