Merge branch 'dev' into going_atomic

7 files changed, 109 insertions, 31 deletions

diff --git a/docs/Changelog.md b/docs/Changelog.md
index d8e96bf3..d8ffe498 100644
--- a/docs/Changelog.md
+++ b/docs/Changelog.md
@@ -9,7 +9,9 @@ Want to stay in the loop on major new features? Join our mailing list by
 sending a mail to <afl-users+subscribe@googlegroups.com>.
 
 ### Version ++3.13a (development)
+  - Note: plot_data switched to relative time from unix time in 3.10
   - frida_mode - new mode that uses frida to fuzz binary-only targets,
+    it currently supports persistent mode and cmplog.
     thanks to @WorksButNotTested!
   - create a fuzzing dictionary with the help of CodeQL thanks to
     @microsvuln! see utils/autodict_ql
@@ -19,6 +21,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
     - add recording of previous fuzz attempts for persistent mode
       to allow replay of non-reproducable crashes, see
       AFL_PERSISTENT_RECORD in config.h and docs/envs.h
+    - fixed a bug when trimming for stdin targets
     - default cmplog level (-l) is now 2, better efficiency.
     - cmplog level 3 (-l 3) now performs redqueen on everything.
       use with care.
@@ -26,16 +29,39 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
     - ensure one fuzzer sync per cycle
     - fix afl_custom_queue_new_entry original file name when syncing
       from fuzzers
+    - on a crashing seed potentially the wrong input was disabled
     - added AFL_EXIT_ON_SEED_ISSUES env that will exit if a seed in
       -i dir crashes the target or results in a timeout. By default
       afl++ ignores these and uses them for splicing instead.
+    - added AFL_EXIT_ON_TIME env that will make afl-fuzz exit fuzzing
+      after no new paths have been found for n seconds
+    - when AFL_FAST_CAL is set a variable path will now be calibrated
+      8 times instead of originally 40. Long calibration is now 20.
+    - added AFL_TRY_AFFINITY to try to bind to CPUs but don't error if
+      it fails
   - afl-cc:
+    - We do not support llvm versions prior 6.0 anymore
+    - Fix for -pie compiled binaries with default afl-clang-fast PCGUARD
     - Leak Sanitizer (AFL_USE_LSAN) added by Joshua Rogers, thanks!
     - Removed InsTrim instrumentation as it is not as good as PCGUARD
     - Removed automatic linking with -lc++ for LTO mode
-  - utils/aflpp_driver/aflpp_qemu_driver_hook fixed to work with qemu mode
+    - Fixed a crash in llvm dict2file when a strncmp length was -1
+    - added --afl-noopt support
+  - utils/aflpp_driver:
+    - aflpp_qemu_driver_hook fixed to work with qemu_mode
+    - aflpp_driver now compiled with -fPIC
+  - unicornafl:
+    - fix MIPS delay slot caching, thanks @JackGrence
+    - fixed aarch64 exit address
+    - execution no longer stops at address 0x0
+  - updated afl-system-config to support Arch Linux weirdness and increase
+    MacOS shared memory
+  - updated the grammar custom mutator to the newest version
   - add -d (add dead fuzzer stats) to afl-whatsup
   - add thread safe counters for LLVM CLASSIC (set AFL_LLVM_THREADSAFE_INST)
+  - added AFL_PRINT_FILENAMES to afl-showmap/cmin to print the
+    current filename
+  - afl-showmap/cmin will now process queue items in alphabetical order
 
 ### Version ++3.12c (release)
   - afl-fuzz:
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
index e3c06c9d..80d452f7 100644
--- a/docs/INSTALL.md
+++ b/docs/INSTALL.md
@@ -103,6 +103,41 @@ The llvm instrumentation requires a fully-operational installation of clang. The
 comes with Xcode is missing some of the essential headers and helper tools.
 See README.llvm.md for advice on how to build the compiler from scratch.
 
+MacOS X supports SYSV shared memory used by AFL's instrumentation, but the
+default settings aren't usable with AFL++. The default settings on 10.14 seem
+to be:
+
+```bash
+$ ipcs -M
+IPC status from <running system> as of XXX
+shminfo:
+        shmmax: 4194304 (max shared memory segment size)
+        shmmin:       1 (min shared memory segment size)
+        shmmni:      32 (max number of shared memory identifiers)
+        shmseg:       8 (max shared memory segments per process)
+        shmall:    1024 (max amount of shared memory in pages)
+```
+
+To temporarily change your settings to something minimally usable with AFL++,
+run these commands as root:
+
+```bash
+sysctl kern.sysv.shmmax=8388608
+sysctl kern.sysv.shmall=4096
+```
+
+If you're running more than one instance of AFL you likely want to make `shmall`
+bigger and increase `shmseg` as well:
+
+```bash
+sysctl kern.sysv.shmmax=8388608
+sysctl kern.sysv.shmseg=48
+sysctl kern.sysv.shmall=98304
+```
+
+See http://www.spy-hill.com/help/apple/SharedMemory.html for documentation for
+these settings and how to make them permanent.
+
 ## 4. Linux or *BSD on non-x86 systems
 
 Standard build will fail on non-x86 systems, but you should be able to
diff --git a/docs/QuickStartGuide.md b/docs/QuickStartGuide.md
index 10be409a..d1966170 100644
--- a/docs/QuickStartGuide.md
+++ b/docs/QuickStartGuide.md
@@ -1,6 +1,6 @@
 # AFL quick start guide
 
-You should read [README.md](README.md) - it's pretty short. If you really can't, here's
+You should read [README.md](../README.md) - it's pretty short. If you really can't, here's
 how to hit the ground running:
 
 1) Compile AFL with 'make'. If build fails, see [INSTALL.md](INSTALL.md) for tips.
diff --git a/docs/binaryonly_fuzzing.md b/docs/binaryonly_fuzzing.md
index 787d970d..11e1dbeb 100644
--- a/docs/binaryonly_fuzzing.md
+++ b/docs/binaryonly_fuzzing.md
@@ -41,6 +41,33 @@
 
   As it is included in afl++ this needs no URL.
 
+  If you like to code a customized fuzzer without much work, we highly
+  recommend to check out our sister project libafl which will support QEMU
+  too:
+  [https://github.com/AFLplusplus/LibAFL](https://github.com/AFLplusplus/LibAFL)
+
+
+## AFL FRIDA
+
+  In frida_mode you can fuzz binary-only targets easily like with QEMU,
+  with the advantage that frida_mode also works on MacOS (both intel and M1).
+
+  If you want to fuzz a binary-only library then you can fuzz it with
+  frida-gum via utils/afl_frida/, you will have to write a harness to
+  call the target function in the library, use afl-frida.c as a template.
+
+  Both come with afl++ so this needs no URL.
+
+  You can also perform remote fuzzing with frida, e.g. if you want to fuzz
+  on iPhone or Android devices, for this you can use
+  [https://github.com/ttdennis/fpicker/](https://github.com/ttdennis/fpicker/)
+  as an intermediate that uses afl++ for fuzzing.
+
+  If you like to code a customized fuzzer without much work, we highly
+  recommend to check out our sister project libafl which supports Frida too:
+  [https://github.com/AFLplusplus/LibAFL](https://github.com/AFLplusplus/LibAFL)
+  Working examples already exist :-)
+
 
 ## WINE+QEMU
 
@@ -62,13 +89,6 @@
   As it is included in afl++ this needs no URL.
 
 
-## AFL FRIDA
-
-   If you want to fuzz a binary-only shared library then you can fuzz it with
-   frida-gum via utils/afl_frida/, you will have to write a harness to
-   call the target function in the library, use afl-frida.c as a template.
-
-
 ## AFL UNTRACER
 
    If you want to fuzz a binary-only shared library then you can fuzz it with
@@ -157,19 +177,6 @@
   If anyone finds any coresight implementation for afl please ping me: vh@thc.org
 
 
-## FRIDA
-
-  Frida is a dynamic instrumentation engine like Pintool, Dyninst and Dynamorio.
-  What is special is that it is written Python, and scripted with Javascript.
-  It is mostly used to reverse binaries on mobile phones however can be used
-  everywhere.
-
-  There is a WIP fuzzer available at [https://github.com/andreafioraldi/frida-fuzzer](https://github.com/andreafioraldi/frida-fuzzer)
-
-  There is also an early implementation in an AFL++ test branch:
-  [https://github.com/AFLplusplus/AFLplusplus/tree/frida](https://github.com/AFLplusplus/AFLplusplus/tree/frida)
-
-
 ## PIN & DYNAMORIO
 
   Pintool and Dynamorio are dynamic instrumentation engines, and they can be
@@ -205,7 +212,8 @@
   * QSYM: [https://github.com/sslab-gatech/qsym](https://github.com/sslab-gatech/qsym)
   * Manticore: [https://github.com/trailofbits/manticore](https://github.com/trailofbits/manticore)
   * S2E: [https://github.com/S2E](https://github.com/S2E)
-  * Tinyinst [https://github.com/googleprojectzero/TinyInst](https://github.com/googleprojectzero/TinyInst) (Mac/Windows only)
+  * Tinyinst: [https://github.com/googleprojectzero/TinyInst](https://github.com/googleprojectzero/TinyInst) (Mac/Windows only)
+  * Jackalope: [https://github.com/googleprojectzero/Jackalope](https://github.com/googleprojectzero/Jackalope)
   *  ... please send me any missing that are good
 
 
diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md
index 62e01f83..3e3ae01d 100644
--- a/docs/custom_mutators.md
+++ b/docs/custom_mutators.md
@@ -92,6 +92,9 @@ def queue_new_entry(filename_new_queue, filename_orig_queue):
 
 def introspection():
     return string
+
+def deinit():  # optional for Python
+    pass
 ```
 
 ### Custom Mutation
@@ -285,8 +288,8 @@ afl-fuzz /path/to/program
 
 ## 4) Example
 
-Please see [example.c](../utils/custom_mutators/example.c) and
-[example.py](../utils/custom_mutators/example.py)
+Please see [example.c](../custom_mutators/examples/example.c) and
+[example.py](../custom_mutators/examples/example.py)
 
 ## 5) Other Resources
 
diff --git a/docs/env_variables.md b/docs/env_variables.md
index d9a774aa..b4b866ab 100644
--- a/docs/env_variables.md
+++ b/docs/env_variables.md
@@ -289,6 +289,10 @@ checks or alter some of the more exotic semantics of the tool:
     normally indicated by the cycle counter in the UI turning green. May be
     convenient for some types of automated jobs.
 
+  - `AFL_EXIT_ON_TIME` Causes afl-fuzz to terminate if no new paths were 
+    found within a specified period of time (in seconds). May be convenient 
+    for some types of automated jobs.
+
   - `AFL_EXIT_ON_SEED_ISSUES` will restore the vanilla afl-fuzz behaviour
     which does not allow crashes or timeout seeds in the initial -i corpus.
 
@@ -313,14 +317,12 @@ checks or alter some of the more exotic semantics of the tool:
     on Linux systems. This slows things down, but lets you run more instances
     of afl-fuzz than would be prudent (if you really want to).
 
+  - Setting `AFL_TRY_AFFINITY` tries to attempt binding to a specific CPU core
+    on Linux systems, but will not terminate if that fails.
+
   - Setting `AFL_NO_AUTODICT` will not load an LTO generated auto dictionary
     that is compiled into the target.
 
-  - `AFL_SKIP_CRASHES` causes AFL++ to tolerate crashing files in the input
-    queue. This can help with rare situations where a program crashes only
-    intermittently, but it's not really recommended under normal operating
-    conditions.
-
   - Setting `AFL_HANG_TMOUT` allows you to specify a different timeout for
     deciding if a particular test case is a "hang". The default is 1 second
     or the value of the `-t` parameter, whichever is larger. Dialing the value
@@ -356,6 +358,7 @@ checks or alter some of the more exotic semantics of the tool:
     and shell scripts; and `AFL_DUMB_FORKSRV` in conjunction with the `-n`
     setting to instruct afl-fuzz to still follow the fork server protocol
     without expecting any instrumentation data in return.
+    Note that this also turns off auto map size detection.
 
   - When running in the `-M` or `-S` mode, setting `AFL_IMPORT_FIRST` causes the
     fuzzer to import test cases from other instances before doing anything
@@ -568,6 +571,9 @@ The corpus minimization script offers very little customization:
     a modest security risk on multi-user systems with rogue users, but should
     be safe on dedicated fuzzing boxes.
 
+  - `AFL_PRINT_FILENAMES` prints each filename to stdout, as it gets processed.
+    This can help when embedding `afl-cmin` or `afl-showmap` in other scripts scripting.
+
 ## 7) Settings for afl-tmin
 
 Virtually nothing to play with. Well, in QEMU mode (`-Q`), `AFL_PATH` will be
diff --git a/docs/life_pro_tips.md b/docs/life_pro_tips.md
index 50ad75d4..13ffcea0 100644
--- a/docs/life_pro_tips.md
+++ b/docs/life_pro_tips.md
@@ -83,5 +83,5 @@ You can find a simple solution in utils/argv_fuzzing.
 ## Attacking a format that uses checksums? 
 
 Remove the checksum-checking code or use a postprocessor!
-See utils/custom_mutators/ for more.
+See `afl_custom_post_process` in custom_mutators/examples/example.c for more.