diff options
44 files changed, 1515 insertions, 326 deletions
diff --git a/.gitignore b/.gitignore index bb3c82eb..2ee40f62 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,4 @@ qemu_mode/qemu-3.1.0 qemu_mode/qemu-3.1.0.tar.xz unicorn_mode/unicorn unicorn_mode/unicorn-* +unicorn_mode/*.tar.gz diff --git a/Android.mk b/Android.mk new file mode 120000 index 00000000..33ceb8f0 --- /dev/null +++ b/Android.mk @@ -0,0 +1 @@ +Makefile \ No newline at end of file diff --git a/README.md b/README.md index dff6463b..14e1ae59 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,10 @@ afl++ is maintained by Marc Heuse <mh@mh-sec.de>, Heiko Eißfeldt <heiko.eissfeldt@hexco.de> and Andrea Fioraldi <andreafioraldi@gmail.com>. + Note that although afl now has a Google afl repository [https://github.com/Google/afl](https://github.com/Google/afl), + it is unlikely to receive any noteable enhancements: [https://twitter.com/Dor3s/status/1154737061787660288](https://twitter.com/Dor3s/status/1154737061787660288) + + ## The enhancements compared to the original stock afl Many improvements were made over the official afl release - which did not @@ -19,7 +23,8 @@ Among others afl++ has, e.g. more performant llvm_mode, supporting llvm up to version 9, Qemu 3.1, more speed and crashfixes for Qemu, - laf-intel feature for Qemu (with libcompcov) and more. + laf-intel feature for Qemu (with libcompcov), better *BSD and Android + support and more. Additionally the following patches have been integrated: @@ -50,7 +55,6 @@ ## 1) Challenges of guided fuzzing -------------------------------- Fuzzing is one of the most powerful and proven strategies for identifying security issues in real-world software; it is responsible for the vast @@ -173,7 +177,6 @@ file for important caveats. ## 4) Instrumenting binary-only apps ---------------------------------- When source code is *NOT* available, the fuzzer offers experimental support for fast, on-the-fly instrumentation of black-box binaries. This is accomplished @@ -201,7 +204,6 @@ A more comprehensive description of these and other options can be found in ## 5) Power schedules ------------------- The power schedules were copied from Marcel Böhme's excellent AFLfast implementation and expands on the ability to discover new paths and @@ -233,7 +235,6 @@ Computer and Communications Security (CCS'16): ## 6) Choosing initial test cases ------------------------------- To operate correctly, the fuzzer requires one or more starting file that contains a good example of the input data normally expected by the targeted @@ -255,7 +256,6 @@ exercise different code paths in the target binary. ## 7) Fuzzing binaries -------------------- The fuzzing process itself is carried out by the afl-fuzz utility. This program requires a read-only directory with initial test cases, a separate place to @@ -294,7 +294,6 @@ fuzzers - add the -d option to the command line. ## 8) Interpreting output ----------------------- See the [docs/status_screen.txt](docs/status_screen.txt) file for information on how to interpret the displayed stats and monitor the health of the process. Be @@ -356,7 +355,6 @@ see [http://lcamtuf.coredump.cx/afl/plot/](http://lcamtuf.coredump.cx/afl/plot/) ## 9) Parallelized fuzzing ------------------------ Every instance of afl-fuzz takes up roughly one core. This means that on multi-core systems, parallelization is necessary to fully utilize the hardware. @@ -369,7 +367,6 @@ last section of [docs/parallel_fuzzing.txt](docs/parallel_fuzzing.txt) for tips. ## 10) Fuzzer dictionaries ----------------------- By default, afl-fuzz mutation engine is optimized for compact data formats - say, images, multimedia, compressed data, regular expression syntax, or shell @@ -406,7 +403,6 @@ utility with AFL. For that, see [libtokencap/README.tokencap](libtokencap/README ## 11) Crash triage ----------------- The coverage-based grouping of crashes usually produces a small data set that can be quickly triaged manually or with a very simple GDB or Valgrind script. @@ -455,7 +451,6 @@ near the end of [docs/technical_details.txt](docs/technical_details.txt). ## 12) Going beyond crashes ------------------------- Fuzzing is a wonderful and underutilized technique for discovering non-crashing design and implementation errors, too. Quite a few interesting bugs have been @@ -480,7 +475,6 @@ shared with libfuzzer) or `#ifdef __AFL_COMPILER` (this one is just for AFL). ## 13) Common-sense risks ----------------------- Please keep in mind that, similarly to many other computationally-intensive tasks, fuzzing may put strain on your hardware and on the OS. In particular: @@ -511,7 +505,6 @@ tasks, fuzzing may put strain on your hardware and on the OS. In particular: ## 14) Known limitations & areas for improvement ---------------------------------------------- Here are some of the most important caveats for AFL: @@ -553,10 +546,9 @@ Beyond this, see INSTALL for platform-specific tips. ## 15) Special thanks ------------------- -Many of the improvements to the original afl wouldn't be possible without -feedback, bug reports, or patches from: +Many of the improvements to the original afl and afl++ wouldn't be possible +without feedback, bug reports, or patches from: ``` Jann Horn Hanno Boeck @@ -598,14 +590,15 @@ feedback, bug reports, or patches from: Rene Freingruber Sergey Davidoff Sami Liedes Craig Young Andrzej Jackowski Daniel Hodson - Nathan Voss Dominik Maier + Nathan Voss Dominik Maier + Andrea Biondo Vincent Le Garrec + Khaled Yakdan Kuang-che Wu ``` Thank you! ## 16) Contact ------------ Questions? Concerns? Bug reports? The contributors can be reached via [https://github.com/vanhauser-thc/AFLplusplus](https://github.com/vanhauser-thc/AFLplusplus) diff --git a/TODO b/TODO index 08dc1bb8..c2cf10a5 100644 --- a/TODO +++ b/TODO @@ -6,23 +6,13 @@ all: (vh: tried, the variable definion look very ugly then, what to do?) afl-fuzz: - - modularize: forkserver is in a module - others: - mutator - is deeply integrated and would loose performance if split - scheduler - is within this and as the values it operates on are afl - specific it does not make sense to seperate this - input - if we get different input vectors then this would make sense, - e.g. network (which we have seen is super non-performant and using - desock is much faster) - so for the moment we are done? (vh) - -docs/: - - update docs/sister_projects.txt - - doc + example for AFL_CUSTOM_MUTATOR_LIBRARY + - put mutator, scheduler, forkserver and input channels in individual files + - reuse forkserver for showmap, afl-cmin, etc. + - custom mutator lib: example and readme + - env var to exclusively run the custom lib/py mutator gcc_plugin: - needs to be rewritten - - fix crashes when compiling :( - whitelist support - skip over uninteresting blocks - laf-intel @@ -42,7 +32,8 @@ Problem: Average targets (tiff, jpeg, unrar) go through 1500 edges. At afl's default map that means ~16 collisions and ~3 wrappings. Solution #1: increase map size. every +1 decreases fuzzing speed by ~10% and halfs the collisions - birthday paradox predicts at collisions at this # of edges: + birthday paradox predicts collisions at this # of edges: + mapsize => collisions 2^16 = 302 2^17 = 427 2^18 = 603 @@ -52,10 +43,10 @@ Problem: Average targets (tiff, jpeg, unrar) go through 1500 edges. 2^22 = 2412 2^23 = 3411 2^24 = 4823 - Its an easy solution but also not a good one. + Increasing the map is an easy solution but also not a good one. Solution #2: use dynamic map size and collision free basic block IDs This only works in llvm_mode and llvm >= 9 though - A potential good future solution + A potential good future solution. Heiko/hexcoder follows this up Solution #3: write instruction pointers to a big shared map 512kb/1MB shared map and the instrumented code writes the instruction pointer into the map. Map must be big enough but could be command line @@ -64,9 +55,7 @@ Problem: Average targets (tiff, jpeg, unrar) go through 1500 edges. impacts speed, but this can be decided by user options Neutral: a little bit slower but no loss of coverage Bad: completely changes how afl uses the map and the scheduling. - Overall another very good solution + Overall another very good solution, Marc Heuse/vanHauser follows this up qemu_mode: - persistent mode patching the return address (WinAFL style) - - instrument only comparison with immediate values by default when using compcov (done) - diff --git a/afl-system-config b/afl-system-config index 366762ef..28793c5b 100755 --- a/afl-system-config +++ b/afl-system-config @@ -1,9 +1,11 @@ #!/bin/sh +PLATFORM=`uname -s` echo This reconfigures the system to have a better fuzzing performance if [ '!' "$EUID" = 0 ] && [ '!' `id -u` = 0 ] ; then echo Error you need to be root to run this exit 1 fi +if [ "$PLATFORM" = "Linux" ] ; then sysctl -w kernel.core_pattern=core sysctl -w kernel.randomize_va_space=0 sysctl -w kernel.sched_child_runs_first=1 @@ -19,5 +21,19 @@ test -e /sys/devices/system/cpu/cpufreq/boost && echo 1 > /sys/devices/system/cp echo echo It is recommended to boot the kernel with lots of security off - if you are running a machine that is in a secured network - so set this: echo '/etc/default/grub:GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"' +fi +if [ "$PLATFORM" = "FreeBSD" ] ; then +sysctl kern.elf32.aslr.enable=0 +sysctl kern.elf64.aslr.enable=0 +echo +echo It is recommended to boot the kernel with lots of security off - if you are running a machine that is in a secured network - so set this: +echo 'sysctl hw.ibrs_disable=1' +echo +echo 'Setting kern.pmap.pg_ps_enabled=0 into /boot/loader.conf might be helpful too.' +fi +if [ "$PLATFORM" = "OpenBSD" ] ; then +echo +echo 'System security features cannot be disabled on OpenBSD.' +fi echo echo Also use AFL_TMPDIR to use a tmpfs for the input file diff --git a/afl-whatsup b/afl-whatsup index a4d30418..c1e41529 100755 --- a/afl-whatsup +++ b/afl-whatsup @@ -54,7 +54,7 @@ fi CUR_TIME=`date +%s` -TMP=`mktemp -t .afl-whatsup-XXXXXXXX` || exit 1 +TMP=`mktemp -t .afl-whatsup-XXXXXXXX` || TMP=`mktemp -p /data/local/tmp .afl-whatsup-XXXXXXXX` || exit 1 ALIVE_CNT=0 DEAD_CNT=0 diff --git a/android-ashmem.h b/android-ashmem.h new file mode 100644 index 00000000..a787c04b --- /dev/null +++ b/android-ashmem.h @@ -0,0 +1,81 @@ +#ifndef _ANDROID_ASHMEM_H +#define _ANDROID_ASHMEM_H + +#include <fcntl.h> +#include <linux/shm.h> +#include <linux/ashmem.h> +#include <sys/ioctl.h> +#include <sys/mman.h> + +#if __ANDROID_API__ >= 26 +#define shmat bionic_shmat +#define shmctl bionic_shmctl +#define shmdt bionic_shmdt +#define shmget bionic_shmget +#endif + #include <sys/shm.h> +#undef shmat +#undef shmctl +#undef shmdt +#undef shmget +#include <stdio.h> + +#define ASHMEM_DEVICE "/dev/ashmem" + +static inline int shmctl(int __shmid, int __cmd, struct shmid_ds *__buf) +{ + int ret = 0; + if (__cmd == IPC_RMID) { + int length = ioctl(__shmid, ASHMEM_GET_SIZE, NULL); + struct ashmem_pin pin = {0, length}; + ret = ioctl(__shmid, ASHMEM_UNPIN, &pin); + close(__shmid); + } + + return ret; +} + +static inline int shmget (key_t __key, size_t __size, int __shmflg) +{ + int fd,ret; + char ourkey[11]; + + fd = open(ASHMEM_DEVICE, O_RDWR); + if (fd < 0) + return fd; + + sprintf(ourkey,"%d",__key); + ret = ioctl(fd, ASHMEM_SET_NAME, ourkey); + if (ret < 0) + goto error; + + ret = ioctl(fd, ASHMEM_SET_SIZE, __size); + if (ret < 0) + goto error; + + return fd; + +error: + close(fd); + return ret; +} + +static inline void *shmat (int __shmid, const void *__shmaddr, int __shmflg) +{ + int size; + void *ptr; + + size = ioctl(__shmid, ASHMEM_GET_SIZE, NULL); + if (size < 0) { + return NULL; + } + + ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, __shmid, 0); + if (ptr == MAP_FAILED) { + return NULL; + } + + return ptr; +} + +#endif diff --git a/config.h b/config.h index 046ab52a..29c33d46 120000..100644 --- a/config.h +++ b/config.h @@ -1 +1,363 @@ -include/config.h \ No newline at end of file +/* + american fuzzy lop plus plus - vaguely configurable bits + ---------------------------------------------- + + Written and maintained by Michal Zalewski <lcamtuf@google.com> + + Copyright 2013, 2014, 2015, 2016 Google Inc. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + */ + +#ifndef _HAVE_CONFIG_H +#define _HAVE_CONFIG_H + +#include "types.h" + +/* Version string: */ + +#define VERSION "++2.53d" // c = release, d = volatile github dev + +/****************************************************** + * * + * Settings that may be of interest to power users: * + * * + ******************************************************/ + +/* Comment out to disable terminal colors (note that this makes afl-analyze + a lot less nice): */ + +#define USE_COLOR + +/* Comment out to disable fancy ANSI boxes and use poor man's 7-bit UI: */ + +#define FANCY_BOXES + +/* Default timeout for fuzzed code (milliseconds). This is the upper bound, + also used for detecting hangs; the actual value is auto-scaled: */ + +#define EXEC_TIMEOUT 1000 + +/* Timeout rounding factor when auto-scaling (milliseconds): */ + +#define EXEC_TM_ROUND 20 + +/* Default memory limit for child process (MB): */ + +#ifndef __x86_64__ +# define MEM_LIMIT 25 +#else +# define MEM_LIMIT 50 +#endif /* ^!__x86_64__ */ + +/* Default memory limit when running in QEMU mode (MB): */ + +#define MEM_LIMIT_QEMU 200 + +/* Default memory limit when running in Unicorn mode (MB): */ + +#define MEM_LIMIT_UNICORN 200 + +/* Number of calibration cycles per every new test case (and for test + cases that show variable behavior): */ + +#define CAL_CYCLES 8 +#define CAL_CYCLES_LONG 40 + +/* Number of subsequent timeouts before abandoning an input file: */ + +#define TMOUT_LIMIT 250 + +/* Maximum number of unique hangs or crashes to record: */ + +#define KEEP_UNIQUE_HANG 500 +#define KEEP_UNIQUE_CRASH 5000 + +/* Baseline number of random tweaks during a single 'havoc' stage: */ + +#define HAVOC_CYCLES 256 +#define HAVOC_CYCLES_INIT 1024 + +/* Maximum multiplier for the above (should be a power of two, beware + of 32-bit int overflows): */ + +#define HAVOC_MAX_MULT 16 +#define HAVOC_MAX_MULT_MOPT 32 + +/* Absolute minimum number of havoc cycles (after all adjustments): */ + +#define HAVOC_MIN 16 + +/* Power Schedule Divisor */ +#define POWER_BETA 1 +#define MAX_FACTOR (POWER_BETA * 32) + +/* Maximum stacking for havoc-stage tweaks. The actual value is calculated + like this: + + n = random between 1 and HAVOC_STACK_POW2 + stacking = 2^n + + In other words, the default (n = 7) produces 2, 4, 8, 16, 32, 64, or + 128 stacked tweaks: */ + +#define HAVOC_STACK_POW2 7 + +/* Caps on block sizes for cloning and deletion operations. Each of these + ranges has a 33% probability of getting picked, except for the first + two cycles where smaller blocks are favored: */ + +#define HAVOC_BLK_SMALL 32 +#define HAVOC_BLK_MEDIUM 128 +#define HAVOC_BLK_LARGE 1500 + +/* Extra-large blocks, selected very rarely (<5% of the time): */ + +#define HAVOC_BLK_XL 32768 + +/* Probabilities of skipping non-favored entries in the queue, expressed as + percentages: */ + +#define SKIP_TO_NEW_PROB 99 /* ...when there are new, pending favorites */ +#define SKIP_NFAV_OLD_PROB 95 /* ...no new favs, cur entry already fuzzed */ +#define SKIP_NFAV_NEW_PROB 75 /* ...no new favs, cur entry not fuzzed yet */ + +/* Splicing cycle count: */ + +#define SPLICE_CYCLES 15 + +/* Nominal per-splice havoc cycle length: */ + +#define SPLICE_HAVOC 32 + +/* Maximum offset for integer addition / subtraction stages: */ + +#define ARITH_MAX 35 + +/* Limits for the test case trimmer. The absolute minimum chunk size; and + the starting and ending divisors for chopping up the input file: */ + +#define TRIM_MIN_BYTES 4 +#define TRIM_START_STEPS 16 +#define TRIM_END_STEPS 1024 + +/* Maximum size of input file, in bytes (keep under 100MB): */ + +#define MAX_FILE (1 * 1024 * 1024) + +/* The same, for the test case minimizer: */ + +#define TMIN_MAX_FILE (10 * 1024 * 1024) + +/* Block normalization steps for afl-tmin: */ + +#define TMIN_SET_MIN_SIZE 4 +#define TMIN_SET_STEPS 128 + +/* Maximum dictionary token size (-x), in bytes: */ + +#define MAX_DICT_FILE 128 + +/* Length limits for auto-detected dictionary tokens: */ + +#define MIN_AUTO_EXTRA 3 +#define MAX_AUTO_EXTRA 32 + +/* Maximum number of user-specified dictionary tokens to use in deterministic + steps; past this point, the "extras/user" step will be still carried out, + but with proportionally lower odds: */ + +#define MAX_DET_EXTRAS 200 + +/* Maximum number of auto-extracted dictionary tokens to actually use in fuzzing + (first value), and to keep in memory as candidates. The latter should be much + higher than the former. */ + +#define USE_AUTO_EXTRAS 50 +#define MAX_AUTO_EXTRAS (USE_AUTO_EXTRAS * 10) + +/* Scaling factor for the effector map used to skip some of the more + expensive deterministic steps. The actual divisor is set to + 2^EFF_MAP_SCALE2 bytes: */ + +#define EFF_MAP_SCALE2 3 + +/* Minimum input file length at which the effector logic kicks in: */ + +#define EFF_MIN_LEN 128 + +/* Maximum effector density past which everything is just fuzzed + unconditionally (%): */ + +#define EFF_MAX_PERC 90 + +/* UI refresh frequency (Hz): */ + +#define UI_TARGET_HZ 5 + +/* Fuzzer stats file and plot update intervals (sec): */ + +#define STATS_UPDATE_SEC 60 +#define PLOT_UPDATE_SEC 5 + +/* Smoothing divisor for CPU load and exec speed stats (1 - no smoothing). */ + +#define AVG_SMOOTHING 16 + +/* Sync interval (every n havoc cycles): */ + +#define SYNC_INTERVAL 5 + +/* Output directory reuse grace period (minutes): */ + +#define OUTPUT_GRACE 25 + +/* Uncomment to use simple file names (id_NNNNNN): */ + +// #define SIMPLE_FILES + +/* List of interesting values to use in fuzzing. */ + +#define INTERESTING_8 \ + -128, /* Overflow signed 8-bit when decremented */ \ + -1, /* */ \ + 0, /* */ \ + 1, /* */ \ + 16, /* One-off with common buffer size */ \ + 32, /* One-off with common buffer size */ \ + 64, /* One-off with common buffer size */ \ + 100, /* One-off with common buffer size */ \ + 127 /* Overflow signed 8-bit when incremented */ + +#define INTERESTING_16 \ + -32768, /* Overflow signed 16-bit when decremented */ \ + -129, /* Overflow signed 8-bit */ \ + 128, /* Overflow signed 8-bit */ \ + 255, /* Overflow unsig 8-bit when incremented */ \ + 256, /* Overflow unsig 8-bit */ \ + 512, /* One-off with common buffer size */ \ + 1000, /* One-off with common buffer size */ \ + 1024, /* One-off with common buffer size */ \ + 4096, /* One-off with common buffer size */ \ + 32767 /* Overflow signed 16-bit when incremented */ + +#define INTERESTING_32 \ + -2147483648LL, /* Overflow signed 32-bit when decremented */ \ + -100663046, /* Large negative number (endian-agnostic) */ \ + -32769, /* Overflow signed 16-bit */ \ + 32768, /* Overflow signed 16-bit */ \ + 65535, /* Overflow unsig 16-bit when incremented */ \ + 65536, /* Overflow unsig 16 bit */ \ + 100663045, /* Large positive number (endian-agnostic) */ \ + 2147483647 /* Overflow signed 32-bit when incremented */ + +/*********************************************************** + * * + * Really exotic stuff you probably don't want to touch: * + * * + ***********************************************************/ + +/* Call count interval between reseeding the libc PRNG from /dev/urandom: */ + +#define RESEED_RNG 10000 + +/* Maximum line length passed from GCC to 'as' and used for parsing + configuration files: */ + +#define MAX_LINE 8192 + +/* Environment variable used to pass SHM ID to the called program. */ + +#define SHM_ENV_VAR "__AFL_SHM_ID" + +/* Other less interesting, internal-only variables. */ + +#define CLANG_ENV_VAR "__AFL_CLANG_MODE" +#define AS_LOOP_ENV_VAR "__AFL_AS_LOOPCHECK" +#define PERSIST_ENV_VAR "__AFL_PERSISTENT" +#define DEFER_ENV_VAR "__AFL_DEFER_FORKSRV" + +/* In-code signatures for deferred and persistent mode. */ + +#define PERSIST_SIG "##SIG_AFL_PERSISTENT##" +#define DEFER_SIG "##SIG_AFL_DEFER_FORKSRV##" + +/* Distinctive bitmap signature used to indicate failed execution: */ + +#define EXEC_FAIL_SIG 0xfee1dead + +/* Distinctive exit code used to indicate MSAN trip condition: */ + +#define MSAN_ERROR 86 + +/* Designated file descriptors for forkserver commands (the application will + use FORKSRV_FD and FORKSRV_FD + 1): */ + +#define FORKSRV_FD 198 + +/* Fork server init timeout multiplier: we'll wait the user-selected + timeout plus this much for the fork server to spin up. */ + +#define FORK_WAIT_MULT 10 + +/* Calibration timeout adjustments, to be a bit more generous when resuming + fuzzing sessions or trying to calibrate already-added internal finds. + The first value is a percentage, the other is in milliseconds: */ + +#define CAL_TMOUT_PERC 125 +#define CAL_TMOUT_ADD 50 + +/* Number of chances to calibrate a case before giving up: */ + +#define CAL_CHANCES 3 + +/* Map size for the traced binary (2^MAP_SIZE_POW2). Must be greater than + 2; you probably want to keep it under 18 or so for performance reasons + (adjusting AFL_INST_RATIO when compiling is probably a better way to solve + problems with complex programs). You need to recompile the target binary + after changing this - otherwise, SEGVs may ensue. */ + +#define MAP_SIZE_POW2 16 +#define MAP_SIZE (1 << MAP_SIZE_POW2) + +/* Maximum allocator request size (keep well under INT_MAX): */ + +#define MAX_ALLOC 0x40000000 + +/* A made-up hashing seed: */ + +#define HASH_CONST 0xa5b35705 + +/* Constants for afl-gotcpu to control busy loop timing: */ + +#define CTEST_TARGET_MS 5000 +#define CTEST_CORE_TRG_MS 1000 +#define CTEST_BUSY_CYCLES (10 * 1000 * 1000) + +/* Enable NeverZero counters in QEMU mode */ + +#define AFL_QEMU_NOT_ZERO + +/* Uncomment this to use inferior block-coverage-based instrumentation. Note + that you need to recompile the target binary for this to have any effect: */ + +// #define COVERAGE_ONLY + +/* Uncomment this to ignore hit counts and output just one bit per tuple. + As with the previous setting, you will need to recompile the target + binary: */ + +// #define SKIP_COUNTS + +/* Uncomment this to use instrumentation data to record newly discovered paths, + but do not use them as seeds for fuzzing. This is useful for conveniently + measuring coverage that could be attained by a "dumb" fuzzing algorithm: */ + +// #define IGNORE_FINDS + +#endif /* ! _HAVE_CONFIG_H */ diff --git a/docs/ChangeLog b/docs/ChangeLog index ed8e0022..782320d6 100644 --- a/docs/ChangeLog +++ b/docs/ChangeLog @@ -18,6 +18,7 @@ Version ++2.53d (dev): ---------------------- - llvm 9 is now supported (still needs testing) + - Android is now supported (thank to JoeyJiao!) - still need to modify the Makefile though - fix building qemu on some Ubuntus (thanks to floyd!) - custom mutator by a loaded library is now supported (thanks to kyakdan!) - more support for *BSD (thanks to devnexen!) @@ -27,6 +28,11 @@ Version ++2.53d (dev): afl never did), plus shows tuple content summary information now - the forkserver is now in its own C file to be easily integratable - small docu updates + - NeverZero counters for QEMU + - NeverZero counters for Unicorn + - CompareCoverage Unicorn + - Immediates-only instrumentation for CompareCoverage + - ... your patch? :) -------------------------- diff --git a/docs/env_variables.txt b/docs/env_variables.txt index 93066dbc..821463ae 100644 --- a/docs/env_variables.txt +++ b/docs/env_variables.txt @@ -245,9 +245,19 @@ The QEMU wrapper used to instrument binary-only code supports several settings: - Setting AFL_INST_LIBS causes the translator to also instrument the code inside any dynamically linked libraries (notably including glibc). + - Setting AFL_COMPCOV_LEVEL enables the CompareCoverage tracing of all cmp + and sub in x86 and x86_64 and memory comparions functions (e.g. strcmp, + memcmp, ...) when libcompcov is preloaded using AFL_PRELOAD. + More info at qemu_mode/libcompcov/README.compcov. + There are two levels at the moment, AFL_COMPCOV_LEVEL=1 that instruments + only comparisons with immediate values / read-only memory and + AFL_COMPCOV_LEVEL=2 that instruments all the comparions. Level 2 is more + accurate but may need a larger shared memory. + - Setting AFL_QEMU_COMPCOV enables the CompareCoverage tracing of all - cmp and sub in x86 and x86_64. Support for other architectures and - comparison functions (mem/strcmp et al.) is planned. + cmp and sub in x86 and x86_64. + This is an alias of AFL_COMPCOV_LEVEL=1 when AFL_COMPCOV_LEVEL is + not specified. - The underlying QEMU binary will recognize any standard "user space emulation" variables (e.g., QEMU_STACK_SIZE), but there should be no @@ -260,10 +270,7 @@ The QEMU wrapper used to instrument binary-only code supports several settings: - AFL_ENTRYPOINT allows you to specify a specific entrypoint into the binary (this can be very good for the performance!). The entrypoint is specified as hex address, e.g. 0x4004110 - - - AFL_QEMU_COMPCOV is for a sub-project in qemu_mode called ./libcompcov - which implements laf-intel for qemu. It also needs AFL_PRELOAD and - you can find more information in qemu_mode/libcompcov/README.compcov + Note that the address must be the address of a basic block. 5) Settings for afl-cmin ------------------------ diff --git a/docs/unicorn_mode.txt b/docs/unicorn_mode.txt deleted file mode 100644 index b691fff8..00000000 --- a/docs/unicorn_mode.txt +++ /dev/null @@ -1,109 +0,0 @@ -========================================================= -Unicorn-based binary-only instrumentation for afl-fuzz -========================================================= - -1) Introduction ---------------- - -The code in ./unicorn_mode allows you to build a standalone feature that -leverages the Unicorn Engine and allows callers to obtain instrumentation -output for black-box, closed-source binary code snippets. This mechanism -can be then used by afl-fuzz to stress-test targets that couldn't be built -with afl-gcc or used in QEMU mode, or with other extensions such as -TriforceAFL. - -There is a significant performance penalty compared to native AFL, -but at least we're able to use AFL on these binaries, right? - -The idea and much of the implementation comes from Nathan Voss <njvoss299@gmail.com>. - -2) How to use -------------- - -Requirements: you need an installed python2 environment. - -*** Building AFL's Unicorn Mode *** - -First, make afl as usual. -Once that completes successfully you need to build and add in the Unicorn Mode -features: - - $ cd unicorn_mode - $ ./build_unicorn_support.sh - -NOTE: This script downloads a recent Unicorn Engine commit that has been tested -and is stable-ish from the Unicorn github page. If you are offline, you'll need -to hack up this script a little bit and supply your own copy of Unicorn's latest -stable release. It's not very hard, just check out the beginning of the -build_unicorn_support.sh script and adjust as necessary. - -Building Unicorn will take a little bit (~5-10 minutes). Once it completes -it automatically compiles a sample application and verify that it works. - -*** Fuzzing with Unicorn Mode *** - -To really use unicorn-mode effectively you need to prepare the following: - - * Relevant binary code to be fuzzed - * Knowledge of the memory map and good starting state - * Folder containing sample inputs to start fuzzing with - - Same ideas as any other AFL inputs - - Quality/speed of results will depend greatly on quality of starting - samples - - See AFL's guidance on how to create a sample corpus - * Unicorn-based test harness which: - - Adds memory map regions - - Loads binary code into memory - - Emulates at least one instruction* - - Yeah, this is lame. See 'Gotchas' section below for more info - - Loads and verifies data to fuzz from a command-line specified file - - AFL will provide mutated inputs by changing the file passed to - the test harness - - Presumably the data to be fuzzed is at a fixed buffer address - - If input constraints (size, invalid bytes, etc.) are known they - should be checked after the file is loaded. If a constraint - fails, just exit the test harness. AFL will treat the input as - 'uninteresting' and move on. - - Sets up registers and memory state for beginning of test - - Emulates the interested code from beginning to end - - If a crash is detected, the test harness must 'crash' by - throwing a signal (SIGSEGV, SIGKILL, SIGABORT, etc.) - -Once you have all those things ready to go you just need to run afl-fuzz in -'unicorn-mode' by passing in the '-U' flag: - - $ afl-fuzz -U -m none -i /path/to/inputs -o /path/to/results -- ./test_harness @@ - -The normal afl-fuzz command line format applies to everything here. Refer to -AFL's main documentation for more info about how to use afl-fuzz effectively. - -For a much clearer vision of what all of this looks like, please refer to the -sample provided in the 'unicorn_mode/samples' directory. There is also a blog -post that goes over the basics at: - -https://medium.com/@njvoss299/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf - -The 'helper_scripts' directory also contains several helper scripts that allow you -to dump context from a running process, load it, and hook heap allocations. For details -on how to use this check out the follow-up blog post to the one linked above. - -A example use of AFL-Unicorn mode is discussed in the Paper Unicorefuzz: -https://www.usenix.org/conference/woot19/presentation/maier - -3) Gotchas, feedback, bugs --------------------------- - -To make sure that AFL's fork server starts up correctly the Unicorn test -harness script must emulate at least one instruction before loading the -data that will be fuzzed from the input file. It doesn't matter what the -instruction is, nor if it is valid. This is an artifact of how the fork-server -is started and could likely be fixed with some clever re-arranging of the -patches applied to Unicorn. - -Running the build script builds Unicorn and its python bindings and installs -them on your system. This installation will supersede any existing Unicorn -installation with the patched afl-unicorn version. - -Refer to the unicorn_mode/samples/arm_example/arm_tester.c for an example -of how to do this properly! If you don't get this right, AFL will not -load any mutated inputs and your fuzzing will be useless! diff --git a/llvm_mode/afl-llvm-pass.so.cc b/llvm_mode/afl-llvm-pass.so.cc index 3fe7f83e..b242163e 100644 --- a/llvm_mode/afl-llvm-pass.so.cc +++ b/llvm_mode/afl-llvm-pass.so.cc @@ -129,9 +129,14 @@ bool AFLCoverage::runOnModule(Module &M) { new GlobalVariable(M, PointerType::get(Int8Ty, 0), false, GlobalValue::ExternalLinkage, 0, "__afl_area_ptr"); +#ifdef __ANDROID__ + GlobalVariable *AFLPrevLoc = new GlobalVariable( + M, Int32Ty, false, GlobalValue::ExternalLinkage, 0, "__afl_prev_loc"); +#else GlobalVariable *AFLPrevLoc = new GlobalVariable( M, Int32Ty, false, GlobalValue::ExternalLinkage, 0, "__afl_prev_loc", 0, GlobalVariable::GeneralDynamicTLSModel, 0, false); +#endif /* Instrument all the things! */ diff --git a/llvm_mode/afl-llvm-rt.o.c b/llvm_mode/afl-llvm-rt.o.c index 1564ae9d..e6d9b993 100644 --- a/llvm_mode/afl-llvm-rt.o.c +++ b/llvm_mode/afl-llvm-rt.o.c @@ -19,6 +19,9 @@ */ +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include "config.h" #include "types.h" @@ -55,7 +58,11 @@ u8 __afl_area_initial[MAP_SIZE]; u8* __afl_area_ptr = __afl_area_initial; +#ifdef __ANDROID__ +u32 __afl_prev_loc; +#else __thread u32 __afl_prev_loc; +#endif /* Running in persistent mode? */ diff --git a/qemu_mode/README.qemu b/qemu_mode/README.md index 754c0259..610f6860 100644 --- a/qemu_mode/README.qemu +++ b/qemu_mode/README.md @@ -1,11 +1,8 @@ -========================================================= -High-performance binary-only instrumentation for afl-fuzz -========================================================= +# High-performance binary-only instrumentation for afl-fuzz (See ../docs/README for the general instruction manual.) -1) Introduction ---------------- +## 1) Introduction The code in this directory allows you to build a standalone feature that leverages the QEMU "user emulation" mode and allows callers to obtain @@ -16,14 +13,15 @@ with afl-gcc. The usual performance cost is 2-5x, which is considerably better than seen so far in experiments with tools such as DynamoRIO and PIN. -The idea and much of the implementation comes from Andrew Griffiths. +The idea and much of the initial implementation comes from Andrew Griffiths. +The actual implementation on QEMU 3 (shipped with afl++) is from +Andrea Fioraldi. Special thanks to abiondo that re-enabled TCG chaining. -2) How to use -------------- +## 2) How to use -The feature is implemented with a fairly simple patch to QEMU 2.10.0. The -simplest way to build it is to run ./build_qemu_support.sh. The script will -download, configure, and compile the QEMU binary for you. +The feature is implemented with a patch to QEMU 3.1.0. The simplest way +to build it is to run ./build_qemu_support.sh. The script will download, +configure, and compile the QEMU binary for you. QEMU is a big project, so this will take a while, and you may have to resolve a couple of dependencies (most notably, you will definitely need @@ -46,20 +44,26 @@ Note: if you want the QEMU helper to be installed on your system for all users, you need to build it before issuing 'make install' in the parent directory. -3) Options ----------- +## 3) Options There is ./libcompcov/ which implements laf-intel (splitting memcmp, strncmp, etc. to make these conditions easier solvable by afl-fuzz). Highly recommended. +The option that enables QEMU CompareCoverage is AFL_COMPCOV_LEVEL. +AFL_COMPCOV_LEVEL=1 is to instrument comparisons with only immediate +values / read-only memory. AFL_COMPCOV_LEVEL=2 instruments all +comparison instructions and memory comparison functions when libcompcov +is preloaded. Comparison instructions are currently instrumented only +on the x86 and x86_64 targets. + Another option is the environment variable AFL_ENTRYPOINT which allows move the forkserver to a different part, e.g. just before the file is opened (e.g. way after command line parsing and config file loading, etc) -which can be a huge speed improvement. +which can be a huge speed improvement. Note that the specified address +must be an address of a basic block. -4) Notes on linking -------------------- +## 4) Notes on linking The feature is supported only on Linux. Supporting BSD may amount to porting the changes made to linux-user/elfload.c and applying them to @@ -80,8 +84,7 @@ practice, this means two things: Setting AFL_INST_LIBS=1 can be used to circumvent the .text detection logic and instrument every basic block encountered. -5) Benchmarking ---------------- +## 5) Benchmarking If you want to compare the performance of the QEMU instrumentation with that of afl-gcc compiled code against the same target, you need to build the @@ -96,8 +99,7 @@ Comparative measurements of execution speed or instrumentation coverage will be fairly meaningless if the optimization levels or instrumentation scopes don't match. -6) Gotchas, feedback, bugs --------------------------- +## 6) Gotchas, feedback, bugs If you need to fix up checksums or do other cleanup on mutated test cases, see experimental/post_library/ for a viable solution. @@ -118,8 +120,7 @@ with -march=core2, can help. Beyond that, this is an early-stage mechanism, so fields reports are welcome. You can send them to <afl-users@googlegroups.com>. -7) Alternatives: static rewriting ---------------------------------- +## 7) Alternatives: static rewriting Statically rewriting binaries just once, instead of attempting to translate them at run time, can be a faster alternative. That said, static rewriting is diff --git a/qemu_mode/build_qemu_support.sh b/qemu_mode/build_qemu_support.sh index 78ad5680..35f5b8ca 100755 --- a/qemu_mode/build_qemu_support.sh +++ b/qemu_mode/build_qemu_support.sh @@ -3,10 +3,17 @@ # american fuzzy lop - QEMU build script # -------------------------------------- # -# Written by Andrew Griffiths <agriffiths@google.com> and -# Michal Zalewski <lcamtuf@google.com> +# Originally written by Andrew Griffiths <agriffiths@google.com> and +# Michal Zalewski <lcamtuf@google.com> +# +# TCG instrumentation and block chaining support by Andrea Biondo +# <andrea.biondo965@gmail.com> +# +# QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero +# counters by Andrea Fioraldi <andreafioraldi@gmail.com> # # Copyright 2015, 2016, 2017 Google Inc. All rights reserved. +# Copyright 2019 AFLplusplus Project. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. diff --git a/qemu_mode/libcompcov/README.compcov b/qemu_mode/libcompcov/README.compcov index 2a4a0ee5..9be13d88 100644 --- a/qemu_mode/libcompcov/README.compcov +++ b/qemu_mode/libcompcov/README.compcov @@ -18,15 +18,19 @@ For optimized binaries this is an issue, those functions are often inlined and this module is not capable to log the coverage in this case. If you have the source code of the fuzzing target you should nto use this -library and QEMU but build ot with afl-clang-fast and the laf-intel options. +library and QEMU but build it with afl-clang-fast and the laf-intel options. To use this library make sure to preload it with AFL_PRELOAD. export AFL_PRELOAD=/path/to/libcompcov.so - export AFL_QEMU_COMPCOV=1 + export AFL_COMPCOV_LEVEL=1 afl-fuzz -Q -i input -o output <your options> -- <target args> +The AFL_COMPCOV_LEVEL tells to QEMU and libcompcov how to log comaprisons. +Level 1 logs just comparison with immediates / read-only memory and level 2 +logs all the comparisons. + The library make use of https://github.com/ouadev/proc_maps_parser and so it is Linux specific. However this is not a strict dependency, other UNIX operating systems can be supported simply replacing the code related to the diff --git a/qemu_mode/libcompcov/libcompcov.so.c b/qemu_mode/libcompcov/libcompcov.so.c index 44045d39..9e44067e 100644 --- a/qemu_mode/libcompcov/libcompcov.so.c +++ b/qemu_mode/libcompcov/libcompcov.so.c @@ -5,7 +5,7 @@ Written and maintained by Andrea Fioraldi <andreafioraldi@gmail.com> - Copyright 2019 Andrea Fioraldi. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -45,6 +45,8 @@ static void *__compcov_code_start, static u8 *__compcov_afl_map; +static u32 __compcov_level; + static int (*__libc_strcmp)(const char*, const char*); static int (*__libc_strncmp)(const char*, const char*, size_t); static int (*__libc_strcasecmp)(const char*, const char*); @@ -54,6 +56,28 @@ static int (*__libc_memcmp)(const void*, const void*, size_t); static int debug_fd = -1; +#define MAX_MAPPINGS 1024 + +static struct mapping { + void *st, *en; +} __compcov_ro[MAX_MAPPINGS]; + +static u32 __compcov_ro_cnt; + + +/* Check an address against the list of read-only mappings. */ + +static u8 __compcov_is_ro(const void* ptr) { + + u32 i; + + for (i = 0; i < __compcov_ro_cnt; i++) + if (ptr >= __compcov_ro[i].st && ptr <= __compcov_ro[i].en) return 1; + + return 0; +} + + static size_t __strlen2(const char *s1, const char *s2, size_t max_length) { // from https://github.com/googleprojectzero/CompareCoverage @@ -71,6 +95,15 @@ static void __compcov_load(void) { __libc_strcasecmp = dlsym(RTLD_NEXT, "strcasecmp"); __libc_strncasecmp = dlsym(RTLD_NEXT, "strncasecmp"); __libc_memcmp = dlsym(RTLD_NEXT, "memcmp"); + + if (getenv("AFL_QEMU_COMPCOV")) { + + __compcov_level = 1; + } + if (getenv("AFL_COMPCOV_LEVEL")) { + + __compcov_level = atoi(getenv("AFL_COMPCOV_LEVEL")); + } char *id_str = getenv(SHM_ENV_VAR); int shm_id; @@ -110,6 +143,12 @@ static void __compcov_load(void) { __compcov_code_end = maps_tmp->addr_end; } } + + if ((maps_tmp->is_w && !maps_tmp->is_r) || __compcov_ro_cnt == MAX_MAPPINGS) + continue; + + __compcov_ro[__compcov_ro_cnt].st = maps_tmp->addr_start; + __compcov_ro[__compcov_ro_cnt].en = maps_tmp->addr_end; } pmparser_free(maps); @@ -149,7 +188,8 @@ int strcmp(const char* str1, const char* str2) { void* retaddr = __builtin_return_address(0); - if (__compcov_is_in_bound(retaddr)) { + if (__compcov_is_in_bound(retaddr) && !(__compcov_level < 2 && + !__compcov_is_ro(str1) && !__compcov_is_ro(str2))) { size_t n = __strlen2(str1, str2, MAX_CMP_LENGTH +1); @@ -173,7 +213,8 @@ int strncmp(const char* str1, const char* str2, size_t len) { void* retaddr = __builtin_return_address(0); - if (__compcov_is_in_bound(retaddr)) { + if (__compcov_is_in_bound(retaddr) && !(__compcov_level < 2 && + !__compcov_is_ro(str1) && !__compcov_is_ro(str2))) { size_t n = __strlen2(str1, str2, MAX_CMP_LENGTH +1); n = MIN(n, len); @@ -198,7 +239,8 @@ int strcasecmp(const char* str1, const char* str2) { void* retaddr = __builtin_return_address(0); - if (__compcov_is_in_bound(retaddr)) { + if (__compcov_is_in_bound(retaddr) && !(__compcov_level < 2 && + !__compcov_is_ro(str1) && !__compcov_is_ro(str2))) { /* Fallback to strcmp, maybe improve in future */ size_t n = __strlen2(str1, str2, MAX_CMP_LENGTH +1); @@ -223,7 +265,8 @@ int strncasecmp(const char* str1, const char* str2, size_t len) { void* retaddr = __builtin_return_address(0); - if (__compcov_is_in_bound(retaddr)) { + if (__compcov_is_in_bound(retaddr) && !(__compcov_level < 2 && + !__compcov_is_ro(str1) && !__compcov_is_ro(str2))) { /* Fallback to strncmp, maybe improve in future */ size_t n = __strlen2(str1, str2, MAX_CMP_LENGTH +1); @@ -249,7 +292,8 @@ int memcmp(const void* mem1, const void* mem2, size_t len) { void* retaddr = __builtin_return_address(0); - if (__compcov_is_in_bound(retaddr)) { + if (__compcov_is_in_bound(retaddr) && !(__compcov_level < 2 && + !__compcov_is_ro(mem1) && !__compcov_is_ro(mem2))) { size_t n = len; diff --git a/qemu_mode/patches/afl-qemu-common.h b/qemu_mode/patches/afl-qemu-common.h new file mode 100644 index 00000000..c475cb58 --- /dev/null +++ b/qemu_mode/patches/afl-qemu-common.h @@ -0,0 +1,51 @@ +/* + american fuzzy lop++ - high-performance binary-only instrumentation + ------------------------------------------------------------------- + + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> + + TCG instrumentation and block chaining support by Andrea Biondo + <andrea.biondo965@gmail.com> + + QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero + counters by Andrea Fioraldi <andreafioraldi@gmail.com> + + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + This code is a shim patched into the separately-distributed source + code of QEMU 3.1.0. It leverages the built-in QEMU tracing functionality + to implement AFL-style instrumentation and to take care of the remaining + parts of the AFL fork server logic. + + The resulting QEMU binary is essentially a standalone instrumentation + tool; for an example of how to leverage it for other purposes, you can + have a look at afl-showmap.c. + + */ + +#include "../../config.h" + +/* NeverZero */ + +#if (defined(__x86_64__) || defined(__i386__)) && defined(AFL_QEMU_NOT_ZERO) +# define INC_AFL_AREA(loc) \ + asm volatile ( \ + "incb (%0, %1, 1)\n" \ + "adcb $0, (%0, %1, 1)\n" \ + : /* no out */ \ + : "r" (afl_area_ptr), "r" (loc) \ + : "memory", "eax" \ + ) +#else +# define INC_AFL_AREA(loc) \ + afl_area_ptr[loc]++ +#endif + diff --git a/qemu_mode/patches/afl-qemu-cpu-inl.h b/qemu_mode/patches/afl-qemu-cpu-inl.h index 86203a5b..4ad31b60 100644 --- a/qemu_mode/patches/afl-qemu-cpu-inl.h +++ b/qemu_mode/patches/afl-qemu-cpu-inl.h @@ -1,19 +1,18 @@ /* - american fuzzy lop - high-performance binary-only instrumentation - ----------------------------------------------------------------- + american fuzzy lop++ - high-performance binary-only instrumentation + ------------------------------------------------------------------- - Written by Andrew Griffiths <agriffiths@google.com> and - Michal Zalewski <lcamtuf@google.com> - - Idea & design very much by Andrew Griffiths. + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> TCG instrumentation and block chaining support by Andrea Biondo <andrea.biondo965@gmail.com> - QEMU 3.1.0 port, TCG thread-safety and CompareCoverage by Andrea Fioraldi - <andreafioraldi@gmail.com> + QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero + counters by Andrea Fioraldi <andreafioraldi@gmail.com> Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -66,7 +65,7 @@ abi_ulong afl_entry_point, /* ELF entry point (_start) */ afl_start_code, /* .text start pointer */ afl_end_code; /* .text end pointer */ -u8 afl_enable_compcov; +u8 afl_compcov_level; /* Set in the child process in forkserver mode: */ @@ -159,9 +158,14 @@ static void afl_setup(void) { } + /* Maintain for compatibility */ if (getenv("AFL_QEMU_COMPCOV")) { - afl_enable_compcov = 1; + afl_compcov_level = 1; + } + if (getenv("AFL_COMPCOV_LEVEL")) { + + afl_compcov_level = atoi(getenv("AFL_COMPCOV_LEVEL")); } /* pthread_atfork() seems somewhat broken in util/rcu.c, and I'm @@ -327,7 +331,7 @@ static void afl_wait_tsl(CPUState *cpu, int fd) { if (is_valid_addr(t.tb.pc)) { mmap_lock(); - tb = tb_gen_code(cpu, t.tb.pc, t.tb.cs_base, t.tb.flags, 0); + tb = tb_gen_code(cpu, t.tb.pc, t.tb.cs_base, t.tb.flags, t.tb.cf_mask); mmap_unlock(); } else { diff --git a/qemu_mode/patches/afl-qemu-cpu-translate-inl.h b/qemu_mode/patches/afl-qemu-cpu-translate-inl.h index 0ca89c98..09ecb9d2 100644 --- a/qemu_mode/patches/afl-qemu-cpu-translate-inl.h +++ b/qemu_mode/patches/afl-qemu-cpu-translate-inl.h @@ -1,19 +1,18 @@ /* - american fuzzy lop - high-performance binary-only instrumentation - ----------------------------------------------------------------- + american fuzzy lop++ - high-performance binary-only instrumentation + ------------------------------------------------------------------- - Written by Andrew Griffiths <agriffiths@google.com> and - Michal Zalewski <lcamtuf@google.com> - - Idea & design very much by Andrew Griffiths. + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> TCG instrumentation and block chaining support by Andrea Biondo <andrea.biondo965@gmail.com> - - QEMU 3.1.0 port, TCG thread-safety and CompareCoverage by Andrea Fioraldi - <andreafioraldi@gmail.com> + + QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero + counters by Andrea Fioraldi <andreafioraldi@gmail.com> Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -32,7 +31,7 @@ */ -#include "../../config.h" +#include "afl-qemu-common.h" #include "tcg.h" #include "tcg-op.h" @@ -40,7 +39,7 @@ extern unsigned char *afl_area_ptr; extern unsigned int afl_inst_rms; extern abi_ulong afl_start_code, afl_end_code; -extern u8 afl_enable_compcov; +extern u8 afl_compcov_level; void tcg_gen_afl_compcov_log_call(void *func, target_ulong cur_loc, TCGv_i64 arg1, TCGv_i64 arg2); @@ -49,7 +48,7 @@ static void afl_compcov_log_16(target_ulong cur_loc, target_ulong arg1, target_ulong arg2) { if ((arg1 & 0xff) == (arg2 & 0xff)) { - afl_area_ptr[cur_loc]++; + INC_AFL_AREA(cur_loc); } } @@ -57,11 +56,11 @@ static void afl_compcov_log_32(target_ulong cur_loc, target_ulong arg1, target_ulong arg2) { if ((arg1 & 0xff) == (arg2 & 0xff)) { - afl_area_ptr[cur_loc]++; + INC_AFL_AREA(cur_loc); if ((arg1 & 0xffff) == (arg2 & 0xffff)) { - afl_area_ptr[cur_loc +1]++; + INC_AFL_AREA(cur_loc +1); if ((arg1 & 0xffffff) == (arg2 & 0xffffff)) { - afl_area_ptr[cur_loc +2]++; + INC_AFL_AREA(cur_loc +2); } } } @@ -71,19 +70,19 @@ static void afl_compcov_log_64(target_ulong cur_loc, target_ulong arg1, target_ulong arg2) { if ((arg1 & 0xff) == (arg2 & 0xff)) { - afl_area_ptr[cur_loc]++; + INC_AFL_AREA(cur_loc); if ((arg1 & 0xffff) == (arg2 & 0xffff)) { - afl_area_ptr[cur_loc +1]++; + INC_AFL_AREA(cur_loc +1); if ((arg1 & 0xffffff) == (arg2 & 0xffffff)) { - afl_area_ptr[cur_loc +2]++; + INC_AFL_AREA(cur_loc +2); if ((arg1 & 0xffffffff) == (arg2 & 0xffffffff)) { - afl_area_ptr[cur_loc +3]++; + INC_AFL_AREA(cur_loc +3); if ((arg1 & 0xffffffffff) == (arg2 & 0xffffffffff)) { - afl_area_ptr[cur_loc +4]++; + INC_AFL_AREA(cur_loc +4); if ((arg1 & 0xffffffffffff) == (arg2 & 0xffffffffffff)) { - afl_area_ptr[cur_loc +5]++; + INC_AFL_AREA(cur_loc +5); if ((arg1 & 0xffffffffffffff) == (arg2 & 0xffffffffffffff)) { - afl_area_ptr[cur_loc +6]++; + INC_AFL_AREA(cur_loc +6); } } } @@ -95,11 +94,14 @@ static void afl_compcov_log_64(target_ulong cur_loc, target_ulong arg1, static void afl_gen_compcov(target_ulong cur_loc, TCGv_i64 arg1, TCGv_i64 arg2, - TCGMemOp ot) { + TCGMemOp ot, int is_imm) { void *func; - if (!afl_enable_compcov || cur_loc > afl_end_code || cur_loc < afl_start_code) + if (!afl_compcov_level || cur_loc > afl_end_code || cur_loc < afl_start_code) + return; + + if (!is_imm && afl_compcov_level < 2) return; switch (ot) { @@ -117,7 +119,7 @@ static void afl_gen_compcov(target_ulong cur_loc, TCGv_i64 arg1, TCGv_i64 arg2, } cur_loc = (cur_loc >> 4) ^ (cur_loc << 8); - cur_loc &= MAP_SIZE - 1; + cur_loc &= MAP_SIZE - 7; if (cur_loc >= afl_inst_rms) return; diff --git a/qemu_mode/patches/afl-qemu-tcg-inl.h b/qemu_mode/patches/afl-qemu-tcg-inl.h index ff90d1b9..a9c53b8c 100644 --- a/qemu_mode/patches/afl-qemu-tcg-inl.h +++ b/qemu_mode/patches/afl-qemu-tcg-inl.h @@ -1,19 +1,18 @@ /* - american fuzzy lop - high-performance binary-only instrumentation - ----------------------------------------------------------------- + american fuzzy lop++ - high-performance binary-only instrumentation + ------------------------------------------------------------------- - Written by Andrew Griffiths <agriffiths@google.com> and - Michal Zalewski <lcamtuf@google.com> - - Idea & design very much by Andrew Griffiths. + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> TCG instrumentation and block chaining support by Andrea Biondo <andrea.biondo965@gmail.com> - QEMU 3.1.0 port, TCG thread-safety and CompareCoverage by Andrea Fioraldi - <andreafioraldi@gmail.com> + QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero + counters by Andrea Fioraldi <andreafioraldi@gmail.com> Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/qemu_mode/patches/afl-qemu-translate-inl.h b/qemu_mode/patches/afl-qemu-translate-inl.h index bfb2897e..ffe43dba 100644 --- a/qemu_mode/patches/afl-qemu-translate-inl.h +++ b/qemu_mode/patches/afl-qemu-translate-inl.h @@ -1,19 +1,18 @@ /* - american fuzzy lop - high-performance binary-only instrumentation - ----------------------------------------------------------------- + american fuzzy lop++ - high-performance binary-only instrumentation + ------------------------------------------------------------------- - Written by Andrew Griffiths <agriffiths@google.com> and - Michal Zalewski <lcamtuf@google.com> - - Idea & design very much by Andrew Griffiths. + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> TCG instrumentation and block chaining support by Andrea Biondo <andrea.biondo965@gmail.com> - QEMU 3.1.0 port, TCG thread-safety and CompareCoverage by Andrea Fioraldi - <andreafioraldi@gmail.com> + QEMU 3.1.0 port, TCG thread-safety, CompareCoverage and NeverZero + counters by Andrea Fioraldi <andreafioraldi@gmail.com> Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -32,7 +31,7 @@ */ -#include "../../config.h" +#include "afl-qemu-common.h" #include "tcg-op.h" /* Declared in afl-qemu-cpu-inl.h */ @@ -42,11 +41,14 @@ extern abi_ulong afl_start_code, afl_end_code; void tcg_gen_afl_maybe_log_call(target_ulong cur_loc); -void afl_maybe_log(target_ulong cur_loc) { +void afl_maybe_log(target_ulong cur_loc) { static __thread abi_ulong prev_loc; - afl_area_ptr[cur_loc ^ prev_loc]++; + register uintptr_t afl_idx = cur_loc ^ prev_loc; + + INC_AFL_AREA(afl_idx); + prev_loc = cur_loc >> 1; } diff --git a/qemu_mode/patches/i386-translate.diff b/qemu_mode/patches/i386-translate.diff index 0bc48828..239b2404 100644 --- a/qemu_mode/patches/i386-translate.diff +++ b/qemu_mode/patches/i386-translate.diff @@ -15,11 +15,11 @@ index 0dd5fbe4..b95d341e 100644 tcg_gen_atomic_fetch_add_tl(s1->cc_srcT, s1->A0, s1->T0, s1->mem_index, ot | MO_LE); tcg_gen_sub_tl(s1->T0, s1->cc_srcT, s1->T1); -+ afl_gen_compcov(s1->pc, s1->cc_srcT, s1->T1, ot); ++ afl_gen_compcov(s1->pc, s1->cc_srcT, s1->T1, ot, d == OR_EAX); } else { tcg_gen_mov_tl(s1->cc_srcT, s1->T0); tcg_gen_sub_tl(s1->T0, s1->T0, s1->T1); -+ afl_gen_compcov(s1->pc, s1->T0, s1->T1, ot); ++ afl_gen_compcov(s1->pc, s1->T0, s1->T1, ot, d == OR_EAX); gen_op_st_rm_T0_A0(s1, ot, d); } gen_op_update2_cc(s1); @@ -27,7 +27,7 @@ index 0dd5fbe4..b95d341e 100644 tcg_gen_mov_tl(cpu_cc_src, s1->T1); tcg_gen_mov_tl(s1->cc_srcT, s1->T0); tcg_gen_sub_tl(cpu_cc_dst, s1->T0, s1->T1); -+ afl_gen_compcov(s1->pc, s1->T0, s1->T1, ot); ++ afl_gen_compcov(s1->pc, s1->T0, s1->T1, ot, d == OR_EAX); set_cc_op(s1, CC_OP_SUBB + ot); break; } diff --git a/src/afl-analyze.c b/src/afl-analyze.c index e12f9194..5bb96154 100644 --- a/src/afl-analyze.c +++ b/src/afl-analyze.c @@ -21,6 +21,9 @@ #define AFL_MAIN +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include "config.h" #include "types.h" #include "debug.h" diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index 0285a242..1e8c5777 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -28,6 +28,10 @@ #endif #define _FILE_OFFSET_BITS 64 +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif + #include "config.h" #include "types.h" #include "debug.h" @@ -63,6 +67,7 @@ #if defined(__APPLE__) || defined(__FreeBSD__) || defined (__OpenBSD__) # include <sys/sysctl.h> +# define HAVE_ARC4RANDOM 1 #endif /* __APPLE__ || __FreeBSD__ || __OpenBSD__ */ /* For systems that have sched_setaffinity; right now just Linux, but one @@ -219,8 +224,10 @@ EXP_ST u8 skip_deterministic, /* Skip deterministic stages? */ fast_cal; /* Try to calibrate faster? */ u8 uses_asan; /* Target uses ASAN? */ - s32 out_fd, /* Persistent fd for out_file */ +static s32 out_fd, /* Persistent fd for out_file */ +#ifndef HAVE_ARC4RANDOM dev_urandom_fd = -1, /* Persistent fd for /dev/urandom */ +#endif dev_null_fd = -1, /* Persistent fd for /dev/null */ fsrv_ctl_fd, /* Fork server control pipe (write) */ fsrv_st_fd; /* Fork server status pipe (read) */ @@ -298,7 +305,9 @@ static u8 stage_val_type; /* Value type (STAGE_VAL_*) */ static u64 stage_finds[32], /* Patterns found per fuzz stage */ stage_cycles[32]; /* Execs per fuzz stage */ +#ifndef HAVE_ARC4RANDOM static u32 rand_cnt; /* Random number counter */ +#endif static u64 total_cal_us, /* Total calibration time (us) */ total_cal_cycles; /* Total calibration cycles */ @@ -642,14 +651,8 @@ static void trim_py(char** ret, size_t* retlen) { int select_algorithm(void) { int i_puppet, j_puppet; - u32 seed[2]; - - if (!fixed_seed) { - ck_read(dev_urandom_fd, &seed, sizeof(seed), "/dev/urandom"); - srandom(seed[0]); - } - double sele = ((double)(random()%10000)*0.0001); + double sele = ((double)(UR(10000))*0.0001); j_puppet = 0; for (i_puppet = 0; i_puppet < operator_num; ++i_puppet) { if (unlikely(i_puppet == 0)) { @@ -700,7 +703,15 @@ static u64 get_cur_time_us(void) { have slight bias. */ static inline u32 UR(u32 limit) { +#ifdef HAVE_ARC4RANDOM + if (fixed_seed) { + return random() % limit; + } + /* The boundary not being necessarily a power of 2, + we need to ensure the result uniformity. */ + return arc4random_uniform(limit); +#else if (!fixed_seed && unlikely(!rand_cnt--)) { u32 seed[2]; @@ -710,6 +721,7 @@ static inline u32 UR(u32 limit) { } return random() % limit; +#endif } @@ -2309,7 +2321,6 @@ static void destroy_extras(void) { } - /* Execute target application, monitoring for timeouts. Return status information. The called program will update trace_bits[]. */ @@ -2388,7 +2399,9 @@ static u8 run_target(char** argv, u32 timeout) { close(dev_null_fd); close(out_dir_fd); +#ifndef HAVE_ARC4RANDOM close(dev_urandom_fd); +#endif close(fileno(plot_file)); /* Set sane defaults for ASAN if nothing else specified. */ @@ -10988,6 +11001,7 @@ static void check_term_size(void) { if (ioctl(1, TIOCGWINSZ, &ws)) return; + if (ws.ws_row == 0 || ws.ws_col == 0) return; if (ws.ws_row < 24 || ws.ws_col < 79) term_too_small = 1; } @@ -11155,8 +11169,10 @@ EXP_ST void setup_dirs_fds(void) { dev_null_fd = open("/dev/null", O_RDWR); if (dev_null_fd < 0) PFATAL("Unable to open /dev/null"); +#ifndef HAVE_ARC4RANDOM dev_urandom_fd = open("/dev/urandom", O_RDONLY); if (dev_urandom_fd < 0) PFATAL("Unable to open /dev/urandom"); +#endif /* Gnuplot output file. */ @@ -12045,8 +12061,8 @@ int main(int argc, char** argv) { if (unicorn_mode) FATAL("-U and -n are mutually exclusive"); } - - if (index(argv[optind], '/') == NULL) WARNF(cLRD "Target binary called without a prefixed path, make sure you are fuzzing the right binary: " cRST "%s", argv[optind]); + + if (strchr(argv[optind], '/') == NULL) WARNF(cLRD "Target binary called without a prefixed path, make sure you are fuzzing the right binary: " cRST "%s", argv[optind]); OKF("afl++ is maintained by Marc \"van Hauser\" Heuse, Heiko \"hexcoder\" Eissfeldt and Andrea Fioraldi"); OKF("afl++ is open source, get it at https://github.com/vanhauser-thc/AFLplusplus"); diff --git a/src/afl-gotcpu.c b/src/afl-gotcpu.c index 9db86933..fa629eb7 100644 --- a/src/afl-gotcpu.c +++ b/src/afl-gotcpu.c @@ -30,6 +30,9 @@ # define _GNU_SOURCE #endif +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include <stdio.h> #include <stdlib.h> #include <unistd.h> diff --git a/src/afl-sharedmem.c b/src/afl-sharedmem.c index 3fd38444..ce3b76e6 100644 --- a/src/afl-sharedmem.c +++ b/src/afl-sharedmem.c @@ -4,6 +4,9 @@ #define AFL_MAIN +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include "config.h" #include "types.h" #include "debug.h" diff --git a/src/afl-showmap.c b/src/afl-showmap.c index 66c77094..ee00bf22 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -23,6 +23,9 @@ #define AFL_MAIN +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include "config.h" #include "types.h" #include "debug.h" diff --git a/src/afl-tmin.c b/src/afl-tmin.c index 663bb510..529720ca 100644 --- a/src/afl-tmin.c +++ b/src/afl-tmin.c @@ -21,6 +21,9 @@ #define AFL_MAIN +#ifdef __ANDROID__ + #include "android-ashmem.h" +#endif #include "config.h" #include "types.h" diff --git a/unicorn_mode/README.md b/unicorn_mode/README.md index 9ee975ef..ea3e3c9b 100644 --- a/unicorn_mode/README.md +++ b/unicorn_mode/README.md @@ -1,23 +1,119 @@ -``` - __ _ _ - __ _ / _| | _ _ _ __ (_) ___ ___ _ __ _ __ - / _` | |_| |___| | | | '_ \| |/ __/ _ \| '__| '_ \ -| (_| | _| |___| |_| | | | | | (_| (_) | | | | | | - \__,_|_| |_| \__,_|_| |_|_|\___\___/|_| |_| |_| - -``` +# Unicorn-based binary-only instrumentation for afl-fuzz -afl-unicorn lets you fuzz any piece of binary that can be emulated by -[Unicorn Engine](http://www.unicorn-engine.org/). +The idea and much of the original implementation comes from Nathan Voss <njvoss299@gmail.com>. -Requirements: Python2 +The port to afl++ if by Dominik Maier <mail@dmnk.co>. -For the full readme please see docs/unicorn_mode.txt +The CompareCoverage and NeverZero counters features by Andrea Fioraldi <andreafioraldi@gmail.com>. -For an in-depth description of what this is, how to install it, and how to use -it check out this [blog post](https://medium.com/@njvoss299/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf). +## 1) Introduction -For general help with AFL, please refer to the documents in the ./docs/ directory. +The code in ./unicorn_mode allows you to build a standalone feature that +leverages the Unicorn Engine and allows callers to obtain instrumentation +output for black-box, closed-source binary code snippets. This mechanism +can be then used by afl-fuzz to stress-test targets that couldn't be built +with afl-gcc or used in QEMU mode, or with other extensions such as +TriforceAFL. -Created by Nathan Voss, originally funded by -[Battelle](https://www.battelle.org/cyber). +There is a significant performance penalty compared to native AFL, +but at least we're able to use AFL on these binaries, right? + +## 2) How to use + +Requirements: you need an installed python2 environment. + +### Building AFL's Unicorn Mode + +First, make afl++ as usual. +Once that completes successfully you need to build and add in the Unicorn Mode +features: + + $ cd unicorn_mode + $ ./build_unicorn_support.sh + +NOTE: This script downloads a Unicorn Engine commit that has been tested +and is stable-ish from the Unicorn github page. If you are offline, you'll need +to hack up this script a little bit and supply your own copy of Unicorn's latest +stable release. It's not very hard, just check out the beginning of the +build_unicorn_support.sh script and adjust as necessary. + +Building Unicorn will take a little bit (~5-10 minutes). Once it completes +it automatically compiles a sample application and verify that it works. + +### Fuzzing with Unicorn Mode + +To really use unicorn-mode effectively you need to prepare the following: + + * Relevant binary code to be fuzzed + * Knowledge of the memory map and good starting state + * Folder containing sample inputs to start fuzzing with + + Same ideas as any other AFL inputs + + Quality/speed of results will depend greatly on quality of starting + samples + + See AFL's guidance on how to create a sample corpus + * Unicorn-based test harness which: + + Adds memory map regions + + Loads binary code into memory + + Emulates at least one instruction* + + Yeah, this is lame. See 'Gotchas' section below for more info + + Loads and verifies data to fuzz from a command-line specified file + + AFL will provide mutated inputs by changing the file passed to + the test harness + + Presumably the data to be fuzzed is at a fixed buffer address + + If input constraints (size, invalid bytes, etc.) are known they + should be checked after the file is loaded. If a constraint + fails, just exit the test harness. AFL will treat the input as + 'uninteresting' and move on. + + Sets up registers and memory state for beginning of test + + Emulates the interested code from beginning to end + + If a crash is detected, the test harness must 'crash' by + throwing a signal (SIGSEGV, SIGKILL, SIGABORT, etc.) + +Once you have all those things ready to go you just need to run afl-fuzz in +'unicorn-mode' by passing in the '-U' flag: + + $ afl-fuzz -U -m none -i /path/to/inputs -o /path/to/results -- ./test_harness @@ + +The normal afl-fuzz command line format applies to everything here. Refer to +AFL's main documentation for more info about how to use afl-fuzz effectively. + +For a much clearer vision of what all of this looks like, please refer to the +sample provided in the 'unicorn_mode/samples' directory. There is also a blog +post that goes over the basics at: + +https://medium.com/@njvoss299/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf + +The 'helper_scripts' directory also contains several helper scripts that allow you +to dump context from a running process, load it, and hook heap allocations. For details +on how to use this check out the follow-up blog post to the one linked above. + +A example use of AFL-Unicorn mode is discussed in the Paper Unicorefuzz: +https://www.usenix.org/conference/woot19/presentation/maier + +## 3) Options + +As for the QEMU-based instrumentation, the afl-unicorn twist of afl++ +comes with a sub-instruction based instrumentation similar in purpose to laf-intel. + +The options that enables Unicorn CompareCoverage are the same used for QEMU. +AFL_COMPCOV_LEVEL=1 is to instrument comparisons with only immediate +values. QEMU_COMPCOV_LEVEL=2 instruments all +comparison instructions. Comparison instructions are currently instrumented only +on the x86 and x86_64 targets. + +## 4) Gotchas, feedback, bugs + +To make sure that AFL's fork server starts up correctly the Unicorn test +harness script must emulate at least one instruction before loading the +data that will be fuzzed from the input file. It doesn't matter what the +instruction is, nor if it is valid. This is an artifact of how the fork-server +is started and could likely be fixed with some clever re-arranging of the +patches applied to Unicorn. + +Running the build script builds Unicorn and its python bindings and installs +them on your system. This installation will supersede any existing Unicorn +installation with the patched afl-unicorn version. + +Refer to the unicorn_mode/samples/arm_example/arm_tester.c for an example +of how to do this properly! If you don't get this right, AFL will not +load any mutated inputs and your fuzzing will be useless! diff --git a/unicorn_mode/build_unicorn_support.sh b/unicorn_mode/build_unicorn_support.sh index 9dcf6773..1575f66c 100644..100755 --- a/unicorn_mode/build_unicorn_support.sh +++ b/unicorn_mode/build_unicorn_support.sh @@ -1,16 +1,20 @@ #!/bin/sh # -# american fuzzy lop - Unicorn-Mode build script -# -------------------------------------- +# american fuzzy lop++ - unicorn mode build script +# ------------------------------------------------ # -# Written by Nathan Voss <njvoss99@gmail.com> +# Originally written by Nathan Voss <njvoss99@gmail.com> # # Adapted from code by Andrew Griffiths <agriffiths@google.com> and # Michal Zalewski <lcamtuf@google.com> # -# Adapted for Afl++ by Dominik Maier <mail@dmnk.co> +# Adapted for AFLplusplus by Dominik Maier <mail@dmnk.co> +# +# CompareCoverage and NeverZero counters by Andrea Fioraldi +# <andreafioraldi@gmail.com> # # Copyright 2017 Battelle Memorial Institute. All rights reserved. +# Copyright 2019 AFLplusplus Project. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -127,12 +131,13 @@ tar xzf "$ARCHIVE" -C ./unicorn --strip-components=1 || exit 1 echo "[+] Unpacking successful." -rm -rf "$ARCHIVE" || exit 1 +#rm -rf "$ARCHIVE" || exit 1 echo "[*] Applying patches..." -cp patches/afl-unicorn-cpu-inl.h unicorn || exit 1 -patch -p1 --directory unicorn <patches/patches.diff || exit 1 +cp patches/*.h unicorn || exit 1 +patch -p1 --directory unicorn < patches/patches.diff || exit 1 +patch -p1 --directory unicorn < patches/compcov.diff || exit 1 echo "[+] Patching done." @@ -144,7 +149,7 @@ echo "[+] Configuration complete." echo "[*] Attempting to build Unicorn (fingers crossed!)..." -UNICORN_QEMU_FLAGS='--python=python2' make || exit 1 +UNICORN_QEMU_FLAGS='--python=python2' make -j `nproc` || exit 1 echo "[+] Build process successful!" diff --git a/unicorn_mode/patches/afl-unicorn-common.h b/unicorn_mode/patches/afl-unicorn-common.h new file mode 100644 index 00000000..6798832c --- /dev/null +++ b/unicorn_mode/patches/afl-unicorn-common.h @@ -0,0 +1,50 @@ +/* + american fuzzy lop++ - unicorn instrumentation + ---------------------------------------------- + + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> + + Adapted for afl-unicorn by Dominik Maier <mail@dmnk.co> + + CompareCoverage and NeverZero counters by Andrea Fioraldi + <andreafioraldi@gmail.com> + + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + This code is a shim patched into the separately-distributed source + code of Unicorn 1.0.1. It leverages the built-in QEMU tracing functionality + to implement AFL-style instrumentation and to take care of the remaining + parts of the AFL fork server logic. + + The resulting libunicorn binary is essentially a standalone instrumentation + tool; for an example of how to leverage it for other purposes, you can + have a look at afl-showmap.c. + + */ + +#include "../../config.h" + +/* NeverZero */ + +#if (defined(__x86_64__) || defined(__i386__)) && defined(AFL_QEMU_NOT_ZERO) +# define INC_AFL_AREA(loc) \ + asm volatile ( \ + "incb (%0, %1, 1)\n" \ + "adcb $0, (%0, %1, 1)\n" \ + : /* no out */ \ + : "r" (afl_area_ptr), "r" (loc) \ + : "memory", "eax" \ + ) +#else +# define INC_AFL_AREA(loc) \ + afl_area_ptr[loc]++ +#endif + diff --git a/unicorn_mode/patches/afl-unicorn-cpu-inl.h b/unicorn_mode/patches/afl-unicorn-cpu-inl.h index 892c3f72..a713e4ca 100644 --- a/unicorn_mode/patches/afl-unicorn-cpu-inl.h +++ b/unicorn_mode/patches/afl-unicorn-cpu-inl.h @@ -1,17 +1,17 @@ /* - american fuzzy lop - high-performance binary-only instrumentation - ----------------------------------------------------------------- + american fuzzy lop++ - unicorn instrumentation + ---------------------------------------------- - Written by Andrew Griffiths <agriffiths@google.com> and - Michal Zalewski <lcamtuf@google.com> + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> - TCG instrumentation and block chaining support by Andrea Biondo - <andrea.biondo965@gmail.com> Adapted for afl-unicorn by Dominik Maier <mail@dmnk.co> - Idea & design very much by Andrew Griffiths. + CompareCoverage and NeverZero counters by Andrea Fioraldi + <andreafioraldi@gmail.com> - Copyright 2015, 2016 Google Inc. All rights reserved. + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -24,7 +24,7 @@ to implement AFL-style instrumentation and to take care of the remaining parts of the AFL fork server logic. - The resulting QEMU binary is essentially a standalone instrumentation + The resulting libunicorn binary is essentially a standalone instrumentation tool; for an example of how to leverage it for other purposes, you can have a look at afl-showmap.c. @@ -33,7 +33,7 @@ #include <sys/shm.h> #include <sys/types.h> #include <sys/wait.h> -#include "../../config.h" +#include "afl-unicorn-common.h" /*************************** * VARIOUS AUXILIARY STUFF * @@ -54,11 +54,11 @@ #define AFL_UNICORN_CPU_SNIPPET2 do { \ if(unlikely(afl_first_instr == 0)) { \ - afl_setup(); \ + afl_setup(env->uc); \ afl_forkserver(env); \ afl_first_instr = 1; \ } \ - afl_maybe_log(tb->pc); \ + afl_maybe_log(env->uc, tb->pc); \ } while (0) /* We use one additional file descriptor to relay "needs translation" @@ -66,24 +66,16 @@ #define TSL_FD (FORKSRV_FD - 1) -/* This is equivalent to afl-as.h: */ - -static unsigned char *afl_area_ptr; - /* Set in the child process in forkserver mode: */ static unsigned char afl_fork_child; static unsigned int afl_forksrv_pid; -/* Instrumentation ratio: */ - -static unsigned int afl_inst_rms = MAP_SIZE; - /* Function declarations. */ -static void afl_setup(void); +static void afl_setup(struct uc_struct* uc); static void afl_forkserver(CPUArchState*); -static inline void afl_maybe_log(unsigned long); +static inline void afl_maybe_log(struct uc_struct* uc, unsigned long); static void afl_wait_tsl(CPUArchState*, int); static void afl_request_tsl(target_ulong, target_ulong, uint64_t); @@ -105,7 +97,7 @@ struct afl_tsl { /* Set up SHM region and initialize other stuff. */ -static void afl_setup(void) { +static void afl_setup(struct uc_struct* uc) { char *id_str = getenv(SHM_ENV_VAR), *inst_r = getenv("AFL_INST_RATIO"); @@ -121,21 +113,35 @@ static void afl_setup(void) { if (r > 100) r = 100; if (!r) r = 1; - afl_inst_rms = MAP_SIZE * r / 100; + uc->afl_inst_rms = MAP_SIZE * r / 100; + } else { + + uc->afl_inst_rms = MAP_SIZE; + } if (id_str) { shm_id = atoi(id_str); - afl_area_ptr = shmat(shm_id, NULL, 0); + uc->afl_area_ptr = shmat(shm_id, NULL, 0); - if (afl_area_ptr == (void*)-1) exit(1); + if (uc->afl_area_ptr == (void*)-1) exit(1); /* With AFL_INST_RATIO set to a low value, we want to touch the bitmap so that the parent doesn't give up on us. */ - if (inst_r) afl_area_ptr[0] = 1; + if (inst_r) uc->afl_area_ptr[0] = 1; + } + + /* Maintain for compatibility */ + if (getenv("AFL_QEMU_COMPCOV")) { + + uc->afl_compcov_level = 1; + } + if (getenv("AFL_COMPCOV_LEVEL")) { + + uc->afl_compcov_level = atoi(getenv("AFL_COMPCOV_LEVEL")); } } @@ -145,7 +151,7 @@ static void afl_forkserver(CPUArchState *env) { static unsigned char tmp[4]; - if (!afl_area_ptr) return; + if (!env->uc->afl_area_ptr) return; /* Tell the parent that we're alive. If the parent doesn't want to talk, assume that we're not running in forkserver mode. */ @@ -208,21 +214,15 @@ static void afl_forkserver(CPUArchState *env) { /* The equivalent of the tuple logging routine from afl-as.h. */ -static inline void afl_maybe_log(unsigned long cur_loc) { +static inline void afl_maybe_log(struct uc_struct* uc, unsigned long cur_loc) { static __thread unsigned long prev_loc; - // DEBUG - //printf("IN AFL_MAYBE_LOG 0x%lx\n", cur_loc); + u8* afl_area_ptr = uc->afl_area_ptr; - // MODIFIED FOR UNICORN MODE -> We want to log all addresses, - // so the checks for 'start < addr < end' are removed if(!afl_area_ptr) return; - // DEBUG - //printf("afl_area_ptr = %p\n", afl_area_ptr); - /* Looks like QEMU always maps to fixed locations, so ASAN is not a concern. Phew. But instruction addresses may be aligned. Let's mangle the value to get something quasi-uniform. */ @@ -233,15 +233,12 @@ static inline void afl_maybe_log(unsigned long cur_loc) { /* Implement probabilistic instrumentation by looking at scrambled block address. This keeps the instrumented locations stable across runs. */ - // DEBUG - //printf("afl_inst_rms = 0x%lx\n", afl_inst_rms); + if (cur_loc >= uc->afl_inst_rms) return; - if (cur_loc >= afl_inst_rms) return; + register uintptr_t afl_idx = cur_loc ^ prev_loc; - // DEBUG - //printf("cur_loc = 0x%lx\n", cur_loc); + INC_AFL_AREA(afl_idx); - afl_area_ptr[cur_loc ^ prev_loc]++; prev_loc = cur_loc >> 1; } diff --git a/unicorn_mode/patches/afl-unicorn-cpu-translate-inl.h b/unicorn_mode/patches/afl-unicorn-cpu-translate-inl.h new file mode 100644 index 00000000..69877c6b --- /dev/null +++ b/unicorn_mode/patches/afl-unicorn-cpu-translate-inl.h @@ -0,0 +1,62 @@ +/* + american fuzzy lop++ - unicorn instrumentation + ---------------------------------------------- + + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> + + Adapted for afl-unicorn by Dominik Maier <mail@dmnk.co> + + CompareCoverage and NeverZero counters by Andrea Fioraldi + <andreafioraldi@gmail.com> + + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + This code is a shim patched into the separately-distributed source + code of Unicorn 1.0.1. It leverages the built-in QEMU tracing functionality + to implement AFL-style instrumentation and to take care of the remaining + parts of the AFL fork server logic. + + The resulting libunicorn binary is essentially a standalone instrumentation + tool; for an example of how to leverage it for other purposes, you can + have a look at afl-showmap.c. + + */ + +#include "../../config.h" + +static void afl_gen_compcov(TCGContext *s, uint64_t cur_loc, TCGv_i64 arg1, + TCGv_i64 arg2, TCGMemOp ot, int is_imm) { + + if (!s->uc->afl_compcov_level || !s->uc->afl_area_ptr) + return; + + if (!is_imm && s->uc->afl_compcov_level < 2) + return; + + cur_loc = (cur_loc >> 4) ^ (cur_loc << 8); + cur_loc &= MAP_SIZE - 7; + + if (cur_loc >= s->uc->afl_inst_rms) return; + + switch (ot) { + case MO_64: + gen_afl_compcov_log_64(s, cur_loc, arg1, arg2); + break; + case MO_32: + gen_afl_compcov_log_32(s, cur_loc, arg1, arg2); + break; + case MO_16: + gen_afl_compcov_log_16(s, cur_loc, arg1, arg2); + break; + default: + return; + } +} diff --git a/unicorn_mode/patches/afl-unicorn-tcg-op-inl.h b/unicorn_mode/patches/afl-unicorn-tcg-op-inl.h new file mode 100644 index 00000000..fa4974d6 --- /dev/null +++ b/unicorn_mode/patches/afl-unicorn-tcg-op-inl.h @@ -0,0 +1,56 @@ +/* + american fuzzy lop++ - unicorn instrumentation + ---------------------------------------------- + + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> + + Adapted for afl-unicorn by Dominik Maier <mail@dmnk.co> + + CompareCoverage and NeverZero counters by Andrea Fioraldi + <andreafioraldi@gmail.com> + + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + This code is a shim patched into the separately-distributed source + code of Unicorn 1.0.1. It leverages the built-in QEMU tracing functionality + to implement AFL-style instrumentation and to take care of the remaining + parts of the AFL fork server logic. + + The resulting libunicorn binary is essentially a standalone instrumentation + tool; for an example of how to leverage it for other purposes, you can + have a look at afl-showmap.c. + + */ + +static inline void gen_afl_compcov_log_16(TCGContext *tcg_ctx, uint64_t cur_loc, + TCGv_i64 arg1, TCGv_i64 arg2) +{ + TCGv_ptr tuc = tcg_const_ptr(tcg_ctx, tcg_ctx->uc); + TCGv_i64 tcur_loc = tcg_const_i64(tcg_ctx, cur_loc); + gen_helper_afl_compcov_log_16(tcg_ctx, tuc, tcur_loc, arg1, arg2); +} + +static inline void gen_afl_compcov_log_32(TCGContext *tcg_ctx, uint64_t cur_loc, + TCGv_i64 arg1, TCGv_i64 arg2) +{ + TCGv_ptr tuc = tcg_const_ptr(tcg_ctx, tcg_ctx->uc); + TCGv_i64 tcur_loc = tcg_const_i64(tcg_ctx, cur_loc); + gen_helper_afl_compcov_log_32(tcg_ctx, tuc, tcur_loc, arg1, arg2); +} + +static inline void gen_afl_compcov_log_64(TCGContext *tcg_ctx, uint64_t cur_loc, + TCGv_i64 arg1, TCGv_i64 arg2) +{ + TCGv_ptr tuc = tcg_const_ptr(tcg_ctx, tcg_ctx->uc); + TCGv_i64 tcur_loc = tcg_const_i64(tcg_ctx, cur_loc); + gen_helper_afl_compcov_log_64(tcg_ctx, tuc, tcur_loc, arg1, arg2); +} + diff --git a/unicorn_mode/patches/afl-unicorn-tcg-runtime-inl.h b/unicorn_mode/patches/afl-unicorn-tcg-runtime-inl.h new file mode 100644 index 00000000..1f0667ce --- /dev/null +++ b/unicorn_mode/patches/afl-unicorn-tcg-runtime-inl.h @@ -0,0 +1,89 @@ +/* + american fuzzy lop++ - unicorn instrumentation + ---------------------------------------------- + + Originally written by Andrew Griffiths <agriffiths@google.com> and + Michal Zalewski <lcamtuf@google.com> + + Adapted for afl-unicorn by Dominik Maier <mail@dmnk.co> + + CompareCoverage and NeverZero counters by Andrea Fioraldi + <andreafioraldi@gmail.com> + + Copyright 2015, 2016, 2017 Google Inc. All rights reserved. + Copyright 2019 AFLplusplus Project. All rights reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at: + + http://www.apache.org/licenses/LICENSE-2.0 + + This code is a shim patched into the separately-distributed source + code of Unicorn 1.0.1. It leverages the built-in QEMU tracing functionality + to implement AFL-style instrumentation and to take care of the remaining + parts of the AFL fork server logic. + + The resulting libunicorn binary is essentially a standalone instrumentation + tool; for an example of how to leverage it for other purposes, you can + have a look at afl-showmap.c. + + */ + +#include "uc_priv.h" +#include "afl-unicorn-common.h" + +void HELPER(afl_compcov_log_16)(void* uc_ptr, uint64_t cur_loc, uint64_t arg1, + uint64_t arg2) { + + u8* afl_area_ptr = ((struct uc_struct*)uc_ptr)->afl_area_ptr; + + if ((arg1 & 0xff) == (arg2 & 0xff)) { + INC_AFL_AREA(cur_loc); + } +} + +void HELPER(afl_compcov_log_32)(void* uc_ptr, uint64_t cur_loc, uint64_t arg1, + uint64_t arg2) { + + u8* afl_area_ptr = ((struct uc_struct*)uc_ptr)->afl_area_ptr; + + if ((arg1 & 0xff) == (arg2 & 0xff)) { + INC_AFL_AREA(cur_loc); + if ((arg1 & 0xffff) == (arg2 & 0xffff)) { + INC_AFL_AREA(cur_loc +1); + if ((arg1 & 0xffffff) == (arg2 & 0xffffff)) { + INC_AFL_AREA(cur_loc +2); + } + } + } +} + +void HELPER(afl_compcov_log_64)(void* uc_ptr, uint64_t cur_loc, uint64_t arg1, + uint64_t arg2) { + + u8* afl_area_ptr = ((struct uc_struct*)uc_ptr)->afl_area_ptr; + + if ((arg1 & 0xff) == (arg2 & 0xff)) { + INC_AFL_AREA(cur_loc); + if ((arg1 & 0xffff) == (arg2 & 0xffff)) { + INC_AFL_AREA(cur_loc +1); + if ((arg1 & 0xffffff) == (arg2 & 0xffffff)) { + INC_AFL_AREA(cur_loc +2); + if ((arg1 & 0xffffffff) == (arg2 & 0xffffffff)) { + INC_AFL_AREA(cur_loc +3); + if ((arg1 & 0xffffffffff) == (arg2 & 0xffffffffff)) { + INC_AFL_AREA(cur_loc +4); + if ((arg1 & 0xffffffffffff) == (arg2 & 0xffffffffffff)) { + INC_AFL_AREA(cur_loc +5); + if ((arg1 & 0xffffffffffffff) == (arg2 & 0xffffffffffffff)) { + INC_AFL_AREA(cur_loc +6); + } + } + } + } + } + } + } +} + diff --git a/unicorn_mode/patches/compcov.diff b/unicorn_mode/patches/compcov.diff new file mode 100644 index 00000000..8ec867d1 --- /dev/null +++ b/unicorn_mode/patches/compcov.diff @@ -0,0 +1,113 @@ +diff --git a/include/uc_priv.h b/include/uc_priv.h +index 22f494e..1aa7b3a 100644 +--- a/include/uc_priv.h ++++ b/include/uc_priv.h +@@ -245,6 +245,12 @@ struct uc_struct { + uint32_t target_page_align; + uint64_t next_pc; // save next PC for some special cases + bool hook_insert; // insert new hook at begin of the hook list (append by default) ++ ++#ifdef UNICORN_AFL ++ unsigned char *afl_area_ptr; ++ int afl_compcov_level; ++ unsigned int afl_inst_rms; ++#endif + }; + + // Metadata stub for the variable-size cpu context used with uc_context_*() +diff --git a/qemu/target-i386/translate.c b/qemu/target-i386/translate.c +index 36fae09..196d346 100644 +--- a/qemu/target-i386/translate.c ++++ b/qemu/target-i386/translate.c +@@ -33,6 +33,12 @@ + + #include "uc_priv.h" + ++#if defined(UNICORN_AFL) ++#include "../../afl-unicorn-cpu-translate-inl.h" ++#else ++#define afl_gen_compcov(a,b,c,d,e,f) do {} while (0) ++#endif ++ + #define PREFIX_REPZ 0x01 + #define PREFIX_REPNZ 0x02 + #define PREFIX_LOCK 0x04 +@@ -1555,6 +1561,7 @@ static void gen_op(DisasContext *s, int op, TCGMemOp ot, int d) + case OP_SUBL: + tcg_gen_mov_tl(tcg_ctx, cpu_cc_srcT, *cpu_T[0]); + tcg_gen_sub_tl(tcg_ctx, *cpu_T[0], *cpu_T[0], *cpu_T[1]); ++ afl_gen_compcov(tcg_ctx, s->pc, *cpu_T[0], *cpu_T[1], ot, d == OR_EAX); + gen_op_st_rm_T0_A0(s, ot, d); + gen_op_update2_cc(tcg_ctx); + set_cc_op(s, CC_OP_SUBB + ot); +@@ -1582,6 +1589,7 @@ static void gen_op(DisasContext *s, int op, TCGMemOp ot, int d) + tcg_gen_mov_tl(tcg_ctx, cpu_cc_src, *cpu_T[1]); + tcg_gen_mov_tl(tcg_ctx, cpu_cc_srcT, *cpu_T[0]); + tcg_gen_sub_tl(tcg_ctx, cpu_cc_dst, *cpu_T[0], *cpu_T[1]); ++ afl_gen_compcov(tcg_ctx, s->pc, *cpu_T[0], *cpu_T[1], ot, d == OR_EAX); + set_cc_op(s, CC_OP_SUBB + ot); + break; + } +diff --git a/qemu/tcg-runtime.c b/qemu/tcg-runtime.c +index 21b022a..14d7891 100644 +--- a/qemu/tcg-runtime.c ++++ b/qemu/tcg-runtime.c +@@ -31,9 +31,14 @@ + + #define DEF_HELPER_FLAGS_2(name, flags, ret, t1, t2) \ + dh_ctype(ret) HELPER(name) (dh_ctype(t1), dh_ctype(t2)); ++#define DEF_HELPER_FLAGS_4(name, flags, ret, t1, t2, t3, t4) \ ++ dh_ctype(ret) HELPER(name) (dh_ctype(t1), dh_ctype(t2), dh_ctype(t3), dh_ctype(t4)); + + #include "tcg-runtime.h" + ++#ifdef UNICORN_AFL ++#include "../afl-unicorn-tcg-runtime-inl.h" ++#endif + + /* 32-bit helpers */ + +diff --git a/qemu/tcg/tcg-op.h b/qemu/tcg/tcg-op.h +index 38b7dd9..c5a9af9 100644 +--- a/qemu/tcg/tcg-op.h ++++ b/qemu/tcg/tcg-op.h +@@ -27,6 +27,10 @@ + + int gen_new_label(TCGContext *); + ++#ifdef UNICORN_AFL ++#include "../../afl-unicorn-tcg-op-inl.h" ++#endif ++ + static inline void gen_uc_tracecode(TCGContext *tcg_ctx, int32_t size, int32_t type, void *uc, uint64_t pc) + { + TCGv_i32 tsize = tcg_const_i32(tcg_ctx, size); +diff --git a/qemu/tcg/tcg-runtime.h b/qemu/tcg/tcg-runtime.h +index 23a0c37..90b993c 100644 +--- a/qemu/tcg/tcg-runtime.h ++++ b/qemu/tcg/tcg-runtime.h +@@ -14,3 +14,9 @@ DEF_HELPER_FLAGS_2(sar_i64, TCG_CALL_NO_RWG_SE, s64, s64, s64) + + DEF_HELPER_FLAGS_2(mulsh_i64, TCG_CALL_NO_RWG_SE, s64, s64, s64) + DEF_HELPER_FLAGS_2(muluh_i64, TCG_CALL_NO_RWG_SE, i64, i64, i64) ++ ++#ifdef UNICORN_AFL ++DEF_HELPER_FLAGS_4(afl_compcov_log_16, 0, void, ptr, i64, i64, i64) ++DEF_HELPER_FLAGS_4(afl_compcov_log_32, 0, void, ptr, i64, i64, i64) ++DEF_HELPER_FLAGS_4(afl_compcov_log_64, 0, void, ptr, i64, i64, i64) ++#endif +diff --git a/qemu/unicorn_common.h b/qemu/unicorn_common.h +index 8dcbb3e..11e18b4 100644 +--- a/qemu/unicorn_common.h ++++ b/qemu/unicorn_common.h +@@ -84,6 +84,10 @@ static inline void uc_common_init(struct uc_struct* uc) + + if (!uc->release) + uc->release = release_common; ++ ++#ifdef UNICORN_AFL ++ uc->afl_area_ptr = 0; ++#endif + } + + #endif diff --git a/unicorn_mode/samples/compcov_x64/COMPILE.md b/unicorn_mode/samples/compcov_x64/COMPILE.md new file mode 100644 index 00000000..35de7ad8 --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/COMPILE.md @@ -0,0 +1,19 @@ +# Compiling compcov_target.c + +compcov_target.c was compiled without optimization, position-independent, +and without standard libraries using the following command line: + +``` +gcc -o compcov_target.elf compcov_target.c -fPIC -O0 -nostdlib +``` + +The .text section from the resulting ELF binary was then extracted to create +the raw binary blob that is loaded and emulated by compcov_test_harness.py: + +``` +objcopy -O binary --only-section=.text compcov_target.elf compcov_target.bin +``` + +Note that the output of this is padded with nulls for 16-byte alignment. This is +important when emulating it, as NOPs will be added after the return of main() +as necessary. diff --git a/unicorn_mode/samples/compcov_x64/compcov_target.bin b/unicorn_mode/samples/compcov_x64/compcov_target.bin new file mode 100644 index 00000000..091bf1db --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/compcov_target.bin Binary files differdiff --git a/unicorn_mode/samples/compcov_x64/compcov_target.c b/unicorn_mode/samples/compcov_x64/compcov_target.c new file mode 100644 index 00000000..eb1205b1 --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/compcov_target.c @@ -0,0 +1,28 @@ +/* + * Sample target file to test afl-unicorn fuzzing capabilities. + * This is a very trivial example that will crash pretty easily + * in several different exciting ways. + * + * Input is assumed to come from a buffer located at DATA_ADDRESS + * (0x00300000), so make sure that your Unicorn emulation of this + * puts user data there. + * + * Written by Andrea Fioraldi + */ + +// Magic address where mutated data will be placed +#define DATA_ADDRESS 0x00300000 + +int main(void) { + unsigned int *data_buf = (unsigned int *) DATA_ADDRESS; + + if (data_buf[0] == 0xabadcafe) { + // Cause an 'invalid read' crash if data[0..3] == '\x01\x02\x03\x04' + unsigned char invalid_read = *(unsigned char *) 0x00000000; + } else if (data_buf[1] == data_buf[2] + 0x4141) { + // Cause an 'invalid read' crash if (0x10 < data[0] < 0x20) and data[1] > data[2] + unsigned char invalid_read = *(unsigned char *) 0x00000000; + } + + return 0; +} diff --git a/unicorn_mode/samples/compcov_x64/compcov_target.elf b/unicorn_mode/samples/compcov_x64/compcov_target.elf new file mode 100755 index 00000000..7015fb46 --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/compcov_target.elf Binary files differdiff --git a/unicorn_mode/samples/compcov_x64/compcov_test_harness.py b/unicorn_mode/samples/compcov_x64/compcov_test_harness.py new file mode 100644 index 00000000..5698cbc8 --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/compcov_test_harness.py @@ -0,0 +1,170 @@ +""" + Simple test harness for AFL's Unicorn Mode. + + This loads the compcov_target.bin binary (precompiled as MIPS code) into + Unicorn's memory map for emulation, places the specified input into + compcov_target's buffer (hardcoded to be at 0x300000), and executes 'main()'. + If any crashes occur during emulation, this script throws a matching signal + to tell AFL that a crash occurred. + + Run under AFL as follows: + + $ cd <afl_path>/unicorn_mode/samples/simple/ + $ ../../../afl-fuzz -U -m none -i ./sample_inputs -o ./output -- python compcov_test_harness.py @@ +""" + +import argparse +import os +import signal + +from unicorn import * +from unicorn.x86_const import * + +# Path to the file containing the binary to emulate +BINARY_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'compcov_target.bin') + +# Memory map for the code to be tested +CODE_ADDRESS = 0x00100000 # Arbitrary address where code to test will be loaded +CODE_SIZE_MAX = 0x00010000 # Max size for the code (64kb) +STACK_ADDRESS = 0x00200000 # Address of the stack (arbitrarily chosen) +STACK_SIZE = 0x00010000 # Size of the stack (arbitrarily chosen) +DATA_ADDRESS = 0x00300000 # Address where mutated data will be placed +DATA_SIZE_MAX = 0x00010000 # Maximum allowable size of mutated data + +try: + # If Capstone is installed then we'll dump disassembly, otherwise just dump the binary. + from capstone import * + cs = Cs(CS_ARCH_X86, CS_MODE_64) + def unicorn_debug_instruction(uc, address, size, user_data): + mem = uc.mem_read(address, size) + for (cs_address, cs_size, cs_mnemonic, cs_opstr) in cs.disasm_lite(bytes(mem), size): + print(" Instr: {:#016x}:\t{}\t{}".format(address, cs_mnemonic, cs_opstr)) +except ImportError: + def unicorn_debug_instruction(uc, address, size, user_data): + print(" Instr: addr=0x{0:016x}, size=0x{1:016x}".format(address, size)) + +def unicorn_debug_block(uc, address, size, user_data): + print("Basic Block: addr=0x{0:016x}, size=0x{1:016x}".format(address, size)) + +def unicorn_debug_mem_access(uc, access, address, size, value, user_data): + if access == UC_MEM_WRITE: + print(" >>> Write: addr=0x{0:016x} size={1} data=0x{2:016x}".format(address, size, value)) + else: + print(" >>> Read: addr=0x{0:016x} size={1}".format(address, size)) + +def unicorn_debug_mem_invalid_access(uc, access, address, size, value, user_data): + if access == UC_MEM_WRITE_UNMAPPED: + print(" >>> INVALID Write: addr=0x{0:016x} size={1} data=0x{2:016x}".format(address, size, value)) + else: + print(" >>> INVALID Read: addr=0x{0:016x} size={1}".format(address, size)) + +def force_crash(uc_error): + # This function should be called to indicate to AFL that a crash occurred during emulation. + # Pass in the exception received from Uc.emu_start() + mem_errors = [ + UC_ERR_READ_UNMAPPED, UC_ERR_READ_PROT, UC_ERR_READ_UNALIGNED, + UC_ERR_WRITE_UNMAPPED, UC_ERR_WRITE_PROT, UC_ERR_WRITE_UNALIGNED, + UC_ERR_FETCH_UNMAPPED, UC_ERR_FETCH_PROT, UC_ERR_FETCH_UNALIGNED, + ] + if uc_error.errno in mem_errors: + # Memory error - throw SIGSEGV + os.kill(os.getpid(), signal.SIGSEGV) + elif uc_error.errno == UC_ERR_INSN_INVALID: + # Invalid instruction - throw SIGILL + os.kill(os.getpid(), signal.SIGILL) + else: + # Not sure what happened - throw SIGABRT + os.kill(os.getpid(), signal.SIGABRT) + +def main(): + + parser = argparse.ArgumentParser(description="Test harness for compcov_target.bin") + parser.add_argument('input_file', type=str, help="Path to the file containing the mutated input to load") + parser.add_argument('-d', '--debug', default=False, action="store_true", help="Enables debug tracing") + args = parser.parse_args() + + # Instantiate a MIPS32 big endian Unicorn Engine instance + uc = Uc(UC_ARCH_X86, UC_MODE_64) + + if args.debug: + uc.hook_add(UC_HOOK_BLOCK, unicorn_debug_block) + uc.hook_add(UC_HOOK_CODE, unicorn_debug_instruction) + uc.hook_add(UC_HOOK_MEM_WRITE | UC_HOOK_MEM_READ, unicorn_debug_mem_access) + uc.hook_add(UC_HOOK_MEM_WRITE_UNMAPPED | UC_HOOK_MEM_READ_INVALID, unicorn_debug_mem_invalid_access) + + #--------------------------------------------------- + # Load the binary to emulate and map it into memory + + print("Loading data input from {}".format(args.input_file)) + binary_file = open(BINARY_FILE, 'rb') + binary_code = binary_file.read() + binary_file.close() + + # Apply constraints to the mutated input + if len(binary_code) > CODE_SIZE_MAX: + print("Binary code is too large (> {} bytes)".format(CODE_SIZE_MAX)) + return + + # Write the mutated command into the data buffer + uc.mem_map(CODE_ADDRESS, CODE_SIZE_MAX) + uc.mem_write(CODE_ADDRESS, binary_code) + + # Set the program counter to the start of the code + start_address = CODE_ADDRESS # Address of entry point of main() + end_address = CODE_ADDRESS + 0x55 # Address of last instruction in main() + uc.reg_write(UC_X86_REG_RIP, start_address) + + #----------------- + # Setup the stack + + uc.mem_map(STACK_ADDRESS, STACK_SIZE) + uc.reg_write(UC_X86_REG_RSP, STACK_ADDRESS + STACK_SIZE) + + #----------------------------------------------------- + # Emulate 1 instruction to kick off AFL's fork server + # THIS MUST BE DONE BEFORE LOADING USER DATA! + # If this isn't done every single run, the AFL fork server + # will not be started appropriately and you'll get erratic results! + # It doesn't matter what this returns with, it just has to execute at + # least one instruction in order to get the fork server started. + + # Execute 1 instruction just to startup the forkserver + print("Starting the AFL forkserver by executing 1 instruction") + try: + uc.emu_start(uc.reg_read(UC_X86_REG_RIP), 0, 0, count=1) + except UcError as e: + print("ERROR: Failed to execute a single instruction (error: {})!".format(e)) + return + + #----------------------------------------------- + # Load the mutated input and map it into memory + + # Load the mutated input from disk + print("Loading data input from {}".format(args.input_file)) + input_file = open(args.input_file, 'rb') + input = input_file.read() + input_file.close() + + # Apply constraints to the mutated input + if len(input) > DATA_SIZE_MAX: + print("Test input is too long (> {} bytes)".format(DATA_SIZE_MAX)) + return + + # Write the mutated command into the data buffer + uc.mem_map(DATA_ADDRESS, DATA_SIZE_MAX) + uc.mem_write(DATA_ADDRESS, input) + + #------------------------------------------------------------ + # Emulate the code, allowing it to process the mutated input + + print("Executing until a crash or execution reaches 0x{0:016x}".format(end_address)) + try: + result = uc.emu_start(uc.reg_read(UC_X86_REG_RIP), end_address, timeout=0, count=0) + except UcError as e: + print("Execution failed with error: {}".format(e)) + force_crash(e) + + print("Done.") + +if __name__ == "__main__": + main() diff --git a/unicorn_mode/samples/compcov_x64/sample_inputs/sample1.bin b/unicorn_mode/samples/compcov_x64/sample_inputs/sample1.bin new file mode 100644 index 00000000..445c7245 --- /dev/null +++ b/unicorn_mode/samples/compcov_x64/sample_inputs/sample1.bin @@ -0,0 +1 @@ +00000000000000000000000000000000 \ No newline at end of file diff --git a/unicorn_mode/samples/simple/COMPILE.md b/unicorn_mode/samples/simple/COMPILE.md index bd4a66c6..f7bf5b50 100644 --- a/unicorn_mode/samples/simple/COMPILE.md +++ b/unicorn_mode/samples/simple/COMPILE.md @@ -1,5 +1,4 @@ -Compiling simple_target.c -========================== +# Compiling simple_target.c You shouldn't need to compile simple_target.c since a MIPS binary version is pre-built and shipped with afl-unicorn. This file documents how the binary @@ -38,4 +37,4 @@ mips-linux-gnu-gcc -o simple_target.elf simple_target.c -fPIC -O0 -nostdlib Note that the output of this is padded with nulls for 16-byte alignment. This is important when emulating it, as NOPs will be added after the return of main() -as necessary. \ No newline at end of file +as necessary. |