diff options
| -rw-r--r-- | README.md | 6 | ||||
| -rw-r--r-- | TODO.md | 2 | ||||
| -rw-r--r-- | docs/Changelog.md | 14 | ||||
| -rw-r--r-- | docs/env_variables.md | 76 | ||||
| -rw-r--r-- | gcc_plugin/GNUmakefile | 2 | ||||
| -rw-r--r-- | gcc_plugin/Makefile | 161 | ||||
| -rw-r--r-- | gcc_plugin/afl-gcc-rt.o.c | 3 | ||||
| -rw-r--r-- | libdislocator/GNUmakefile | 44 | ||||
| -rw-r--r-- | libdislocator/Makefile | 45 | ||||
| -rw-r--r-- | llvm_mode/GNUmakefile | 29 | ||||
| -rw-r--r-- | llvm_mode/README.ctx.md | 22 | ||||
| -rw-r--r-- | llvm_mode/README.instrim.md | 7 | ||||
| -rw-r--r-- | llvm_mode/README.md | 30 | ||||
| -rw-r--r-- | llvm_mode/README.ngram.md | 12 | ||||
| -rw-r--r-- | llvm_mode/afl-clang-fast.c | 340 | ||||
| -rw-r--r-- | llvm_mode/afl-llvm-pass.so.cc | 89 | ||||
| -rw-r--r-- | llvm_mode/afl-llvm-rt.o.c | 15 | ||||
| -rw-r--r-- | llvm_mode/llvm-ngram-coverage.h | 2 | ||||
| -rw-r--r-- | llvm_mode/split-compares-pass.so.cc | 4 | ||||
| -rw-r--r-- | src/afl-common.c | 15 |
20 files changed, 664 insertions, 254 deletions
diff --git a/README.md b/README.md index e1ca5949..8d167f7e 100644 --- a/README.md +++ b/README.md @@ -74,10 +74,12 @@ | Non-colliding coverage | | x(4) | | (x)(5) | | | InsTrim | | x | | | | | Ngram prev_loc coverage | | x(6) | | | | + | Context coverage | | x | | | | + | Snapshot LKM support | | x | | (x)(5) | | neverZero: - (1) only in LLVM >= 9.0 due to a bug in LLVM in previous versions + (1) default for LLVM >= 9.0, env var for older version due an efficiency bug in llvm <= 8 (2) GCC creates non-performant code, hence it is disabled in gcc_plugin @@ -87,7 +89,7 @@ (5) upcoming, development in the branch - (6) not compatible with LTO and InsTrim modes + (6) not compatible with LTO and InsTrim and needs at least LLVM >= 4.1 So all in all this is the best-of afl that is currently out there :-) diff --git a/TODO.md b/TODO.md index 91297332..ffb8f647 100644 --- a/TODO.md +++ b/TODO.md @@ -11,7 +11,7 @@ - context sensitive branch coverage in llvm_mode - random crc32 HASH_CONST per run? because with 65536 paths we have collisions - + - namespace for targets? e.g. network ## Further down the road diff --git a/docs/Changelog.md b/docs/Changelog.md index 72336b11..7af8a62e 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -34,16 +34,22 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. the last 5 queue entries - rare: puts focus on queue entries that hits rare branches, also ignores runtime - - llvm_mode: added Control Flow Integrity sanitizer (AFL_USE_CFISAN) + - llvm_mode: + - added Control Flow Integrity sanitizer (AFL_USE_CFISAN) + - added AFL_LLVM_INSTRUMENT option to control the instrumentation type + easier: DEFAULT, CFG (INSTRIM), LTO, CTX, NGRAM-x (x=2-16) + - made USE_TRACE_PC compile obsolete - LTO collision free instrumented added in llvm_mode with afl-clang-lto - note that this mode is amazing, but quite some targets won't compile - Added llvm_mode NGRAM prev_loc coverage by Adrean Herrera (https://github.com/adrianherrera/afl-ngram-pass/), activate by setting - AFL_LLVM_NGRAM_SIZE + AFL_LLVM_INSTRUMENT=NGRAM-<value> or AFL_LLVM_NGRAM_SIZE=<value> + - Added llvm_mode context sensitive branch coverage, activated by setting + AFL_LLVM_INSTRUMENT=CTX or AFL_LLVM_CTX=1 - llvm_mode InsTrim mode: - removed workaround for bug where paths were not instrumented and imported fix by author - - made skipping 1 block functions an option and is disable by default, + - made skipping 1 block functions an option and is disabled by default, set AFL_LLVM_INSTRIM_SKIPSINGLEBLOCK=1 to re-enable this - qemu_mode: - qemu_mode now uses solely the internal capstone version to fix builds @@ -53,6 +59,8 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - now supports hang mode `-H` to minimize hangs - fixed potential afl-tmin missbehavior for targets with multiple hangs - Pressing Control-c in afl-cmin did not terminate it for some OS + - the custom API was rewritten and is now the same for Python and shared + libraries. ### Version ++2.62c (release): diff --git a/docs/env_variables.md b/docs/env_variables.md index ae283b1c..802e7bd0 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -93,6 +93,17 @@ of the settings discussed in section #1, with the exception of: Then there are a few specific features that are only available in llvm_mode: +### Select the instrumentation mode + + - AFL_LLVM_INSTRUMENT - this configures the instrumentation mode. + Available options: + DEFAULT - classic AFL (map[cur_loc ^ prev_loc >> 1]++) + CFG - InsTrim instrumentation (see below) + LTO - LTO instrumentation (see below) + CTX - context sensitive instrumentation (see below) + NGRAM-x - deeper previous location coverage (from NGRAM-2 up to NGRAM-16) + Only one can be used. + ### LTO This is a different kind way of instrumentation: first it compiles all @@ -112,9 +123,46 @@ Then there are a few specific features that are only available in llvm_mode: - AFL_LLVM_LTO_DONTWRITEID prevents that the highest location ID written into the instrumentation is set in a global variable - Instrim, LTO and ngram modes can not be used together. See llvm_mode/README.LTO.md for more information. +### INSTRIM + + This feature increases the speed by ~15% without any disadvantages. + + - Setting AFL_LLVM_INSTRIM or AFL_LLVM_INSTRUMENT=CFG to activates this mode + + - Setting AFL_LLVM_INSTRIM_LOOPHEAD=1 expands on INSTRIM to optimize loops. + afl-fuzz will only be able to see the path the loop took, but not how + many times it was called (unless it is a complex loop). + + - Setting AFL_LLVM_INSTRIM_SKIPSINGLEBLOCK=1 will skip instrumenting + functions with a single basic block. This is useful for most C and + some C++ targets. + + See llvm_mode/README.instrim.md + +### NGRAM + + - Setting AFL_LLVM_NGRAM_SIZE or AFL_LLVM_INSTRUMENT=NGRAM-{value} + activates ngram prev_loc coverage, good values are 2, 4 or 8 + (any value between 2 and 16 is valid). + It is highly recommended to increase the MAP_SIZE_POW2 definition in + config.h to at least 18 and maybe up to 20 for this as otherwise too + many map collisions occur. + + See llvm_mode/README.ctx.md + +### CTX + + - Setting AFL_LLVM_CTX or AFL_LLVM_INSTRUMENT=CTX + activates context sensitive branch coverage - meaning that each edge + is additionally combined with its caller. + It is highly recommended to increase the MAP_SIZE_POW2 definition in + config.h to at least 18 and maybe up to 20 for this as otherwise too + many map collisions occur. + + See llvm_mode/README.ngram.md + ### LAF-INTEL This great feature will split compares to series of single byte comparisons @@ -139,32 +187,6 @@ Then there are a few specific features that are only available in llvm_mode: See llvm_mode/README.whitelist.md for more information. -### INSTRIM - - This feature increases the speed by whopping 20% but at the cost of a - lower path discovery and therefore coverage. - - - Setting AFL_LLVM_INSTRIM activates this mode - - - Setting AFL_LLVM_INSTRIM_LOOPHEAD=1 expands on INSTRIM to optimize loops. - afl-fuzz will only be able to see the path the loop took, but not how - many times it was called (unless it is a complex loop). - - - Setting AFL_LLVM_INSTRIM_SKIPSINGLEBLOCK=1 will skip instrumenting - functions with a single basic block. This is useful for most C and - some C++ targets. - - Instrim, LTO and ngram modes can not be used together. - See llvm_mode/README.instrim.md - -### NGRAM - - - Setting AFL_LLVM_NGRAM_SIZE activates ngram prev_loc coverage, good - values are 2, 4 or 8. - - Instrim, LTO and ngram modes can not be used together. - See llvm_mode/README.ngram.md - ### NOT_ZERO - Setting AFL_LLVM_NOT_ZERO=1 during compilation will use counters diff --git a/gcc_plugin/GNUmakefile b/gcc_plugin/GNUmakefile index 4c7a0313..9a404966 100644 --- a/gcc_plugin/GNUmakefile +++ b/gcc_plugin/GNUmakefile @@ -29,7 +29,7 @@ MAN_PATH ?= $(PREFIX)/man/man8 VERSION = $(shell grep '^$(HASH)define VERSION ' ../config.h | cut -d '"' -f2) CFLAGS ?= -O3 -g -funroll-loops -D_FORTIFY_SOURCE=2 -override CFLAGS = -Wall -I../include -Wno-pointer-sign \ +CFLAGS = -Wall -I../include -Wno-pointer-sign \ -DAFL_PATH=\"$(HELPER_PATH)\" -DBIN_PATH=\"$(BIN_PATH)\" \ -DGCC_VERSION=\"$(GCCVER)\" -DGCC_BINDIR=\"$(GCCBINDIR)\" \ -Wno-unused-function diff --git a/gcc_plugin/Makefile b/gcc_plugin/Makefile index 0b306dde..1c6f365f 100644 --- a/gcc_plugin/Makefile +++ b/gcc_plugin/Makefile @@ -1,2 +1,159 @@ -all: - @echo please use GNU make, thanks! +# +# american fuzzy lop++ - GCC plugin instrumentation +# ----------------------------------------------- +# +# Written by Austin Seipp <aseipp@pobox.com> and +# Laszlo Szekeres <lszekeres@google.com> and +# Michal Zalewski and +# Heiko Eißfeldt <heiko@hexco.de> +# +# GCC integration design is based on the LLVM design, which comes +# from Laszlo Szekeres. +# +# Copyright 2015 Google Inc. All rights reserved. +# Copyright 2019-2020 AFLplusplus Project. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# + +PREFIX ?= /usr/local +HELPER_PATH ?= $(PREFIX)/lib/afl +BIN_PATH ?= $(PREFIX)/bin +DOC_PATH ?= $(PREFIX)/share/doc/afl +MAN_PATH ?= $(PREFIX)/man/man8 + +VERSION = $(shell grep '^$(HASH)define VERSION ' ../config.h | cut -d '"' -f2) +VERSION:sh= grep '^$(HASH)define VERSION ' ../config.h | cut -d '"' -f2 + +CFLAGS ?= -O3 -g -funroll-loops -D_FORTIFY_SOURCE=2 +CFLAGS = -Wall -I../include -Wno-pointer-sign \ + -DAFL_PATH=\"$(HELPER_PATH)\" -DBIN_PATH=\"$(BIN_PATH)\" \ + -DGCC_VERSION=\"$(GCCVER)\" -DGCC_BINDIR=\"$(GCCBINDIR)\" \ + -Wno-unused-function + +CXXFLAGS ?= -O3 -g -funroll-loops -D_FORTIFY_SOURCE=2 +CXXEFLAGS := $(CXXFLAGS) -Wall + +CC ?= gcc +CXX ?= g++ + +MYCC=$(CC:clang=gcc) +MYCXX=$(CXX:clang++=g++) + +PLUGIN_PATH = $(shell $(MYCC) -print-file-name=plugin) +PLUGIN_PATH:sh= $(MYCC) -print-file-name=plugin +PLUGIN_FLAGS = -fPIC -fno-rtti -I"$(PLUGIN_PATH)/include" +HASH=\# + +GCCVER = $(shell $(MYCC) --version 2>/dev/null | awk 'NR == 1 {print $$NF}') +GCCBINDIR = $(shell dirname `command -v $(MYCC)` 2>/dev/null ) + +_SHMAT_OK= $(shell echo '$(HASH)include <sys/ipc.h>@$(HASH)include <sys/shm.h>@int main() { int _id = shmget(IPC_PRIVATE, 65536, IPC_CREAT | IPC_EXCL | 0600); shmctl(_id, IPC_RMID, 0); return 0;}' | tr @ '\n' | $(MYCC) -x c - -o .test2 2>/dev/null && echo 1 || echo 0 ; rm -f .test2 ) +_SHMAT_OK:sh= echo '$(HASH)include <sys/ipc.h>@$(HASH)include <sys/shm.h>@int main() { int _id = shmget(IPC_PRIVATE, 65536, IPC_CREAT | IPC_EXCL | 0600); shmctl(_id, IPC_RMID, 0); return 0;}' | tr @ '\n' | $(MYCC) -x c - -o .test2 2>/dev/null && echo 1 || echo 0 ; rm -f .test2 + +IGNORE_MMAP=$(TEST_MMAP:1=0) +__SHMAT_OK=$(_SHMAT_OK)$(IGNORE_MMAP) +___SHMAT_OK=$(__SHMAT_OK:10=0) +SHMAT_OK=$(___SHMAT_OK:1=1) +_CFLAGS_ADD=$(SHMAT_OK:1=) +CFLAGS_ADD=$(_CFLAGS_ADD:0=-DUSEMMAP=1) + +_LDFLAGS_ADD=$(SHMAT_OK:1=) +LDFLAGS_ADD=$(_LDFLAGS_ADD:0=-lrt) + +CFLAGS += $(CFLAGS_ADD) +LDFLAGS += $(LDFLAGS_ADD) + +PROGS = ../afl-gcc-fast ../afl-gcc-pass.so ../afl-gcc-rt.o + + +all: test_shm test_deps $(PROGS) afl-gcc-fast.8 test_build all_done + +debug: + @echo _SHMAT_OK = $(_SHMAT_OK) + @echo IGNORE_MMAP = $(IGNORE_MMAP) + @echo __SHMAT_OK = $(__SHMAT_OK) + @echo ___SHMAT_OK = $(___SHMAT_OK) + @echo SHMAT_OK = $(SHMAT_OK) + +test_shm: + @if [ "$(SHMAT_OK)" == "1" ]; then \ + echo "[+] shmat seems to be working."; \ + rm -f .test2; \ + else \ + echo "[-] shmat seems not to be working, switching to mmap implementation"; \ + fi + +test_deps: + @echo "[*] Checking for working '$(MYCC)'..." + @type $(MYCC) >/dev/null 2>&1 || ( echo "[-] Oops, can't find '$(MYCC)'. Make sure that it's in your \$$PATH (or set \$$CC and \$$CXX)."; exit 1 ) +# @echo "[*] Checking for gcc for plugin support..." +# @$(MYCC) -v 2>&1 | grep -q -- --enable-plugin || ( echo "[-] Oops, this gcc has not been configured with plugin support."; exit 1 ) + @echo "[*] Checking for gcc plugin development header files..." + @test -d `$(MYCC) -print-file-name=plugin`/include || ( echo "[-] Oops, can't find gcc header files. Be sure to install 'gcc-X-plugin-dev'."; exit 1 ) + @echo "[*] Checking for '../afl-showmap'..." + @test -f ../afl-showmap || ( echo "[-] Oops, can't find '../afl-showmap'. Be sure to compile AFL first."; exit 1 ) + @echo "[+] All set and ready to build." + +afl-common.o: ../src/afl-common.c + $(MYCC) $(CFLAGS) -c $< -o $@ $(LDFLAGS) + +../afl-gcc-fast: afl-gcc-fast.c afl-common.o | test_deps + $(MYCC) -DAFL_GCC_CC=\"$(MYCC)\" -DAFL_GCC_CXX=\"$(MYCXX)\" $(CFLAGS) $< afl-common.o -o $@ $(LDFLAGS) + ln -sf afl-gcc-fast ../afl-g++-fast + +../afl-gcc-pass.so: afl-gcc-pass.so.cc | test_deps + $(MYCXX) $(CXXEFLAGS) $(PLUGIN_FLAGS) -shared $< -o $@ + +../afl-gcc-rt.o: afl-gcc-rt.o.c | test_deps + $(MYCC) $(CFLAGS) -fPIC -c $< -o $@ + +test_build: $(PROGS) + @echo "[*] Testing the CC wrapper and instrumentation output..." + unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. AFL_CC=$(CC) ../afl-gcc-fast $(CFLAGS) ../test-instr.c -o test-instr $(LDFLAGS) +# unset AFL_USE_ASAN AFL_USE_MSAN; AFL_INST_RATIO=100 AFL_PATH=. AFL_CC=$(CC) ../afl-gcc-fast $(CFLAGS) ../test-instr.c -o test-instr $(LDFLAGS) + ASAN_OPTIONS=detect_leaks=0 ../afl-showmap -m none -q -o .test-instr0 ./test-instr </dev/null + echo 1 | ASAN_OPTIONS=detect_leaks=0 ../afl-showmap -m none -q -o .test-instr1 ./test-instr + @rm -f test-instr + @cmp -s .test-instr0 .test-instr1; DR="$$?"; rm -f .test-instr0 .test-instr1; if [ "$$DR" = "0" ]; then echo; echo "Oops, the instrumentation does not seem to be behaving correctly!"; echo; echo "Please post to https://github.com/AFLplusplus/AFLplusplus/issues to troubleshoot the issue."; echo; exit 1; fi + @echo "[+] All right, the instrumentation seems to be working!" + +all_done: test_build + @echo "[+] All done! You can now use '../afl-gcc-fast' to compile programs." + +.NOTPARALLEL: clean + +vpath % .. +%.8: % + @echo .TH $* 8 `date "+%Y-%m-%d"` "afl++" > ../$@ + @echo .SH NAME >> ../$@ + @echo .B $* >> ../$@ + @echo >> ../$@ + @echo .SH SYNOPSIS >> ../$@ + @../$* -h 2>&1 | head -n 3 | tail -n 1 | sed 's/^\.\///' >> ../$@ + @echo >> ../$@ + @echo .SH OPTIONS >> ../$@ + @echo .nf >> ../$@ + @../$* -h 2>&1 | tail -n +4 >> ../$@ + @echo >> ../$@ + @echo .SH AUTHOR >> ../$@ + @echo "afl++ was written by Michal \"lcamtuf\" Zalewski and is maintained by Marc \"van Hauser\" Heuse <mh@mh-sec.de>, Heiko \"hexcoder-\" Eissfeldt <heiko.eissfeldt@hexco.de>, Andrea Fioraldi <andreafioraldi@gmail.com> and Dominik Maier <domenukk@gmail.com>" >> ../$@ + @echo The homepage of afl++ is: https://github.com/AFLplusplus/AFLplusplus >> ../$@ + @echo >> ../$@ + @echo .SH LICENSE >> ../$@ + @echo Apache License Version 2.0, January 2004 >> ../$@ + ln -sf afl-gcc-fast.8 ../afl-g++-fast.8 + +install: all + install -m 755 ../afl-gcc-fast $${DESTDIR}$(BIN_PATH) + install -m 755 ../afl-gcc-pass.so ../afl-gcc-rt.o $${DESTDIR}$(HELPER_PATH) + install -m 644 -T README.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.md + install -m 644 -T README.whitelist.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.whitelist.md + +clean: + rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 .test2 + rm -f $(PROGS) afl-common.o ../afl-g++-fast ../afl-g*-fast.8 diff --git a/gcc_plugin/afl-gcc-rt.o.c b/gcc_plugin/afl-gcc-rt.o.c index 77bb5325..30606150 100644 --- a/gcc_plugin/afl-gcc-rt.o.c +++ b/gcc_plugin/afl-gcc-rt.o.c @@ -25,6 +25,9 @@ #include "../config.h" #include "../types.h" +#ifdef USEMMAP +#include <stdio.h> +#endif #include <stdlib.h> #include <signal.h> #include <unistd.h> diff --git a/libdislocator/GNUmakefile b/libdislocator/GNUmakefile deleted file mode 100644 index 3ee37088..00000000 --- a/libdislocator/GNUmakefile +++ /dev/null @@ -1,44 +0,0 @@ -# -# american fuzzy lop++ - libdislocator -# ---------------------------------- -# -# Originally written by Michal Zalewski -# -# Copyright 2016 Google Inc. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# - -PREFIX ?= /usr/local -HELPER_PATH = $(PREFIX)/lib/afl - -VERSION = $(shell grep '^\#define VERSION ' ../config.h | cut -d '"' -f2) - -CFLAGS ?= -O3 -funroll-loops -override CFLAGS += -I ../include/ -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign - -ifdef USEHUGEPAGE - CFLAGS += -DUSEHUGEPAGE -endif - -all: libdislocator.so - -VPATH = .. -libdislocator.so: libdislocator.so.c ../config.h - $(CC) $(CFLAGS) -shared -fPIC $< -o ../$@ $(LDFLAGS) - -.NOTPARALLEL: clean - -clean: - rm -f *.o *.so *~ a.out core core.[1-9][0-9]* - rm -f ../libdislocator.so - -install: all - install -m 755 -d $${DESTDIR}$(HELPER_PATH) - install -m 755 ../libdislocator.so $${DESTDIR}$(HELPER_PATH) - install -m 644 README.dislocator.md $${DESTDIR}$(HELPER_PATH) - diff --git a/libdislocator/Makefile b/libdislocator/Makefile index 0b306dde..201f1e92 100644 --- a/libdislocator/Makefile +++ b/libdislocator/Makefile @@ -1,2 +1,43 @@ -all: - @echo please use GNU make, thanks! +# +# american fuzzy lop++ - libdislocator +# ---------------------------------- +# +# Originally written by Michal Zalewski +# +# Copyright 2016 Google Inc. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# + +PREFIX ?= /usr/local +HELPER_PATH = $(PREFIX)/lib/afl + +VERSION = $(shell grep '^\#define VERSION ' ../config.h | cut -d '"' -f2) + +CFLAGS ?= -O3 -funroll-loops -D_FORTIFY_SOURCE=2 +CFLAGS += -I ../include/ -Wall -g -Wno-pointer-sign + +CFLAGS_ADD=$(USEHUGEPAGE:1=-DUSEHUGEPAGE) +CFLAGS += $(CFLAGS_ADD) + +all: libdislocator.so + +VPATH = .. +libdislocator.so: libdislocator.so.c ../config.h + $(CC) $(CFLAGS) -shared -fPIC $< -o ../$@ $(LDFLAGS) + +.NOTPARALLEL: clean + +clean: + rm -f *.o *.so *~ a.out core core.[1-9][0-9]* + rm -f ../libdislocator.so + +install: all + install -m 755 -d $${DESTDIR}$(HELPER_PATH) + install -m 755 ../libdislocator.so $${DESTDIR}$(HELPER_PATH) + install -m 644 README.dislocator.md $${DESTDIR}$(HELPER_PATH) + diff --git a/llvm_mode/GNUmakefile b/llvm_mode/GNUmakefile index 006d115d..e3708efa 100644 --- a/llvm_mode/GNUmakefile +++ b/llvm_mode/GNUmakefile @@ -134,15 +134,22 @@ ifeq "$(AFL_REAL_LD)" "" endif endif +AFL_CLANG_FUSELD= +ifneq "$(AFL_CLANG_FLTO)" "" +ifeq "$(shell echo 'int main() {return 0; }' | $(CC) -x c - -fuse-ld=`command -v ld` -o .test 2>/dev/null && echo 1 || echo 0 ; rm -f .test )" "1" + AFL_CLANG_FUSELD=1 +endif +endif + CFLAGS ?= -O3 -funroll-loops -D_FORTIFY_SOURCE=2 override CFLAGS = -Wall \ -g -Wno-pointer-sign -I ../include/ \ -DAFL_PATH=\"$(HELPER_PATH)\" -DBIN_PATH=\"$(BIN_PATH)\" \ -DLLVM_BINDIR=\"$(LLVM_BINDIR)\" -DVERSION=\"$(VERSION)\" \ -DLLVM_VERSION=\"$(LLVMVER)\" -DAFL_CLANG_FLTO=\"$(AFL_CLANG_FLTO)\" \ - -DAFL_REAL_LD=\"$(AFL_REAL_LD)\" -Wno-unused-function + -DAFL_REAL_LD=\"$(AFL_REAL_LD)\" -DAFL_CLANG_FUSELD=\"$(AFL_CLANG_FUSELD)\" -Wno-unused-function ifdef AFL_TRACE_PC - CFLAGS += -DUSE_TRACE_PC=1 + $(info Compile option AFL_TRACE_PC is deprecated, just set AFL_LLVM_INSTRUMENT=PCGUARD to activate when compiling targets ) endif CXXFLAGS ?= -O3 -funroll-loops -D_FORTIFY_SOURCE=2 @@ -162,10 +169,6 @@ ifeq "$(shell uname)" "OpenBSD" CLANG_LFL += `$(LLVM_CONFIG) --libdir`/libLLVM.so endif -ifeq "$(shell echo 'int main() {return 0; }' | $(CC) -x c - -fuse-ld=`command -v ld` -o .test 2>/dev/null && echo 1 || echo 0 ; rm -f .test )" "1" - CFLAGS += -DAFL_CLANG_FUSELD=1 -endif - ifeq "$(shell echo '$(HASH)include <sys/ipc.h>@$(HASH)include <sys/shm.h>@int main() { int _id = shmget(IPC_PRIVATE, 65536, IPC_CREAT | IPC_EXCL | 0600); shmctl(_id, IPC_RMID, 0); return 0;}' | tr @ '\n' | $(CC) -x c - -o .test2 2>/dev/null && echo 1 || echo 0 ; rm -f .test2 )" "1" SHMAT_OK=1 else @@ -180,11 +183,7 @@ ifeq "$(TEST_MMAP)" "1" LDFLAGS += -lrt endif -ifndef AFL_TRACE_PC PROGS = ../afl-clang-fast ../afl-ld ../afl-llvm-pass.so ../afl-llvm-lto-whitelist.so ../afl-llvm-lto-instrumentation.so ../libLLVMInsTrim.so ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../compare-transform-pass.so ../split-compares-pass.so ../split-switches-pass.so ../cmplog-routines-pass.so ../cmplog-instructions-pass.so -else - PROGS = ../afl-clang-fast ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../compare-transform-pass.so ../split-compares-pass.so ../split-switches-pass.so ../cmplog-routines-pass.so ../cmplog-instructions-pass.so -endif # If prerequisites are not given, warn, do not build anything, and exit with code 0 ifeq "$(LLVMVER)" "" @@ -222,14 +221,10 @@ no_build: @printf "%b\\n" "\\033[0;31mPrerequisites are not met, skipping build llvm_mode\\033[0m" test_deps: -ifndef AFL_TRACE_PC @echo "[*] Checking for working 'llvm-config'..." ifneq "$(LLVM_APPLE)" "1" @type $(LLVM_CONFIG) >/dev/null 2>&1 || ( echo "[-] Oops, can't find 'llvm-config'. Install clang or set \$$LLVM_CONFIG or \$$PATH beforehand."; echo " (Sometimes, the binary will be named llvm-config-3.5 or something like that.)"; exit 1 ) endif -else - @echo "[!] Note: using -fsanitize=trace-pc mode (this will fail with older LLVM)." -endif @echo "[*] Checking for working '$(CC)'..." @type $(CC) >/dev/null 2>&1 || ( echo "[-] Oops, can't find '$(CC)'. Make sure that it's in your \$$PATH (or set \$$CC and \$$CXX)."; exit 1 ) @echo "[*] Checking for matching versions of '$(CC)' and '$(LLVM_CONFIG)'" @@ -327,12 +322,8 @@ all_done: test_build install: all install -d -m 755 $${DESTDIR}$(BIN_PATH) $${DESTDIR}$(HELPER_PATH) $${DESTDIR}$(DOC_PATH) $${DESTDIR}$(MISC_PATH) -ifndef AFL_TRACE_PC if [ -f ../afl-clang-fast -a -f ../libLLVMInsTrim.so -a -f ../afl-llvm-rt.o ]; then set -e; install -m 755 ../afl-clang-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-fast++; install -m 755 ../libLLVMInsTrim.so ../afl-llvm-pass.so ../afl-llvm-rt.o $${DESTDIR}$(HELPER_PATH); fi - if [ -f afl-clang-lto -a -f afl-ld ]; then set -e; install -m 755 afl-clang-lto $${DESTDIR}$(BIN_PATH); ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto++; install -m 755 afl-ld $${DESTDIR}$(HELPER_PATH); ln -sf afl-ld $${DESTDIR}$(HELPER_PATH)/ld; install -m 755 afl-llvm-lto-instrumentation.so $${DESTDIR}$(HELPER_PATH); install -m 755 afl-llvm-lto-whitelist.so $${DESTDIR}$(HELPER_PATH); fi -else - if [ -f ../afl-clang-fast -a -f ../afl-llvm-rt.o ]; then set -e; install -m 755 ../afl-clang-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-fast++; install -m 755 ../afl-llvm-rt.o $${DESTDIR}$(HELPER_PATH); fi -endif + if [ -f ../afl-clang-lto -a -f ../afl-ld ]; then set -e; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto++; install -m 755 ../afl-ld $${DESTDIR}$(HELPER_PATH); ln -sf afl-ld $${DESTDIR}$(HELPER_PATH)/ld; install -m 755 ../afl-llvm-lto-instrumentation.so $${DESTDIR}$(HELPER_PATH); install -m 755 ../afl-llvm-lto-whitelist.so $${DESTDIR}$(HELPER_PATH); fi if [ -f ../afl-llvm-rt-32.o ]; then set -e; install -m 755 ../afl-llvm-rt-32.o $${DESTDIR}$(HELPER_PATH); fi if [ -f ../afl-llvm-rt-64.o ]; then set -e; install -m 755 ../afl-llvm-rt-64.o $${DESTDIR}$(HELPER_PATH); fi if [ -f ../compare-transform-pass.so ]; then set -e; install -m 755 ../compare-transform-pass.so $${DESTDIR}$(HELPER_PATH); fi diff --git a/llvm_mode/README.ctx.md b/llvm_mode/README.ctx.md new file mode 100644 index 00000000..14255313 --- /dev/null +++ b/llvm_mode/README.ctx.md @@ -0,0 +1,22 @@ +# AFL Context Sensitive Branch Coverage + +## What is this? + +This is an LLVM-based implementation of the context sensitive branch coverage. + +Basically every function gets it's own ID and that ID is combined with the +edges of the called functions. + +So if both function A and function B call a function C, the coverage +collected in C will be different. + +In math the coverage is collected as follows: +`map[current_location_ID ^ previous_location_ID >> 1 ^ previous_callee_ID] += 1` + +## Usage + +Set the `AFL_LLVM_INSTRUMENT=CTX` or `AFL_LLVM_CTX=1` environment variable. + +It is highly recommended to increase the MAP_SIZE_POW2 definition in +config.h to at least 18 and maybe up to 20 for this as otherwise too +many map collisions occur. diff --git a/llvm_mode/README.instrim.md b/llvm_mode/README.instrim.md index 5c3f32c8..b905af11 100644 --- a/llvm_mode/README.instrim.md +++ b/llvm_mode/README.instrim.md @@ -5,13 +5,12 @@ InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing ## Introduction InsTrim uses CFG and markers to instrument just what is necessary in the -binary in llvm_mode. It is about 20-25% faster but as a cost has a lower -path discovery. +binary in llvm_mode. It is about 10-15% faster without disadvantages. ## Usage -Set the environment variable `AFL_LLVM_INSTRIM=1` during compilation of -the target. +Set the environment variable `AFL_LLVM_INSTRUMENT=CFG` or `AFL_LLVM_INSTRIM=1` +during compilation of the target. There is also an advanced mode which instruments loops in a way so that afl-fuzz can see which loop path has been selected but not being able to diff --git a/llvm_mode/README.md b/llvm_mode/README.md index e6c47c9c..805bb659 100644 --- a/llvm_mode/README.md +++ b/llvm_mode/README.md @@ -92,13 +92,33 @@ which C/C++ files to actually instrument. See [README.whitelist](README.whitelis For splitting memcmp, strncmp, etc. please see [README.laf-intel](README.laf-intel.md) -Then there is an optimized instrumentation strategy that uses CFGs and -markers to just instrument what is needed. This increases speed by 20-25% -however has a lower path discovery. -If you want to use this, set AFL_LLVM_INSTRIM=1 +Then there are different ways of instrumenting the target: + +1. There is an optimized instrumentation strategy that uses CFGs and +markers to just instrument what is needed. This increases speed by 10-15% +without any disadvantages +If you want to use this, set AFL_LLVM_INSTRUMENT=CFG or AFL_LLVM_INSTRIM=1 See [README.instrim](README.instrim.md) -A new instrumentation called CmpLog is also available as an alternative to +2. An even better instrumentation strategy uses LTO and link time +instrumentation. Note that not all targets can compile in this mode, however +if it works it is the best option you can use. +Simply use afl-clang-lto/afl-clang-lto++ to use this option. +See [README.lto](README.lto.md) + +3. Alternativly you can choose a completely different coverage method: + +3a. N-GRAM coverage - which combines the previous visited edges with the +current one. This explodes the map but on the other hand has proven to be +effective for fuzzing. +See [README.ngram](README.ngram.md) + +3b. Context sensitive coverage - which combines the visited edges with an +individual caller ID (the function that called the current one) +[README.ctx](README.ctx.md) + +Then - additionally to one of the instrumentation options above - there is +a very effective new instrumentation option called CmpLog as an alternative to laf-intel that allow AFL++ to apply mutations similar to Redqueen. See [README.cmplog](README.cmplog.md) diff --git a/llvm_mode/README.ngram.md b/llvm_mode/README.ngram.md index ef248198..de3ba432 100644 --- a/llvm_mode/README.ngram.md +++ b/llvm_mode/README.ngram.md @@ -13,8 +13,16 @@ is built on top of AFL's QEMU mode. This is essentially a port that uses LLVM vectorized instructions to achieve the same results when compiling source code. +In math the branch coverage is performed as follows: +`map[current_location ^ prev_location[0] >> 1 ^ prev_location[1] >> 1 ^ ... up to n-1`] += 1` + ## Usage The size of `n` (i.e., the number of branches to remember) is an option -that is specified in the `AFL_LLVM_NGRAM_SIZE` environment variable. -Good values are 2, 4 or 8. +that is specified either in the `AFL_LLVM_INSTRUMENT=NGRAM-{value}` or the +`AFL_LLVM_NGRAM_SIZE` environment variable. +Good values are 2, 4 or 8, valid are 2-16. + +It is highly recommended to increase the MAP_SIZE_POW2 definition in +config.h to at least 18 and maybe up to 20 for this as otherwise too +many map collisions occur. diff --git a/llvm_mode/afl-clang-fast.c b/llvm_mode/afl-clang-fast.c index 5fc09252..657d1a84 100644 --- a/llvm_mode/afl-clang-fast.c +++ b/llvm_mode/afl-clang-fast.c @@ -29,11 +29,13 @@ #include "types.h" #include "debug.h" #include "alloc-inl.h" +#include "llvm-ngram-coverage.h" #include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <string.h> +#include <strings.h> #include <limits.h> #include <assert.h> @@ -41,7 +43,7 @@ static u8 * obj_path; /* Path to runtime libraries */ static u8 **cc_params; /* Parameters passed to the real CC */ static u32 cc_par_cnt = 1; /* Param count, including argv0 */ static u8 llvm_fullpath[PATH_MAX]; -static u8 lto_mode; +static u8 instrument_mode; static u8 * lto_flag = AFL_CLANG_FLTO; static u8 * march_opt = CFLAGS_OPT; static u8 debug; @@ -49,6 +51,26 @@ static u8 cwd[4096]; static u8 cmplog_mode; u8 use_stdin = 0; /* dummy */ +enum { + + INSTRUMENT_CLASSIC = 0, + INSTRUMENT_AFL = 0, + INSTRUMENT_DEFAULT = 0, + INSTRUMENT_PCGUARD = 1, + INSTRUMENT_INSTRIM = 2, + INSTRUMENT_CFG = 2, + INSTRUMENT_LTO = 3, + INSTRUMENT_CTX = 4, + INSTRUMENT_NGRAM = 5 // + ngram value of 2-16 = 7 - 21 + +}; + +char instrument_mode_string[6][16] = { + + "DEFAULT", "PCGUARD", "CFG", "LTO", "CTX", + +}; + u8 *getthecwd() { static u8 fail[] = ""; @@ -150,26 +172,11 @@ static void edit_params(u32 argc, char **argv, char **envp) { has_llvm_config = (strlen(LLVM_BINDIR) > 0); - if (!strncmp(name, "afl-clang-lto", strlen("afl-clang-lto"))) { - -#ifdef USE_TRACE_PC - FATAL("afl-clang-lto does not work with TRACE_PC mode"); -#endif + if (instrument_mode == INSTRUMENT_LTO) if (lto_flag[0] != '-') FATAL( "Using afl-clang-lto is not possible because Makefile magic did not " "identify the correct -flto flag"); - if (getenv("AFL_LLVM_INSTRIM") != NULL) - FATAL("afl-clang-lto does not work with InsTrim mode"); - if (getenv("AFL_LLVM_NGRAM_SIZE") != NULL) - FATAL("afl-clang-lto does not work with ngram coverage mode"); - lto_mode = 1; - - } - - if (getenv("AFL_LLVM_NGRAM_SIZE") != NULL && - getenv("AFL_LLVM_INSTRIM") != NULL) - FATAL("AFL_LLVM_NGRAM_SIZE and AFL_LLVM_INSTRIM cannot be used together"); if (!strcmp(name, "afl-clang-fast++") || !strcmp(name, "afl-clang-lto++")) { @@ -260,17 +267,7 @@ static void edit_params(u32 argc, char **argv, char **envp) { } -#ifdef USE_TRACE_PC - - cc_params[cc_par_cnt++] = - "-fsanitize-coverage=trace-pc-guard"; // edge coverage by default - // cc_params[cc_par_cnt++] = "-mllvm"; - // cc_params[cc_par_cnt++] = - // "-fsanitize-coverage=trace-cmp,trace-div,trace-gep"; - // cc_params[cc_par_cnt++] = "-sanitizer-coverage-block-threshold=0"; -#else - - if (lto_mode) { + if (instrument_mode == INSTRUMENT_LTO) { char *old_path = getenv("PATH"); char *new_path = alloc_printf("%s:%s", AFL_PATH, old_path); @@ -299,8 +296,7 @@ static void edit_params(u32 argc, char **argv, char **envp) { } else - if (getenv("USE_TRACE_PC") || getenv("AFL_USE_TRACE_PC") || - getenv("AFL_LLVM_USE_TRACE_PC") || getenv("AFL_TRACE_PC")) { + if (instrument_mode == INSTRUMENT_PCGUARD) { cc_params[cc_par_cnt++] = "-fsanitize-coverage=trace-pc-guard"; // edge coverage by default @@ -310,15 +306,13 @@ static void edit_params(u32 argc, char **argv, char **envp) { cc_params[cc_par_cnt++] = "-Xclang"; cc_params[cc_par_cnt++] = "-load"; cc_params[cc_par_cnt++] = "-Xclang"; - if (getenv("AFL_LLVM_INSTRIM") != NULL || getenv("INSTRIM_LIB") != NULL) + if (instrument_mode == INSTRUMENT_CFG) cc_params[cc_par_cnt++] = alloc_printf("%s/libLLVMInsTrim.so", obj_path); else cc_params[cc_par_cnt++] = alloc_printf("%s/afl-llvm-pass.so", obj_path); } -#endif /* ^USE_TRACE_PC */ - cc_params[cc_par_cnt++] = "-Qunused-arguments"; /* Detect stray -v calls from ./configure scripts. */ @@ -389,7 +383,7 @@ static void edit_params(u32 argc, char **argv, char **envp) { if (getenv("AFL_USE_CFISAN")) { - if (!lto_mode) { + if (instrument_mode != INSTRUMENT_LTO) { uint32_t i = 0, found = 0; while (envp[i] != NULL && !found) @@ -403,15 +397,6 @@ static void edit_params(u32 argc, char **argv, char **envp) { } -#ifdef USE_TRACE_PC - - if (getenv("USE_TRACE_PC") || getenv("AFL_USE_TRACE_PC") || - getenv("AFL_LLVM_USE_TRACE_PC") || getenv("AFL_TRACE_PC")) - if (getenv("AFL_INST_RATIO")) - FATAL("AFL_INST_RATIO not available at compile time with 'trace-pc'."); - -#endif /* USE_TRACE_PC */ - if (!getenv("AFL_DONT_OPTIMIZE")) { cc_params[cc_par_cnt++] = "-g"; @@ -534,7 +519,7 @@ static void edit_params(u32 argc, char **argv, char **envp) { int main(int argc, char **argv, char **envp) { int i; - char *callname = "afl-clang-fast"; + char *callname = "afl-clang-fast", *ptr; if (getenv("AFL_DEBUG")) { @@ -545,83 +530,203 @@ int main(int argc, char **argv, char **envp) { be_quiet = 1; - if (strstr(argv[0], "afl-clang-lto") != NULL) callname = "afl-clang-lto"; +#ifdef USE_TRACE_PC + instrument_mode = INSTRUMENT_PCGUARD; +#endif - if (argc < 2 || strcmp(argv[1], "-h") == 0) { + if ((ptr = getenv("AFL_LLVM_INSTRUMENT")) != NULL) { + + if (strncasecmp(ptr, "cfg", strlen("cfg")) == 0 || + strncasecmp(ptr, "instrim", strlen("instrim")) == 0) + instrument_mode = INSTRUMENT_CFG; + else if (strncasecmp(ptr, "pc-guard", strlen("pc-guard")) == 0 || + strncasecmp(ptr, "pcguard", strlen("pcgard")) == 0) + instrument_mode = INSTRUMENT_PCGUARD; + else if (strncasecmp(ptr, "lto", strlen("lto")) == 0) + instrument_mode = INSTRUMENT_LTO; + else if (strncasecmp(ptr, "ctx", strlen("ctx")) == 0) { + + instrument_mode = INSTRUMENT_CTX; + setenv("AFL_LLVM_CTX", "1", 1); + + } else if (strncasecmp(ptr, "ngram", strlen("ngram")) == 0) { + + ptr += strlen("ngram"); + while (*ptr && (*ptr < '0' || *ptr > '9')) + ptr++; + if (!*ptr) + if ((ptr = getenv("AFL_LLVM_NGRAM_SIZE")) != NULL) + FATAL( + "you must set the NGRAM size with (e.g. for value 2) " + "AFL_LLVM_INSTRUMENT=ngram-2"); + instrument_mode = INSTRUMENT_NGRAM + atoi(ptr); + if (instrument_mode < INSTRUMENT_NGRAM + 2 || + instrument_mode > INSTRUMENT_NGRAM + NGRAM_SIZE_MAX) + FATAL( + "NGRAM instrumentation mode must be between 2 and NGRAM_SIZE_MAX " + "(%u)", + NGRAM_SIZE_MAX); + + ptr = alloc_printf("%u", instrument_mode - INSTRUMENT_NGRAM); + setenv("AFL_LLVM_NGRAM_SIZE", ptr, 1); + + } else if (strncasecmp(ptr, "classic", strlen("classic")) != 0 || + + strncasecmp(ptr, "default", strlen("default")) != 0 || + strncasecmp(ptr, "afl", strlen("afl")) != 0) + FATAL("unknown AFL_LLVM_INSTRUMENT value: %s", ptr); -#ifdef USE_TRACE_PC - printf(cCYA "afl-clang-fast" VERSION cRST - " [tpcg] by <lszekeres@google.com>\n") -#else - if (strstr(argv[0], "afl-clang-lto") == NULL) + } - printf("afl-clang-fast" VERSION " by <lszekeres@google.com>\n"); + if (getenv("USE_TRACE_PC") || getenv("AFL_USE_TRACE_PC") || + getenv("AFL_LLVM_USE_TRACE_PC") || getenv("AFL_TRACE_PC")) { - else { + if (instrument_mode == 0) + instrument_mode = INSTRUMENT_PCGUARD; + else if (instrument_mode != INSTRUMENT_PCGUARD) + FATAL("you can not set AFL_LLVM_INSTRUMENT and AFL_TRACE_PC together"); - printf("afl-clang-lto" VERSION - " by Marc \"vanHauser\" Heuse <mh@mh-sec.de>\n"); + } + + if (getenv("AFL_LLVM_INSTRIM") || getenv("INSTRIM") || + getenv("INSTRIM_LIB")) { + + if (instrument_mode == 0) + instrument_mode = INSTRUMENT_CFG; + else if (instrument_mode != INSTRUMENT_CFG) + FATAL( + "you can not set AFL_LLVM_INSTRUMENT and AFL_LLVM_INSTRIM together"); + + } + + if (getenv("AFL_LLVM_CTX")) { + + if (instrument_mode == 0) + instrument_mode = INSTRUMENT_CTX; + else if (instrument_mode != INSTRUMENT_CTX) + FATAL("you can not set AFL_LLVM_INSTRUMENT and AFL_LLVM_CTX together"); + + } + + if (getenv("AFL_LLVM_NGRAM_SIZE")) { + + if (instrument_mode == 0) { + + instrument_mode = INSTRUMENT_NGRAM + atoi(getenv("AFL_LLVM_NGRAM_SIZE")); + if (instrument_mode < INSTRUMENT_NGRAM + 2 || + instrument_mode > INSTRUMENT_NGRAM + NGRAM_SIZE_MAX) + FATAL( + "NGRAM instrumentation mode must be between 2 and NGRAM_SIZE_MAX " + "(%u)", + NGRAM_SIZE_MAX); + + } else if (instrument_mode != INSTRUMENT_NGRAM) + + FATAL( + "you can not set AFL_LLVM_INSTRUMENT and AFL_LLVM_NGRAM_SIZE " + "together"); + + } + + if (instrument_mode < INSTRUMENT_NGRAM) + ptr = instrument_mode_string[instrument_mode]; + else + ptr = alloc_printf("NGRAM-%u", instrument_mode - INSTRUMENT_NGRAM); + + if (strstr(argv[0], "afl-clang-lto") != NULL) { + + if (instrument_mode == 0 || instrument_mode == INSTRUMENT_LTO) { + + callname = "afl-clang-lto"; + instrument_mode = INSTRUMENT_LTO; + ptr = instrument_mode_string[instrument_mode]; + + } else { + + if (!be_quiet) + WARNF("afl-clang-lto called with mode %s, using that mode instead", + ptr); } -#endif /* ^USE_TRACE_PC */ - - SAYF( - "\n" - "%s[++] [options]\n" - "\n" - "This is a helper application for afl-fuzz. It serves as a drop-in " - "replacement\n" - "for clang, letting you recompile third-party code with the " - "required " - "runtime\n" - "instrumentation. A common use pattern would be one of the " - "following:\n\n" - - " CC=%s/afl-clang-fast ./configure\n" - " CXX=%s/afl-clang-fast++ ./configure\n\n" - - "In contrast to the traditional afl-clang tool, this version is " - "implemented as\n" - "an LLVM pass and tends to offer improved performance with slow " - "programs.\n\n" - - "Environment variables used:\n" - "AFL_CC: path to the C compiler to use\n" - "AFL_CXX: path to the C++ compiler to use\n" - "AFL_PATH: path to instrumenting pass and runtime " - "(afl-llvm-rt.*o)\n" - "AFL_DONT_OPTIMIZE: disable optimization instead of -O3\n" - "AFL_NO_BUILTIN: compile for use with libtokencap.so\n" - "AFL_INST_RATIO: percentage of branches to instrument\n" - "AFL_QUIET: suppress verbose output\n" - "AFL_DEBUG: enable developer debugging output\n" - "AFL_HARDEN: adds code hardening to catch memory bugs\n" - "AFL_USE_ASAN: activate address sanitizer\n" - "AFL_USE_MSAN: activate memory sanitizer\n" - "AFL_USE_UBSAN: activate undefined behaviour sanitizer\n" - "AFL_USE_CFISAN: activate control flow sanitizer\n" - "AFL_LLVM_WHITELIST: enable whitelisting (selective " - "instrumentation)\n" - "AFL_LLVM_NOT_ZERO: use cycling trace counters that skip zero\n" - "AFL_LLVM_USE_TRACE_PC: use LLVM trace-pc-guard instrumentation\n" - "AFL_LLVM_LAF_SPLIT_COMPARES: enable cascaded comparisons\n" - "AFL_LLVM_LAF_SPLIT_SWITCHES: casc. comp. in 'switch'\n" - "AFL_LLVM_LAF_TRANSFORM_COMPARES: transform library comparison " - "function calls\n" - " to cascaded comparisons\n" - "AFL_LLVM_LAF_SPLIT_FLOATS: transform floating point comp. to " - "cascaded " - "comp.\n" - "AFL_LLVM_LAF_SPLIT_COMPARES_BITW: size limit (default 8)\n", - callname, BIN_PATH, BIN_PATH); + } + +#ifndef AFL_CLANG_FLTO + if (instrument_mode == INSTRUMENT_LTO) + FATAL("instrumentation mode LTO specified but LLVM support not available"); +#endif + + if (argc < 2 || strcmp(argv[1], "-h") == 0) { + + if (instrument_mode != INSTRUMENT_LTO) + printf("afl-clang-fast" VERSION " by <lszekeres@google.com> in %s mode\n", + ptr); + else + printf("afl-clang-lto" VERSION + " by Marc \"vanHauser\" Heuse <mh@mh-sec.de> in %s mode\n", + ptr); + + SAYF( + "\n" + "%s[++] [options]\n" + "\n" + "This is a helper application for afl-fuzz. It serves as a drop-in " + "replacement\n" + "for clang, letting you recompile third-party code with the " + "required " + "runtime\n" + "instrumentation. A common use pattern would be one of the " + "following:\n\n" + + " CC=%s/afl-clang-fast ./configure\n" + " CXX=%s/afl-clang-fast++ ./configure\n\n" + + "In contrast to the traditional afl-clang tool, this version is " + "implemented as\n" + "an LLVM pass and tends to offer improved performance with slow " + "programs.\n\n" + + "Environment variables used:\n" + "AFL_CC: path to the C compiler to use\n" + "AFL_CXX: path to the C++ compiler to use\n" + "AFL_PATH: path to instrumenting pass and runtime " + "(afl-llvm-rt.*o)\n" + "AFL_DONT_OPTIMIZE: disable optimization instead of -O3\n" + "AFL_NO_BUILTIN: compile for use with libtokencap.so\n" + "AFL_INST_RATIO: percentage of branches to instrument\n" + "AFL_QUIET: suppress verbose output\n" + "AFL_DEBUG: enable developer debugging output\n" + "AFL_HARDEN: adds code hardening to catch memory bugs\n" + "AFL_USE_ASAN: activate address sanitizer\n" + "AFL_USE_MSAN: activate memory sanitizer\n" + "AFL_USE_UBSAN: activate undefined behaviour sanitizer\n" + "AFL_USE_CFISAN: activate control flow sanitizer\n" + "AFL_LLVM_WHITELIST: enable whitelisting (selective " + "instrumentation)\n" + "AFL_LLVM_NOT_ZERO: use cycling trace counters that skip zero\n" + "AFL_LLVM_LAF_SPLIT_COMPARES: enable cascaded comparisons\n" + "AFL_LLVM_LAF_SPLIT_SWITCHES: casc. comp. in 'switch'\n" + "AFL_LLVM_LAF_TRANSFORM_COMPARES: transform library comparison " + "function calls\n" + " to cascaded comparisons\n" + "AFL_LLVM_LAF_SPLIT_FLOATS: transform floating point comp. to " + "cascaded " + "comp.\n" + "AFL_LLVM_LAF_SPLIT_COMPARES_BITW: size limit (default 8)\n", + callname, BIN_PATH, BIN_PATH); SAYF( "\nafl-clang-fast specific environment variables:\n" + "AFL_LLVM_CMPLOG: log operands of comparisons (RedQueen mutator)\n" + "AFL_LLVM_INSTRUMENT: set instrumentation mode: DEFAULT, CFG " + "(INSTRIM), LTO, CTX, NGRAM-2 ... NGRAM-16\n" + "You can also use the old environment variables:" + "AFL_LLVM_CTX: use context sensitive coverage\n" + "AFL_LLVM_USE_TRACE_PC: use LLVM trace-pc-guard instrumentation\n" + "AFL_LLVM_NGRAM_SIZE: use ngram prev_loc count coverage\n" "AFL_LLVM_INSTRIM: use light weight instrumentation InsTrim\n" - "AFL_LLVM_INSTRIM_LOOPHEAD: optimize loop tracing for speed\n" - "AFL_LLVM_NGRAM_SIZE: use ngram prev_loc coverage\n" - "AFL_LLVM_CMPLOG: log operands of comparisons (RedQueen mutator)\n"); + "AFL_LLVM_INSTRIM_LOOPHEAD: optimize loop tracing for speed (sub " + "option to INSTRIM)\n"); #ifdef AFL_CLANG_FLTO SAYF( @@ -652,22 +757,17 @@ int main(int argc, char **argv, char **envp) { getenv("AFL_DEBUG") != NULL) { -#ifdef USE_TRACE_PC - SAYF(cCYA "afl-clang-fast" VERSION cRST - " [tpcg] by <lszekeres@google.com>\n"); -#warning \ - "You do not need to specifically compile with USE_TRACE_PC anymore, setting the environment variable AFL_LLVM_USE_TRACE_PC is enough." -#else - if (strstr(argv[0], "afl-clang-lto") == NULL) + if (instrument_mode != INSTRUMENT_LTO) - SAYF(cCYA "afl-clang-fast" VERSION cRST " by <lszekeres@google.com>\n"); + SAYF(cCYA "afl-clang-fast" VERSION cRST + " by <lszekeres@google.com> in %s mode\n", + ptr); else SAYF(cCYA "afl-clang-lto" VERSION cRST - " by Marc \"vanHauser\" Heuse <mh@mh-sec.de>\n"); - -#endif /* ^USE_TRACE_PC */ + " by Marc \"vanHauser\" Heuse <mh@mh-sec.de> in mode %s\n", + ptr); } diff --git a/llvm_mode/afl-llvm-pass.so.cc b/llvm_mode/afl-llvm-pass.so.cc index 5fe98d8b..31d00fec 100644 --- a/llvm_mode/afl-llvm-pass.so.cc +++ b/llvm_mode/afl-llvm-pass.so.cc @@ -124,6 +124,8 @@ class AFLCoverage : public ModulePass { protected: std::list<std::string> myWhitelist; uint32_t ngram_size = 0; + uint32_t debug = 0; + char * ctx_str = NULL; }; @@ -179,6 +181,8 @@ bool AFLCoverage::runOnModule(Module &M) { char be_quiet = 0; + if (getenv("AFL_DEBUG")) debug = 1; + if ((isatty(2) && !getenv("AFL_QUIET")) || getenv("AFL_DEBUG") != NULL) { SAYF(cCYA "afl-llvm-pass" VERSION cRST @@ -209,6 +213,7 @@ bool AFLCoverage::runOnModule(Module &M) { char *ngram_size_str = getenv("AFL_LLVM_NGRAM_SIZE"); if (!ngram_size_str) ngram_size_str = getenv("AFL_NGRAM_SIZE"); + ctx_str = getenv("AFL_LLVM_CTX"); #ifdef AFL_HAVE_VECTOR_INTRINSICS /* Decide previous location vector size (must be a power of two) */ @@ -216,11 +221,11 @@ bool AFLCoverage::runOnModule(Module &M) { if (ngram_size_str) if (sscanf(ngram_size_str, "%u", &ngram_size) != 1 || ngram_size < 2 || - ngram_size > MAX_NGRAM_SIZE) + ngram_size > NGRAM_SIZE_MAX) FATAL( - "Bad value of AFL_NGRAM_SIZE (must be between 2 and MAX_NGRAM_SIZE " + "Bad value of AFL_NGRAM_SIZE (must be between 2 and NGRAM_SIZE_MAX " "(%u))", - MAX_NGRAM_SIZE); + NGRAM_SIZE_MAX); if (ngram_size == 1) ngram_size = 0; if (ngram_size) @@ -228,9 +233,8 @@ bool AFLCoverage::runOnModule(Module &M) { else #else if (ngram_size_str) - FATAL( - "Sorry, n-gram branch coverage is not supported with llvm version %s!", - LLVM_VERSION_STRING); + FATAL("Sorry, NGRAM branch coverage is not supported with llvm version %s!", + LLVM_VERSION_STRING); #endif PrevLocSize = 1; @@ -239,6 +243,9 @@ bool AFLCoverage::runOnModule(Module &M) { if (ngram_size) PrevLocTy = VectorType::get(IntLocTy, PrevLocVecSize); #endif + if (ctx_str && ngram_size_str) + FATAL("you must decide between NGRAM and CTX instrumentation"); + /* Get globals for the SHM region and the previous location. Note that __afl_prev_loc is thread-local. */ @@ -246,6 +253,17 @@ bool AFLCoverage::runOnModule(Module &M) { new GlobalVariable(M, PointerType::get(Int8Ty, 0), false, GlobalValue::ExternalLinkage, 0, "__afl_area_ptr"); GlobalVariable *AFLPrevLoc; + GlobalVariable *AFLContext; + + if (ctx_str) +#ifdef __ANDROID__ + AFLContext = new GlobalVariable( + M, Int32Ty, false, GlobalValue::ExternalLinkage, 0, "__afl_prev_ctx"); +#else + AFLContext = new GlobalVariable( + M, Int32Ty, false, GlobalValue::ExternalLinkage, 0, "__afl_prev_ctx", 0, + GlobalVariable::GeneralDynamicTLSModel, 0, false); +#endif #ifdef AFL_HAVE_VECTOR_INTRINSICS if (ngram_size) @@ -291,14 +309,70 @@ bool AFLCoverage::runOnModule(Module &M) { ConstantInt *Zero = ConstantInt::get(Int8Ty, 0); ConstantInt *One = ConstantInt::get(Int8Ty, 1); + LoadInst *PrevCtx; // CTX sensitive coverage + /* Instrument all the things! */ int inst_blocks = 0; for (auto &F : M) { + if (debug) + fprintf(stderr, "FUNCTION: %s (%zu)\n", F.getName().str().c_str(), + F.size()); + if (isBlacklisted(&F)) continue; + // AllocaInst *CallingContext = nullptr; + + if (ctx_str && F.size() > 1) { // Context sensitive coverage + // load the context ID of the previous function and write to to a local + // variable on the stack + auto bb = &F.getEntryBlock(); + BasicBlock::iterator IP = bb->getFirstInsertionPt(); + IRBuilder<> IRB(&(*IP)); + PrevCtx = IRB.CreateLoad(AFLContext); + PrevCtx->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None)); + + // does the function have calls? and is any of the calls larger than one + // basic block? + int has_calls = 0; + for (auto &BB : F) { + + if (has_calls) break; + for (auto &IN : BB) { + + CallInst *callInst = nullptr; + if ((callInst = dyn_cast<CallInst>(&IN))) { + + Function *Callee = callInst->getCalledFunction(); + if (!Callee || Callee->size() < 2) + continue; + else { + + has_calls = 1; + break; + + } + + } + + } + + } + + // if yes we store a context ID for this function in the global var + if (has_calls) { + + ConstantInt *NewCtx = ConstantInt::get(Int32Ty, AFL_R(MAP_SIZE)); + StoreInst * StoreCtx = IRB.CreateStore(NewCtx, AFLContext); + StoreCtx->setMetadata(M.getMDKindID("nosanitize"), + MDNode::get(C, None)); + + } + + } + for (auto &BB : F) { BasicBlock::iterator IP = BB.getFirstInsertionPt(); @@ -484,6 +558,9 @@ bool AFLCoverage::runOnModule(Module &M) { PrevLocTrans = IRB.CreateXorReduce(PrevLoc); else #endif + if (ctx_str) + PrevLocTrans = IRB.CreateZExt(IRB.CreateXor(PrevLoc, PrevCtx), Int32Ty); + else PrevLocTrans = IRB.CreateZExt(PrevLoc, IRB.getInt32Ty()); /* Load SHM pointer */ diff --git a/llvm_mode/afl-llvm-rt.o.c b/llvm_mode/afl-llvm-rt.o.c index 4d1f846c..e2fd5190 100644 --- a/llvm_mode/afl-llvm-rt.o.c +++ b/llvm_mode/afl-llvm-rt.o.c @@ -67,15 +67,18 @@ u8 __afl_area_initial[MAP_SIZE]; u8 *__afl_area_ptr = __afl_area_initial; #ifdef __ANDROID__ -PREV_LOC_T __afl_prev_loc[MAX_NGRAM_SIZE]; +PREV_LOC_T __afl_prev_loc[NGRAM_SIZE_MAX]; u32 __afl_final_loc; +u32 __afl_prev_ctx; +u32 __afl_cmp_counter #else -__thread PREV_LOC_T __afl_prev_loc[MAX_NGRAM_SIZE]; +__thread PREV_LOC_T __afl_prev_loc[NGRAM_SIZE_MAX]; __thread u32 __afl_final_loc; +__thread u32 __afl_prev_ctx; +__thread u32 __afl_cmp_counter; #endif -struct cmp_map *__afl_cmp_map; -__thread u32 __afl_cmp_counter; + struct cmp_map *__afl_cmp_map; /* Running in persistent mode? */ @@ -387,7 +390,7 @@ int __afl_persistent_loop(unsigned int max_cnt) { memset(__afl_area_ptr, 0, MAP_SIZE); __afl_area_ptr[0] = 1; - memset(__afl_prev_loc, 0, MAX_NGRAM_SIZE * sizeof(PREV_LOC_T)); + memset(__afl_prev_loc, 0, NGRAM_SIZE_MAX * sizeof(PREV_LOC_T)); } @@ -404,7 +407,7 @@ int __afl_persistent_loop(unsigned int max_cnt) { raise(SIGSTOP); __afl_area_ptr[0] = 1; - memset(__afl_prev_loc, 0, MAX_NGRAM_SIZE * sizeof(PREV_LOC_T)); + memset(__afl_prev_loc, 0, NGRAM_SIZE_MAX * sizeof(PREV_LOC_T)); return 1; diff --git a/llvm_mode/llvm-ngram-coverage.h b/llvm_mode/llvm-ngram-coverage.h index 488b4fe7..12b666e9 100644 --- a/llvm_mode/llvm-ngram-coverage.h +++ b/llvm_mode/llvm-ngram-coverage.h @@ -12,7 +12,7 @@ typedef u64 PREV_LOC_T; #endif /* Maximum ngram size */ -#define MAX_NGRAM_SIZE 128U +#define NGRAM_SIZE_MAX 16U #endif diff --git a/llvm_mode/split-compares-pass.so.cc b/llvm_mode/split-compares-pass.so.cc index d296ba3d..f0615f85 100644 --- a/llvm_mode/split-compares-pass.so.cc +++ b/llvm_mode/split-compares-pass.so.cc @@ -1235,8 +1235,8 @@ bool SplitComparesTransform::runOnModule(Module &M) { int bitw = 64; - char *bitw_env = getenv("LAF_SPLIT_COMPARES_BITW"); - if (!bitw_env) bitw_env = getenv("AFL_LLVM_LAF_SPLIT_COMPARES_BITW"); + char *bitw_env = getenv("AFL_LLVM_LAF_SPLIT_COMPARES_BITW"); + if (!bitw_env) bitw_env = getenv("LAF_SPLIT_COMPARES_BITW"); if (bitw_env) { bitw = atoi(bitw_env); } enableFPSplit = getenv("AFL_LLVM_LAF_SPLIT_FLOATS") != NULL; diff --git a/src/afl-common.c b/src/afl-common.c index 9502d53f..73b3fa8a 100644 --- a/src/afl-common.c +++ b/src/afl-common.c @@ -57,13 +57,14 @@ char *afl_environment_variables[] = { "AFL_INST_LIBS", "AFL_INST_RATIO", "AFL_KEEP_TRACES", "AFL_KEEP_ASSEMBLY", "AFL_LD_HARD_FAIL", "AFL_LD_LIMIT_MB", "AFL_LD_NO_CALLOC_OVER", "AFL_LD_PRELOAD", "AFL_LD_VERBOSE", "AFL_LLVM_CMPLOG", "AFL_LLVM_INSTRIM", - "AFL_LLVM_INSTRIM_LOOPHEAD", "AFL_LLVM_INSTRIM_SKIPSINGLEBLOCK", - "AFL_LLVM_LAF_SPLIT_COMPARES", "AFL_LLVM_LAF_SPLIT_COMPARES_BITW", - "AFL_LLVM_LAF_SPLIT_FLOATS", "AFL_LLVM_LAF_SPLIT_SWITCHES", - "AFL_LLVM_LAF_TRANSFORM_COMPARES", "AFL_LLVM_NGRAM_SIZE", "AFL_NGRAM_SIZE", - "AFL_LLVM_NOT_ZERO", "AFL_LLVM_WHITELIST", "AFL_NO_AFFINITY", - "AFL_LLVM_LTO_STARTID", "AFL_LLVM_LTO_DONTWRITEID", "AFL_NO_ARITH", - "AFL_NO_BUILTIN", "AFL_NO_CPU_RED", "AFL_NO_FORKSRV", "AFL_NO_UI", + "AFL_LLVM_CTX", "AFL_LLVM_INSTRUMENT", "AFL_LLVM_INSTRIM_LOOPHEAD", + "AFL_LLVM_INSTRIM_SKIPSINGLEBLOCK", "AFL_LLVM_LAF_SPLIT_COMPARES", + "AFL_LLVM_LAF_SPLIT_COMPARES_BITW", "AFL_LLVM_LAF_SPLIT_FLOATS", + "AFL_LLVM_LAF_SPLIT_SWITCHES", "AFL_LLVM_LAF_TRANSFORM_COMPARES", + "AFL_LLVM_NGRAM_SIZE", "AFL_NGRAM_SIZE", "AFL_LLVM_NOT_ZERO", + "AFL_LLVM_WHITELIST", "AFL_NO_AFFINITY", "AFL_LLVM_LTO_STARTID", + "AFL_LLVM_LTO_DONTWRITEID", "AFL_NO_ARITH", "AFL_NO_BUILTIN", + "AFL_NO_CPU_RED", "AFL_NO_FORKSRV", "AFL_NO_UI", "AFL_NO_X86", // not really an env but we dont want to warn on it "AFL_PATH", "AFL_PERFORMANCE_FILE", //"AFL_PERSISTENT", // not implemented anymore, so warn additionally |