From f7bdb6555d74cde1632497c6bcb92b07aac1b3b6 Mon Sep 17 00:00:00 2001 From: hexcoder Date: Thu, 10 Oct 2019 21:38:12 +0200 Subject: Update README.md Found Apple's statement on not supporting statically linked binaries and mentioned it. --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'README.md') diff --git a/README.md b/README.md index f818e271..4c04e1b6 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,9 @@ These build options exist: * clean: cleans everything. for qemu_mode and unicorn_mode it means it deletes all downloads as well * help: shows these build options -You can also build statically linked versions of the afl++ binaries by passing the STATIC=1 argument to make: +[https://developer.apple.com/library/archive/qa/qa1118/_index.html](Unless +you are on Mac OS X) you can also build statically linked versions of the +afl++ binaries by passing the STATIC=1 argument to make: ```shell $ make all STATIC=1 -- cgit 1.4.1 From 61a84e15ea5a798a5667c67ccfc843499e288a9b Mon Sep 17 00:00:00 2001 From: hexcoder Date: Thu, 10 Oct 2019 21:46:15 +0200 Subject: Update README.md --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 4c04e1b6..9acddd4c 100644 --- a/README.md +++ b/README.md @@ -105,8 +105,7 @@ These build options exist: * clean: cleans everything. for qemu_mode and unicorn_mode it means it deletes all downloads as well * help: shows these build options -[https://developer.apple.com/library/archive/qa/qa1118/_index.html](Unless -you are on Mac OS X) you can also build statically linked versions of the +[Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html) you can also build statically linked versions of the afl++ binaries by passing the STATIC=1 argument to make: ```shell -- cgit 1.4.1 From bccaf93f8b444e3cddda76057303e0ab28c1c5b3 Mon Sep 17 00:00:00 2001 From: hexcoder Date: Thu, 10 Oct 2019 23:21:52 +0200 Subject: Update README.md typos in names --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 9acddd4c..bf8f69dc 100644 --- a/README.md +++ b/README.md @@ -620,7 +620,7 @@ without feedback, bug reports, or patches from: Jonathan Gray Filipe Cabecinhas Nico Weber Jodie Cunningham Andrew Griffiths Parker Thompson - Jonathan Neuschfer Tyler Nighswander + Jonathan Neuschaefer Tyler Nighswander Ben Nagy Samir Aguiar Aidan Thornton Aleksandar Nikolich Sam Hakim Laszlo Szekeres @@ -639,7 +639,7 @@ without feedback, bug reports, or patches from: Austin Seipp Daniel Komaromy Daniel Binderman Jonathan Metzman Vegard Nossum Jan Kneschke - Kurt Roeckx Marcel Bohme + Kurt Roeckx Marcel Boehme Van-Thuan Pham Abhik Roychoudhury Joshua J. Drake Toby Hutton Rene Freingruber Sergey Davidoff -- cgit 1.4.1 From 97f5ce52d129c460e4a8af3d55aefd05092a79a9 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Tue, 15 Oct 2019 16:48:05 +0200 Subject: v2.57c release --- README.md | 7 ++++--- TODO | 4 ++-- docs/ChangeLog | 9 +++++---- include/config.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index bf8f69dc..f0e6faef 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # american fuzzy lop plus plus (afl++) - Release Version: 2.54c + Release Version: 2.57c - Github Version: 2.54d + Github Version: 2.57d includes all necessary/interesting changes from Google's afl 2.56b @@ -22,7 +22,7 @@ ## The enhancements compared to the original stock afl Many improvements were made over the official afl release - which did not - get any improvements since November 2017. + get any feature improvements since November 2017. Among other changes afl++ has a more performant llvm_mode, supports llvm up to version 9, QEMU 3.1, more speed and crashfixes for QEMU, @@ -103,6 +103,7 @@ These build options exist: * distrib: everything (for both binary-only and source code fuzzing) * install: installs everything you have compiled with the build options above * clean: cleans everything. for qemu_mode and unicorn_mode it means it deletes all downloads as well +* tests: runs test cases to ensure that all features are still working as they should * help: shows these build options [Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html) you can also build statically linked versions of the diff --git a/TODO b/TODO index 696e9372..3ca13d6e 100644 --- a/TODO +++ b/TODO @@ -1,6 +1,6 @@ -Roadmap 2.55d: -============== +Roadmap 2.60: +============= afl-fuzz: - radamsa mutator diff --git a/docs/ChangeLog b/docs/ChangeLog index 7ccc8b66..aafb7bff 100644 --- a/docs/ChangeLog +++ b/docs/ChangeLog @@ -13,12 +13,13 @@ Want to stay in the loop on major new features? Join our mailing list by sending a mail to . ----------------------- -Version ++2.54d (dev): ----------------------- +-------------------------------- +Version ++2.54d-2.57c (release): +-------------------------------- + - we jump to 2.57 instead of 2.55 to catch up with Google's versioning - persistent mode for QEMU (see qemu_mode/README.md) - - custom mutator library is now a standard mutator, to exclusivly use it + - custom mutator library is now an additional mutator, to exclusivly use it add AFL_CUSTOM_MUTATOR_ONLY (that will trigger the previous behaviour) - new library qemu_mode/unsigaction which filters sigaction events - afl-fuzz: new command line option -I to execute a command on a new crash diff --git a/include/config.h b/include/config.h index f2732ad4..d331cd93 100644 --- a/include/config.h +++ b/include/config.h @@ -26,7 +26,7 @@ /* Version string: */ -#define VERSION "++2.54d" // c = release, d = volatile github dev +#define VERSION "++2.57c" // c = release, d = volatile github dev /****************************************************** * * -- cgit 1.4.1 From be6bc155ebd891eebd21eac8982784e7f2be4f41 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Mon, 21 Oct 2019 11:28:32 +0200 Subject: v2.58c --- Makefile | 9 ++++++++- README.md | 18 +++++++++--------- TODO | 5 +---- docs/ChangeLog | 14 ++++++++------ gcc_plugin/README.gcc.md | 4 ---- gcc_plugin/README.whitelist.md | 3 +-- include/config.h | 2 +- 7 files changed, 28 insertions(+), 27 deletions(-) (limited to 'README.md') diff --git a/Makefile b/Makefile index cfcd0890..87c7cdef 100644 --- a/Makefile +++ b/Makefile @@ -104,6 +104,13 @@ man: $(MANPAGES) tests: source-only @cd test ; ./test.sh +performance-tests: performance-test +test-performance: performance-test + +performance-test: source-only + @cd test ; ./test-performance.sh + + help: @echo "HELP --- the following make targets exist:" @echo "==========================================" @@ -307,7 +314,7 @@ install: all $(MANPAGES) install -m 755 $(PROGS) $(SH_PROGS) $${DESTDIR}$(BIN_PATH) rm -f $${DESTDIR}$(BIN_PATH)/afl-as if [ -f afl-qemu-trace ]; then install -m 755 afl-qemu-trace $${DESTDIR}$(BIN_PATH); fi - #if [ -f afl-gcc-fast ]; then set e; install -m 755 afl-gcc-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-gcc-fast $${DESTDIR}$(BIN_PATH)/afl-g++-fast; install -m 755 afl-gcc-pass.so afl-gcc-rt.o $${DESTDIR}$(HELPER_PATH); fi + if [ -f afl-gcc-fast ]; then set e; install -m 755 afl-gcc-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-gcc-fast $${DESTDIR}$(BIN_PATH)/afl-g++-fast; install -m 755 afl-gcc-pass.so afl-gcc-rt.o $${DESTDIR}$(HELPER_PATH); fi ifndef AFL_TRACE_PC if [ -f afl-clang-fast -a -f libLLVMInsTrim.so -a -f afl-llvm-rt.o ]; then set -e; install -m 755 afl-clang-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-fast++; install -m 755 libLLVMInsTrim.so afl-llvm-pass.so afl-llvm-rt.o $${DESTDIR}$(HELPER_PATH); fi else diff --git a/README.md b/README.md index f0e6faef..4b9537d2 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # american fuzzy lop plus plus (afl++) - Release Version: 2.57c + Release Version: 2.58c - Github Version: 2.57d + Github Version: 2.58d includes all necessary/interesting changes from Google's afl 2.56b @@ -52,13 +52,13 @@ A more thorough list is available in the PATCHES file. - | Feature/Instrumentation | LLVM | GCC | QEMU | Unicorn | - | ----------------------- |:----:|:---:|:----:| -------:| - | laf-intel / CompCov | x | | x | x | - | NeverZero | x(1)| x | x | x | - | Persistent mode | x | | x | | - | Whitelist | x | | | | - | InsTrim | x | | | | + | Feature/Instrumentation | AFL-GCC | LLVM_MODE | GCC_PLUGIN | QEMU_MODE | Unicorn | + | ----------------------- |:-------:|:---------:|:----------:|:---------:|:-------:| + | laf-intel / CompCov | | x | | x | x | + | NeverZero | X | x(1) | | x | x | + | Persistent mode | | x | X | x | | + | Whitelist | | x | X | | | + | InsTrim | | x | | | | (1) only in LLVM >= 9.0 due to a bug in llvm in previous versions diff --git a/TODO b/TODO index 3ca13d6e..f2642b1a 100644 --- a/TODO +++ b/TODO @@ -7,11 +7,8 @@ afl-fuzz: - test the libmutator actually works and does not run infinite (need an example though) gcc_plugin: - - needs to be rewritten - - whitelist support - - skip over uninteresting blocks - - laf-intel - neverZero + - laf-intel qemu_mode: - update to 4.x (probably this will be skipped :( ) diff --git a/docs/ChangeLog b/docs/ChangeLog index 46b9e330..c2d46e4d 100644 --- a/docs/ChangeLog +++ b/docs/ChangeLog @@ -13,14 +13,16 @@ Want to stay in the loop on major new features? Join our mailing list by sending a mail to . ----------------------- -Version ++2.57d (dev): ----------------------- +-------------------------- +Version ++2.58c (release): +-------------------------- - - reverted patch to not unlink and recreate the input file, it resulted in performance loss + - reverted patch to not unlink and recreate the input file, it resulted in + performance loss of ~10% - added test/test-performance.sh script - - (re)added gcc_plugin, fast inline instrumentation is not yet finished - - added gcc_plugin tests + - (re)added gcc_plugin, fast inline instrumentation is not yet finished, + however it includes the whitelisting and persistance feature! by hexcoder- + - gcc_plugin tests added to testing framework -------------------------------- diff --git a/gcc_plugin/README.gcc.md b/gcc_plugin/README.gcc.md index 2e19e911..676ef427 100644 --- a/gcc_plugin/README.gcc.md +++ b/gcc_plugin/README.gcc.md @@ -156,7 +156,3 @@ depending on whether the input loop is being entered for the first time or executed again. To avoid spurious warnings, the feature implies AFL_NO_VAR_CHECK and hides the "variable path" warnings in the UI. -PS. Because there are task switches still involved, the mode isn't as fast as -"pure" in-process fuzzing offered, say, by LLVM's LibFuzzer; but it is a lot -faster than the normal fork() model, and compared to in-process fuzzing, -should be a lot more robust. diff --git a/gcc_plugin/README.whitelist.md b/gcc_plugin/README.whitelist.md index bcc02693..8ad2068d 100644 --- a/gcc_plugin/README.whitelist.md +++ b/gcc_plugin/README.whitelist.md @@ -6,8 +6,7 @@ Using afl++ with partial instrumentation that are interesting to you using the gcc instrumentation provided by afl++. - Originally developed by Christian Holler (:decoder) , - adapted to gcc plugin by hexcoder-. + Plugin by hexcoder-. ## 1) Description and purpose diff --git a/include/config.h b/include/config.h index 2e993a42..f121025d 100644 --- a/include/config.h +++ b/include/config.h @@ -26,7 +26,7 @@ /* Version string: */ -#define VERSION "++2.57d" // c = release, d = volatile github dev +#define VERSION "++2.58c" // c = release, d = volatile github dev /****************************************************** * * -- cgit 1.4.1 From 6bc874088fff699d170c89dd864c23507150fe3f Mon Sep 17 00:00:00 2001 From: van Hauser Date: Wed, 23 Oct 2019 02:58:16 +0200 Subject: readme update --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README.md') diff --git a/README.md b/README.md index 4b9537d2..d58b49e2 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ | Feature/Instrumentation | AFL-GCC | LLVM_MODE | GCC_PLUGIN | QEMU_MODE | Unicorn | | ----------------------- |:-------:|:---------:|:----------:|:---------:|:-------:| | laf-intel / CompCov | | x | | x | x | - | NeverZero | X | x(1) | | x | x | + | NeverZero | X | x(1) | X | x | x | | Persistent mode | | x | X | x | | | Whitelist | | x | X | | | | InsTrim | | x | | | | -- cgit 1.4.1 From 1398d8d5a64b11a62e8ed25732e2114a5dc47564 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Wed, 23 Oct 2019 03:23:14 +0200 Subject: gcc sucks --- README.md | 4 +++- gcc_plugin/afl-gcc-rt.o.c | 9 +++++---- 2 files changed, 8 insertions(+), 5 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index d58b49e2..de012e62 100644 --- a/README.md +++ b/README.md @@ -55,12 +55,14 @@ | Feature/Instrumentation | AFL-GCC | LLVM_MODE | GCC_PLUGIN | QEMU_MODE | Unicorn | | ----------------------- |:-------:|:---------:|:----------:|:---------:|:-------:| | laf-intel / CompCov | | x | | x | x | - | NeverZero | X | x(1) | X | x | x | + | NeverZero | X | x(1) | (2) | x | x | | Persistent mode | | x | X | x | | | Whitelist | | x | X | | | | InsTrim | | x | | | | + neverZero: (1) only in LLVM >= 9.0 due to a bug in llvm in previous versions + (2) gcc create non-performant code, hence it is disabled in gcc_plugin So all in all this is the best-of AFL that is currently out there :-) diff --git a/gcc_plugin/afl-gcc-rt.o.c b/gcc_plugin/afl-gcc-rt.o.c index 1dfca9e3..47049167 100644 --- a/gcc_plugin/afl-gcc-rt.o.c +++ b/gcc_plugin/afl-gcc-rt.o.c @@ -58,10 +58,11 @@ __thread u32 __afl_prev_loc; void __afl_trace(u32 x) { u32 l = __afl_prev_loc; - const u32 n = l ^ x; - u8 *const bitmap_ptr = __afl_area_ptr + n; - *bitmap_ptr += 1 + (*bitmap_ptr == (u8)~0); /* neverZero */ - + __afl_area_ptr[l ^ x]++; + /* // neverZero is disable as gcc creates non-performant code. shame on you gcc + if (__afl_area_ptr[l ^ x] == 0) + __afl_area_ptr[l ^ x]++; + */ __afl_prev_loc = (x >> 1); return; -- cgit 1.4.1 From 4de38fe40a4e0dac34c178c4ef6f4cb213fb3ee3 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sun, 27 Oct 2019 09:04:40 +0100 Subject: docs update --- README.md | 6 +++--- docs/perf_tips.txt | 9 ++++++--- 2 files changed, 9 insertions(+), 6 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index de012e62..88a86aaa 100644 --- a/README.md +++ b/README.md @@ -55,9 +55,9 @@ | Feature/Instrumentation | AFL-GCC | LLVM_MODE | GCC_PLUGIN | QEMU_MODE | Unicorn | | ----------------------- |:-------:|:---------:|:----------:|:---------:|:-------:| | laf-intel / CompCov | | x | | x | x | - | NeverZero | X | x(1) | (2) | x | x | - | Persistent mode | | x | X | x | | - | Whitelist | | x | X | | | + | NeverZero | x | x(1) | (2) | x | x | + | Persistent mode | | x | x | x | | + | Whitelist | | x | x | | | | InsTrim | | x | | | | neverZero: diff --git a/docs/perf_tips.txt b/docs/perf_tips.txt index 2fa19234..0cac8f7b 100644 --- a/docs/perf_tips.txt +++ b/docs/perf_tips.txt @@ -56,9 +56,9 @@ a file. 3) Use LLVM instrumentation --------------------------- -When fuzzing slow targets, you can gain 2x performance improvement by using -the LLVM-based instrumentation mode described in llvm_mode/README.llvm. Note -that this mode requires the use of clang and will not work with GCC. +When fuzzing slow targets, you can gain 20-100% performance improvement by +using the LLVM-based instrumentation mode described in llvm_mode/README.llvm. +Note that this mode requires the use of clang and will not work with GCC. The LLVM mode also offers a "persistent", in-process fuzzing mode that can work well for certain types of self-contained libraries, and for fast targets, @@ -75,6 +75,9 @@ If you are only intested in specific parts of the code being fuzzed, you can whitelist the files that are actually relevant. This improves the speed and accuracy of afl. See llvm_mode/README.whitelist +Also use the InsTrim mode on larger binaries, this improves performance and +coverage a lot. + 4) Profile and optimize the binary ---------------------------------- -- cgit 1.4.1 From b33bb0943ac4957eaf7b16ef694a4e4b4a538212 Mon Sep 17 00:00:00 2001 From: David Carlier Date: Thu, 31 Oct 2019 15:50:58 +0000 Subject: libtokencap/libdislocator README rename proposals and fixing the install tasks in the process. --- README.md | 2 +- docs/notes_for_asan.txt | 2 +- libdislocator/Makefile | 2 +- libdislocator/README.dislocator.md | 60 +++++++++++++++++++++++++++++++++++ libdislocator/README.md | 60 ----------------------------------- libdislocator/libdislocator.so.c | 2 +- libtokencap/Makefile | 2 +- libtokencap/README.md | 64 -------------------------------------- libtokencap/README.tokencap.md | 64 ++++++++++++++++++++++++++++++++++++++ libtokencap/libtokencap.so.c | 2 +- 10 files changed, 130 insertions(+), 130 deletions(-) create mode 100644 libdislocator/README.dislocator.md delete mode 100644 libdislocator/README.md delete mode 100644 libtokencap/README.md create mode 100644 libtokencap/README.tokencap.md (limited to 'README.md') diff --git a/README.md b/README.md index 88a86aaa..e8d4e6a8 100644 --- a/README.md +++ b/README.md @@ -457,7 +457,7 @@ parsers and grammars, but isn't nearly as good as the -x mode. If a dictionary is really hard to come by, another option is to let AFL run for a while, and then use the token capture library that comes as a companion -utility with AFL. For that, see [libtokencap/README.md](libtokencap/README.md). +utility with AFL. For that, see [libtokencap/README.md](libtokencap/README.tokencap.md). ## 11) Crash triage diff --git a/docs/notes_for_asan.txt b/docs/notes_for_asan.txt index 972ca909..09ca172e 100644 --- a/docs/notes_for_asan.txt +++ b/docs/notes_for_asan.txt @@ -34,7 +34,7 @@ Note that ASAN is incompatible with -static, so be mindful of that. There is also the option of generating a corpus using a non-ASAN binary, and then feeding it to an ASAN-instrumented one to check for bugs. This is faster, and can give you somewhat comparable results. You can also try using -libdislocator (see libdislocator/README.dislocator in the parent directory) as a +libdislocator (see libdislocator/README.dislocator.md in the parent directory) as a lightweight and hassle-free (but less thorough) alternative. 2) Long version diff --git a/libdislocator/Makefile b/libdislocator/Makefile index 91efba07..05ba26b3 100644 --- a/libdislocator/Makefile +++ b/libdislocator/Makefile @@ -34,5 +34,5 @@ clean: install: all install -m 755 ../libdislocator.so $${DESTDIR}$(HELPER_PATH) - install -m 644 README.dislocator $${DESTDIR}$(HELPER_PATH) + install -m 644 README.dislocator.md $${DESTDIR}$(HELPER_PATH) diff --git a/libdislocator/README.dislocator.md b/libdislocator/README.dislocator.md new file mode 100644 index 00000000..5d5a1464 --- /dev/null +++ b/libdislocator/README.dislocator.md @@ -0,0 +1,60 @@ +# libdislocator, an abusive allocator + + (See ../docs/README for the general instruction manual.) + +This is a companion library that can be used as a drop-in replacement for the +libc allocator in the fuzzed binaries. It improves the odds of bumping into +heap-related security bugs in several ways: + + - It allocates all buffers so that they are immediately adjacent to a + subsequent PROT_NONE page, causing most off-by-one reads and writes to + immediately segfault, + + - It adds a canary immediately below the allocated buffer, to catch writes + to negative offsets (won't catch reads, though), + + - It sets the memory returned by malloc() to garbage values, improving the + odds of crashing when the target accesses uninitialized data, + + - It sets freed memory to PROT_NONE and does not actually reuse it, causing + most use-after-free bugs to segfault right away, + + - It forces all realloc() calls to return a new address - and sets + PROT_NONE on the original block. This catches use-after-realloc bugs, + + - It checks for calloc() overflows and can cause soft or hard failures + of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, + AFL_LD_HARD_FAIL). + +Basically, it is inspired by some of the non-default options available for the +OpenBSD allocator - see malloc.conf(5) on that platform for reference. It is +also somewhat similar to several other debugging libraries, such as gmalloc +and DUMA - but is simple, plug-and-play, and designed specifically for fuzzing +jobs. + +Note that it does nothing for stack-based memory handling errors. The +-fstack-protector-all setting for GCC / clang, enabled when using AFL_HARDEN, +can catch some subset of that. + +The allocator is slow and memory-intensive (even the tiniest allocation uses up +4 kB of physical memory and 8 kB of virtual mem), making it completely unsuitable +for "production" uses; but it can be faster and more hassle-free than ASAN / MSAN +when fuzzing small, self-contained binaries. + +To use this library, run AFL like so: + +``` +AFL_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] +``` + +You *have* to specify path, even if it's just ./libdislocator.so or +$PWD/libdislocator.so. + +Similarly to afl-tmin, the library is not "proprietary" and can be used with +other fuzzers or testing tools without the need for any code tweaks. It does not +require AFL-instrumented binaries to work. + +Note that the AFL_PRELOAD approach (which AFL internally maps to LD_PRELOAD or +DYLD_INSERT_LIBRARIES, depending on the OS) works only if the target binary is +dynamically linked. Otherwise, attempting to use the library will have no +effect. diff --git a/libdislocator/README.md b/libdislocator/README.md deleted file mode 100644 index 5d5a1464..00000000 --- a/libdislocator/README.md +++ /dev/null @@ -1,60 +0,0 @@ -# libdislocator, an abusive allocator - - (See ../docs/README for the general instruction manual.) - -This is a companion library that can be used as a drop-in replacement for the -libc allocator in the fuzzed binaries. It improves the odds of bumping into -heap-related security bugs in several ways: - - - It allocates all buffers so that they are immediately adjacent to a - subsequent PROT_NONE page, causing most off-by-one reads and writes to - immediately segfault, - - - It adds a canary immediately below the allocated buffer, to catch writes - to negative offsets (won't catch reads, though), - - - It sets the memory returned by malloc() to garbage values, improving the - odds of crashing when the target accesses uninitialized data, - - - It sets freed memory to PROT_NONE and does not actually reuse it, causing - most use-after-free bugs to segfault right away, - - - It forces all realloc() calls to return a new address - and sets - PROT_NONE on the original block. This catches use-after-realloc bugs, - - - It checks for calloc() overflows and can cause soft or hard failures - of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, - AFL_LD_HARD_FAIL). - -Basically, it is inspired by some of the non-default options available for the -OpenBSD allocator - see malloc.conf(5) on that platform for reference. It is -also somewhat similar to several other debugging libraries, such as gmalloc -and DUMA - but is simple, plug-and-play, and designed specifically for fuzzing -jobs. - -Note that it does nothing for stack-based memory handling errors. The --fstack-protector-all setting for GCC / clang, enabled when using AFL_HARDEN, -can catch some subset of that. - -The allocator is slow and memory-intensive (even the tiniest allocation uses up -4 kB of physical memory and 8 kB of virtual mem), making it completely unsuitable -for "production" uses; but it can be faster and more hassle-free than ASAN / MSAN -when fuzzing small, self-contained binaries. - -To use this library, run AFL like so: - -``` -AFL_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] -``` - -You *have* to specify path, even if it's just ./libdislocator.so or -$PWD/libdislocator.so. - -Similarly to afl-tmin, the library is not "proprietary" and can be used with -other fuzzers or testing tools without the need for any code tweaks. It does not -require AFL-instrumented binaries to work. - -Note that the AFL_PRELOAD approach (which AFL internally maps to LD_PRELOAD or -DYLD_INSERT_LIBRARIES, depending on the OS) works only if the target binary is -dynamically linked. Otherwise, attempting to use the library will have no -effect. diff --git a/libdislocator/libdislocator.so.c b/libdislocator/libdislocator.so.c index 7fe40afa..106b44f4 100644 --- a/libdislocator/libdislocator.so.c +++ b/libdislocator/libdislocator.so.c @@ -14,7 +14,7 @@ http://www.apache.org/licenses/LICENSE-2.0 This is a companion library that can be used as a drop-in replacement - for the libc allocator in the fuzzed binaries. See README.dislocator for + for the libc allocator in the fuzzed binaries. See README.dislocator.md for more info. */ diff --git a/libtokencap/Makefile b/libtokencap/Makefile index df2426ed..6e1319d8 100644 --- a/libtokencap/Makefile +++ b/libtokencap/Makefile @@ -49,5 +49,5 @@ clean: install: all install -m 755 ../libtokencap.so $${DESTDIR}$(HELPER_PATH) - install -m 644 README.tokencap $${DESTDIR}$(HELPER_PATH) + install -m 644 README.tokencap.md $${DESTDIR}$(HELPER_PATH) diff --git a/libtokencap/README.md b/libtokencap/README.md deleted file mode 100644 index 8aae38bf..00000000 --- a/libtokencap/README.md +++ /dev/null @@ -1,64 +0,0 @@ -# strcmp() / memcmp() token capture library - - (See ../docs/README for the general instruction manual.) - -This companion library allows you to instrument `strcmp()`, `memcmp()`, -and related functions to automatically extract syntax tokens passed to any of -these libcalls. The resulting list of tokens may be then given as a starting -dictionary to afl-fuzz (the -x option) to improve coverage on subsequent -fuzzing runs. - -This may help improving coverage in some targets, and do precisely nothing in -others. In some cases, it may even make things worse: if libtokencap picks up -syntax tokens that are not used to process the input data, but that are a part -of - say - parsing a config file... well, you're going to end up wasting a lot -of CPU time on trying them out in the input stream. In other words, use this -feature with care. Manually screening the resulting dictionary is almost -always a necessity. - -As for the actual operation: the library stores tokens, without any deduping, -by appending them to a file specified via AFL_TOKEN_FILE. If the variable is not -set, the tool uses stderr (which is probably not what you want). - -Similarly to afl-tmin, the library is not "proprietary" and can be used with -other fuzzers or testing tools without the need for any code tweaks. It does not -require AFL-instrumented binaries to work. - -To use the library, you *need* to make sure that your fuzzing target is compiled -with -fno-builtin and is linked dynamically. If you wish to automate the first -part without mucking with CFLAGS in Makefiles, you can set AFL_NO_BUILTIN=1 -when using afl-gcc. This setting specifically adds the following flags: - -``` - -fno-builtin-strcmp -fno-builtin-strncmp -fno-builtin-strcasecmp - -fno-builtin-strcasencmp -fno-builtin-memcmp -fno-builtin-strstr - -fno-builtin-strcasestr -``` - -The next step is simply loading this library via LD_PRELOAD. The optimal usage -pattern is to allow afl-fuzz to fuzz normally for a while and build up a corpus, -and then fire off the target binary, with libtokencap.so loaded, on every file -found by AFL in that earlier run. This demonstrates the basic principle: - -``` - export AFL_TOKEN_FILE=$PWD/temp_output.txt - - for i in /queue/id*; do - LD_PRELOAD=/path/to/libtokencap.so \ - /path/to/target/program [...params, including $i...] - done - - sort -u temp_output.txt >afl_dictionary.txt -``` - -If you don't get any results, the target library is probably not using strcmp() -and memcmp() to parse input; or you haven't compiled it with -fno-builtin; or -the whole thing isn't dynamically linked, and LD_PRELOAD is having no effect. - -Portability hints: There is probably no particularly portable and non-invasive -way to distinguish between read-only and read-write memory mappings. -The `__tokencap_load_mappings()` function is the only thing that would -need to be changed for other OSes. - -Current supported OSes are: Linux, Darwin, FreeBSD (thanks to @devnexen) - diff --git a/libtokencap/README.tokencap.md b/libtokencap/README.tokencap.md new file mode 100644 index 00000000..8aae38bf --- /dev/null +++ b/libtokencap/README.tokencap.md @@ -0,0 +1,64 @@ +# strcmp() / memcmp() token capture library + + (See ../docs/README for the general instruction manual.) + +This companion library allows you to instrument `strcmp()`, `memcmp()`, +and related functions to automatically extract syntax tokens passed to any of +these libcalls. The resulting list of tokens may be then given as a starting +dictionary to afl-fuzz (the -x option) to improve coverage on subsequent +fuzzing runs. + +This may help improving coverage in some targets, and do precisely nothing in +others. In some cases, it may even make things worse: if libtokencap picks up +syntax tokens that are not used to process the input data, but that are a part +of - say - parsing a config file... well, you're going to end up wasting a lot +of CPU time on trying them out in the input stream. In other words, use this +feature with care. Manually screening the resulting dictionary is almost +always a necessity. + +As for the actual operation: the library stores tokens, without any deduping, +by appending them to a file specified via AFL_TOKEN_FILE. If the variable is not +set, the tool uses stderr (which is probably not what you want). + +Similarly to afl-tmin, the library is not "proprietary" and can be used with +other fuzzers or testing tools without the need for any code tweaks. It does not +require AFL-instrumented binaries to work. + +To use the library, you *need* to make sure that your fuzzing target is compiled +with -fno-builtin and is linked dynamically. If you wish to automate the first +part without mucking with CFLAGS in Makefiles, you can set AFL_NO_BUILTIN=1 +when using afl-gcc. This setting specifically adds the following flags: + +``` + -fno-builtin-strcmp -fno-builtin-strncmp -fno-builtin-strcasecmp + -fno-builtin-strcasencmp -fno-builtin-memcmp -fno-builtin-strstr + -fno-builtin-strcasestr +``` + +The next step is simply loading this library via LD_PRELOAD. The optimal usage +pattern is to allow afl-fuzz to fuzz normally for a while and build up a corpus, +and then fire off the target binary, with libtokencap.so loaded, on every file +found by AFL in that earlier run. This demonstrates the basic principle: + +``` + export AFL_TOKEN_FILE=$PWD/temp_output.txt + + for i in /queue/id*; do + LD_PRELOAD=/path/to/libtokencap.so \ + /path/to/target/program [...params, including $i...] + done + + sort -u temp_output.txt >afl_dictionary.txt +``` + +If you don't get any results, the target library is probably not using strcmp() +and memcmp() to parse input; or you haven't compiled it with -fno-builtin; or +the whole thing isn't dynamically linked, and LD_PRELOAD is having no effect. + +Portability hints: There is probably no particularly portable and non-invasive +way to distinguish between read-only and read-write memory mappings. +The `__tokencap_load_mappings()` function is the only thing that would +need to be changed for other OSes. + +Current supported OSes are: Linux, Darwin, FreeBSD (thanks to @devnexen) + diff --git a/libtokencap/libtokencap.so.c b/libtokencap/libtokencap.so.c index 2fe9ae63..7495180d 100644 --- a/libtokencap/libtokencap.so.c +++ b/libtokencap/libtokencap.so.c @@ -15,7 +15,7 @@ This Linux-only companion library allows you to instrument strcmp(), memcmp(), and related functions to automatically extract tokens. - See README.tokencap for more info. + See README.tokencap.md for more info. */ -- cgit 1.4.1 From dede2e976400cec53099ab42fa3f4fa73e324d2c Mon Sep 17 00:00:00 2001 From: van Hauser Date: Tue, 5 Nov 2019 09:16:04 +0100 Subject: dockerfile additions --- Dockerfile | 4 +++- README.md | 8 ++++++++ docs/ChangeLog | 1 + 3 files changed, 12 insertions(+), 1 deletion(-) (limited to 'README.md') diff --git a/Dockerfile b/Dockerfile index 7defa8e9..558968d8 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,6 @@ FROM ubuntu:eoan +MAINTAINER David Carlier +LABEL "about"="AFLplusplus docker image" RUN apt-get update && apt-get install -y --no-install-recommends \ automake \ bison \ @@ -22,4 +24,4 @@ COPY . . ENV CC=gcc-9 ENV CXX=g++-9 ENV LLVM_CONFIG=llvm-config-9 -RUN make clean && make distrib && make tests +RUN make clean && make distrib && make install diff --git a/README.md b/README.md index e8d4e6a8..583b7df8 100644 --- a/README.md +++ b/README.md @@ -115,6 +115,14 @@ afl++ binaries by passing the STATIC=1 argument to make: $ make all STATIC=1 ``` +Note that afl++ is faster and better the newer the compilers used. +Hence gcc-9 and especially llvm-9 should be the compilers of choice. +If your distribution does not have them, you can use the Dockerfile: + +```shell +$ docker build -t aflplusplus +``` + ## 1) Challenges of guided fuzzing diff --git a/docs/ChangeLog b/docs/ChangeLog index 3a5961ad..3924226e 100644 --- a/docs/ChangeLog +++ b/docs/ChangeLog @@ -23,6 +23,7 @@ Version ++2.58d (dev): - llvm_mode: float splitting is now configured via AFL_LLVM_LAF_SPLIT_FLOATS - libtokencap: support for *BSD/OSX added - libcompcov floating point splitting support for qemu and unicorn + - Dockerfile by courtesy of devnexen - ripped regex.dictionary from Google afl PR - removed unnecessary warnings -- cgit 1.4.1