From 500a378fdf8664aea42f557f60c9842bb15f06a0 Mon Sep 17 00:00:00 2001 From: Andrea Fioraldi Date: Sat, 31 Aug 2019 11:23:48 +0200 Subject: modernize some readmes --- libdislocator/README.dislocator | 60 ----------------------------------- libdislocator/README.md | 60 +++++++++++++++++++++++++++++++++++ libtokencap/README.md | 63 +++++++++++++++++++++++++++++++++++++ libtokencap/README.tokencap | 61 ----------------------------------- qemu_mode/libcompcov/README.compcov | 37 ---------------------- qemu_mode/libcompcov/README.md | 37 ++++++++++++++++++++++ 6 files changed, 160 insertions(+), 158 deletions(-) delete mode 100644 libdislocator/README.dislocator create mode 100644 libdislocator/README.md create mode 100644 libtokencap/README.md delete mode 100644 libtokencap/README.tokencap delete mode 100644 qemu_mode/libcompcov/README.compcov create mode 100644 qemu_mode/libcompcov/README.md diff --git a/libdislocator/README.dislocator b/libdislocator/README.dislocator deleted file mode 100644 index 837e7466..00000000 --- a/libdislocator/README.dislocator +++ /dev/null @@ -1,60 +0,0 @@ -=================================== -libdislocator, an abusive allocator -=================================== - - (See ../docs/README for the general instruction manual.) - -This is a companion library that can be used as a drop-in replacement for the -libc allocator in the fuzzed binaries. It improves the odds of bumping into -heap-related security bugs in several ways: - - - It allocates all buffers so that they are immediately adjacent to a - subsequent PROT_NONE page, causing most off-by-one reads and writes to - immediately segfault, - - - It adds a canary immediately below the allocated buffer, to catch writes - to negative offsets (won't catch reads, though), - - - It sets the memory returned by malloc() to garbage values, improving the - odds of crashing when the target accesses uninitialized data, - - - It sets freed memory to PROT_NONE and does not actually reuse it, causing - most use-after-free bugs to segfault right away, - - - It forces all realloc() calls to return a new address - and sets - PROT_NONE on the original block. This catches use-after-realloc bugs, - - - It checks for calloc() overflows and can cause soft or hard failures - of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, - AFL_LD_HARD_FAIL). - -Basically, it is inspired by some of the non-default options available for the -OpenBSD allocator - see malloc.conf(5) on that platform for reference. It is -also somewhat similar to several other debugging libraries, such as gmalloc -and DUMA - but is simple, plug-and-play, and designed specifically for fuzzing -jobs. - -Note that it does nothing for stack-based memory handling errors. The --fstack-protector-all setting for GCC / clang, enabled when using AFL_HARDEN, -can catch some subset of that. - -The allocator is slow and memory-intensive (even the tiniest allocation uses up -4 kB of physical memory and 8 kB of virtual mem), making it completely unsuitable -for "production" uses; but it can be faster and more hassle-free than ASAN / MSAN -when fuzzing small, self-contained binaries. - -To use this library, run AFL like so: - -AFL_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] - -You *have* to specify path, even if it's just ./libdislocator.so or -$PWD/libdislocator.so. - -Similarly to afl-tmin, the library is not "proprietary" and can be used with -other fuzzers or testing tools without the need for any code tweaks. It does not -require AFL-instrumented binaries to work. - -Note that the AFL_PRELOAD approach (which AFL internally maps to LD_PRELOAD or -DYLD_INSERT_LIBRARIES, depending on the OS) works only if the target binary is -dynamically linked. Otherwise, attempting to use the library will have no -effect. diff --git a/libdislocator/README.md b/libdislocator/README.md new file mode 100644 index 00000000..5d5a1464 --- /dev/null +++ b/libdislocator/README.md @@ -0,0 +1,60 @@ +# libdislocator, an abusive allocator + + (See ../docs/README for the general instruction manual.) + +This is a companion library that can be used as a drop-in replacement for the +libc allocator in the fuzzed binaries. It improves the odds of bumping into +heap-related security bugs in several ways: + + - It allocates all buffers so that they are immediately adjacent to a + subsequent PROT_NONE page, causing most off-by-one reads and writes to + immediately segfault, + + - It adds a canary immediately below the allocated buffer, to catch writes + to negative offsets (won't catch reads, though), + + - It sets the memory returned by malloc() to garbage values, improving the + odds of crashing when the target accesses uninitialized data, + + - It sets freed memory to PROT_NONE and does not actually reuse it, causing + most use-after-free bugs to segfault right away, + + - It forces all realloc() calls to return a new address - and sets + PROT_NONE on the original block. This catches use-after-realloc bugs, + + - It checks for calloc() overflows and can cause soft or hard failures + of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, + AFL_LD_HARD_FAIL). + +Basically, it is inspired by some of the non-default options available for the +OpenBSD allocator - see malloc.conf(5) on that platform for reference. It is +also somewhat similar to several other debugging libraries, such as gmalloc +and DUMA - but is simple, plug-and-play, and designed specifically for fuzzing +jobs. + +Note that it does nothing for stack-based memory handling errors. The +-fstack-protector-all setting for GCC / clang, enabled when using AFL_HARDEN, +can catch some subset of that. + +The allocator is slow and memory-intensive (even the tiniest allocation uses up +4 kB of physical memory and 8 kB of virtual mem), making it completely unsuitable +for "production" uses; but it can be faster and more hassle-free than ASAN / MSAN +when fuzzing small, self-contained binaries. + +To use this library, run AFL like so: + +``` +AFL_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] +``` + +You *have* to specify path, even if it's just ./libdislocator.so or +$PWD/libdislocator.so. + +Similarly to afl-tmin, the library is not "proprietary" and can be used with +other fuzzers or testing tools without the need for any code tweaks. It does not +require AFL-instrumented binaries to work. + +Note that the AFL_PRELOAD approach (which AFL internally maps to LD_PRELOAD or +DYLD_INSERT_LIBRARIES, depending on the OS) works only if the target binary is +dynamically linked. Otherwise, attempting to use the library will have no +effect. diff --git a/libtokencap/README.md b/libtokencap/README.md new file mode 100644 index 00000000..baf69da1 --- /dev/null +++ b/libtokencap/README.md @@ -0,0 +1,63 @@ +# strcmp() / memcmp() token capture library + + (See ../docs/README for the general instruction manual.) + +This Linux-only companion library allows you to instrument `strcmp()`, `memcmp()`, +and related functions to automatically extract syntax tokens passed to any of +these libcalls. The resulting list of tokens may be then given as a starting +dictionary to afl-fuzz (the -x option) to improve coverage on subsequent +fuzzing runs. + +This may help improving coverage in some targets, and do precisely nothing in +others. In some cases, it may even make things worse: if libtokencap picks up +syntax tokens that are not used to process the input data, but that are a part +of - say - parsing a config file... well, you're going to end up wasting a lot +of CPU time on trying them out in the input stream. In other words, use this +feature with care. Manually screening the resulting dictionary is almost +always a necessity. + +As for the actual operation: the library stores tokens, without any deduping, +by appending them to a file specified via AFL_TOKEN_FILE. If the variable is not +set, the tool uses stderr (which is probably not what you want). + +Similarly to afl-tmin, the library is not "proprietary" and can be used with +other fuzzers or testing tools without the need for any code tweaks. It does not +require AFL-instrumented binaries to work. + +To use the library, you *need* to make sure that your fuzzing target is compiled +with -fno-builtin and is linked dynamically. If you wish to automate the first +part without mucking with CFLAGS in Makefiles, you can set AFL_NO_BUILTIN=1 +when using afl-gcc. This setting specifically adds the following flags: + +``` + -fno-builtin-strcmp -fno-builtin-strncmp -fno-builtin-strcasecmp + -fno-builtin-strcasencmp -fno-builtin-memcmp -fno-builtin-strstr + -fno-builtin-strcasestr +``` + +The next step is simply loading this library via LD_PRELOAD. The optimal usage +pattern is to allow afl-fuzz to fuzz normally for a while and build up a corpus, +and then fire off the target binary, with libtokencap.so loaded, on every file +found by AFL in that earlier run. This demonstrates the basic principle: + +``` + export AFL_TOKEN_FILE=$PWD/temp_output.txt + + for i in /queue/id*; do + LD_PRELOAD=/path/to/libtokencap.so \ + /path/to/target/program [...params, including $i...] + done + + sort -u temp_output.txt >afl_dictionary.txt +``` + +If you don't get any results, the target library is probably not using strcmp() +and memcmp() to parse input; or you haven't compiled it with -fno-builtin; or +the whole thing isn't dynamically linked, and LD_PRELOAD is having no effect. + +PS. The library is Linux-only because there is probably no particularly portable +and non-invasive way to distinguish between read-only and read-write memory +mappings. The `__tokencap_load_mappings()` function is the only thing that would +need to be changed for other OSes. Porting to platforms with /proc//maps +(e.g., FreeBSD) should be trivial. + diff --git a/libtokencap/README.tokencap b/libtokencap/README.tokencap deleted file mode 100644 index 650739f2..00000000 --- a/libtokencap/README.tokencap +++ /dev/null @@ -1,61 +0,0 @@ -========================================= -strcmp() / memcmp() token capture library -========================================= - - (See ../docs/README for the general instruction manual.) - -This Linux-only companion library allows you to instrument strcmp(), memcmp(), -and related functions to automatically extract syntax tokens passed to any of -these libcalls. The resulting list of tokens may be then given as a starting -dictionary to afl-fuzz (the -x option) to improve coverage on subsequent -fuzzing runs. - -This may help improving coverage in some targets, and do precisely nothing in -others. In some cases, it may even make things worse: if libtokencap picks up -syntax tokens that are not used to process the input data, but that are a part -of - say - parsing a config file... well, you're going to end up wasting a lot -of CPU time on trying them out in the input stream. In other words, use this -feature with care. Manually screening the resulting dictionary is almost -always a necessity. - -As for the actual operation: the library stores tokens, without any deduping, -by appending them to a file specified via AFL_TOKEN_FILE. If the variable is not -set, the tool uses stderr (which is probably not what you want). - -Similarly to afl-tmin, the library is not "proprietary" and can be used with -other fuzzers or testing tools without the need for any code tweaks. It does not -require AFL-instrumented binaries to work. - -To use the library, you *need* to make sure that your fuzzing target is compiled -with -fno-builtin and is linked dynamically. If you wish to automate the first -part without mucking with CFLAGS in Makefiles, you can set AFL_NO_BUILTIN=1 -when using afl-gcc. This setting specifically adds the following flags: - - -fno-builtin-strcmp -fno-builtin-strncmp -fno-builtin-strcasecmp - -fno-builtin-strcasencmp -fno-builtin-memcmp -fno-builtin-strstr - -fno-builtin-strcasestr - -The next step is simply loading this library via LD_PRELOAD. The optimal usage -pattern is to allow afl-fuzz to fuzz normally for a while and build up a corpus, -and then fire off the target binary, with libtokencap.so loaded, on every file -found by AFL in that earlier run. This demonstrates the basic principle: - - export AFL_TOKEN_FILE=$PWD/temp_output.txt - - for i in /queue/id*; do - LD_PRELOAD=/path/to/libtokencap.so \ - /path/to/target/program [...params, including $i...] - done - - sort -u temp_output.txt >afl_dictionary.txt - -If you don't get any results, the target library is probably not using strcmp() -and memcmp() to parse input; or you haven't compiled it with -fno-builtin; or -the whole thing isn't dynamically linked, and LD_PRELOAD is having no effect. - -PS. The library is Linux-only because there is probably no particularly portable -and non-invasive way to distinguish between read-only and read-write memory -mappings. The __tokencap_load_mappings() function is the only thing that would -need to be changed for other OSes. Porting to platforms with /proc//maps -(e.g., FreeBSD) should be trivial. - diff --git a/qemu_mode/libcompcov/README.compcov b/qemu_mode/libcompcov/README.compcov deleted file mode 100644 index 9be13d88..00000000 --- a/qemu_mode/libcompcov/README.compcov +++ /dev/null @@ -1,37 +0,0 @@ -================================================================ -strcmp() / memcmp() CompareCoverage library for AFLplusplus-QEMU -================================================================ - - Written by Andrea Fioraldi - -This Linux-only companion library allows you to instrument strcmp(), memcmp(), -and related functions to log the CompareCoverage of these libcalls. - -Use this with caution. While this can speedup a lot the bypass of hard -branch conditions it can also waste a lot of time and take up unnecessary space -in the shared memory when logging the coverage related to functions that -doesn't process input-related data. - -To use the library, you *need* to make sure that your fuzzing target is linked -dynamically and make use of strcmp(), memcmp(), and related functions. -For optimized binaries this is an issue, those functions are often inlined -and this module is not capable to log the coverage in this case. - -If you have the source code of the fuzzing target you should nto use this -library and QEMU but build it with afl-clang-fast and the laf-intel options. - -To use this library make sure to preload it with AFL_PRELOAD. - - export AFL_PRELOAD=/path/to/libcompcov.so - export AFL_COMPCOV_LEVEL=1 - - afl-fuzz -Q -i input -o output -- - -The AFL_COMPCOV_LEVEL tells to QEMU and libcompcov how to log comaprisons. -Level 1 logs just comparison with immediates / read-only memory and level 2 -logs all the comparisons. - -The library make use of https://github.com/ouadev/proc_maps_parser and so it is -Linux specific. However this is not a strict dependency, other UNIX operating -systems can be supported simply replacing the code related to the -/proc/self/maps parsing. diff --git a/qemu_mode/libcompcov/README.md b/qemu_mode/libcompcov/README.md new file mode 100644 index 00000000..5aaa3dd8 --- /dev/null +++ b/qemu_mode/libcompcov/README.md @@ -0,0 +1,37 @@ +# strcmp() / memcmp() CompareCoverage library for afl++ QEMU + + Written by Andrea Fioraldi + +This Linux-only companion library allows you to instrument `strcmp()`, `memcmp()`, +and related functions to log the CompareCoverage of these libcalls. + +Use this with caution. While this can speedup a lot the bypass of hard +branch conditions it can also waste a lot of time and take up unnecessary space +in the shared memory when logging the coverage related to functions that +doesn't process input-related data. + +To use the library, you *need* to make sure that your fuzzing target is linked +dynamically and make use of strcmp(), memcmp(), and related functions. +For optimized binaries this is an issue, those functions are often inlined +and this module is not capable to log the coverage in this case. + +If you have the source code of the fuzzing target you should nto use this +library and QEMU but build it with afl-clang-fast and the laf-intel options. + +To use this library make sure to preload it with AFL_PRELOAD. + +``` + export AFL_PRELOAD=/path/to/libcompcov.so + export AFL_COMPCOV_LEVEL=1 + + afl-fuzz -Q -i input -o output -- +``` + +The AFL_COMPCOV_LEVEL tells to QEMU and libcompcov how to log comaprisons. +Level 1 logs just comparison with immediates / read-only memory and level 2 +logs all the comparisons. + +The library make use of https://github.com/ouadev/proc_maps_parser and so it is +Linux specific. However this is not a strict dependency, other UNIX operating +systems can be supported simply replacing the code related to the +/proc/self/maps parsing. -- cgit 1.4.1