diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/ChangeLog | 12 | ||||
| -rw-r--r-- | docs/binaryonly_fuzzing.md | 161 | ||||
| -rw-r--r-- | docs/binaryonly_fuzzing.txt | 144 |
3 files changed, 170 insertions, 147 deletions
diff --git a/docs/ChangeLog b/docs/ChangeLog index 5347d244..5017a803 100644 --- a/docs/ChangeLog +++ b/docs/ChangeLog @@ -21,9 +21,15 @@ Version ++2.60d (develop): - afl-fuzz: - now prints the real python version support compiled in - set stronger performance compile options and little tweaks - - afl-clang-fast now shows in the help output for which llvm version it - was compiled for - - added blacklisted function check in llvm_mode + - Android: prefer bigcores when selecting a CPU + - afl-clang-fast: + - show in the help output for which llvm version it was compiled for + - now does not need to be recompiled between trace-pc and pass + instrumentation. compile normally and set AFL_LLVM_USE_TRACE_PC :) + - llvm 11 is supported + - afl-cmin is now a sh script (invoking awk) instead of bash for portability + the original script is still present as afl-cmin.bash + - added blacklist and whitelisting function check in all modules of llvm_mode - added fix from Debian project to compile libdislocator and libtokencap diff --git a/docs/binaryonly_fuzzing.md b/docs/binaryonly_fuzzing.md new file mode 100644 index 00000000..6eff30d7 --- /dev/null +++ b/docs/binaryonly_fuzzing.md @@ -0,0 +1,161 @@ +# Fuzzing binary-only programs with afl++ + + afl++, libfuzzer and others are great if you have the source code, and + it allows for very fast and coverage guided fuzzing. + + However, if there is only the binary program and no source code available, + then standard `afl-fuzz -n` (dumb mode) is not effective. + + The following is a description of how these binaries can be fuzzed with afl++ + + !!!!! + TL;DR: try DYNINST with afl-dyninst. If it produces too many crashes then + use afl -Q qemu_mode, or better: use both in parallel. + !!!!! + + +## QEMU + + Qemu is the "native" solution to the program. + It is available in the ./qemu_mode/ directory and once compiled it can + be accessed by the afl-fuzz -Q command line option. + The speed decrease is at about 50%. + It is the easiest to use alternative and even works for cross-platform binaries. + + Note that there is also honggfuzz: [https://github.com/google/honggfuzz](https://github.com/google/honggfuzz) + which now has a qemu_mode, but its performance is just 1.5%! + + As it is included in afl++ this needs no URL. + + +## WINE+QEMU + + Wine mode can run Win32 PE binaries with the QEMU instrumentation. + It needs Wine, python3 and the pefile python package installed. + + As it is included in afl++ this needs no URL. + + +## UNICORN + + Unicorn is a fork of QEMU. The instrumentation is, therefore, very similar. + In contrast to QEMU, Unicorn does not offer a full system or even userland + emulation. Runtime environment and/or loaders have to be written from scratch, + if needed. On top, block chaining has been removed. This means the speed boost + introduced in the patched QEMU Mode of afl++ cannot simply be ported over to + Unicorn. For further information, check out ./unicorn_mode.txt. + + As it is included in afl++ this needs no URL. + + +## DYNINST + + Dyninst is a binary instrumentation framework similar to Pintool and + Dynamorio (see far below). However whereas Pintool and Dynamorio work at + runtime, dyninst instruments the target at load time, and then let it run - + or save the binary with the changes. + This is great for some things, e.g. fuzzing, and not so effective for others, + e.g. malware analysis. + + So what we can do with dyninst is taking every basic block, and put afl's + instrumention code in there - and then save the binary. + Afterwards we can just fuzz the newly saved target binary with afl-fuzz. + Sounds great? It is. The issue though - it is a non-trivial problem to + insert instructions, which change addresses in the process space, so that + everything is still working afterwards. Hence more often than not binaries + crash when they are run. + + The speed decrease is about 15-35%, depending on the optimization options + used with afl-dyninst. + + So if Dyninst works, it is the best option available. Otherwise it just + doesn't work well. + + [https://github.com/vanhauser-thc/afl-dyninst](https://github.com/vanhauser-thc/afl-dyninst) + + +## INTEL-PT + + If you have a newer Intel CPU, you can make use of Intels processor trace. + The big issue with Intel's PT is the small buffer size and the complex + encoding of the debug information collected through PT. + This makes the decoding very CPU intensive and hence slow. + As a result, the overall speed decrease is about 70-90% (depending on + the implementation and other factors). + + There are two afl intel-pt implementations: + + 1. [https://github.com/junxzm1990/afl-pt](https://github.com/junxzm1990/afl-pt) + => this needs Ubuntu 14.04.05 without any updates and the 4.4 kernel. + + 2. [https://github.com/hunter-ht-2018/ptfuzzer](https://github.com/hunter-ht-2018/ptfuzzer) + => this needs a 4.14 or 4.15 kernel. the "nopti" kernel boot option must + be used. This one is faster than the other. + + Note that there is also honggfuzz: https://github.com/google/honggfuzz + But its IPT performance is just 6%! + + +## CORESIGHT + + Coresight is ARM's answer to Intel's PT. + There is no implementation so far which handle coresight and getting + it working on an ARM Linux is very difficult due to custom kernel building + on embedded systems is difficult. And finding one that has coresight in + the ARM chip is difficult too. + My guess is that it is slower than Qemu, but faster than Intel PT. + + If anyone finds any coresight implementation for afl please ping me: vh@thc.org + + +## FRIDA + + Frida is a dynamic instrumentation engine like Pintool, Dyninst and Dynamorio. + What is special is that it is written Python, and scripted with Javascript. + It is mostly used to reverse binaries on mobile phones however can be used + everywhere. + + There is a WIP fuzzer available at [https://github.com/andreafioraldi/frida-fuzzer](https://github.com/andreafioraldi/frida-fuzzer) + + +## PIN & DYNAMORIO + + Pintool and Dynamorio are dynamic instrumentation engines, and they can be + used for getting basic block information at runtime. + Pintool is only available for Intel x32/x64 on Linux, Mac OS and Windows + whereas Dynamorio is additionally available for ARM and AARCH64. + Dynamorio is also 10x faster than Pintool. + + The big issue with Dynamorio (and therefore Pintool too) is speed. + Dynamorio has a speed decrease of 98-99% + Pintool has a speed decrease of 99.5% + + Hence Dynamorio is the option to go for if everything fails, and Pintool + only if Dynamorio fails too. + + Dynamorio solutions: + * [https://github.com/vanhauser-thc/afl-dynamorio](https://github.com/vanhauser-thc/afl-dynamorio) + * [https://github.com/mxmssh/drAFL](https://github.com/mxmssh/drAFL) + * [https://github.com/googleprojectzero/winafl/](https://github.com/googleprojectzero/winafl/) <= very good but windows only + + Pintool solutions: + * [https://github.com/vanhauser-thc/afl-pin](https://github.com/vanhauser-thc/afl-pin) + * [https://github.com/mothran/aflpin](https://github.com/mothran/aflpin) + * [https://github.com/spinpx/afl_pin_mode](https://github.com/spinpx/afl_pin_mode) <= only old Pintool version supported + + +## Non-AFL solutions + + There are many binary-only fuzzing frameworks. + Some are great for CTFs but don't work with large binaries, others are very + slow but have good path discovery, some are very hard to set-up ... + + * QSYM: [https://github.com/sslab-gatech/qsym](https://github.com/sslab-gatech/qsym) + * Manticore: [https://github.com/trailofbits/manticore](https://github.com/trailofbits/manticore) + * S2E: [https://github.com/S2E](https://github.com/S2E) + * ... please send me any missing that are good + + +## Closing words + + That's it! News, corrections, updates? Send an email to vh@thc.org diff --git a/docs/binaryonly_fuzzing.txt b/docs/binaryonly_fuzzing.txt deleted file mode 100644 index 239fb4b0..00000000 --- a/docs/binaryonly_fuzzing.txt +++ /dev/null @@ -1,144 +0,0 @@ - -Fuzzing binary-only programs with afl++ -======================================= - -afl++, libfuzzer and others are great if you have the source code, and -it allows for very fast and coverage guided fuzzing. - -However, if there is only the binary program and not source code available, -then standard afl++ (dumb mode) is not effective. - -The following is a description of how these can be fuzzed with afl++ - -!!!!! -TL;DR: try DYNINST with afl-dyninst. If it produces too many crashes then - use afl -Q qemu_mode, or better: use both in parallel. -!!!!! - - -QEMU ----- -Qemu is the "native" solution to the program. -It is available in the ./qemu_mode/ directory and once compiled it can -be accessed by the afl-fuzz -Q command line option. -The speed decrease is at about 50% -It is the easiest to use alternative and even works for cross-platform binaries. - -As it is included in afl++ this needs no URL. - -WINE+QEMU ---------- -Wine mode can run Win32 PE with the QEMU instrumentation. -It needs Wine, python3 and the pefile python package installed. - -UNICORN -------- -Unicorn is a fork of QEMU. The instrumentation is, therefore, very similar. -In contrast to QEMU, Unicorn does not offer a full system or even userland emulation. -Runtime environment and/or loaders have to be written from scratch, if needed. -On top, block chaining has been removed. This means the speed boost introduced in -to the patched QEMU Mode of afl++ cannot simply be ported over to Unicorn. -For further information, check out ./unicorn_mode.txt. - - -DYNINST -------- -Dyninst is a binary instrumentation framework similar to Pintool and Dynamorio -(see far below). However whereas Pintool and Dynamorio work at runtime, dyninst -instruments the target at load time, and then let it run. -This is great for some things, e.g. fuzzing, and not so effective for others, -e.g. malware analysis. - -So what we can do with dyninst is taking every basic block, and put afl's -instrumention code in there - and then save the binary. -Afterwards we can just fuzz the newly saved target binary with afl-fuzz. -Sounds great? It is. The issue though - it is a non-trivial problem to -insert instructions, which change addresses in the process space, so -everything is still working afterwards. Hence more often than not binaries -crash when they are run (because of instrumentation). - -The speed decrease is about 15-35%, depending on the optimization options -used with afl-dyninst. - -So if dyninst works, it is the best option available. Otherwise it just doesn't -work well. - -https://github.com/vanhauser-thc/afl-dyninst - - -INTEL-PT --------- -If you have a newer Intel CPU, you can make use of Intels processor trace. -The big issue with Intel's PT is the small buffer size and the complex -encoding of the debug information collected through PT. -This makes the decoding very CPU intensive and hence slow. -As a result, the overall speed decrease is about 70-90% (depending on -the implementation and other factors). - -There are two afl intel-pt implementations: - -1. https://github.com/junxzm1990/afl-pt - => this needs Ubuntu 14.04.05 without any updates and the 4.4 kernel. - -2. https://github.com/hunter-ht-2018/ptfuzzer - => this needs a 4.14 or 4.15 kernel. the "nopti" kernel boot option must - be used. This one is faster than the other. - - -CORESIGHT ---------- - -Coresight is ARM's answer to Intel's PT. -There is no implementation so far which handle coresight and getting -it working on an ARM Linux is very difficult due to custom kernel building -on embedded systems is difficult. And finding one that has coresight in -the ARM chip is difficult too. -My guess is that it is slower than Qemu, but faster than Intel PT. -If anyone finds any coresight implementation for afl please ping me: -vh@thc.org - - -PIN & DYNAMORIO ---------------- - -Pintool and Dynamorio are dynamic instrumentation engines, and they can be -used for getting basic block information at runtime. -Pintool is only available for Intel x32/x64 on Linux, Mac OS and Windows -whereas Dynamorio is additionally available for ARM and AARCH64. -Dynamorio is also 10x faster than Pintool. - -The big issue with Dynamorio (and therefore Pintool too) is speed. -Dynamorio has a speed decrease of 98-99% -Pintool has a speed decrease of 99.5% - -Hence Dynamorio is the option to go for if everything fails, and Pintool -only if Dynamorio fails too. - -Dynamorio solutions: - https://github.com/vanhauser-thc/afl-dynamorio - https://github.com/mxmssh/drAFL - https://github.com/googleprojectzero/winafl/ <= very good but windows only - -Pintool solutions: - https://github.com/vanhauser-thc/afl-pin - https://github.com/mothran/aflpin - https://github.com/spinpx/afl_pin_mode <= only old Pintool version supported - - -Non-AFL solutions ------------------ - -There are many binary-only fuzzing frameworks. Some are great for CTFs but don't -work with large binaries, others are very slow but have good path discovery, -some are very hard to set-up ... - -QSYM: https://github.com/sslab-gatech/qsym -Manticore: https://github.com/trailofbits/manticore -S2E: https://github.com/S2E -<please send me any missing that are good> - - - -That's it! -News, corrections, updates? -Email vh@thc.org |