diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/Changelog.md | 27 | ||||
| -rw-r--r-- | docs/FAQ.md | 20 | ||||
| -rw-r--r-- | docs/INSTALL.md | 18 | ||||
| -rw-r--r-- | docs/env_variables.md | 16 | ||||
| -rw-r--r-- | docs/features.md | 102 | ||||
| -rw-r--r-- | docs/fuzzing_binary-only_targets.md | 23 | ||||
| -rw-r--r-- | docs/fuzzing_in_depth.md | 142 | ||||
| -rw-r--r-- | docs/important_changes.md | 35 | ||||
| -rw-r--r-- | docs/tutorials.md | 2 |
9 files changed, 210 insertions, 175 deletions
diff --git a/docs/Changelog.md b/docs/Changelog.md index bdd6c405..7f539556 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -1,15 +1,16 @@ # Changelog - This is the list of all noteworthy changes made in every public release of - the tool. See README.md for the general instruction manual. + This is the list of all noteworthy changes made in every public + release of the tool. See README.md for the general instruction manual. ## Staying informed Want to stay in the loop on major new features? Join our mailing list by sending a mail to <afl-users+subscribe@googlegroups.com>. -### Version ++3.15a (dev) - - documentation restructuring, made possible by Google Season of Docs +### Version ++4.00c (release) + - complete documentation restructuring, made possible by Google Season + of Docs :) thank you Jana! - we renamed several UI and fuzzer_stat entries to be more precise, e.g. "unique crashes" -> "saved crashes", "total paths" -> "corpus count", "current path" -> "current item". @@ -17,14 +18,14 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - Nyx mode (full system emulation with snapshot capability) has been added - thanks to @schumilo and @eqv! - unicorn_mode: - - Moved to unicorn2! By Ziqiao Kong (@lazymio) - - Faster, more accurate emulation (newer QEMU base), riscv support + - Moved to unicorn2! by Ziqiao Kong (@lazymio) + - Faster, more accurate emulation (newer QEMU base), risc-v support - removed indirections in rust callbacks - new binary-only fuzzing mode: coresight_mode for aarch64 CPUs :) thanks to RICSecLab submitting! - if instrumented libaries are dlopen()'ed after the forkserver you - will now see crashes. before you would have colliding coverage. - we changed this to force fixing a broken setup rather then allowing + will now see a crash. Before you would have colliding coverage. + We changed this to force fixing a broken setup rather then allowing ineffective fuzzing. See docs/best_practices.md how to fix such setups. - afl-fuzz: @@ -32,10 +33,11 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. (it is better!) - fix a regression introduced in 3.10 that resulted in less coverage being detected. thanks to Collin May for reporting! + - ensure all spawned targets are killed on exit - added AFL_IGNORE_PROBLEMS, plus checks to identify and abort on incorrect LTO usage setups and enhanced the READMEs for better information on how to deal with instrumenting libraries - - fix -n dumb mode (nobody should use this) + - fix -n dumb mode (nobody should use this mode though) - fix stability issue with LTO and cmplog - better banner - more effective cmplog mode @@ -54,6 +56,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - Prevent accidentaly killing non-afl/fuzz services when aborting afl-showmap and other tools. - afl-cc: + - detect overflow reads on initial input buffer for asan - new cmplog mode (incompatible with older afl++ versions) - support llvm IR select instrumentation for default PCGUARD and LTO - fix for shared linking on MacOS @@ -62,7 +65,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - fixed a potential crash in targets for LAF string handling - fixed a bad assert in LAF split switches - added AFL_USE_TSAN thread sanitizer support - - llvm and LTO mode modified to work with new llvm 14-dev (again. again.) + - llvm and LTO mode modified to work with new llvm 14-dev (again.) - fix for AFL_REAL_LD - more -z defs filtering - make -v without options work @@ -73,7 +76,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - added afl-persistent-config script to set perform permanent system configuration settings for fuzzing, for Linux and Macos. thanks to jhertz! - - added xml, curl and exotic string functions to llvm dictionary features + - added xml, curl & exotic string functions to llvm dictionary feature - fix AFL_PRELOAD issues on MacOS - removed utils/afl_frida because frida_mode/ is now so much better - added uninstall target to makefile (todo: update new readme!) @@ -96,7 +99,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>. - Fix to instrument global namespace functions in c++ - Fix for llvm 13 - support partial linking - - do honor AFL_LLVM_{ALLOW/DENY}LIST for LTO autodictionary and DICT2FILE + - do honor AFL_LLVM_{ALLOW/DENY}LIST for LTO autodictionary andDICT2FILE - We do support llvm versions from 3.8 to 5.0 again - frida_mode: - several fixes for cmplog diff --git a/docs/FAQ.md b/docs/FAQ.md index 73328d6e..1822e46b 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -188,8 +188,9 @@ If you find an interesting or important question missing, submit it via A power schedule measures how "interesting" a value is, and depending on the calculated value spends more or less time mutating it. - AFL++ comes with several power schedules, initially ported from [AFLFast](https://github.com/mboehme/aflfast) - however modified to be more effective and several more modes added. + AFL++ comes with several power schedules, initially ported from + [AFLFast](https://github.com/mboehme/aflfast), however, modified to be more + effective and several more modes added. The most effective modes are `-p fast` (default) and `-p explore`. @@ -200,6 +201,7 @@ If you find an interesting or important question missing, submit it via It does not make sense to explain the details of the calculation and reasoning behind all of the schedules. If you are interested, read the source code and the AFLFast paper. +</p></details> ## Troubleshooting @@ -215,16 +217,18 @@ If you find an interesting or important question missing, submit it via To ignore this set AFL_IGNORE_PROBLEMS=1. ``` - As the error describes, a dlopen() call is happening in the target that is loading an instrumented library after the forkserver is already in place, - This is a problem for afl-fuzz because when the forkserver is started we must know the map size already and it can't be changed later. + As the error describes, a dlopen() call is happening in the target that is + loading an instrumented library after the forkserver is already in place. This + is a problem for afl-fuzz because when the forkserver is started, we must know + the map size already and it can't be changed later. - The best solution is to simply set `AFL_PRELOAD=foo.so` the libraries that - are dlopen'ed (e.g. use `strace` to see which), or to set a manual forkserver + The best solution is to simply set `AFL_PRELOAD=foo.so` to the libraries that + are dlopen'ed (e.g., use `strace` to see which), or to set a manual forkserver after the final dlopen(). - If this is not a viable option you can set `AFL_IGNORE_PROBLEMS=1` but then + If this is not a viable option, you can set `AFL_IGNORE_PROBLEMS=1` but then the existing map will be used also for the newly loaded libraries, which - allows it to work, however the efficiency of the fuzzing will be partially + allows it to work, however, the efficiency of the fuzzing will be partially degraded. </p></details> diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 3c96a4fd..2847ca2a 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -33,10 +33,10 @@ sudo make install It is recommended to install the newest available gcc, clang and llvm-dev possible in your distribution! -Note that `make distrib` also builds FRIDA mode, QEMU mode, unicorn_mode -and more. If you just want plain AFL++, then do `make all`. If you want -some assisting tooling compiled but are not interested in binary-only targets -then instead choose: +Note that `make distrib` also builds FRIDA mode, QEMU mode, unicorn_mode, and +more. If you just want plain AFL++, then do `make all`. If you want some +assisting tooling compiled but are not interested in binary-only targets, then +instead choose: ```shell make source-only @@ -46,7 +46,8 @@ These build targets exist: * all: the main afl++ binaries and llvm/gcc instrumentation * binary-only: everything for binary-only fuzzing: frida_mode, nyx_mode, - qemu_mode, frida_mode, unicorn_mode, coresight_mode, libdislocator, libtokencap + qemu_mode, frida_mode, unicorn_mode, coresight_mode, libdislocator, + libtokencap * source-only: everything for source code fuzzing: nyx_mode, libdislocator, libtokencap * distrib: everything (for both binary-only and source code fuzzing) @@ -116,10 +117,9 @@ sudo gmake install ``` `afl-gcc` will fail unless you have GCC installed, but that is using outdated -instrumentation anyway. `afl-clang` might fail too depending on your PATH -setup. But you don't want neither, you want `afl-clang-fast` anyway :) -Note that `afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on -MacOS. +instrumentation anyway. `afl-clang` might fail too depending on your PATH setup. +But you don't want neither, you want `afl-clang-fast` anyway :) Note that +`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on MacOS. The crash reporting daemon that comes by default with MacOS X will cause problems with fuzzing. You need to turn it off: diff --git a/docs/env_variables.md b/docs/env_variables.md index 76a64bd2..2a8fbcb7 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -285,12 +285,12 @@ mode. ``` - GCC_PLUGIN mode only: Setting `AFL_GCC_INSTRUMENT_FILE` or - `AFL_GCC_ALLOWLIST` with a filename will only instrument those files - that match the names listed in this file (one filename per line). - - Setting `AFL_GCC_DENYLIST` or `AFL_GCC_BLOCKLIST` - with a file name and/or function will only skip those files that match - the names listed in the specified file. See + `AFL_GCC_ALLOWLIST` with a filename will only instrument those files that + match the names listed in this file (one filename per line). + + Setting `AFL_GCC_DENYLIST` or `AFL_GCC_BLOCKLIST` with a file name and/or + function will only skip those files that match the names listed in the + specified file. See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information. @@ -298,8 +298,8 @@ mode. code with calls to an injected subroutine instead of the much more efficient inline instrumentation. - Setting `AFL_GCC_SKIP_NEVERZERO=1` will not implement the skip zero test. - If the target performs only a few loops, then this will give a small + Setting `AFL_GCC_SKIP_NEVERZERO=1` will not implement the skip zero test. If + the target performs only a few loops, then this will give a small performance boost. ## 4) Settings for afl-fuzz diff --git a/docs/features.md b/docs/features.md index 628f9383..dd3d2bcb 100644 --- a/docs/features.md +++ b/docs/features.md @@ -1,70 +1,88 @@ # Important features of AFL++ AFL++ supports llvm from 3.8 up to version 12, very fast binary fuzzing with -QEMU 5.1 with laf-intel and redqueen, FRIDA mode, unicorn mode, gcc plugin, full +QEMU 5.1 with laf-intel and Redqueen, FRIDA mode, unicorn mode, gcc plugin, full *BSD, Mac OS, Solaris and Android support and much, much, much more. -| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) |unicorn_mode(10) |nyx_mode(12)|coresight_mode(11)| -| ------------------------------|:-------:|:---------:|:----------:|:----------------:|:----------------:|:----------------:|:----------:|:----------------:| -| Threadsafe counters [A] | | x(3) | | | | | x | | -| NeverZero [B] | x86[_64]| x(1) | x | x | x | x | | | -| Persistent Mode [C] | | x | x | x86[_64]/arm64 | x86[_64]/arm[64] | x | | | -| LAF-Intel / CompCov [D] | | x | | | x86[_64]/arm[64] | x86[_64]/arm[64] | x86[_64] | | -| CmpLog [E] | | x | | x86[_64]/arm64 | x86[_64]/arm[64] | | | | -| Selective Instrumentation [F] | | x | x | x | x | | | | -| Non-Colliding Coverage [G] | | x(4) | | | (x)(5) | | | | -| Ngram prev_loc Coverage [H] | | x(6) | | | | | | | -| Context Coverage [I] | | x(6) | | | | | | | -| Auto Dictionary [J] | | x(7) | | | | | | | -| Snapshot Support (K) | | (x)(8) | (x)(8) | | (x)(5) | | x | | -| Shared Memory Test cases [L] | | x | x | x86[_64]/arm64 | x | x | x | | +## Features and instrumentation + +| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) | unicorn_mode(10) | nyx_mode(12) | coresight_mode(11) | +| ------------------------------|:--------:|:---------:|:----------:|:--------------:|:----------------:|:----------------:|:------------:|:------------------:| +| Threadsafe counters [A] | | x(3) | | | | | x | | +| NeverZero [B] | x86[_64] | x(1) | x | x | x | x | | | +| Persistent Mode [C] | | x | x | x86[_64]/arm64 | x86[_64]/arm[64] | x | | | +| LAF-Intel / CompCov [D] | | x | | | x86[_64]/arm[64] | x86[_64]/arm[64] | x86[_64] | | +| CmpLog [E] | | x | | x86[_64]/arm64 | x86[_64]/arm[64] | | | | +| Selective Instrumentation [F] | | x | x | x | x | | | | +| Non-Colliding Coverage [G] | | x(4) | | | (x)(5) | | | | +| Ngram prev_loc Coverage [H] | | x(6) | | | | | | | +| Context Coverage [I] | | x(6) | | | | | | | +| Auto Dictionary [J] | | x(7) | | | | | | | +| Snapshot Support [K] | | (x)(8) | (x)(8) | | (x)(5) | | x | | +| Shared Memory Test cases [L] | | x | x | x86[_64]/arm64 | x | x | x | | + +## More information about features A. Default is not thread-safe coverage counter updates for better performance, see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) -B. On wrapping coverage counters (255 + 1) skip the 0 value and jump to 1 - instead. This has shown to give better coverage data and is the default; - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + +B. On wrapping coverage counters (255 + 1), skip the 0 value and jump to 1 + instead. This has shown to give better coverage data and is the default; see + [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + C. Instead of forking, reiterate the fuzz target function in a loop (like - `LLVMFuzzerTestOneInput`. Great speed increase but only work with target - functions that does not keep state, leak memory or exit; - see [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) -D. Split any non-8-bit comparison to 8 bit comparison; - see [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) -E. CmpLog is our enhanced [Redqueen](https://www.ndss-symposium.org/ndss-paper/redqueen-fuzzing-with-input-to-state-correspondence/) - implementation, see see [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md) + `LLVMFuzzerTestOneInput`. Great speed increase but only works with target + functions that do not keep state, leak memory, or exit; see + [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) + +D. Split any non-8-bit comparison to 8-bit comparison; see + [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) + +E. CmpLog is our enhanced + [Redqueen](https://www.ndss-symposium.org/ndss-paper/redqueen-fuzzing-with-input-to-state-correspondence/) + implementation, see + [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md) + F. Similar and compatible to clang 13+ sancov sanitize-coverage-allow/deny but for all llvm versions and all our compile modes, only instrument what should - be instrumented, for more speed, directed fuzzing and less instability; - see [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) + be instrumented, for more speed, directed fuzzing and less instability; see + [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) + G. Vanilla AFL uses coverage where edges could collide to the same coverage bytes the larger the target is. Our default instrumentation in LTO and afl-clang-fast (PCGUARD) uses non-colliding coverage that also makes it - faster. Vanilla AFL style is available with `AFL_LLVM_INSTRUMENT=AFL`; - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + faster. Vanilla AFL style is available with `AFL_LLVM_INSTRUMENT=AFL`; see + [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + H.+I. Alternative coverage based on previous edges (NGRAM) or depending on the - caller (CTX), based on + caller (CTX), based on [https://www.usenix.org/system/files/raid2019-wang-jinghan.pdf](https://www.usenix.org/system/files/raid2019-wang-jinghan.pdf); - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + J. An LTO feature that creates a fuzzing dictionary based on comparisons found - during compilation/instrumentation. Automatic feature :) - See [instrumentation/README.lto.md](../instrumentation/README.lto.md) + during compilation/instrumentation. Automatic feature :) See + [instrumentation/README.lto.md](../instrumentation/README.lto.md) + K. The snapshot feature requires a kernel module that was a lot of work to get right and maintained so it is no longer supported. We have [nyx_mode](../nyx_mode/README.md) instead. + L. Faster fuzzing and less kernel syscall overhead by in-memory fuzz testcase delivery, see [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) -1. default for LLVM >= 9.0, environment variable for older version due an +## More information about instrumentation + +1. Default for LLVM >= 9.0, environment variable for older version due an efficiency bug in previous llvm versions 2. GCC creates non-performant code, hence it is disabled in gcc_plugin -3. with `AFL_LLVM_THREADSAFE_INST`, disables NeverZero -4. with pcguard mode and LTO mode for LLVM 11 and newer -5. upcoming, development in the branch -6. not compatible with LTO instrumentation and needs at least LLVM v4.1 -7. automatic in LTO mode with LLVM 11 and newer, an extra pass for all LLVM +3. With `AFL_LLVM_THREADSAFE_INST`, disables NeverZero +4. With pcguard mode and LTO mode for LLVM 11 and newer +5. Upcoming, development in the branch +6. Not compatible with LTO instrumentation and needs at least LLVM v4.1 +7. Automatic in LTO mode with LLVM 11 and newer, an extra pass for all LLVM versions that write to a file to use with afl-fuzz' `-x` -8. the snapshot LKM is currently unmaintained due to too many kernel changes +8. The snapshot LKM is currently unmaintained due to too many kernel changes coming too fast :-( 9. FRIDA mode is supported on Linux and MacOS for Intel and ARM 10. QEMU/Unicorn is only supported on Linux @@ -72,6 +90,8 @@ L. Faster fuzzing and less kernel syscall overhead by in-memory fuzz testcase extension 12. Nyx mode is only supported on Linux and currently restricted to x86_x64 +## Integrated features and patches + Among others, the following features and patches have been integrated: * NeverZero patch for afl-gcc, instrumentation, QEMU mode and unicorn_mode which @@ -80,7 +100,7 @@ Among others, the following features and patches have been integrated: * Unicorn mode which allows fuzzing of binaries from completely different platforms (integration provided by domenukk) * The new CmpLog instrumentation for LLVM and QEMU inspired by - [Redqueen](https://www.syssec.ruhr-uni-bochum.de/media/emma/veroeffentlichungen/2018/12/17/NDSS19-Redqueen.pdf) + [Redqueen](https://github.com/RUB-SysSec/redqueen) * Win32 PE binary-only fuzzing with QEMU and Wine * AFLfast's power schedules by Marcel Böhme: [https://github.com/mboehme/aflfast](https://github.com/mboehme/aflfast) diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index 32e6c6c2..1a2b27c7 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -14,6 +14,7 @@ fuzzed with AFL++. FRIDA mode and QEMU mode in persistent mode are the fastest - if persistent mode is possible and the stability is high enough. + Otherwise, try Zafl, RetroWrite, Dyninst, and if these fail, too, then try standard FRIDA/QEMU mode with `AFL_ENTRYPOINT` to where you need it. @@ -127,13 +128,13 @@ Working examples already exist :-) ### Nyx mode -Nyx is a full system emulation fuzzing environment with snapshot support that -is built upon KVM and QEMU. -It is only available on Linux and currently restricted to x86_x64. +Nyx is a full system emulation fuzzing environment with snapshot support that is +built upon KVM and QEMU. It is only available on Linux and currently restricted +to x86_x64. For binary-only fuzzing a special 5.10 kernel is required. -See [nyx_mode/README.md](../nyx_mode/README.md) +See [nyx_mode/README.md](../nyx_mode/README.md). ### Unicorn @@ -198,15 +199,15 @@ afl-clang-fast's. ### RetroWrite -RetroWrite is a static binary rewriter that can be combined with AFL++. -If you have an x86_64 binary that still has its symbols (i.e., not stripped binary), -is compiled with position independent code (PIC/PIE), and does not contain C++ exceptions, -then the RetroWrite solution might be for you. It decompiles to ASM files which -can then be instrumented with afl-gcc. +RetroWrite is a static binary rewriter that can be combined with AFL++. If you +have an x86_64 binary that still has its symbols (i.e., not stripped binary), is +compiled with position independent code (PIC/PIE), and does not contain C++ +exceptions, then the RetroWrite solution might be for you. It decompiles to ASM +files which can then be instrumented with afl-gcc. Binaries that are statically instrumented for fuzzing using RetroWrite are close -in performance to compiler-instrumented binaries and outperform -the QEMU-based instrumentation. +in performance to compiler-instrumented binaries and outperform the QEMU-based +instrumentation. [https://github.com/HexHive/retrowrite](https://github.com/HexHive/retrowrite) diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 446c4466..ac72c757 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -141,37 +141,39 @@ options are available: [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md). If you use LTO, LLVM, or GCC_PLUGIN mode -(afl-clang-fast/afl-clang-lto/afl-gcc-fast), you have the option to -selectively instrument _parts_ of the target that you are interested in. -For afl-clang-fast you have to use an llvm version newer than 10.0.0 or a mode -other than DEFAULT/PCGUARD. +(afl-clang-fast/afl-clang-lto/afl-gcc-fast), you have the option to selectively +instrument _parts_ of the target that you are interested in. For afl-clang-fast, +you have to use an llvm version newer than 10.0.0 or a mode other than +DEFAULT/PCGUARD. -This step can be done either by explicitly including parts to be instrumented -or by explicitly excluding parts from instrumentation. +This step can be done either by explicitly including parts to be instrumented or +by explicitly excluding parts from instrumentation. -* To instrument _only specified parts_ - create a file (eg `allowlist.txt`) with all the filenames and/or functions of - the source code that should be instrumented and then: +* To instrument _only specified parts_, create a file (e.g., `allowlist.txt`) + with all the filenames and/or functions of the source code that should be + instrumented and then: - 1. just put one filename or function (prefixing with `fun: `) per line (no - directory information necessary for filenames) in the file `allowlist.txt`. - Example: -``` -foo.cpp # will match foo/foo.cpp, bar/foo.cpp, barfoo.cpp etc. -fun: foo_func # will match the function foo_func -``` + 1. Just put one filename or function (prefixing with `fun: `) per line (no + directory information necessary for filenames) in the file `allowlist.txt`. - 2. set `export AFL_LLVM_ALLOWLIST=allowlist.txt` to enable selective positive + Example: + + ``` + foo.cpp # will match foo/foo.cpp, bar/foo.cpp, barfoo.cpp etc. + fun: foo_func # will match the function foo_func + ``` + + 2. Set `export AFL_LLVM_ALLOWLIST=allowlist.txt` to enable selective positive instrumentation. -* Similarly to _exclude_ specified parts from instrumentation - create a file (eg `denylist.txt`) with all the filenames of the source code - that should be skipped during instrumentation and then +* Similarly to _exclude_ specified parts from instrumentation, create a file + (e.g., `denylist.txt`) with all the filenames of the source code that should + be skipped during instrumentation and then: - 1. same as above just put one filename or function per line in the file - `denylist.txt` + 1. Same as above. Just put one filename or function per line in the file + `denylist.txt`. - 2. set `export AFL_LLVM_DENYLIST=denylist.txt` to enable selective negative + 2. Set `export AFL_LLVM_DENYLIST=denylist.txt` to enable selective negative instrumentation. **NOTE:** During optimization functions might be @@ -195,8 +197,8 @@ allows you to find bugs that would not necessarily result in a crash. Note that sanitizers have a huge impact on CPU (= less executions per second) and RAM usage. Also, you should only run one afl-fuzz instance per sanitizer -type. This is enough because e.g. a use-after-free bug will be picked up by -ASAN (address sanitizer) anyway after syncing test cases from other fuzzing +type. This is enough because e.g. a use-after-free bug will be picked up by ASAN +(address sanitizer) anyway after syncing test cases from other fuzzing instances, so running more than one address sanitized target would be a waste. The following sanitizers have built-in support in AFL++: @@ -208,9 +210,9 @@ The following sanitizers have built-in support in AFL++: local variable that is defined and read before it is even set. Enabled with `export AFL_USE_MSAN=1` before compiling. * UBSAN = Undefined Behavior SANitizer, finds instances where - by the C and C++ - standards - undefined behavior happens, e.g., adding two signed integers - where the result is larger than what a signed integer can hold. Enabled - with `export AFL_USE_UBSAN=1` before compiling. + standards - undefined behavior happens, e.g., adding two signed integers where + the result is larger than what a signed integer can hold. Enabled with `export + AFL_USE_UBSAN=1` before compiling. * CFISAN = Control Flow Integrity SANitizer, finds instances where the control flow is found to be illegal. Originally this was rather to prevent return oriented programming (ROP) exploit chains from functioning. In fuzzing, this @@ -224,7 +226,7 @@ The following sanitizers have built-in support in AFL++: the other sanitizers above this needs `__AFL_LEAK_CHECK();` added to all areas of the target source code where you find a leak check necessary! Enabled with `export AFL_USE_LSAN=1` before compiling. To ignore the memory-leaking check - for certain allocations, `__AFL_LSAN_OFF();` can be used before memory is + for certain allocations, `__AFL_LSAN_OFF();` can be used before memory is allocated, and `__AFL_LSAN_ON();` afterwards. Memory allocated between these two macros will not be checked for memory leaks. @@ -286,8 +288,8 @@ Then build the target. (Usually with `make`.) 3. In case the configure/build system complains about AFL++'s compiler and aborts, then set `export AFL_NOOPT=1` which will then just behave like the - real compiler and run the configure step separately. - For building the target afterwards this option has to be unset again! + real compiler and run the configure step separately. For building the target + afterwards this option has to be unset again! #### configure @@ -397,11 +399,12 @@ You can find many good examples of starting files in the Use the AFL++ tool `afl-cmin` to remove inputs from the corpus that do not produce a new path/coverage in the target: -1. Put all files from [step a](#a-collecting-inputs) into one directory, e.g., INPUTS. +1. Put all files from [step a](#a-collecting-inputs) into one directory, e.g., + `INPUTS`. 2. Run afl-cmin: - * If the target program is to be called by fuzzing as `bin/target - INPUTFILE`, replace the INPUTFILE argument that the target program would read - from with `@@`: + * If the target program is to be called by fuzzing as `bin/target INPUTFILE`, + replace the INPUTFILE argument that the target program would read from with + `@@`: ``` afl-cmin -i INPUTS -o INPUTS_UNIQUE -- bin/target -someopt @@ @@ -448,8 +451,8 @@ In this final step, fuzz the target. There are not that many important options to run the target - unless you want to use many CPU cores/threads for the fuzzing, which will make the fuzzing much more useful. -If you just use one instance for fuzzing, then you are fuzzing just for fun and not -seriously :-) +If you just use one instance for fuzzing, then you are fuzzing just for fun and +not seriously :-) ### a) Running afl-fuzz @@ -482,14 +485,14 @@ afl-fuzz -i input -o output -- bin/target -someopt @@ Note that the directory specified with `-o` will be created if it does not exist. -It can be valuable to run afl-fuzz in a `screen` or `tmux` shell so you can log off, -or afl-fuzz is not aborted if you are running it in a remote ssh session where -the connection fails in between. Only do that though once you have verified that -your fuzzing setup works! Run it like `screen -dmS afl-main -- afl-fuzz -M -main-$HOSTNAME -i ...` and it will start away in a screen session. To enter this -session, type `screen -r afl-main`. You see - it makes sense to name the screen -session same as the afl-fuzz -M/-S naming :-) For more information on screen or -tmux, check their documentation. +It can be valuable to run afl-fuzz in a `screen` or `tmux` shell so you can log +off, or afl-fuzz is not aborted if you are running it in a remote ssh session +where the connection fails in between. Only do that though once you have +verified that your fuzzing setup works! Run it like `screen -dmS afl-main -- +afl-fuzz -M main-$HOSTNAME -i ...` and it will start away in a screen session. +To enter this session, type `screen -r afl-main`. You see - it makes sense to +name the screen session same as the afl-fuzz `-M`/`-S` naming :-) For more +information on screen or tmux, check their documentation. If you need to stop and re-start the fuzzing, use the same command line options (or even change them by selecting a different power schedule or another mutation @@ -499,18 +502,21 @@ mode!) and switch the input directory with a dash (`-`): afl-fuzz -i - -o output -- bin/target -someopt @@ ``` -Adding a dictionary is helpful. See the directory -[dictionaries/](../dictionaries/) if something is already included for your data -format, and tell afl-fuzz to load that dictionary by adding `-x +Adding a dictionary is helpful. You have to following options: + +* See the directory +[dictionaries/](../dictionaries/), if something is already included for your +data format, and tell afl-fuzz to load that dictionary by adding `-x dictionaries/FORMAT.dict`. -With `afl-clang-lto`, you have an autodictionary generation for which you need -to do nothing except to use afl-clang-lto as the compiler. -With `afl-clang-fast` you can set -`AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a -dictionary during target compilation. -You also have the option to generate a dictionary yourself during an independant -run of the target, see [utils/libtokencap/README.md](../utils/libtokencap/README.md). -Finally you can also write a dictionary file manually, of course. +* With `afl-clang-lto`, you have an autodictionary generation for which you need + to do nothing except to use afl-clang-lto as the compiler. +* With `afl-clang-fast`, you can set + `AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a + dictionary during target compilation. +* You also have the option to generate a dictionary yourself during an + independent run of the target, see + [utils/libtokencap/README.md](../utils/libtokencap/README.md). +* Finally, you can also write a dictionary file manually, of course. afl-fuzz has a variety of options that help to workaround target quirks like very specific locations for the input file (`-f`), performing deterministic @@ -572,8 +578,8 @@ can set the cache size (in MB) by setting the environment variable There should be one main fuzzer (`-M main-$HOSTNAME` option) and as many secondary fuzzers (e.g., `-S variant1`) as you have cores that you use. Every --M/-S entry needs a unique name (that can be whatever), however, the same -o -output directory location has to be used for all instances. +`-M`/`-S` entry needs a unique name (that can be whatever), however, the same +`-o` output directory location has to be used for all instances. For every secondary fuzzer there should be a variation, e.g.: * one should fuzz the target that was compiled differently: with sanitizers @@ -588,10 +594,10 @@ For every secondary fuzzer there should be a variation, e.g.: All other secondaries should be used like this: * a quarter to a third with the MOpt mutator enabled: `-L 0` -* run with a different power schedule, recommended are: - `fast (default), explore, coe, lin, quad, exploit and rare` which you can set - with the `-p` option, e.g., `-p explore`. See the [FAQ](FAQ.md#what-are-power-schedules) - for details. +* run with a different power schedule, recommended are: `fast` (default), + `explore`, `coe`, `lin`, `quad`, `exploit`, and `rare` which you can set with + the `-p` option, e.g., `-p explore`. See the + [FAQ](FAQ.md#what-are-power-schedules) for details. * a few instances should use the old queue cycling with `-Z` Also, it is recommended to set `export AFL_IMPORT_FIRST=1` to load test cases @@ -615,7 +621,7 @@ A long list can be found at [https://github.com/Microsvuln/Awesome-AFL](https://github.com/Microsvuln/Awesome-AFL). However, you can also sync AFL++ with honggfuzz, libfuzzer with `-entropic=1`, -etc. Just show the main fuzzer (-M) with the `-F` option where the queue/work +etc. Just show the main fuzzer (`-M`) with the `-F` option where the queue/work directory of a different fuzzer is, e.g., `-F /src/target/honggfuzz`. Using honggfuzz (with `-n 1` or `-n 2`) and libfuzzer in parallel is highly recommended! @@ -877,9 +883,9 @@ normal fuzzing campaigns as these are much shorter runnings. 1. Always: * LTO has a much longer compile time which is diametrical to short fuzzing - hence use afl-clang-fast instead. - * If you compile with CMPLOG, then you can save compilation time and reuse that - compiled target with the `-c` option and as the main fuzz target. This - will impact the speed by ~15% though. + * If you compile with CMPLOG, then you can save compilation time and reuse + that compiled target with the `-c` option and as the main fuzz target. + This will impact the speed by ~15% though. * `AFL_FAST_CAL` - enables fast calibration, this halves the time the saturated corpus needs to be loaded. * `AFL_CMPLOG_ONLY_NEW` - only perform cmplog on new finds, not the initial @@ -909,8 +915,8 @@ and ## The End -Check out the [FAQ](FAQ.md). Maybe it answers your question (that you might -not even have known you had ;-) ). +Check out the [FAQ](FAQ.md). Maybe it answers your question (that you might not +even have known you had ;-) ). This is basically all you need to know to professionally run fuzzing campaigns. If you want to know more, the tons of texts in [docs/](./) will have you diff --git a/docs/important_changes.md b/docs/important_changes.md index 39fe56b9..e847f360 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -7,23 +7,23 @@ changes. With AFL++ 4.00, we introduced the following changes from previous behaviors: * the complete documentation was overhauled and restructured thanks to @llzmb! - * a new CMPLOG target format requires recompiling CMPLOG targets for use - with afl++ 4.0 onwards + * a new CMPLOG target format requires recompiling CMPLOG targets for use with + AFL++ 4.0 onwards * better naming for several fields in the UI With AFL++ 3.15, we introduced the following changes from previous behaviors: - * afl-cmin and afl-showmap -Ci now descend into subdirectories like afl-fuzz - -i does (but note that afl-cmin.bash does not) + * afl-cmin and afl-showmap `-Ci` now descend into subdirectories like afl-fuzz + `-i` does (but note that afl-cmin.bash does not) With AFL++ 3.14, we introduced the following changes from previous behaviors: - * afl-fuzz: deterministic fuzzing is not a default for -M main anymore + * afl-fuzz: deterministic fuzzing is not a default for `-M main` anymore * afl-cmin/afl-showmap -i now descends into subdirectories (afl-cmin.bash, however, does not) With AFL++ 3.10, we introduced the following changes from previous behaviors: - * The '+' feature of the '-t' option now means to auto-calculate the timeout + * The '+' feature of the `-t` option now means to auto-calculate the timeout with the value given being the maximum timeout. The original meaning of - "skipping timeouts instead of abort" is now inherent to the -t option. + "skipping timeouts instead of abort" is now inherent to the `-t` option. With AFL++ 3.00, we introduced changes that break some previous AFL and AFL++ behaviors and defaults: @@ -41,19 +41,20 @@ behaviors and defaults: if any were given. This allows to fuzz targets build regularly like those for debug or release versions. * afl-fuzz: - * if neither -M or -S is specified, `-S default` is assumed, so more fuzzers - can easily be added later + * if neither `-M` or `-S` is specified, `-S default` is assumed, so more + fuzzers can easily be added later * `-i` input directory option now descends into subdirectories. It also does not fail on crashes and too large files, instead it skips them and uses them for splicing mutations - * -m none is now the default, set memory limits (in MB) with, e.g., -m 250 - * deterministic fuzzing is now disabled by default (unless using -M) and can - be enabled with -D + * `-m` none is now the default, set memory limits (in MB) with, e.g., `-m + 250` + * deterministic fuzzing is now disabled by default (unless using `-M`) and + can be enabled with `-D` * a caching of test cases can now be performed and can be modified by - editing config.h for TESTCASE_CACHE or by specifying the environment + editing config.h for `TESTCASE_CACHE` or by specifying the environment variable `AFL_TESTCACHE_SIZE` (in MB). Good values are between 50-500 (default: 50). - * -M mains do not perform trimming - * examples/ got renamed to utils/ - * libtokencap/ libdislocator/ and qdbi_mode/ were moved to utils/ - * afl-cmin/afl-cmin.bash now search first in PATH and last in AFL_PATH + * `-M` mains do not perform trimming + * `examples/` got renamed to `utils/` + * `libtokencap/`, `libdislocator/`, and `qdbi_mode/` were moved to `utils/` + * afl-cmin/afl-cmin.bash now search first in `PATH` and last in `AFL_PATH` diff --git a/docs/tutorials.md b/docs/tutorials.md index 9d4f50eb..64d2b376 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -28,7 +28,7 @@ structure is), these links have you covered: * Superion for AFL++: [https://github.com/adrian-rt/superion-mutator](https://github.com/adrian-rt/superion-mutator) -# Video Tutorials +## Video Tutorials * [Install AFL++ Ubuntu](https://www.youtube.com/watch?v=5dCvhkbi3RA) * [[Fuzzing with AFLplusplus] Installing AFLPlusplus and fuzzing a simple C program](https://www.youtube.com/watch?v=9wRVo0kYSlc) * [[Fuzzing with AFLplusplus] How to fuzz a binary with no source code on Linux in persistent mode](https://www.youtube.com/watch?v=LGPJdEO02p4) |