diff options
author | van Hauser <vh@thc.org> | 2022-01-21 13:08:20 +0100 |
---|---|---|
committer | GitHub <noreply@github.com> | 2022-01-21 13:08:20 +0100 |
commit | b427a53a6b203dd73aabb25f6068e55691339b03 (patch) | |
tree | 020fb8d0b012d5ebfd7ded0b32c00b79851a0532 | |
parent | 26a3d1b53b0a3ae690735c550c2985f079f3c672 (diff) | |
parent | 2bb86863e6d15dbd5bb0c381afa51f476302784e (diff) | |
download | afl++-b427a53a6b203dd73aabb25f6068e55691339b03.tar.gz |
Merge pull request #1299 from llzmb/docs_quality_assurance_3
Docs content - quality assurance - 3rd run
-rw-r--r-- | README.md | 18 | ||||
-rw-r--r-- | docs/FAQ.md | 20 | ||||
-rw-r--r-- | docs/INSTALL.md | 18 | ||||
-rw-r--r-- | docs/env_variables.md | 16 | ||||
-rw-r--r-- | docs/features.md | 102 | ||||
-rw-r--r-- | docs/fuzzing_binary-only_targets.md | 23 | ||||
-rw-r--r-- | docs/fuzzing_in_depth.md | 142 | ||||
-rw-r--r-- | docs/important_changes.md | 35 | ||||
-rw-r--r-- | docs/tutorials.md | 2 | ||||
-rw-r--r-- | instrumentation/README.cmplog.md | 5 | ||||
-rw-r--r-- | nyx_mode/README.md | 204 | ||||
-rw-r--r-- | qemu_mode/README.md | 2 | ||||
-rw-r--r-- | src/README.md | 2 |
13 files changed, 333 insertions, 256 deletions
diff --git a/README.md b/README.md index 5c2262cf..966e8036 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,10 @@ Repository: AFL++ is maintained by: -* Marc "van Hauser" Heuse <mh@mh-sec.de>, -* Heiko "hexcoder-" Eißfeldt <heiko.eissfeldt@hexco.de>, -* Andrea Fioraldi <andreafioraldi@gmail.com> and -* Dominik Maier <mail@dmnk.co>. +* Marc "van Hauser" Heuse <mh@mh-sec.de> +* Heiko "hexcoder-" Eißfeldt <heiko.eissfeldt@hexco.de> +* Andrea Fioraldi <andreafioraldi@gmail.com> +* Dominik Maier <mail@dmnk.co> * Documentation: Jana Aydinbas <jana.aydinbas@gmail.com> Originally developed by Michał "lcamtuf" Zalewski. @@ -29,13 +29,13 @@ terms of the Apache-2.0 License. See the [LICENSE](LICENSE) for details. Here is some information to get you started: -* For an overview of the AFL++ documentation and a very helpful graphical - guide please visit [docs/README.md](docs/README.md) +* For an overview of the AFL++ documentation and a very helpful graphical guide, + please visit [docs/README.md](docs/README.md). * To get you started with tutorials, go to [docs/tutorials.md](docs/tutorials.md). * For releases, see the [Releases tab](https://github.com/AFLplusplus/AFLplusplus/releases) and - [branches](#branches). The best branches to use are however `stable` or + [branches](#branches). The best branches to use are, however, `stable` or `dev` - depending on your risk appetite. Also take a look at the list of [important changes in AFL++](docs/important_changes.md) and the list of [features](docs/features.md). @@ -127,13 +127,13 @@ Step-by-step quick start: Questions? Concerns? Bug reports? -* The contributors can be reached via (e.g. by creating an issue): +* The contributors can be reached via (e.g., by creating an issue): [https://github.com/AFLplusplus/AFLplusplus](https://github.com/AFLplusplus/AFLplusplus). * Take a look at our [FAQ](docs/FAQ.md). If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/discussions](https://github.com/AFLplusplus/AFLplusplus/discussions). * Best: join the [Awesome Fuzzing](https://discord.gg/gCraWct) Discord server. -* There is a (not really used) mailing list for the AFL/AFL++ project +* There is a (not really used) mailing list for the AFL/AFL++ project ([browse archive](https://groups.google.com/group/afl-users)). To compare notes with other users or to get notified about major new features, send an email to <afl-users+subscribe@googlegroups.com>, but note that this is not diff --git a/docs/FAQ.md b/docs/FAQ.md index 73328d6e..1822e46b 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -188,8 +188,9 @@ If you find an interesting or important question missing, submit it via A power schedule measures how "interesting" a value is, and depending on the calculated value spends more or less time mutating it. - AFL++ comes with several power schedules, initially ported from [AFLFast](https://github.com/mboehme/aflfast) - however modified to be more effective and several more modes added. + AFL++ comes with several power schedules, initially ported from + [AFLFast](https://github.com/mboehme/aflfast), however, modified to be more + effective and several more modes added. The most effective modes are `-p fast` (default) and `-p explore`. @@ -200,6 +201,7 @@ If you find an interesting or important question missing, submit it via It does not make sense to explain the details of the calculation and reasoning behind all of the schedules. If you are interested, read the source code and the AFLFast paper. +</p></details> ## Troubleshooting @@ -215,16 +217,18 @@ If you find an interesting or important question missing, submit it via To ignore this set AFL_IGNORE_PROBLEMS=1. ``` - As the error describes, a dlopen() call is happening in the target that is loading an instrumented library after the forkserver is already in place, - This is a problem for afl-fuzz because when the forkserver is started we must know the map size already and it can't be changed later. + As the error describes, a dlopen() call is happening in the target that is + loading an instrumented library after the forkserver is already in place. This + is a problem for afl-fuzz because when the forkserver is started, we must know + the map size already and it can't be changed later. - The best solution is to simply set `AFL_PRELOAD=foo.so` the libraries that - are dlopen'ed (e.g. use `strace` to see which), or to set a manual forkserver + The best solution is to simply set `AFL_PRELOAD=foo.so` to the libraries that + are dlopen'ed (e.g., use `strace` to see which), or to set a manual forkserver after the final dlopen(). - If this is not a viable option you can set `AFL_IGNORE_PROBLEMS=1` but then + If this is not a viable option, you can set `AFL_IGNORE_PROBLEMS=1` but then the existing map will be used also for the newly loaded libraries, which - allows it to work, however the efficiency of the fuzzing will be partially + allows it to work, however, the efficiency of the fuzzing will be partially degraded. </p></details> diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 3c96a4fd..2847ca2a 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -33,10 +33,10 @@ sudo make install It is recommended to install the newest available gcc, clang and llvm-dev possible in your distribution! -Note that `make distrib` also builds FRIDA mode, QEMU mode, unicorn_mode -and more. If you just want plain AFL++, then do `make all`. If you want -some assisting tooling compiled but are not interested in binary-only targets -then instead choose: +Note that `make distrib` also builds FRIDA mode, QEMU mode, unicorn_mode, and +more. If you just want plain AFL++, then do `make all`. If you want some +assisting tooling compiled but are not interested in binary-only targets, then +instead choose: ```shell make source-only @@ -46,7 +46,8 @@ These build targets exist: * all: the main afl++ binaries and llvm/gcc instrumentation * binary-only: everything for binary-only fuzzing: frida_mode, nyx_mode, - qemu_mode, frida_mode, unicorn_mode, coresight_mode, libdislocator, libtokencap + qemu_mode, frida_mode, unicorn_mode, coresight_mode, libdislocator, + libtokencap * source-only: everything for source code fuzzing: nyx_mode, libdislocator, libtokencap * distrib: everything (for both binary-only and source code fuzzing) @@ -116,10 +117,9 @@ sudo gmake install ``` `afl-gcc` will fail unless you have GCC installed, but that is using outdated -instrumentation anyway. `afl-clang` might fail too depending on your PATH -setup. But you don't want neither, you want `afl-clang-fast` anyway :) -Note that `afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on -MacOS. +instrumentation anyway. `afl-clang` might fail too depending on your PATH setup. +But you don't want neither, you want `afl-clang-fast` anyway :) Note that +`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on MacOS. The crash reporting daemon that comes by default with MacOS X will cause problems with fuzzing. You need to turn it off: diff --git a/docs/env_variables.md b/docs/env_variables.md index 76a64bd2..2a8fbcb7 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -285,12 +285,12 @@ mode. ``` - GCC_PLUGIN mode only: Setting `AFL_GCC_INSTRUMENT_FILE` or - `AFL_GCC_ALLOWLIST` with a filename will only instrument those files - that match the names listed in this file (one filename per line). - - Setting `AFL_GCC_DENYLIST` or `AFL_GCC_BLOCKLIST` - with a file name and/or function will only skip those files that match - the names listed in the specified file. See + `AFL_GCC_ALLOWLIST` with a filename will only instrument those files that + match the names listed in this file (one filename per line). + + Setting `AFL_GCC_DENYLIST` or `AFL_GCC_BLOCKLIST` with a file name and/or + function will only skip those files that match the names listed in the + specified file. See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information. @@ -298,8 +298,8 @@ mode. code with calls to an injected subroutine instead of the much more efficient inline instrumentation. - Setting `AFL_GCC_SKIP_NEVERZERO=1` will not implement the skip zero test. - If the target performs only a few loops, then this will give a small + Setting `AFL_GCC_SKIP_NEVERZERO=1` will not implement the skip zero test. If + the target performs only a few loops, then this will give a small performance boost. ## 4) Settings for afl-fuzz diff --git a/docs/features.md b/docs/features.md index 628f9383..dd3d2bcb 100644 --- a/docs/features.md +++ b/docs/features.md @@ -1,70 +1,88 @@ # Important features of AFL++ AFL++ supports llvm from 3.8 up to version 12, very fast binary fuzzing with -QEMU 5.1 with laf-intel and redqueen, FRIDA mode, unicorn mode, gcc plugin, full +QEMU 5.1 with laf-intel and Redqueen, FRIDA mode, unicorn mode, gcc plugin, full *BSD, Mac OS, Solaris and Android support and much, much, much more. -| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) |unicorn_mode(10) |nyx_mode(12)|coresight_mode(11)| -| ------------------------------|:-------:|:---------:|:----------:|:----------------:|:----------------:|:----------------:|:----------:|:----------------:| -| Threadsafe counters [A] | | x(3) | | | | | x | | -| NeverZero [B] | x86[_64]| x(1) | x | x | x | x | | | -| Persistent Mode [C] | | x | x | x86[_64]/arm64 | x86[_64]/arm[64] | x | | | -| LAF-Intel / CompCov [D] | | x | | | x86[_64]/arm[64] | x86[_64]/arm[64] | x86[_64] | | -| CmpLog [E] | | x | | x86[_64]/arm64 | x86[_64]/arm[64] | | | | -| Selective Instrumentation [F] | | x | x | x | x | | | | -| Non-Colliding Coverage [G] | | x(4) | | | (x)(5) | | | | -| Ngram prev_loc Coverage [H] | | x(6) | | | | | | | -| Context Coverage [I] | | x(6) | | | | | | | -| Auto Dictionary [J] | | x(7) | | | | | | | -| Snapshot Support (K) | | (x)(8) | (x)(8) | | (x)(5) | | x | | -| Shared Memory Test cases [L] | | x | x | x86[_64]/arm64 | x | x | x | | +## Features and instrumentation + +| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) | unicorn_mode(10) | nyx_mode(12) | coresight_mode(11) | +| ------------------------------|:--------:|:---------:|:----------:|:--------------:|:----------------:|:----------------:|:------------:|:------------------:| +| Threadsafe counters [A] | | x(3) | | | | | x | | +| NeverZero [B] | x86[_64] | x(1) | x | x | x | x | | | +| Persistent Mode [C] | | x | x | x86[_64]/arm64 | x86[_64]/arm[64] | x | | | +| LAF-Intel / CompCov [D] | | x | | | x86[_64]/arm[64] | x86[_64]/arm[64] | x86[_64] | | +| CmpLog [E] | | x | | x86[_64]/arm64 | x86[_64]/arm[64] | | | | +| Selective Instrumentation [F] | | x | x | x | x | | | | +| Non-Colliding Coverage [G] | | x(4) | | | (x)(5) | | | | +| Ngram prev_loc Coverage [H] | | x(6) | | | | | | | +| Context Coverage [I] | | x(6) | | | | | | | +| Auto Dictionary [J] | | x(7) | | | | | | | +| Snapshot Support [K] | | (x)(8) | (x)(8) | | (x)(5) | | x | | +| Shared Memory Test cases [L] | | x | x | x86[_64]/arm64 | x | x | x | | + +## More information about features A. Default is not thread-safe coverage counter updates for better performance, see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) -B. On wrapping coverage counters (255 + 1) skip the 0 value and jump to 1 - instead. This has shown to give better coverage data and is the default; - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + +B. On wrapping coverage counters (255 + 1), skip the 0 value and jump to 1 + instead. This has shown to give better coverage data and is the default; see + [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + C. Instead of forking, reiterate the fuzz target function in a loop (like - `LLVMFuzzerTestOneInput`. Great speed increase but only work with target - functions that does not keep state, leak memory or exit; - see [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) -D. Split any non-8-bit comparison to 8 bit comparison; - see [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) -E. CmpLog is our enhanced [Redqueen](https://www.ndss-symposium.org/ndss-paper/redqueen-fuzzing-with-input-to-state-correspondence/) - implementation, see see [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md) + `LLVMFuzzerTestOneInput`. Great speed increase but only works with target + functions that do not keep state, leak memory, or exit; see + [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) + +D. Split any non-8-bit comparison to 8-bit comparison; see + [instrumentation/README.laf-intel.md](../instrumentation/README.laf-intel.md) + +E. CmpLog is our enhanced + [Redqueen](https://www.ndss-symposium.org/ndss-paper/redqueen-fuzzing-with-input-to-state-correspondence/) + implementation, see + [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md) + F. Similar and compatible to clang 13+ sancov sanitize-coverage-allow/deny but for all llvm versions and all our compile modes, only instrument what should - be instrumented, for more speed, directed fuzzing and less instability; - see [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) + be instrumented, for more speed, directed fuzzing and less instability; see + [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) + G. Vanilla AFL uses coverage where edges could collide to the same coverage bytes the larger the target is. Our default instrumentation in LTO and afl-clang-fast (PCGUARD) uses non-colliding coverage that also makes it - faster. Vanilla AFL style is available with `AFL_LLVM_INSTRUMENT=AFL`; - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + faster. Vanilla AFL style is available with `AFL_LLVM_INSTRUMENT=AFL`; see + [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + H.+I. Alternative coverage based on previous edges (NGRAM) or depending on the - caller (CTX), based on + caller (CTX), based on [https://www.usenix.org/system/files/raid2019-wang-jinghan.pdf](https://www.usenix.org/system/files/raid2019-wang-jinghan.pdf); - see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md) + see [instrumentation/README.llvm.md](../instrumentation/README.llvm.md). + J. An LTO feature that creates a fuzzing dictionary based on comparisons found - during compilation/instrumentation. Automatic feature :) - See [instrumentation/README.lto.md](../instrumentation/README.lto.md) + during compilation/instrumentation. Automatic feature :) See + [instrumentation/README.lto.md](../instrumentation/README.lto.md) + K. The snapshot feature requires a kernel module that was a lot of work to get right and maintained so it is no longer supported. We have [nyx_mode](../nyx_mode/README.md) instead. + L. Faster fuzzing and less kernel syscall overhead by in-memory fuzz testcase delivery, see [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) -1. default for LLVM >= 9.0, environment variable for older version due an +## More information about instrumentation + +1. Default for LLVM >= 9.0, environment variable for older version due an efficiency bug in previous llvm versions 2. GCC creates non-performant code, hence it is disabled in gcc_plugin -3. with `AFL_LLVM_THREADSAFE_INST`, disables NeverZero -4. with pcguard mode and LTO mode for LLVM 11 and newer -5. upcoming, development in the branch -6. not compatible with LTO instrumentation and needs at least LLVM v4.1 -7. automatic in LTO mode with LLVM 11 and newer, an extra pass for all LLVM +3. With `AFL_LLVM_THREADSAFE_INST`, disables NeverZero +4. With pcguard mode and LTO mode for LLVM 11 and newer +5. Upcoming, development in the branch +6. Not compatible with LTO instrumentation and needs at least LLVM v4.1 +7. Automatic in LTO mode with LLVM 11 and newer, an extra pass for all LLVM versions that write to a file to use with afl-fuzz' `-x` -8. the snapshot LKM is currently unmaintained due to too many kernel changes +8. The snapshot LKM is currently unmaintained due to too many kernel changes coming too fast :-( 9. FRIDA mode is supported on Linux and MacOS for Intel and ARM 10. QEMU/Unicorn is only supported on Linux @@ -72,6 +90,8 @@ L. Faster fuzzing and less kernel syscall overhead by in-memory fuzz testcase extension 12. Nyx mode is only supported on Linux and currently restricted to x86_x64 +## Integrated features and patches + Among others, the following features and patches have been integrated: * NeverZero patch for afl-gcc, instrumentation, QEMU mode and unicorn_mode which @@ -80,7 +100,7 @@ Among others, the following features and patches have been integrated: * Unicorn mode which allows fuzzing of binaries from completely different platforms (integration provided by domenukk) * The new CmpLog instrumentation for LLVM and QEMU inspired by - [Redqueen](https://www.syssec.ruhr-uni-bochum.de/media/emma/veroeffentlichungen/2018/12/17/NDSS19-Redqueen.pdf) + [Redqueen](https://github.com/RUB-SysSec/redqueen) * Win32 PE binary-only fuzzing with QEMU and Wine * AFLfast's power schedules by Marcel Böhme: [https://github.com/mboehme/aflfast](https://github.com/mboehme/aflfast) diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index 32e6c6c2..1a2b27c7 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -14,6 +14,7 @@ fuzzed with AFL++. FRIDA mode and QEMU mode in persistent mode are the fastest - if persistent mode is possible and the stability is high enough. + Otherwise, try Zafl, RetroWrite, Dyninst, and if these fail, too, then try standard FRIDA/QEMU mode with `AFL_ENTRYPOINT` to where you need it. @@ -127,13 +128,13 @@ Working examples already exist :-) ### Nyx mode -Nyx is a full system emulation fuzzing environment with snapshot support that -is built upon KVM and QEMU. -It is only available on Linux and currently restricted to x86_x64. +Nyx is a full system emulation fuzzing environment with snapshot support that is +built upon KVM and QEMU. It is only available on Linux and currently restricted +to x86_x64. For binary-only fuzzing a special 5.10 kernel is required. -See [nyx_mode/README.md](../nyx_mode/README.md) +See [nyx_mode/README.md](../nyx_mode/README.md). ### Unicorn @@ -198,15 +199,15 @@ afl-clang-fast's. ### RetroWrite -RetroWrite is a static binary rewriter that can be combined with AFL++. -If you have an x86_64 binary that still has its symbols (i.e., not stripped binary), -is compiled with position independent code (PIC/PIE), and does not contain C++ exceptions, -then the RetroWrite solution might be for you. It decompiles to ASM files which -can then be instrumented with afl-gcc. +RetroWrite is a static binary rewriter that can be combined with AFL++. If you +have an x86_64 binary that still has its symbols (i.e., not stripped binary), is +compiled with position independent code (PIC/PIE), and does not contain C++ +exceptions, then the RetroWrite solution might be for you. It decompiles to ASM +files which can then be instrumented with afl-gcc. Binaries that are statically instrumented for fuzzing using RetroWrite are close -in performance to compiler-instrumented binaries and outperform -the QEMU-based instrumentation. +in performance to compiler-instrumented binaries and outperform the QEMU-based +instrumentation. [https://github.com/HexHive/retrowrite](https://github.com/HexHive/retrowrite) diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 446c4466..ac72c757 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -141,37 +141,39 @@ options are available: [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md). If you use LTO, LLVM, or GCC_PLUGIN mode -(afl-clang-fast/afl-clang-lto/afl-gcc-fast), you have the option to -selectively instrument _parts_ of the target that you are interested in. -For afl-clang-fast you have to use an llvm version newer than 10.0.0 or a mode -other than DEFAULT/PCGUARD. +(afl-clang-fast/afl-clang-lto/afl-gcc-fast), you have the option to selectively +instrument _parts_ of the target that you are interested in. For afl-clang-fast, +you have to use an llvm version newer than 10.0.0 or a mode other than +DEFAULT/PCGUARD. -This step can be done either by explicitly including parts to be instrumented -or by explicitly excluding parts from instrumentation. +This step can be done either by explicitly including parts to be instrumented or +by explicitly excluding parts from instrumentation. -* To instrument _only specified parts_ - create a file (eg `allowlist.txt`) with all the filenames and/or functions of - the source code that should be instrumented and then: +* To instrument _only specified parts_, create a file (e.g., `allowlist.txt`) + with all the filenames and/or functions of the source code that should be + instrumented and then: - 1. just put one filename or function (prefixing with `fun: `) per line (no - directory information necessary for filenames) in the file `allowlist.txt`. - Example: -``` -foo.cpp # will match foo/foo.cpp, bar/foo.cpp, barfoo.cpp etc. -fun: foo_func # will match the function foo_func -``` + 1. Just put one filename or function (prefixing with `fun: `) per line (no + directory information necessary for filenames) in the file `allowlist.txt`. - 2. set `export AFL_LLVM_ALLOWLIST=allowlist.txt` to enable selective positive + Example: + + ``` + foo.cpp # will match foo/foo.cpp, bar/foo.cpp, barfoo.cpp etc. + fun: foo_func # will match the function foo_func + ``` + + 2. Set `export AFL_LLVM_ALLOWLIST=allowlist.txt` to enable selective positive instrumentation. -* Similarly to _exclude_ specified parts from instrumentation - create a file (eg `denylist.txt`) with all the filenames of the source code - that should be skipped during instrumentation and then +* Similarly to _exclude_ specified parts from instrumentation, create a file + (e.g., `denylist.txt`) with all the filenames of the source code that should + be skipped during instrumentation and then: - 1. same as above just put one filename or function per line in the file - `denylist.txt` + 1. Same as above. Just put one filename or function per line in the file + `denylist.txt`. - 2. set `export AFL_LLVM_DENYLIST=denylist.txt` to enable selective negative + 2. Set `export AFL_LLVM_DENYLIST=denylist.txt` to enable selective negative instrumentation. **NOTE:** During optimization functions might be @@ -195,8 +197,8 @@ allows you to find bugs that would not necessarily result in a crash. Note that sanitizers have a huge impact on CPU (= less executions per second) and RAM usage. Also, you should only run one afl-fuzz instance per sanitizer -type. This is enough because e.g. a use-after-free bug will be picked up by -ASAN (address sanitizer) anyway after syncing test cases from other fuzzing +type. This is enough because e.g. a use-after-free bug will be picked up by ASAN +(address sanitizer) anyway after syncing test cases from other fuzzing instances, so running more than one address sanitized target would be a waste. The following sanitizers have built-in support in AFL++: @@ -208,9 +210,9 @@ The following sanitizers have built-in support in AFL++: local variable that is defined and read before it is even set. Enabled with `export AFL_USE_MSAN=1` before compiling. * UBSAN = Undefined Behavior SANitizer, finds instances where - by the C and C++ - standards - undefined behavior happens, e.g., adding two signed integers - where the result is larger than what a signed integer can hold. Enabled - with `export AFL_USE_UBSAN=1` before compiling. + standards - undefined behavior happens, e.g., adding two signed integers where + the result is larger than what a signed integer can hold. Enabled with `export + AFL_USE_UBSAN=1` before compiling. * CFISAN = Control Flow Integrity SANitizer, finds instances where the control flow is found to be illegal. Originally this was rather to prevent return oriented programming (ROP) exploit chains from functioning. In fuzzing, this @@ -224,7 +226,7 @@ The following sanitizers have built-in support in AFL++: the other sanitizers above this needs `__AFL_LEAK_CHECK();` added to all areas of the target source code where you find a leak check necessary! Enabled with `export AFL_USE_LSAN=1` before compiling. To ignore the memory-leaking check - for certain allocations, `__AFL_LSAN_OFF();` can be used before memory is + for certain allocations, `__AFL_LSAN_OFF();` can be used before memory is allocated, and `__AFL_LSAN_ON();` afterwards. Memory allocated between these two macros will not be checked for memory leaks. @@ -286,8 +288,8 @@ Then build the target. (Usually with `make`.) 3. In case the configure/build system complains about AFL++'s compiler and aborts, then set `export AFL_NOOPT=1` which will then just behave like the - real compiler and run the configure step separately. - For building the target afterwards this option has to be unset again! + real compiler and run the configure step separately. For building the target + afterwards this option has to be unset again! #### configure @@ -397,11 +399,12 @@ You can find many good examples of starting files in the Use the AFL++ tool `afl-cmin` to remove inputs from the corpus that do not produce a new path/coverage in the target: -1. Put all files from [step a](#a-collecting-inputs) into one directory, e.g., INPUTS. +1. Put all files from [step a](#a-collecting-inputs) into one directory, e.g., + `INPUTS`. 2. Run afl-cmin: - * If the target program is to be called by fuzzing as `bin/target - INPUTFILE`, replace the INPUTFILE argument that the target program would read - from with `@@`: + * If the target program is to be called by fuzzing as `bin/target INPUTFILE`, + replace the INPUTFILE argument that the target program would read from with + `@@`: ``` afl-cmin -i INPUTS -o INPUTS_UNIQUE -- bin/target -someopt @@ @@ -448,8 +451,8 @@ In this final step, fuzz the target. There are not that many important options to run the target - unless you want to use many CPU cores/threads for the fuzzing, which will make the fuzzing much more useful. -If you just use one instance for fuzzing, then you are fuzzing just for fun and not -seriously :-) +If you just use one instance for fuzzing, then you are fuzzing just for fun and +not seriously :-) ### a) Running afl-fuzz @@ -482,14 +485,14 @@ afl-fuzz -i input -o output -- bin/target -someopt @@ Note that the directory specified with `-o` will be created if it does not exist. -It can be valuable to run afl-fuzz in a `screen` or `tmux` shell so you can log off, -or afl-fuzz is not aborted if you are running it in a remote ssh session where -the connection fails in between. Only do that though once you have verified that -your fuzzing setup works! Run it like `screen -dmS afl-main -- afl-fuzz -M -main-$HOSTNAME -i ...` and it will start away in a screen session. To enter this -session, type `screen -r afl-main`. You see - it makes sense to name the screen -session same as the afl-fuzz -M/-S naming :-) For more information on screen or -tmux, check their documentation. +It can be valuable to run afl-fuzz in a `screen` or `tmux` shell so you can log +off, or afl-fuzz is not aborted if you are running it in a remote ssh session +where the connection fails in between. Only do that though once you have +verified that your fuzzing setup works! Run it like `screen -dmS afl-main -- +afl-fuzz -M main-$HOSTNAME -i ...` and it will start away in a screen session. +To enter this session, type `screen -r afl-main`. You see - it makes sense to +name the screen session same as the afl-fuzz `-M`/`-S` naming :-) For more +information on screen or tmux, check their documentation. If you need to stop and re-start the fuzzing, use the same command line options (or even change them by selecting a different power schedule or another mutation @@ -499,18 +502,21 @@ mode!) and switch the input directory with a dash (`-`): afl-fuzz -i - -o output -- bin/target -someopt @@ ``` -Adding a dictionary is helpful. See the directory -[dictionaries/](../dictionaries/) if something is already included for your data -format, and tell afl-fuzz to load that dictionary by adding `-x +Adding a dictionary is helpful. You have to following options: + +* See the directory +[dictionaries/](../dictionaries/), if something is already included for your +data format, and tell afl-fuzz to load that dictionary by adding `-x dictionaries/FORMAT.dict`. -With `afl-clang-lto`, you have an autodictionary generation for which you need -to do nothing except to use afl-clang-lto as the compiler. -With `afl-clang-fast` you can set -`AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a -dictionary during target compilation. -You also have the option to generate a dictionary yourself during an independant -run of the target, see [utils/libtokencap/README.md](../utils/libtokencap/README.md). -Finally you can also write a dictionary file manually, of course. +* With `afl-clang-lto`, you have an autodictionary generation for which you need + to do nothing except to use afl-clang-lto as the compiler. +* With `afl-clang-fast`, you can set + `AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a + dictionary during target compilation. +* You also have the option to generate a dictionary yourself during an + independent run of the target, see + [utils/libtokencap/README.md](../utils/libtokencap/README.md). +* Finally, you can also write a dictionary file manually, of course. afl-fuzz has a variety of options that help to workaround target quirks like very specific locations for the input file (`-f`), performing deterministic @@ -572,8 +578,8 @@ can set the cache size (in MB) by setting the environment variable There should be one main fuzzer (`-M main-$HOSTNAME` option) and as many secondary fuzzers (e.g., `-S variant1`) as you have cores that you use. Every --M/-S entry needs a unique name (that can be whatever), however, the same -o -output directory location has to be used for all instances. +`-M`/`-S` entry needs a unique name (that can be whatever), however, the same +`-o` output directory location has to be used for all instances. For every secondary fuzzer there should be a variation, e.g.: * one should fuzz the target that was compiled differently: with sanitizers @@ -588,10 +594,10 @@ For every secondary fuzzer there should be a variation, e.g.: All other secondaries should be used like this: * a quarter to a third with the MOpt mutator enabled: `-L 0` -* run with a different power schedule, recommended are: - `fast (default), explore, coe, lin, quad, exploit and rare` which you can set - with the `-p` option, e.g., `-p explore`. See the [FAQ](FAQ.md#what-are-power-schedules) - for details. +* run with a different power schedule, recommended are: `fast` (default), + `explore`, `coe`, `lin`, `quad`, `exploit`, and `rare` which you can set with + the `-p` option, e.g., `-p explore`. See the + [FAQ](FAQ.md#what-are-power-schedules) for details. * a few instances should use the old queue cycling with `-Z` Also, it is recommended to set `export AFL_IMPORT_FIRST=1` to load test cases @@ -615,7 +621,7 @@ A long list can be found at [https://github.com/Microsvuln/Awesome-AFL](https://github.com/Microsvuln/Awesome-AFL). However, you can also sync AFL++ with honggfuzz, libfuzzer with `-entropic=1`, -etc. Just show the main fuzzer (-M) with the `-F` option where the queue/work +etc. Just show the main fuzzer (`-M`) with the `-F` option where the queue/work directory of a different fuzzer is, e.g., `-F /src/target/honggfuzz`. Using honggfuzz (with `-n 1` or `-n 2`) and libfuzzer in parallel is highly recommended! @@ -877,9 +883,9 @@ normal fuzzing campaigns as these are much shorter runnings. 1. Always: * LTO has a much longer compile time which is diametrical to short fuzzing - hence use afl-clang-fast instead. - * If you compile with CMPLOG, then you can save compilation time and reuse that - compiled target with the `-c` option and as the main fuzz target. This - will impact the speed by ~15% though. + * If you compile with CMPLOG, then you can save compilation time and reuse + that compiled target with the `-c` option and as the main fuzz target. + This will impact the speed by ~15% though. * `AFL_FAST_CAL` - enables fast calibration, this halves the time the saturated corpus needs to be loaded. * `AFL_CMPLOG_ONLY_NEW` - only perform cmplog on new finds, not the initial @@ -909,8 +915,8 @@ and ## The End -Check out the [FAQ](FAQ.md). Maybe it answers your question (that you might -not even have known you had ;-) ). +Check out the [FAQ](FAQ.md). Maybe it answers your question (that you might not +even have known you had ;-) ). This is basically all you need to know to professionally run fuzzing campaigns. If you want to know more, the tons of texts in [docs/](./) will have you diff --git a/docs/important_changes.md b/docs/important_changes.md index 39fe56b9..e847f360 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -7,23 +7,23 @@ changes. With AFL++ 4.00, we introduced the following changes from previous behaviors: * the complete documentation was overhauled and restructured thanks to @llzmb! - * a new CMPLOG target format requires recompiling CMPLOG targets for use - with afl++ 4.0 onwards + * a new CMPLOG target format requires recompiling CMPLOG targets for use with + AFL++ 4.0 onwards * better naming for several fields in the UI With AFL++ 3.15, we introduced the following changes from previous behaviors: - * afl-cmin and afl-showmap -Ci now descend into subdirectories like afl-fuzz - -i does (but note that afl-cmin.bash does not) + * afl-cmin and afl-showmap `-Ci` now descend into subdirectories like afl-fuzz + `-i` does (but note that afl-cmin.bash does not) With AFL++ 3.14, we introduced the following changes from previous behaviors: - * afl-fuzz: deterministic fuzzing is not a default for -M main anymore + * afl-fuzz: deterministic fuzzing is not a default for `-M main` anymore * afl-cmin/afl-showmap -i now descends into subdirectories (afl-cmin.bash, however, does not) With AFL++ 3.10, we introduced the following changes from previous behaviors: - * The '+' feature of the '-t' option now means to auto-calculate the timeout + * The '+' feature of the `-t` option now means to auto-calculate the timeout with the value given being the maximum timeout. The original meaning of - "skipping timeouts instead of abort" is now inherent to the -t option. + "skipping timeouts instead of abort" is now inherent to the `-t` option. With AFL++ 3.00, we introduced changes that break some previous AFL and AFL++ behaviors and defaults: @@ -41,19 +41,20 @@ behaviors and defaults: if any were given. This allows to fuzz targets build regularly like those for debug or release versions. * afl-fuzz: - * if neither -M or -S is specified, `-S default` is assumed, so more fuzzers - can easily be added later + * if neither `-M` or `-S` is specified, `-S default` is assumed, so more + fuzzers can easily be added later * `-i` input directory option now descends into subdirectories. It also does not fail on crashes and too large files, instead it skips them and uses them for splicing mutations - * -m none is now the default, set memory limits (in MB) with, e.g., -m 250 - * deterministic fuzzing is now disabled by default (unless using -M) and can - be enabled with -D + * `-m` none is now the default, set memory limits (in MB) with, e.g., `-m + 250` + * deterministic fuzzing is now disabled by default (unless using `-M`) and + can be enabled with `-D` * a caching of test cases can now be performed and can be modified by - editing config.h for TESTCASE_CACHE or by specifying the environment + editing config.h for `TESTCASE_CACHE` or by specifying the environment variable `AFL_TESTCACHE_SIZE` (in MB). Good values are between 50-500 (default: 50). - * -M mains do not perform trimming - * examples/ got renamed to utils/ - * libtokencap/ libdislocator/ and qdbi_mode/ were moved to utils/ - * afl-cmin/afl-cmin.bash now search first in PATH and last in AFL_PATH + * `-M` mains do not perform trimming + * `examples/` got renamed to `utils/` + * `libtokencap/`, `libdislocator/`, and `qdbi_mode/` were moved to `utils/` + * afl-cmin/afl-cmin.bash now search first in `PATH` and last in `AFL_PATH` diff --git a/docs/tutorials.md b/docs/tutorials.md index 9d4f50eb..64d2b376 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -28,7 +28,7 @@ structure is), these links have you covered: * Superion for AFL++: [https://github.com/adrian-rt/superion-mutator](https://github.com/adrian-rt/superion-mutator) -# Video Tutorials +## Video Tutorials * [Install AFL++ Ubuntu](https://www.youtube.com/watch?v=5dCvhkbi3RA) * [[Fuzzing with AFLplusplus] Installing AFLPlusplus and fuzzing a simple C program](https://www.youtube.com/watch?v=9wRVo0kYSlc) * [[Fuzzing with AFLplusplus] How to fuzz a binary with no source code on Linux in persistent mode](https://www.youtube.com/watch?v=LGPJdEO02p4) diff --git a/instrumentation/README.cmplog.md b/instrumentation/README.cmplog.md index a2caca4f..8a9fd372 100644 --- a/instrumentation/README.cmplog.md +++ b/instrumentation/README.cmplog.md @@ -4,9 +4,8 @@ The CmpLog instrumentation enables logging of comparison operands in a shared memory. These values can be used by various mutators built on top of it. At the moment, -we support the RedQueen mutator (input-2-state instructions only), for details -see -[the RedQueen paper](https://www.syssec.ruhr-uni-bochum.de/media/emma/veroeffentlichungen/2018/12/17/NDSS19-Redqueen.pdf). +we support the Redqueen mutator (input-2-state instructions only), for details +see [the Redqueen paper](https://github.com/RUB-SysSec/redqueen). ## Build diff --git a/nyx_mode/README.md b/nyx_mode/README.md index 9c7a1d1f..b75f1793 100644 --- a/nyx_mode/README.md +++ b/nyx_mode/README.md @@ -1,7 +1,7 @@ -# Nyx Mode +# Nyx mode -Nyx is a full system emulation fuzzing mode that supports snapshotting and -can be used for both source code based instrumentation and binary-only targets. +Nyx is a full system emulation fuzzing mode that supports snapshotting and can +be used for both source code based instrumentation and binary-only targets. It is recommended to be used if the target cannot be fuzzed in persistent mode (so default fork mode fuzzing is used). @@ -10,36 +10,36 @@ It is only available on Linux and is currently restricted to x86_x64 however aarch64 support is in the works (but the host must then run on aarch64 too). Underneath it is built upon KVM and QEMU and requires a modern Linux kernel -(5.11+) for fuzzing source code based instrumented targets (e.g. -`afl-clang-fast`). To fuzz binary-only targets, this is done via Intel PT -and requires an Intel processor (6th generation onwards) and a special -5.10 kernel (see [KVM-Nyx](https://github.com/nyx-fuzz/KVM-Nyx)). +(5.11+) for fuzzing source code based instrumented targets (e.g., +`afl-clang-fast`). To fuzz binary-only targets, this is done via Intel PT and +requires an Intel processor (6th generation onwards) and a special 5.10 kernel +(see [KVM-Nyx](https://github.com/nyx-fuzz/KVM-Nyx)). ## Building Nyx mode 1. Install all the packages from [docs/INSTALL.md](../docs/INSTALL.md). -2. Additionally install the following packages: +2. Additionally, install the following packages: -```shell -apt-get install -y libgtk-3-dev pax-utils python3-msgpack python3-jinja2 -``` + ```shell + apt-get install -y libgtk-3-dev pax-utils python3-msgpack python3-jinja2 + ``` -3. As Nyx is written in Rust, install the newest rust compiler (rust packages - in the Linux distribution are usually too old to be able to build Nyx): +3. As Nyx is written in Rust, install the newest rust compiler (rust packages in + the Linux distribution are usually too old to be able to build Nyx): -```shell -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -``` + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + ``` 4. Finally build Nyx mode: -```shell -./build_nyx_support.sh -``` + ```shell + ./build_nyx_support.sh + ``` 5. Optionally, for binary-only fuzzing: set up the required 5.10 kernel, see - [KVM-Nyx](https://github.com/nyx-fuzz/KVM-Nyx). + [KVM-Nyx](https://github.com/nyx-fuzz/KVM-Nyx). ## Preparing to fuzz a target with Nyx mode @@ -52,35 +52,37 @@ option `instrumentation` below with `processor_trace`): ```shell python3 nyx_mode/packer/packer/nyx_packer.py \ - /PATH/TO/TARGET \ - PACKAGE-DIRECTORY \ - afl \ - instrumentation \ - --fast_reload_mode \ - --purge + /PATH/TO/TARGET \ + PACKAGE-DIRECTORY \ + afl \ + instrumentation \ + --fast_reload_mode \ + --purge ``` This will create a directory with all necessary files and the Nyx configuration. -The name of the directory will be whatever you choose for PACKAGE-DIRECTORY +The name of the directory will be whatever you choose for `PACKAGE-DIRECTORY` above. In the final step for the packaging we generate the Nyx configuration: + ```shell python3 nyx_mode/packer/packer/nyx_config_gen.py PACKAGE-DIRECTORY Kernel ``` ## Fuzzing with Nyx mode -All the hard parts are done, fuzzing with Nyx mode is easy - just supply -the PACKAGE-DIRECTORY as fuzzing target and specify the `-X` option to afl-fuzz: +All the hard parts are done, fuzzing with Nyx mode is easy - just supply the +`PACKAGE-DIRECTORY` as fuzzing target and specify the `-X` option to afl-fuzz: ```shell afl-fuzz -i in -o out -X -- ./PACKAGE-DIRECTORY ``` Most likely your first run will fail because the Linux modules have to be -specially set up, but afl-fuzz will tell you this on startup and how to -rectify the situation: +specially set up, but afl-fuzz will tell you this on startup and how to rectify +the situation: + ``` sudo modprobe -r kvm-intel # or kvm-amd for AMD processors sudo modprobe -r kvm @@ -91,10 +93,10 @@ sudo modprobe kvm-intel # or kvm-amd for AMD processors If you want to fuzz in parallel (and you should!), then this has to be done in a special way: - * Instead of `-X` (standalone mode) you specify `-Y` (multi processor mode). - * First a Main afl-fuzz instance has to be started with `-M 0` - * Only afterwards can you start Secondary afl-fuzz instances, which must have - an increasing number value, starting at 1, e.g. `-S 1` +* Instead of `-X` (standalone mode), you specify `-Y` (multi processor mode). +* First, a Main afl-fuzz instance has to be started with `-M 0`. +* Only afterwards you can start Secondary afl-fuzz instances, which must have an + increasing number value, starting at 1, e.g., `-S 1`. ```shell afl-fuzz -i in -o out -Y -M 0 -- ./PACKAGE-DIRECTORY @@ -112,11 +114,13 @@ afl-fuzz -i in -o out -Y -S 2 -- ./PACKAGE-DIRECTORY ### Fuzzing libxml2 with AFL++ in Nyx-mode -This tutorial is based on the [Fuzzing libxml2 with AFL++](https://aflplus.plus/docs/tutorials/libxml2_tutorial/) tutorial. +This tutorial is based on the +[Fuzzing libxml2 with AFL++](https://aflplus.plus/docs/tutorials/libxml2_tutorial/) +tutorial. -### Preparing libxml2 +### Preparing libxml2 -First, get the latest libxml2 source files by using `git`: +First, get the latest libxml2 source files by using `git`: ``` git clone https://gitlab.gnome.org/GNOME/libxml2 @@ -135,43 +139,65 @@ make CC=afl-clang-fast CXX=afl-clang-fast++ LD=afl-clang-fast Nyx expects that the target is provided in a certain format. More specifically, the target is passed as a so-called „share directory“ to a Nyx-frontend implementation. The share directory contains the target as well as a folder containing all dependencies and other files that are copied over to the guest. But more importantly, this share directory also contains a bootstrap script (`fuzz.sh`if you are using `KVM-Nyx`otherwise `fuzz_no_pt.sh`) that is also executed right after launching the fuzzer. Both bootstrap scripts use several tools to communicate with the "outer world": -- `hcat` - this tool copies a given string to the host -- `hget` - this program requests a file from the host's share directory -- `hget_bulk` - an improved version of `hget`. It is quite useful if you want to transfer huge files. But please keep in mind that this version of `hget` has a much larger startup overhead and won't improve your transfer rates on small files (typically files smaller than 100MB). -- `habort` - this tool basically sends an abort signal to the host (useful if something went wrong during bootstrap) -- `hpush` - a tool to transfer a given file to the host (the transfered file will be put in the `dump/` folder of your Nyx workdir) - -Those tools are all using hypercalls which are defined in `packer/nyx.h`. We will give some more examples later on how to use these hypercalls directly to implement custom fuzzing harnesses. +- `hcat` - this tool copies a given string to the host +- `hget` - this program requests a file from the host's share directory +- `hget_bulk` - an improved version of `hget`. It is quite useful if you want to + transfer huge files. But please keep in mind that this version of `hget` has a + much larger startup overhead and won't improve your transfer rates on small + files (typically files smaller than 100 MB). +- `habort` - this tool basically sends an abort signal to the host (useful if + something went wrong during bootstrap) +- `hpush` - a tool to transfer a given file to the host (the transferred file + will be put in the `dump/` folder of your Nyx workdir) + +Those tools are all using hypercalls which are defined in `packer/nyx.h`. We +will give some more examples later on how to use these hypercalls directly to +implement custom fuzzing harnesses. ### Pack libxml2 into Nyx sharedir format -To turn a given linux target into the Nyx format, you can simply use `nyx_packer.py`. To do so, move to the following directory: +To turn a given linux target into the Nyx format, you can simply use +`nyx_packer.py`. To do so, move to the following directory: ``` cd nyx_mode/packer/packer ``` - And run the tool with the following options to pack `libxml2`: +And run the tool with the following options to pack `libxml2`: -```. -python3 ./nyx_packer.py \ - ~/libxml2/xmllint \ - /tmp/nyx_libxml2 \ - afl \ - instrumentation \ - -args "/tmp/input" \ - -file "/tmp/input" \ - --fast_reload_mode \ - --purge ``` - -In this example, the packer will take `xmllint`, recursively get all dependencies and put both into the specified share directory (`/tmp/nyx_libxml2` in this case). Because we have selected the `afl` option, an `ld_preload`-based agent is also automatically built and put into the sharedir. Another option would be `spec`. Without going into too much detail here, the `spec`mode is only used by Nyx's [spec-fuzzer](https://github.com/nyx-fuzz/spec-fuzzer) implementation. Next, since our target is built with compile-time instrumentations, we must select the `instrumentation` option, otherwise we could also use `processor-trace` option to enable Intel-PT fuzzing on targets without instrumentation. - -To specify that the input generated by the fuzzer is passed as a seperate file to the target, we need to set the `-file` option. Otherwise, the input will be passed over to the target via `stdin`. To specify any required `argv` options you can use the `-args`parameter. - -In case you want to fuzz the target only with fast snapshots enabled, you can also set the `--fast_reload_mode` option to improve performance. - -Finally, we need to generate a Nyx configuration file. Simply run the following command and you're good to proceed: +python3 ./nyx_packer.py \ + ~/libxml2/xmllint \ + /tmp/nyx_libxml2 \ + afl \ + instrumentation \ + -args "/tmp/input" \ + -file "/tmp/input" \ + --fast_reload_mode \ + --purge +``` + +In this example, the packer will take `xmllint`, recursively get all +dependencies and put both into the specified share directory (`/tmp/nyx_libxml2` +in this case). Because we have selected the `afl` option, an `ld_preload`-based +agent is also automatically built and put into the sharedir. Another option +would be `spec`. Without going into too much detail here, the `spec` mode is +only used by Nyx's [spec-fuzzer](https://github.com/nyx-fuzz/spec-fuzzer) +implementation. Next, since our target is built with compile-time +instrumentations, we must select the `instrumentation` option, otherwise we +could also use `processor-trace` option to enable Intel-PT fuzzing on targets +without instrumentation. + +To specify that the input generated by the fuzzer is passed as a separate file +to the target, we need to set the `-file` option. Otherwise, the input will be +passed over to the target via `stdin`. To specify any required `argv` options, +you can use the `-args` parameter. + +In case you want to fuzz the target only with fast snapshots enabled, you can +also set the `--fast_reload_mode` option to improve performance. + +Finally, we need to generate a Nyx configuration file. Simply run the following +command and you're good to proceed: ``` python3 ./nyx_config_gen.py /tmp/nyx_libxml2/ Kernel @@ -179,15 +205,17 @@ python3 ./nyx_config_gen.py /tmp/nyx_libxml2/ Kernel ### Run Nyx mode -From here on, we are almost done. Move to the AFL++ top directory and start the fuzzer with the following arguments: +From here on, we are almost done. Move to the AFL++ top directory and start the +fuzzer with the following arguments: ```shell -mkdir /tmp/in/ # create an input folder -echo "AAAA" >> /tmp/in/A # create a dummy input file +mkdir /tmp/in/ # create an input folder +echo "AAAA" >> /tmp/in/A # create a dummy input file ./afl-fuzz -i /tmp/in/ -o /tmp/out -X /tmp/nyx_libxml2/ ``` -If everything has been successfully set up to this point, you will now be welcomed by the following AFL++ screen: +If everything has been successfully set up to this point, you will now be +welcomed by the following AFL++ screen: ``` american fuzzy lop ++3.15a {default} (/tmp/nyx_libxml2/) [fast] - NYX @@ -216,46 +244,64 @@ If everything has been successfully set up to this point, you will now be welcom └────────────────────────────────────────────────────┘ ``` -If you want to run the fuzzer in distributed mode, which might be especially useful if you want to keep your memory footprint low, we got you covered. To start an initiating `parent` process, which will also create the snapshot which is later shared across all other `child`s, simply run AFL++Nyx with the following arguments: +If you want to run the fuzzer in distributed mode, which might be especially +useful if you want to keep your memory footprint low, we got you covered. To +start an initiating `parent` process, which will also create the snapshot which +is later shared across all other `child`s, simply run AFL++Nyx with the +following arguments: ``` ./afl-fuzz -i /tmp/in/ -o /tmp/out -d -Y -M 0 /tmp/nyx_libxml2/ ``` -To attach other child processes adjust the `-S <id>` and run the following command: +To attach other child processes adjust the `-S <id>` and run the following +command: ``` ./afl-fuzz -i /tmp/in/ -o /tmp/out -d -Y -S 1 /tmp/nyx_libxml2/ ``` -If you want to disable fast snapshots (except for crashes), you can simply set the `NYX_DISABLE_SNAPSHOT_MODE` environment variable. +If you want to disable fast snapshots (except for crashes), you can simply set +the `NYX_DISABLE_SNAPSHOT_MODE` environment variable. ### Run AFL++Nyx with a custom agent -Most of the common use-cases for linux userland targets are already handled by our general purpose [agent](https://github.com/nyx-fuzz/packer/blob/main/packer/linux_x86_64-userspace/src/ld_preload_fuzz.c) implementation. But in case you want to build your own agent, or write a custom harness for a specific target or you just want to implement all the hypercall and shared memory communication on your own, you can use our custom harness example as a starting point for that. You can find the code [here](custom_harness/) +Most of the common use-cases for linux userland targets are already handled by +our general purpose +[agent](https://github.com/nyx-fuzz/packer/blob/main/packer/linux_x86_64-userspace/src/ld_preload_fuzz.c) +implementation. But in case you want to build your own agent, or write a custom +harness for a specific target or you just want to implement all the hypercall +and shared memory communication on your own, you can use our custom harness +example as a starting point for that. You can find the code in +[custom_harness/](./custom_harness/). -This custom harness can be statically compiled with by gcc or clang. There is no need to use an AFL compiler, because this agent implements its own very basic coverage tracking by simply setting specific bytes in the "coverage" bitmap after specific branches have been covered. +This custom harness can be statically compiled with by gcc or clang. There is no +need to use an AFL compiler, because this agent implements its own very basic +coverage tracking by simply setting specific bytes in the "coverage" bitmap +after specific branches have been covered. -To prepare this target, we must first create a new folder that will later become the sharedir. +To prepare this target, we must first create a new folder that will later become +the sharedir. ```` mkdir /tmp/nyx_custom_agent/ ```` - To compile this example, run the following command (remove the `-DNO_PT_NYX` option if you are using KVM-Nyx ): +To compile this example, run the following command (remove the `-DNO_PT_NYX` +option if you are using KVM-Nyx): ``` gcc example.c -DNO_PT_NYX -static -I ../packer/ -o /tmp/nyx_custom_agent/target ``` -Copy both bootstrap scripts into the sharedir: +Copy both bootstrap scripts into the sharedir: ``` cp fuzz.sh /tmp/nyx_custom_agent cp fuzz_no_pt.sh /tmp/nyx_custom_agent ``` -Copy all `htools` executables into the sharedir: +Copy all `htools` executables into the sharedir: ``` cd ~/AFLplusplus/packer/packer/linux_x86_64-userspace/ @@ -263,7 +309,7 @@ sh compile_64.sh cp bin64/h* /tmp/nyx_custom_agent/ ``` -And finally, generate a Nyx configuration: +And finally, generate a Nyx configuration: ``` cd ~/AFLplusplus/packer/packer diff --git a/qemu_mode/README.md b/qemu_mode/README.md index a045ef4f..3ebfc54c 100644 --- a/qemu_mode/README.md +++ b/qemu_mode/README.md @@ -135,7 +135,7 @@ Recommended, but not as good as CMPLOG mode (see below). ## 8) CMPLOG mode -Another new feature is CMPLOG, which is based on the redqueen project. Here all +Another new feature is CMPLOG, which is based on the Redqueen project. Here all immediates in CMP instructions are learned and put into a dynamic dictionary and applied to all locations in the input that reached that CMP, trying to solve and pass it. This is a very effective feature and it is available for x86, x86_64, diff --git a/src/README.md b/src/README.md index 35af6ab9..3f332280 100644 --- a/src/README.md +++ b/src/README.md @@ -18,7 +18,7 @@ Quick explanation about the files here: - `afl-fuzz-performance.c` - hash64 and rand functions - `afl-fuzz-python.c` - afl-fuzz the python mutator extension - `afl-fuzz-queue.c` - afl-fuzz handling the queue -- `afl-fuzz-redqueen.c` - afl-fuzz redqueen implemention +- `afl-fuzz-redqueen.c` - afl-fuzz redqueen implementation - `afl-fuzz-run.c` - afl-fuzz running the target - `afl-fuzz-state.c` - afl-fuzz state and globals - `afl-fuzz-stats.c` - afl-fuzz writing the statistics file |