From 185f4436598aacb3f25736ce6eb53309d1d9472f Mon Sep 17 00:00:00 2001 From: van Hauser Date: Fri, 31 Jul 2020 17:53:01 +0200 Subject: add LTO AFL_LLVM_DOCUMENT_IDS feature --- docs/FAQ.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index e09385a8..b09a16ae 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -95,12 +95,13 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! 2. Second step: Find the responsible function. - a) For LTO instrumented binaries just disassemble or decompile the target - and look which edge is writing to that edge ID. Ghidra is a good tool - for this: [https://ghidra-sre.org/](https://ghidra-sre.org/) + a) For LTO instrumented binaries this can be documented during compile + time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/afile`. + This file will have one assigned edge ID and the corresponding function + per line. - b) For PCGUARD instrumented binaries it is more difficult. Here you can - either modify the __sanitizer_cov_trace_pc_guard function in + b) For PCGUARD instrumented binaries it is much more difficult. Here you + can either modify the __sanitizer_cov_trace_pc_guard function in llvm_mode/afl-llvm-rt.o.c to write a backtrace to a file if the ID in __afl_area_ptr[*guard] is one of the unstable edge IDs. Then recompile and reinstall llvm_mode and rebuild your target. Run the recompiled @@ -121,4 +122,3 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! 4. Fourth step: recompile the target Recompile, fuzz it, be happy :) - -- cgit v1.2.3 From 593940c39a3838b072863b7093ed31f92846cfcb Mon Sep 17 00:00:00 2001 From: van Hauser Date: Fri, 31 Jul 2020 20:20:30 +0200 Subject: refer to llvm 12 for partial instrumentation for PCGUARD --- docs/FAQ.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index b09a16ae..ee221d02 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -117,7 +117,8 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! remove from instrumentation. Simply follow this document on how to do this: [llvm_mode/README.instrument_file.md](llvm_mode/README.instrument_file.md) - If PCGUARD is used, then you need to follow this guide: [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) + If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): + [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) 4. Fourth step: recompile the target -- cgit v1.2.3 From e0d1529061a5de9d32066c05f8faedac65b29ea5 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Mon, 3 Aug 2020 10:03:45 +0200 Subject: edge id documentation example for sancov --- docs/FAQ.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index ee221d02..c15cd484 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -103,10 +103,11 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! b) For PCGUARD instrumented binaries it is much more difficult. Here you can either modify the __sanitizer_cov_trace_pc_guard function in llvm_mode/afl-llvm-rt.o.c to write a backtrace to a file if the ID in - __afl_area_ptr[*guard] is one of the unstable edge IDs. Then recompile - and reinstall llvm_mode and rebuild your target. Run the recompiled - target with afl-fuzz for a while and then check the file that you - wrote with the backtrace information. + __afl_area_ptr[*guard] is one of the unstable edge IDs. + (Example code is already there). + Then recompile and reinstall llvm_mode and rebuild your target. + Run the recompiled target with afl-fuzz for a while and then check the + file that you wrote with the backtrace information. Alternatively you can use `gdb` to hook __sanitizer_cov_trace_pc_guard_init on start, check to which memory address the edge ID value is written and set a write breakpoint to that address (`watch 0x.....`). -- cgit v1.2.3 From 6d364dd2cb0ac31797b52e590b57bf9c10cc2302 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Wed, 5 Aug 2020 01:13:51 +0200 Subject: add sancov-like allow/denylist instrument feature --- docs/FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index c15cd484..33ce49e6 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -117,7 +117,7 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! Identify which source code files contain the functions that you need to remove from instrumentation. - Simply follow this document on how to do this: [llvm_mode/README.instrument_file.md](llvm_mode/README.instrument_file.md) + Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) -- cgit v1.2.3 From 3d8f05458041d6007617e383453c7ad66b345144 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Wed, 5 Aug 2020 01:19:31 +0200 Subject: readme fix --- docs/FAQ.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 33ce49e6..e690635a 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -112,10 +112,11 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! on start, check to which memory address the edge ID value is written and set a write breakpoint to that address (`watch 0x.....`). - 3. Third step: create a text file with the filenames + 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to - remove from instrumentation. + remove from instrumentation, or just specify the functions you want to + skip instrumenting. Note that optimization might inline functions! Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): -- cgit v1.2.3 From e94cc1fae0e78cd6c2e7e3cad737ad039148888f Mon Sep 17 00:00:00 2001 From: van Hauser Date: Fri, 14 Aug 2020 07:09:01 +0200 Subject: new faq entries --- docs/FAQ.md | 45 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index e690635a..997f4c40 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -3,9 +3,11 @@ ## Contents 1. [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - 2. [What is an edge?](#what-is-an-edge) - 3. [Why is my stability below 100%?](#why-is-my-stability-below-100) - 4. [How can I improve the stability value](#how-can-i-improve-the-stability-value) + 2. [How do I fuzz a network service?](#how-to-fuzz-a-network-service) + 3. [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) + 4. [What is an edge?](#what-is-an-edge) + 5. [Why is my stability below 100%?](#why-is-my-stability-below-100) + 6. [How can I improve the stability value](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) @@ -20,6 +22,43 @@ If you find an interesting or important question missing, submit it via 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) +## How do I fuzz a network service? + +The short answer is - you cannot, at least "out of the box". + +Using network has a slow-down of x10-20 on the fuzzing speed, does not scale, +and finally usually it is more than one initial data packet but a back-and-forth +which is totally unsupported by most coverage aware fuzzers. + +The established method to fuzz network services is to modify the source code +to read from a file or stdin (fd 0) (or even faster via shared memory, combine +this with persistent mode [llvm_mode/README.persistent_mode.md](llvm_mode/README.persistent_mode.md) +and you have a performance gain of x10 instead of a performance loss of over +x10 - that is a x100 difference! + +If modifying the source is not an option (e.g. because you only have a binary +and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD +to emulate the network. This is also much faster than network would be. +See [examples/socket_fuzzing/](../examples/socket_fuzzing/) + +There is an outdated afl++ branch that implements networking if you are +desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - +however a better option is AFLnet ([https://github.com/aflnet/aflnet](https://github.com/aflnet/aflnet)) +which allows you to define network state with different type of data packets. + +## How do I fuzz a GUI program? + +If the GUI program can read the fuzz data from a file (via the command line, +a fixed location or via an environment variable) without needing any user +interaction then then yes. + +Otherwise it is not possible without modifying the source code - which is a +very good idea anyway as the GUI functionality is a huge CPU/time overhead +for the fuzzing. + +So create a new `main()` that just reads the test case and calls the +functionality for processing the input that the GUI program is using. + ## What is an "edge" A program contains `functions`, `functions` contain the compiled machine code. -- cgit v1.2.3 From 1cf473848789b340134b68b02d793c6d90f746a8 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sat, 15 Aug 2020 10:27:40 +0200 Subject: more FAQ --- docs/FAQ.md | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 65 insertions(+), 13 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 997f4c40..93a87a72 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -2,16 +2,39 @@ ## Contents - 1. [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - 2. [How do I fuzz a network service?](#how-to-fuzz-a-network-service) - 3. [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) - 4. [What is an edge?](#what-is-an-edge) - 5. [Why is my stability below 100%?](#why-is-my-stability-below-100) - 6. [How can I improve the stability value](#how-can-i-improve-the-stability-value) + * [What is the difference between afl and afl++?](#what-is-the-difference-between-afl-and-afl) + * [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) + * [How do I fuzz a network service?](#how-to-fuzz-a-network-service) + * [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) + * [What is an edge?](#what-is-an-edge) + * [Why is my stability below 100%?](#why-is-my-stability-below-100) + * [How can I improve the stability value](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) +## What is the difference between afl and afl++? + +American Fuzzy Lop (AFL) was developed by Michał "lcamtuf" Zalewski starting in +2013/2014, and when he left Google end of 2017 he stopped developing it. + +At the end of 2019 the Google fuzzing team took over maintance of AFL, however +it is only accepting PR from the community and is not developing enhancements +anymore. + +In the second quarter of 2019, 1 1/2 years after no further development of +AFL had happened and it became clear there would be none coming, afl++ +was born, where initially first community patches were collected and applied +for bugs and enhancements. Then from various AFL spin-offs - mostly academic +research - features were integrated. This already resulted in a much advanced +AFL. + +Until the end of 2019 the afl++ team had grown to four active developers which +then implemented their own research and feature, making it now by far the most +flexible and feature rich guided fuzzer available as open source. +And in independent fuzzing benchmarks it is one of the best fuzzers available, +e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) + ## How to improve the fuzzing speed 1. use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) @@ -101,15 +124,15 @@ code example above): ``` Every line between two blocks is an `edge`. -## Why is my stability below 100 +## Why is my stability below 100% Stability is measured by how many percent of the edges in the target are "stable". Sending the same input again and again should take the exact same path through the target every time. If that is the case, the stability is 100%. -If however randomness happens, e.g. a thread reading from shared memory, +If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc. then in some of the re-executions with the same data -will result in the edge information being different accross runs. +the result in the edge information will be different accross runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for afl++ to identify valid new @@ -122,9 +145,25 @@ improve the stability. ## How can I improve the stability value +For fuzzing a 100% stable target that covers all edges is the best. +A 90% stable target that covers all edges is however better than a 100% stable +target that ignores 10% of the edges. + +With instability you basically have a partial coverage loss on an edge, with +ignore you have a full loss on that edge. + +There are functions that are unstable, but also provide value to coverage, eg +init functions that use fuzz data as input for example. +If however it is a function that has nothing to do with the input data is the +source, e.g. checking jitter, or is a hash map function etc. then it should +not be instrumented. + +To be able to make this decision the following process will allow you to +identify the functions with variable edges so you can make this decision. + Four steps are required to do this and requires quite some knowledge of coding and/or disassembly and it is only effectively possible with -afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! +afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. 1. First step: Identify which edge ID numbers are unstable @@ -135,9 +174,9 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! 2. Second step: Find the responsible function. a) For LTO instrumented binaries this can be documented during compile - time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/afile`. - This file will have one assigned edge ID and the corresponding function - per line. + time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. + This file will have one assigned edge ID and the corresponding + function per line. b) For PCGUARD instrumented binaries it is much more difficult. Here you can either modify the __sanitizer_cov_trace_pc_guard function in @@ -151,6 +190,10 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! on start, check to which memory address the edge ID value is written and set a write breakpoint to that address (`watch 0x.....`). + c) in all other instrumentation types this is not possible. So just + recompile with the the two mentioned above. This is just for + identifying the functions that have unstable edges. + 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to @@ -161,6 +204,15 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation! If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) + Only deny those functions from instrumentation that provide no value + for coverage - that is if it does not process any fuzz data directly + or indirectly (e.g. hash maps, thread management etc.). + If however a function directly or indirectly handles fuzz data then you + should not put the function in a deny instrumentation list and rather + live with the instability it comes with. + 4. Fourth step: recompile the target Recompile, fuzz it, be happy :) + + This link explains this process for [Fuzzbench](https://github.com/google/fuzzbench/issues/677) -- cgit v1.2.3 From ed6243df5a284c0e915d577ef45e80218e07907c Mon Sep 17 00:00:00 2001 From: hexcoder Date: Tue, 1 Sep 2020 00:49:26 +0200 Subject: Review FAQ.md --- docs/FAQ.md | 95 +++++++++++++++++++++++++++++++------------------------------ 1 file changed, 49 insertions(+), 46 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 93a87a72..df5cc79c 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -4,11 +4,11 @@ * [What is the difference between afl and afl++?](#what-is-the-difference-between-afl-and-afl) * [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - * [How do I fuzz a network service?](#how-to-fuzz-a-network-service) - * [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) + * [How do I fuzz a network service?](#how-do-i-fuzz-a-network-service) + * [How do I fuzz a GUI program?](#how-do-i-fuzz-a-gui-program) * [What is an edge?](#what-is-an-edge) * [Why is my stability below 100%?](#why-is-my-stability-below-100) - * [How can I improve the stability value](#how-can-i-improve-the-stability-value) + * [How can I improve the stability value?](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) @@ -18,51 +18,54 @@ If you find an interesting or important question missing, submit it via American Fuzzy Lop (AFL) was developed by Michał "lcamtuf" Zalewski starting in 2013/2014, and when he left Google end of 2017 he stopped developing it. -At the end of 2019 the Google fuzzing team took over maintance of AFL, however -it is only accepting PR from the community and is not developing enhancements +At the end of 2019 the Google fuzzing team took over maintenance of AFL, however +it is only accepting PRs from the community and is not developing enhancements anymore. -In the second quarter of 2019, 1 1/2 years after no further development of -AFL had happened and it became clear there would be none coming, afl++ -was born, where initially first community patches were collected and applied -for bugs and enhancements. Then from various AFL spin-offs - mostly academic +In the second quarter of 2019, 1 1/2 year later when no further development of +AFL had happened and it became clear there would none be coming, afl++ +was born, where initially community patches were collected and applied +for bug fixes and enhancements. Then from various AFL spin-offs - mostly academic research - features were integrated. This already resulted in a much advanced AFL. Until the end of 2019 the afl++ team had grown to four active developers which -then implemented their own research and feature, making it now by far the most +then implemented their own research and features, making it now by far the most flexible and feature rich guided fuzzer available as open source. And in independent fuzzing benchmarks it is one of the best fuzzers available, e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) -## How to improve the fuzzing speed +## How to improve the fuzzing speed? - 1. use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) + 1. Use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) 2. Use [persistent mode](llvm_mode/README.persistent_mode.md) (x2-x20 speed increase) 3. Use the [afl++ snapshot module](https://github.com/AFLplusplus/AFL-Snapshot-LKM) (x2 speed increase) - 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to point the input file on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) - 5. Improve kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) + 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input file directory on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) + 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) ## How do I fuzz a network service? -The short answer is - you cannot, at least "out of the box". +The short answer is - you cannot, at least not "out of the box". -Using network has a slow-down of x10-20 on the fuzzing speed, does not scale, -and finally usually it is more than one initial data packet but a back-and-forth -which is totally unsupported by most coverage aware fuzzers. +Using a network channel is inadequate for several reasons: +- it has a slow-down of x10-20 on the fuzzing speed +- it does not scale to multiple connections, +- instead of one initial data packet often a back-and-forth +interplay of packets is needed for stateful protocols +(which is totally unsupported by most coverage aware fuzzers). The established method to fuzz network services is to modify the source code to read from a file or stdin (fd 0) (or even faster via shared memory, combine this with persistent mode [llvm_mode/README.persistent_mode.md](llvm_mode/README.persistent_mode.md) and you have a performance gain of x10 instead of a performance loss of over -x10 - that is a x100 difference! +x10 - that is a x100 difference!). If modifying the source is not an option (e.g. because you only have a binary and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD -to emulate the network. This is also much faster than network would be. -See [examples/socket_fuzzing/](../examples/socket_fuzzing/) +to emulate the network. This is also much faster than the real network would be. +See [examples/socket_fuzzing/](../examples/socket_fuzzing/). There is an outdated afl++ branch that implements networking if you are desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - @@ -73,7 +76,7 @@ which allows you to define network state with different type of data packets. If the GUI program can read the fuzz data from a file (via the command line, a fixed location or via an environment variable) without needing any user -interaction then then yes. +interaction then it would be suitable for fuzzing. Otherwise it is not possible without modifying the source code - which is a very good idea anyway as the GUI functionality is a huge CPU/time overhead @@ -82,13 +85,13 @@ for the fuzzing. So create a new `main()` that just reads the test case and calls the functionality for processing the input that the GUI program is using. -## What is an "edge" +## What is an "edge"? A program contains `functions`, `functions` contain the compiled machine code. The compiled machine code in a `function` can be in a single or many `basic blocks`. A `basic block` is the largest possible number of subsequent machine code -instructions that runs independent, meaning it does not split up to different -locations nor is it jumped into it from a different location: +instructions that has exactly one entry (at the beginning) and runs linearly without +branching or jumping to other addresses (except at the end). ``` function() { A: @@ -98,7 +101,7 @@ function() { if (x) goto C; else goto D; C: some code - goto D + goto E D: some code goto B @@ -108,7 +111,7 @@ function() { ``` Every code block between two jump locations is a `basic block`. -An `edge` is then the unique relationship between two `basic blocks` (from the +An `edge` is then the unique relationship between two directly connected `basic blocks` (from the code example above): ``` Block A @@ -124,7 +127,7 @@ code example above): ``` Every line between two blocks is an `edge`. -## Why is my stability below 100% +## Why is my stability below 100%? Stability is measured by how many percent of the edges in the target are "stable". Sending the same input again and again should take the exact same @@ -132,37 +135,37 @@ path through the target every time. If that is the case, the stability is 100%. If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc. then in some of the re-executions with the same data -the result in the edge information will be different accross runs. +the edge coverage result will be different accross runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for afl++ to identify valid new paths. A value above 90% is usually fine and a value above 80% is also still ok, and -even above 20% can still result in successful finds of bugs. -However, it is recommended that below 90% or 80% you should take measures to -improve the stability. +even a value above 20% can still result in successful finds of bugs. +However, it is recommended that for values below 90% or 80% you should take +countermeasures to improve stability. -## How can I improve the stability value +## How can I improve the stability value? -For fuzzing a 100% stable target that covers all edges is the best. +For fuzzing a 100% stable target that covers all edges is the best case. A 90% stable target that covers all edges is however better than a 100% stable target that ignores 10% of the edges. With instability you basically have a partial coverage loss on an edge, with -ignore you have a full loss on that edge. +ignored functions you have a full loss on that edges. There are functions that are unstable, but also provide value to coverage, eg init functions that use fuzz data as input for example. -If however it is a function that has nothing to do with the input data is the -source, e.g. checking jitter, or is a hash map function etc. then it should -not be instrumented. +If however a function that has nothing to do with the input data is the +source of instability, e.g. checking jitter, or is a hash map function etc. +then it should not be instrumented. -To be able to make this decision the following process will allow you to -identify the functions with variable edges so you can make this decision. +To be able to exclude these functions (based on AFL++'s measured stability) +the following process will allow to identify functions with variable edges. -Four steps are required to do this and requires quite some knowledge of -coding and/or disassembly and it is only effectively possible with +Four steps are required to do this and it also requires quite some knowledge +of coding and/or disassembly and is effectively possible only with afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. 1. First step: Identify which edge ID numbers are unstable @@ -171,7 +174,7 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. The out/fuzzer_stats file will then show the edge IDs that were identified as unstable. - 2. Second step: Find the responsible function. + 2. Second step: Find the responsible function(s). a) For LTO instrumented binaries this can be documented during compile time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. @@ -191,20 +194,20 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. and set a write breakpoint to that address (`watch 0x.....`). c) in all other instrumentation types this is not possible. So just - recompile with the the two mentioned above. This is just for + recompile with the two mentioned above. This is just for identifying the functions that have unstable edges. 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to remove from instrumentation, or just specify the functions you want to - skip instrumenting. Note that optimization might inline functions! + skip for instrumentation. Note that optimization might inline functions! Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) - Only deny those functions from instrumentation that provide no value + Only exclude those functions from instrumentation that provide no value for coverage - that is if it does not process any fuzz data directly or indirectly (e.g. hash maps, thread management etc.). If however a function directly or indirectly handles fuzz data then you -- cgit v1.2.3 From 020b8a49648ec5f58f44dc104f3a3c83624a41d1 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Wed, 2 Sep 2020 20:19:49 +0200 Subject: minor FAQ fixes --- docs/FAQ.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index df5cc79c..064638f4 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -41,7 +41,7 @@ e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) 2. Use [persistent mode](llvm_mode/README.persistent_mode.md) (x2-x20 speed increase) 3. Use the [afl++ snapshot module](https://github.com/AFLplusplus/AFL-Snapshot-LKM) (x2 speed increase) 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input file directory on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) - 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) + 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system less secure) 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) @@ -51,10 +51,8 @@ The short answer is - you cannot, at least not "out of the box". Using a network channel is inadequate for several reasons: - it has a slow-down of x10-20 on the fuzzing speed -- it does not scale to multiple connections, -- instead of one initial data packet often a back-and-forth -interplay of packets is needed for stateful protocols -(which is totally unsupported by most coverage aware fuzzers). +- it does not scale to fuzzing multiple instances easily, +- instead of one initial data packet often a back-and-forth interplay of packets is needed for stateful protocols (which is totally unsupported by most coverage aware fuzzers). The established method to fuzz network services is to modify the source code to read from a file or stdin (fd 0) (or even faster via shared memory, combine @@ -90,8 +88,8 @@ functionality for processing the input that the GUI program is using. A program contains `functions`, `functions` contain the compiled machine code. The compiled machine code in a `function` can be in a single or many `basic blocks`. A `basic block` is the largest possible number of subsequent machine code -instructions that has exactly one entry (at the beginning) and runs linearly without -branching or jumping to other addresses (except at the end). +instructions that has exactly one entrypoint (which can be be entered by multiple other basic blocks) +and runs linearly without branching or jumping to other addresses (except at the end). ``` function() { A: @@ -126,6 +124,7 @@ code example above): Block E ``` Every line between two blocks is an `edge`. +Note that a few basic block loop to itself, this too would be an edge. ## Why is my stability below 100%? -- cgit v1.2.3 From e30b2c6af6e369844c92c00a20ebdd53473a747c Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 5 Sep 2020 13:18:28 +0200 Subject: final changes for pre-3.0 --- docs/FAQ.md | 104 +++++++++++++++++++++++++++++------------------------------- 1 file changed, 51 insertions(+), 53 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 064638f4..24942492 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -4,11 +4,11 @@ * [What is the difference between afl and afl++?](#what-is-the-difference-between-afl-and-afl) * [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - * [How do I fuzz a network service?](#how-do-i-fuzz-a-network-service) - * [How do I fuzz a GUI program?](#how-do-i-fuzz-a-gui-program) + * [How do I fuzz a network service?](#how-to-fuzz-a-network-service) + * [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) * [What is an edge?](#what-is-an-edge) * [Why is my stability below 100%?](#why-is-my-stability-below-100) - * [How can I improve the stability value?](#how-can-i-improve-the-stability-value) + * [How can I improve the stability value](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) @@ -18,52 +18,51 @@ If you find an interesting or important question missing, submit it via American Fuzzy Lop (AFL) was developed by Michał "lcamtuf" Zalewski starting in 2013/2014, and when he left Google end of 2017 he stopped developing it. -At the end of 2019 the Google fuzzing team took over maintenance of AFL, however -it is only accepting PRs from the community and is not developing enhancements +At the end of 2019 the Google fuzzing team took over maintance of AFL, however +it is only accepting PR from the community and is not developing enhancements anymore. -In the second quarter of 2019, 1 1/2 year later when no further development of -AFL had happened and it became clear there would none be coming, afl++ -was born, where initially community patches were collected and applied -for bug fixes and enhancements. Then from various AFL spin-offs - mostly academic +In the second quarter of 2019, 1 1/2 years after no further development of +AFL had happened and it became clear there would be none coming, afl++ +was born, where initially first community patches were collected and applied +for bugs and enhancements. Then from various AFL spin-offs - mostly academic research - features were integrated. This already resulted in a much advanced AFL. Until the end of 2019 the afl++ team had grown to four active developers which -then implemented their own research and features, making it now by far the most +then implemented their own research and feature, making it now by far the most flexible and feature rich guided fuzzer available as open source. And in independent fuzzing benchmarks it is one of the best fuzzers available, e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) -## How to improve the fuzzing speed? +## How to improve the fuzzing speed - 1. Use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) - 2. Use [persistent mode](llvm_mode/README.persistent_mode.md) (x2-x20 speed increase) + 1. use [instrumentation](docs/README.llvm.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) + 2. Use [persistent mode](instrumentation/README.persistent_mode.md) (x2-x20 speed increase) 3. Use the [afl++ snapshot module](https://github.com/AFLplusplus/AFL-Snapshot-LKM) (x2 speed increase) - 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input file directory on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) - 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system less secure) + 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to point the input file on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) + 5. Improve kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) ## How do I fuzz a network service? -The short answer is - you cannot, at least not "out of the box". +The short answer is - you cannot, at least "out of the box". -Using a network channel is inadequate for several reasons: -- it has a slow-down of x10-20 on the fuzzing speed -- it does not scale to fuzzing multiple instances easily, -- instead of one initial data packet often a back-and-forth interplay of packets is needed for stateful protocols (which is totally unsupported by most coverage aware fuzzers). +Using network has a slow-down of x10-20 on the fuzzing speed, does not scale, +and finally usually it is more than one initial data packet but a back-and-forth +which is totally unsupported by most coverage aware fuzzers. The established method to fuzz network services is to modify the source code to read from a file or stdin (fd 0) (or even faster via shared memory, combine -this with persistent mode [llvm_mode/README.persistent_mode.md](llvm_mode/README.persistent_mode.md) +this with persistent mode [instrumentation/README.persistent_mode.md](instrumentation/README.persistent_mode.md) and you have a performance gain of x10 instead of a performance loss of over -x10 - that is a x100 difference!). +x10 - that is a x100 difference! If modifying the source is not an option (e.g. because you only have a binary and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD -to emulate the network. This is also much faster than the real network would be. -See [examples/socket_fuzzing/](../examples/socket_fuzzing/). +to emulate the network. This is also much faster than network would be. +See [examples/socket_fuzzing/](../examples/socket_fuzzing/) There is an outdated afl++ branch that implements networking if you are desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - @@ -74,7 +73,7 @@ which allows you to define network state with different type of data packets. If the GUI program can read the fuzz data from a file (via the command line, a fixed location or via an environment variable) without needing any user -interaction then it would be suitable for fuzzing. +interaction then then yes. Otherwise it is not possible without modifying the source code - which is a very good idea anyway as the GUI functionality is a huge CPU/time overhead @@ -83,13 +82,13 @@ for the fuzzing. So create a new `main()` that just reads the test case and calls the functionality for processing the input that the GUI program is using. -## What is an "edge"? +## What is an "edge" A program contains `functions`, `functions` contain the compiled machine code. The compiled machine code in a `function` can be in a single or many `basic blocks`. A `basic block` is the largest possible number of subsequent machine code -instructions that has exactly one entrypoint (which can be be entered by multiple other basic blocks) -and runs linearly without branching or jumping to other addresses (except at the end). +instructions that runs independent, meaning it does not split up to different +locations nor is it jumped into it from a different location: ``` function() { A: @@ -99,7 +98,7 @@ function() { if (x) goto C; else goto D; C: some code - goto E + goto D D: some code goto B @@ -109,7 +108,7 @@ function() { ``` Every code block between two jump locations is a `basic block`. -An `edge` is then the unique relationship between two directly connected `basic blocks` (from the +An `edge` is then the unique relationship between two `basic blocks` (from the code example above): ``` Block A @@ -124,9 +123,8 @@ code example above): Block E ``` Every line between two blocks is an `edge`. -Note that a few basic block loop to itself, this too would be an edge. -## Why is my stability below 100%? +## Why is my stability below 100% Stability is measured by how many percent of the edges in the target are "stable". Sending the same input again and again should take the exact same @@ -134,37 +132,37 @@ path through the target every time. If that is the case, the stability is 100%. If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc. then in some of the re-executions with the same data -the edge coverage result will be different accross runs. +the result in the edge information will be different accross runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for afl++ to identify valid new paths. A value above 90% is usually fine and a value above 80% is also still ok, and -even a value above 20% can still result in successful finds of bugs. -However, it is recommended that for values below 90% or 80% you should take -countermeasures to improve stability. +even above 20% can still result in successful finds of bugs. +However, it is recommended that below 90% or 80% you should take measures to +improve the stability. -## How can I improve the stability value? +## How can I improve the stability value -For fuzzing a 100% stable target that covers all edges is the best case. +For fuzzing a 100% stable target that covers all edges is the best. A 90% stable target that covers all edges is however better than a 100% stable target that ignores 10% of the edges. With instability you basically have a partial coverage loss on an edge, with -ignored functions you have a full loss on that edges. +ignore you have a full loss on that edge. There are functions that are unstable, but also provide value to coverage, eg init functions that use fuzz data as input for example. -If however a function that has nothing to do with the input data is the -source of instability, e.g. checking jitter, or is a hash map function etc. -then it should not be instrumented. +If however it is a function that has nothing to do with the input data is the +source, e.g. checking jitter, or is a hash map function etc. then it should +not be instrumented. -To be able to exclude these functions (based on AFL++'s measured stability) -the following process will allow to identify functions with variable edges. +To be able to make this decision the following process will allow you to +identify the functions with variable edges so you can make this decision. -Four steps are required to do this and it also requires quite some knowledge -of coding and/or disassembly and is effectively possible only with +Four steps are required to do this and requires quite some knowledge of +coding and/or disassembly and it is only effectively possible with afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. 1. First step: Identify which edge ID numbers are unstable @@ -173,7 +171,7 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. The out/fuzzer_stats file will then show the edge IDs that were identified as unstable. - 2. Second step: Find the responsible function(s). + 2. Second step: Find the responsible function. a) For LTO instrumented binaries this can be documented during compile time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. @@ -182,10 +180,10 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. b) For PCGUARD instrumented binaries it is much more difficult. Here you can either modify the __sanitizer_cov_trace_pc_guard function in - llvm_mode/afl-llvm-rt.o.c to write a backtrace to a file if the ID in + instrumentation/afl-llvm-rt.o.c to write a backtrace to a file if the ID in __afl_area_ptr[*guard] is one of the unstable edge IDs. (Example code is already there). - Then recompile and reinstall llvm_mode and rebuild your target. + Then recompile and reinstall instrumentation and rebuild your target. Run the recompiled target with afl-fuzz for a while and then check the file that you wrote with the backtrace information. Alternatively you can use `gdb` to hook __sanitizer_cov_trace_pc_guard_init @@ -193,20 +191,20 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. and set a write breakpoint to that address (`watch 0x.....`). c) in all other instrumentation types this is not possible. So just - recompile with the two mentioned above. This is just for + recompile with the the two mentioned above. This is just for identifying the functions that have unstable edges. 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to remove from instrumentation, or just specify the functions you want to - skip for instrumentation. Note that optimization might inline functions! + skip instrumenting. Note that optimization might inline functions! - Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) + Simply follow this document on how to do this: [instrumentation/README.instrument_list.md](instrumentation/README.instrument_list.md) If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) - Only exclude those functions from instrumentation that provide no value + Only deny those functions from instrumentation that provide no value for coverage - that is if it does not process any fuzz data directly or indirectly (e.g. hash maps, thread management etc.). If however a function directly or indirectly handles fuzz data then you -- cgit v1.2.3 From efd27bd8cec501798b719d56a4bc516f4703a787 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Fri, 30 Oct 2020 14:01:59 +0100 Subject: recover heikos faq changes --- docs/FAQ.md | 104 +++++++++++++++++++++++++++++++----------------------------- 1 file changed, 53 insertions(+), 51 deletions(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 24942492..064638f4 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -4,11 +4,11 @@ * [What is the difference between afl and afl++?](#what-is-the-difference-between-afl-and-afl) * [How to improve the fuzzing speed?](#how-to-improve-the-fuzzing-speed) - * [How do I fuzz a network service?](#how-to-fuzz-a-network-service) - * [How do I fuzz a GUI program?](#how-to-fuzz-a-gui-program) + * [How do I fuzz a network service?](#how-do-i-fuzz-a-network-service) + * [How do I fuzz a GUI program?](#how-do-i-fuzz-a-gui-program) * [What is an edge?](#what-is-an-edge) * [Why is my stability below 100%?](#why-is-my-stability-below-100) - * [How can I improve the stability value](#how-can-i-improve-the-stability-value) + * [How can I improve the stability value?](#how-can-i-improve-the-stability-value) If you find an interesting or important question missing, submit it via [https://github.com/AFLplusplus/AFLplusplus/issues](https://github.com/AFLplusplus/AFLplusplus/issues) @@ -18,51 +18,52 @@ If you find an interesting or important question missing, submit it via American Fuzzy Lop (AFL) was developed by Michał "lcamtuf" Zalewski starting in 2013/2014, and when he left Google end of 2017 he stopped developing it. -At the end of 2019 the Google fuzzing team took over maintance of AFL, however -it is only accepting PR from the community and is not developing enhancements +At the end of 2019 the Google fuzzing team took over maintenance of AFL, however +it is only accepting PRs from the community and is not developing enhancements anymore. -In the second quarter of 2019, 1 1/2 years after no further development of -AFL had happened and it became clear there would be none coming, afl++ -was born, where initially first community patches were collected and applied -for bugs and enhancements. Then from various AFL spin-offs - mostly academic +In the second quarter of 2019, 1 1/2 year later when no further development of +AFL had happened and it became clear there would none be coming, afl++ +was born, where initially community patches were collected and applied +for bug fixes and enhancements. Then from various AFL spin-offs - mostly academic research - features were integrated. This already resulted in a much advanced AFL. Until the end of 2019 the afl++ team had grown to four active developers which -then implemented their own research and feature, making it now by far the most +then implemented their own research and features, making it now by far the most flexible and feature rich guided fuzzer available as open source. And in independent fuzzing benchmarks it is one of the best fuzzers available, e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html) -## How to improve the fuzzing speed +## How to improve the fuzzing speed? - 1. use [instrumentation](docs/README.llvm.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) - 2. Use [persistent mode](instrumentation/README.persistent_mode.md) (x2-x20 speed increase) + 1. Use [llvm_mode](docs/llvm_mode/README.md): afl-clang-lto (llvm >= 11) or afl-clang-fast (llvm >= 9 recommended) + 2. Use [persistent mode](llvm_mode/README.persistent_mode.md) (x2-x20 speed increase) 3. Use the [afl++ snapshot module](https://github.com/AFLplusplus/AFL-Snapshot-LKM) (x2 speed increase) - 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to point the input file on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) - 5. Improve kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system more insecure) + 4. If you do not use shmem persistent mode, use `AFL_TMPDIR` to put the input file directory on a tempfs location, see [docs/env_variables.md](docs/env_variables.md) + 5. Improve Linux kernel performance: modify `/etc/default/grub`, set `GRUB_CMDLINE_LINUX_DEFAULT="ibpb=off ibrs=off kpti=off l1tf=off mds=off mitigations=off no_stf_barrier noibpb noibrs nopcid nopti nospec_store_bypass_disable nospectre_v1 nospectre_v2 pcid=off pti=off spec_store_bypass_disable=off spectre_v2=off stf_barrier=off"`; then `update-grub` and `reboot` (warning: makes the system less secure) 6. Running on an `ext2` filesystem with `noatime` mount option will be a bit faster than on any other journaling filesystem 7. Use your cores! [README.md:3.b) Using multiple cores/threads](../README.md#b-using-multiple-coresthreads) ## How do I fuzz a network service? -The short answer is - you cannot, at least "out of the box". +The short answer is - you cannot, at least not "out of the box". -Using network has a slow-down of x10-20 on the fuzzing speed, does not scale, -and finally usually it is more than one initial data packet but a back-and-forth -which is totally unsupported by most coverage aware fuzzers. +Using a network channel is inadequate for several reasons: +- it has a slow-down of x10-20 on the fuzzing speed +- it does not scale to fuzzing multiple instances easily, +- instead of one initial data packet often a back-and-forth interplay of packets is needed for stateful protocols (which is totally unsupported by most coverage aware fuzzers). The established method to fuzz network services is to modify the source code to read from a file or stdin (fd 0) (or even faster via shared memory, combine -this with persistent mode [instrumentation/README.persistent_mode.md](instrumentation/README.persistent_mode.md) +this with persistent mode [llvm_mode/README.persistent_mode.md](llvm_mode/README.persistent_mode.md) and you have a performance gain of x10 instead of a performance loss of over -x10 - that is a x100 difference! +x10 - that is a x100 difference!). If modifying the source is not an option (e.g. because you only have a binary and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD -to emulate the network. This is also much faster than network would be. -See [examples/socket_fuzzing/](../examples/socket_fuzzing/) +to emulate the network. This is also much faster than the real network would be. +See [examples/socket_fuzzing/](../examples/socket_fuzzing/). There is an outdated afl++ branch that implements networking if you are desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - @@ -73,7 +74,7 @@ which allows you to define network state with different type of data packets. If the GUI program can read the fuzz data from a file (via the command line, a fixed location or via an environment variable) without needing any user -interaction then then yes. +interaction then it would be suitable for fuzzing. Otherwise it is not possible without modifying the source code - which is a very good idea anyway as the GUI functionality is a huge CPU/time overhead @@ -82,13 +83,13 @@ for the fuzzing. So create a new `main()` that just reads the test case and calls the functionality for processing the input that the GUI program is using. -## What is an "edge" +## What is an "edge"? A program contains `functions`, `functions` contain the compiled machine code. The compiled machine code in a `function` can be in a single or many `basic blocks`. A `basic block` is the largest possible number of subsequent machine code -instructions that runs independent, meaning it does not split up to different -locations nor is it jumped into it from a different location: +instructions that has exactly one entrypoint (which can be be entered by multiple other basic blocks) +and runs linearly without branching or jumping to other addresses (except at the end). ``` function() { A: @@ -98,7 +99,7 @@ function() { if (x) goto C; else goto D; C: some code - goto D + goto E D: some code goto B @@ -108,7 +109,7 @@ function() { ``` Every code block between two jump locations is a `basic block`. -An `edge` is then the unique relationship between two `basic blocks` (from the +An `edge` is then the unique relationship between two directly connected `basic blocks` (from the code example above): ``` Block A @@ -123,8 +124,9 @@ code example above): Block E ``` Every line between two blocks is an `edge`. +Note that a few basic block loop to itself, this too would be an edge. -## Why is my stability below 100% +## Why is my stability below 100%? Stability is measured by how many percent of the edges in the target are "stable". Sending the same input again and again should take the exact same @@ -132,37 +134,37 @@ path through the target every time. If that is the case, the stability is 100%. If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc. then in some of the re-executions with the same data -the result in the edge information will be different accross runs. +the edge coverage result will be different accross runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for afl++ to identify valid new paths. A value above 90% is usually fine and a value above 80% is also still ok, and -even above 20% can still result in successful finds of bugs. -However, it is recommended that below 90% or 80% you should take measures to -improve the stability. +even a value above 20% can still result in successful finds of bugs. +However, it is recommended that for values below 90% or 80% you should take +countermeasures to improve stability. -## How can I improve the stability value +## How can I improve the stability value? -For fuzzing a 100% stable target that covers all edges is the best. +For fuzzing a 100% stable target that covers all edges is the best case. A 90% stable target that covers all edges is however better than a 100% stable target that ignores 10% of the edges. With instability you basically have a partial coverage loss on an edge, with -ignore you have a full loss on that edge. +ignored functions you have a full loss on that edges. There are functions that are unstable, but also provide value to coverage, eg init functions that use fuzz data as input for example. -If however it is a function that has nothing to do with the input data is the -source, e.g. checking jitter, or is a hash map function etc. then it should -not be instrumented. +If however a function that has nothing to do with the input data is the +source of instability, e.g. checking jitter, or is a hash map function etc. +then it should not be instrumented. -To be able to make this decision the following process will allow you to -identify the functions with variable edges so you can make this decision. +To be able to exclude these functions (based on AFL++'s measured stability) +the following process will allow to identify functions with variable edges. -Four steps are required to do this and requires quite some knowledge of -coding and/or disassembly and it is only effectively possible with +Four steps are required to do this and it also requires quite some knowledge +of coding and/or disassembly and is effectively possible only with afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. 1. First step: Identify which edge ID numbers are unstable @@ -171,7 +173,7 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. The out/fuzzer_stats file will then show the edge IDs that were identified as unstable. - 2. Second step: Find the responsible function. + 2. Second step: Find the responsible function(s). a) For LTO instrumented binaries this can be documented during compile time, just set `export AFL_LLVM_DOCUMENT_IDS=/path/to/a/file`. @@ -180,10 +182,10 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. b) For PCGUARD instrumented binaries it is much more difficult. Here you can either modify the __sanitizer_cov_trace_pc_guard function in - instrumentation/afl-llvm-rt.o.c to write a backtrace to a file if the ID in + llvm_mode/afl-llvm-rt.o.c to write a backtrace to a file if the ID in __afl_area_ptr[*guard] is one of the unstable edge IDs. (Example code is already there). - Then recompile and reinstall instrumentation and rebuild your target. + Then recompile and reinstall llvm_mode and rebuild your target. Run the recompiled target with afl-fuzz for a while and then check the file that you wrote with the backtrace information. Alternatively you can use `gdb` to hook __sanitizer_cov_trace_pc_guard_init @@ -191,20 +193,20 @@ afl-clang-fast PCGUARD and afl-clang-lto LTO instrumentation. and set a write breakpoint to that address (`watch 0x.....`). c) in all other instrumentation types this is not possible. So just - recompile with the the two mentioned above. This is just for + recompile with the two mentioned above. This is just for identifying the functions that have unstable edges. 3. Third step: create a text file with the filenames/functions Identify which source code files contain the functions that you need to remove from instrumentation, or just specify the functions you want to - skip instrumenting. Note that optimization might inline functions! + skip for instrumentation. Note that optimization might inline functions! - Simply follow this document on how to do this: [instrumentation/README.instrument_list.md](instrumentation/README.instrument_list.md) + Simply follow this document on how to do this: [llvm_mode/README.instrument_list.md](llvm_mode/README.instrument_list.md) If PCGUARD is used, then you need to follow this guide (needs llvm 12+!): [http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](http://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) - Only deny those functions from instrumentation that provide no value + Only exclude those functions from instrumentation that provide no value for coverage - that is if it does not process any fuzz data directly or indirectly (e.g. hash maps, thread management etc.). If however a function directly or indirectly handles fuzz data then you -- cgit v1.2.3 From c05e4efbe9b4e7d1ff078b7a392621f2ca7572e6 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Tue, 1 Dec 2020 14:40:30 +0100 Subject: renamed examples/ to utils/ --- docs/FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/FAQ.md') diff --git a/docs/FAQ.md b/docs/FAQ.md index 064638f4..714d50eb 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -63,7 +63,7 @@ x10 - that is a x100 difference!). If modifying the source is not an option (e.g. because you only have a binary and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD to emulate the network. This is also much faster than the real network would be. -See [examples/socket_fuzzing/](../examples/socket_fuzzing/). +See [utils/socket_fuzzing/](../utils/socket_fuzzing/). There is an outdated afl++ branch that implements networking if you are desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - -- cgit v1.2.3