diff options
Diffstat (limited to 'docs')
-rw-r--r-- | docs/QuickStartGuide.md | 2 | ||||
-rw-r--r-- | docs/life_pro_tips.txt | 2 | ||||
-rw-r--r-- | docs/parallel_fuzzing.md (renamed from docs/parallel_fuzzing.txt) | 31 | ||||
-rw-r--r-- | docs/perf_tips.txt | 2 | ||||
-rw-r--r-- | docs/status_screen.txt | 4 | ||||
-rw-r--r-- | docs/technical_details.txt | 2 |
6 files changed, 24 insertions, 19 deletions
diff --git a/docs/QuickStartGuide.md b/docs/QuickStartGuide.md index 1e89a6ad..d5ad303e 100644 --- a/docs/QuickStartGuide.md +++ b/docs/QuickStartGuide.md @@ -51,4 +51,4 @@ following files: - README.md - A general introduction to AFL, - docs/perf_tips.txt - Simple tips on how to fuzz more quickly, - docs/status_screen.txt - An explanation of the tidbits shown in the UI, - - docs/parallel_fuzzing.txt - Advice on running AFL on multiple cores. + - docs/parallel_fuzzing.md - Advice on running AFL on multiple cores. diff --git a/docs/life_pro_tips.txt b/docs/life_pro_tips.txt index c8c47636..27c70592 100644 --- a/docs/life_pro_tips.txt +++ b/docs/life_pro_tips.txt @@ -14,7 +14,7 @@ See dictionaries/README.dictionaries to learn how. % You can get the most out of your hardware by parallelizing AFL jobs. -See docs/parallel_fuzzing.txt for step-by-step tips. +See docs/parallel_fuzzing.md for step-by-step tips. % diff --git a/docs/parallel_fuzzing.txt b/docs/parallel_fuzzing.md index 1e65c01f..51fa3986 100644 --- a/docs/parallel_fuzzing.txt +++ b/docs/parallel_fuzzing.md @@ -1,12 +1,9 @@ -========================= -Tips for parallel fuzzing -========================= +# Tips for parallel fuzzing This document talks about synchronizing afl-fuzz jobs on a single machine or across a fleet of systems. See README for the general instruction manual. -1) Introduction ---------------- +## 1) Introduction Every copy of afl-fuzz will take up one CPU core. This means that on an n-core system, you can almost always run around n concurrent fuzzing jobs with @@ -33,8 +30,7 @@ several instances in parallel. See docs/power_schedules.txt Alternatively running other AFL spinoffs in parallel can be of value, e.g. Angora (https://github.com/AngoraFuzzer/Angora/) -2) Single-system parallelization --------------------------------- +## 2) Single-system parallelization If you wish to parallelize a single job across multiple cores on a local system, simply create a new, empty output directory ("sync dir") that will be @@ -43,12 +39,16 @@ for every instance - say, "fuzzer01", "fuzzer02", etc. Run the first one ("master", -M) like this: +``` $ ./afl-fuzz -i testcase_dir -o sync_dir -M fuzzer01 [...other stuff...] +``` ...and then, start up secondary (-S) instances like this: +``` $ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer02 [...other stuff...] $ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer03 [...other stuff...] +``` Each fuzzer will keep its state in a separate subdirectory, like so: @@ -68,9 +68,11 @@ Note that running multiple -M instances is wasteful, although there is an experimental support for parallelizing the deterministic checks. To leverage that, you need to create -M instances like so: +``` $ ./afl-fuzz -i testcase_dir -o sync_dir -M masterA:1/3 [...] $ ./afl-fuzz -i testcase_dir -o sync_dir -M masterB:2/3 [...] $ ./afl-fuzz -i testcase_dir -o sync_dir -M masterC:3/3 [...] +``` ...where the first value after ':' is the sequential ID of a particular master instance (starting at 1), and the second value is the total number of fuzzers to @@ -86,15 +88,16 @@ WARNING: Exercise caution when explicitly specifying the -f option. Each fuzzer must use a separate temporary file; otherwise, things will go south. One safe example may be: +``` $ ./afl-fuzz [...] -S fuzzer10 -f file10.txt ./fuzzed/binary @@ $ ./afl-fuzz [...] -S fuzzer11 -f file11.txt ./fuzzed/binary @@ $ ./afl-fuzz [...] -S fuzzer12 -f file12.txt ./fuzzed/binary @@ +``` This is not a concern if you use @@ without -f and let afl-fuzz come up with the file name. -3) Multi-system parallelization -------------------------------- +## 3) Multi-system parallelization The basic operating principle for multi-system parallelization is similar to the mechanism explained in section 2. The key difference is that you need to @@ -106,18 +109,22 @@ write a simple script that performs two actions: that includes host name in the fuzzer ID, so that you can do something like: + ```sh for s in {1..10}; do ssh user@host${s} "tar -czf - sync/host${s}_fuzzid*/[qf]*" >host${s}.tgz done + ``` - Distributes and unpacks these files on all the remaining machines, e.g.: + ```sh for s in {1..10}; do for d in {1..10}; do test "$s" = "$d" && continue ssh user@host${d} 'tar -kxzf -' <host${s}.tgz done done + ``` There is an example of such a script in experimental/distributed_fuzzing/; you can also find a more featured, experimental tool developed by @@ -167,8 +174,7 @@ It is *not* advisable to skip the synchronization script and run the fuzzers directly on a network filesystem; unexpected latency and unkillable processes in I/O wait state can mess things up. -4) Remote monitoring and data collection ----------------------------------------- +## 4) Remote monitoring and data collection You can use screen, nohup, tmux, or something equivalent to run remote instances of afl-fuzz. If you redirect the program's output to a file, it will @@ -192,8 +198,7 @@ Keep in mind that crashing inputs are *not* automatically propagated to the master instance, so you may still want to monitor for crashes fleet-wide from within your synchronization or health checking scripts (see afl-whatsup). -5) Asymmetric setups --------------------- +## 5) Asymmetric setups It is perhaps worth noting that all of the following is permitted: diff --git a/docs/perf_tips.txt b/docs/perf_tips.txt index 0cac8f7b..b4a8893d 100644 --- a/docs/perf_tips.txt +++ b/docs/perf_tips.txt @@ -140,7 +140,7 @@ options to use non-instrumented system-wide copies instead. The fuzzer is designed to need ~1 core per job. This means that on a, say, 4-core system, you can easily run four parallel fuzzing jobs with relatively -little performance hit. For tips on how to do that, see parallel_fuzzing.txt. +little performance hit. For tips on how to do that, see parallel_fuzzing.md. The afl-gotcpu utility can help you understand if you still have idle CPU capacity on your system. (It won't tell you about memory bandwidth, cache diff --git a/docs/status_screen.txt b/docs/status_screen.txt index c6f9f791..ef27bc76 100644 --- a/docs/status_screen.txt +++ b/docs/status_screen.txt @@ -218,7 +218,7 @@ now. It tells you about the current stage, which can be any of: splices together two random inputs from the queue at some arbitrarily selected midpoint. - - sync - a stage used only when -M or -S is set (see parallel_fuzzing.txt). + - sync - a stage used only when -M or -S is set (see parallel_fuzzing.md). No real fuzzing is involved, but the tool scans the output from other fuzzers and imports test cases as necessary. The first time this is done, it may take several minutes or so. @@ -370,7 +370,7 @@ comparing it to the number of logical cores on the system. If the value is shown in green, you are using fewer CPU cores than available on your system and can probably parallelize to improve performance; for tips on -how to do that, see parallel_fuzzing.txt. +how to do that, see parallel_fuzzing.md. If the value is shown in red, your CPU is *possibly* oversubscribed, and running additional fuzzers may not give you any benefits. diff --git a/docs/technical_details.txt b/docs/technical_details.txt index 1604c4d0..734512a2 100644 --- a/docs/technical_details.txt +++ b/docs/technical_details.txt @@ -485,7 +485,7 @@ This allows for extreme flexibility in fuzzer setup, including running synced instances against different parsers of a common data format, often with synergistic effects. -For more information about this design, see parallel_fuzzing.txt. +For more information about this design, see parallel_fuzzing.md. 12) Binary-only instrumentation ------------------------------- |