From ffad6f3f095fce0a6a570727fb97593873a64b26 Mon Sep 17 00:00:00 2001 From: tocic Date: Sat, 10 Sep 2022 07:41:19 +0300 Subject: Fix typos in docs --- docs/fuzzing_in_depth.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/fuzzing_in_depth.md') diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 92c9910b..a0bf1566 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -523,7 +523,7 @@ mode!) and switch the input directory with a dash (`-`): afl-fuzz -i - -o output -- bin/target -someopt @@ ``` -Adding a dictionary is helpful. You have to following options: +Adding a dictionary is helpful. You have the following options: * See the directory [dictionaries/](../dictionaries/), if something is already included for your @@ -672,7 +672,7 @@ The syncing process itself is very simple. As the `-M main-$HOSTNAME` instance syncs to all `-S` secondaries as well as to other fuzzers, you have to copy only this directory to the other machines. -Lets say all servers have the `-o out` directory in /target/foo/out, and you +Let's say all servers have the `-o out` directory in /target/foo/out, and you created a file `servers.txt` which contains the hostnames of all participating servers, plus you have an ssh key deployed to all of them, then run: -- cgit 1.4.1 From 2c39c51263fd38de50ef41ff30075c1282997e14 Mon Sep 17 00:00:00 2001 From: fedotoff Date: Mon, 21 Nov 2022 15:18:19 +0300 Subject: casr-afl short description in fuzzing_in_depth. --- docs/fuzzing_in_depth.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'docs/fuzzing_in_depth.md') diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index a0bf1566..1645ba5c 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -900,6 +900,32 @@ then color-codes the input based on which sections appear to be critical and which are not; while not bulletproof, it can often offer quick insights into complex file formats. +`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides a +straightforward CASR integration with AFL++. While walking through afl +instances, `casr-afl` generates crash reports depending on target binary. For +binary with ASAN `casr-san` is used, otherwise `casr-gdb`. On the next step +report deduplication is done by `casr-cluster`. Finally, reports are triaged +into clusters. Crash reports contain many useful information: severity +(like [exploitable](https://github.com/jfoote/exploitable)), OS and package +versions, command line, stack trace, register values, disassembly, and even +source code fragment where crash appeared. + +**NOTE:** `casr-gdb` and `casr-san` should be in PATH to make `casr-afl` work. +Before using casr-afl, please, follow the installation +[guide](https://github.com/ispras/casr#getting-started). Using `casr-afl` is +very simple: + +```shell +casr-afl -i /path/to/afl/out/dir -o /path/to/casr/out/dir +``` + +Output directory contains subdirectories (cl1...clN) with report clusters. To +view reports you could use `casr-cli` tool: + +```shell +casr-cli /path/to/casr/out/dir/cl1/report.casrep +``` + ## 5. CI fuzzing Some notes on continuous integration (CI) fuzzing - this fuzzing is different to -- cgit 1.4.1 From a16726039f167548da86ce51d0cf4bd1b04e5374 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 21 Nov 2022 13:28:07 +0100 Subject: shorten text --- docs/fuzzing_in_depth.md | 25 +++---------------------- unicorn_mode/unicornafl | 2 +- 2 files changed, 4 insertions(+), 23 deletions(-) (limited to 'docs/fuzzing_in_depth.md') diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 1645ba5c..87f31a58 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -900,32 +900,13 @@ then color-codes the input based on which sections appear to be critical and which are not; while not bulletproof, it can often offer quick insights into complex file formats. -`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides a -straightforward CASR integration with AFL++. While walking through afl -instances, `casr-afl` generates crash reports depending on target binary. For -binary with ASAN `casr-san` is used, otherwise `casr-gdb`. On the next step -report deduplication is done by `casr-cluster`. Finally, reports are triaged -into clusters. Crash reports contain many useful information: severity -(like [exploitable](https://github.com/jfoote/exploitable)), OS and package -versions, command line, stack trace, register values, disassembly, and even -source code fragment where crash appeared. - -**NOTE:** `casr-gdb` and `casr-san` should be in PATH to make `casr-afl` work. -Before using casr-afl, please, follow the installation -[guide](https://github.com/ispras/casr#getting-started). Using `casr-afl` is -very simple: - +`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides +comfortable triaging for crashes found by AFL++. Reports are clustered and +contain severity and other information. ```shell casr-afl -i /path/to/afl/out/dir -o /path/to/casr/out/dir ``` -Output directory contains subdirectories (cl1...clN) with report clusters. To -view reports you could use `casr-cli` tool: - -```shell -casr-cli /path/to/casr/out/dir/cl1/report.casrep -``` - ## 5. CI fuzzing Some notes on continuous integration (CI) fuzzing - this fuzzing is different to diff --git a/unicorn_mode/unicornafl b/unicorn_mode/unicornafl index 0a31c2b2..6e00ceac 160000 --- a/unicorn_mode/unicornafl +++ b/unicorn_mode/unicornafl @@ -1 +1 @@ -Subproject commit 0a31c2b28bf7037fe8b0ff376521fdbdf28a9efe +Subproject commit 6e00ceac6fd5627e42e1858c543c84f2fbdaedda -- cgit 1.4.1 From 80eabd6e8a30c2ffc0f084ab34df8b9d582419c3 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 13 Feb 2023 11:34:14 +0100 Subject: AFL_LLVM_DICT2FILE_NO_MAIN support --- TODO.md | 3 +-- docs/Changelog.md | 1 + docs/env_variables.md | 3 +++ docs/fuzzing_in_depth.md | 4 +++- include/envs.h | 1 + instrumentation/README.llvm.md | 4 ++++ instrumentation/SanitizerCoverageLTO.so.cc | 11 ++++++++++- instrumentation/afl-llvm-dict2file.so.cc | 17 ++++++++++++++--- src/afl-cc.c | 5 ++++- 9 files changed, 41 insertions(+), 8 deletions(-) (limited to 'docs/fuzzing_in_depth.md') diff --git a/TODO.md b/TODO.md index 862224f0..187fa191 100644 --- a/TODO.md +++ b/TODO.md @@ -9,13 +9,12 @@ - afl-plot to support multiple plot_data - parallel builds for source-only targets - get rid of check_binary, replace with more forkserver communication - - first fuzzer should be a main automatically + - first fuzzer should be a main automatically? not sure. ## Maybe - forkserver tells afl-fuzz if cmplog is supported and if so enable it by default, with AFL_CMPLOG_NO=1 (?) set to skip? - - afl_custom_fuzz_splice_optin() - afl_custom_splice() - cmdline option from-to range for mutations diff --git a/docs/Changelog.md b/docs/Changelog.md index eee88a51..89c37912 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -10,6 +10,7 @@ - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) - support llvm 15 native pcguard changes + - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support - better sanitizer default options support for all tools - unicorn_mode: updated and minor issues fixed - frida_mode: fix issue on MacOS diff --git a/docs/env_variables.md b/docs/env_variables.md index 0a57d190..61fb1e2b 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -129,6 +129,9 @@ subset of the settings discussed in section 1, with the exception of: write all constant string comparisons to this file to be used later with afl-fuzz' `-x` option. + - An option to `AFL_LLVM_DICT2FILE` is `AFL_LLVM_DICT2FILE_NO_MAIN=1` which + skill not parse `main()`. + - `TMPDIR` and `AFL_KEEP_ASSEMBLY`, since no temporary assembly files are created. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 87f31a58..efab0633 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -534,6 +534,8 @@ dictionaries/FORMAT.dict`. * With `afl-clang-fast`, you can set `AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a dictionary during target compilation. + Adding `AFL_LLVM_DICT2FILE_NO_MAIN=1` to not parse main (usually command line + parameter parsing) is often a good idea too. * You also have the option to generate a dictionary yourself during an independent run of the target, see [utils/libtokencap/README.md](../utils/libtokencap/README.md). @@ -935,7 +937,7 @@ phase and start fuzzing at once. 3. Also randomize the afl-fuzz runtime options, e.g.: * 65% for `AFL_DISABLE_TRIM` * 50% for `AFL_KEEP_TIMEOUTS` - * 50% use a dictionary generated by `AFL_LLVM_DICT2FILE` + * 50% use a dictionary generated by `AFL_LLVM_DICT2FILE` + `AFL_LLVM_DICT2FILE_NO_MAIN=1` * 40% use MOpt (`-L 0`) * 40% for `AFL_EXPAND_HAVOC_NOW` * 20% for old queue processing (`-Z`) diff --git a/include/envs.h b/include/envs.h index 0770f94d..5018b0f8 100644 --- a/include/envs.h +++ b/include/envs.h @@ -133,6 +133,7 @@ static char *afl_environment_variables[] = { "AFL_LLVM_CTX", "AFL_LLVM_CTX_K", "AFL_LLVM_DICT2FILE", + "AFL_LLVM_DICT2FILE_NO_MAIN", "AFL_LLVM_DOCUMENT_IDS", "AFL_LLVM_INSTRIM_LOOPHEAD", "AFL_LLVM_INSTRUMENT", diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index 9da1b0f6..c0677474 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -167,6 +167,10 @@ Just specify `AFL_LLVM_DICT2FILE=/absolute/path/file.txt` and during compilation all constant string compare parameters will be written to this file to be used with afl-fuzz' `-x` option. +Adding `AFL_LLVM_DICT2FILE_NO_MAIN=1` will skip parsing `main()` which often +does command line parsing which has string comparisons that are not helpful +for fuzzing. + ## 6) AFL++ Context Sensitive Branch Coverage ### What is this? diff --git a/instrumentation/SanitizerCoverageLTO.so.cc b/instrumentation/SanitizerCoverageLTO.so.cc index 231151f5..f82224ed 100644 --- a/instrumentation/SanitizerCoverageLTO.so.cc +++ b/instrumentation/SanitizerCoverageLTO.so.cc @@ -236,6 +236,7 @@ class ModuleSanitizerCoverageLTO // const SpecialCaseList * Allowlist; // const SpecialCaseList * Blocklist; uint32_t autodictionary = 1; + uint32_t autodictionary_no_main = 0; uint32_t inst = 0; uint32_t afl_global_id = 0; uint32_t unhandled = 0; @@ -411,7 +412,8 @@ bool ModuleSanitizerCoverageLTO::instrumentModule( /* Show a banner */ setvbuf(stdout, NULL, _IONBF, 0); - if (getenv("AFL_DEBUG")) debug = 1; + if (getenv("AFL_DEBUG")) { debug = 1; } + if (getenv("AFL_LLVM_DICT2FILE_NO_MAIN")) { autodictionary_no_main = 1; } if ((isatty(2) && !getenv("AFL_QUIET")) || debug) { @@ -503,6 +505,13 @@ bool ModuleSanitizerCoverageLTO::instrumentModule( if (!isInInstrumentList(&F, MNAME) || !F.size()) { continue; } + if (autodictionary_no_main && + (!F.getName().compare("main") || !F.getName().compare("_main"))) { + + continue; + + } + for (auto &BB : F) { for (auto &IN : BB) { diff --git a/instrumentation/afl-llvm-dict2file.so.cc b/instrumentation/afl-llvm-dict2file.so.cc index bbbbe32c..97f1d47f 100644 --- a/instrumentation/afl-llvm-dict2file.so.cc +++ b/instrumentation/afl-llvm-dict2file.so.cc @@ -182,7 +182,7 @@ bool AFLdict2filePass::runOnModule(Module &M) { DenseMap valueMap; char *ptr; - int found = 0; + int found = 0, handle_main = 1; /* Show a banner */ setvbuf(stdout, NULL, _IONBF, 0); @@ -192,10 +192,14 @@ bool AFLdict2filePass::runOnModule(Module &M) { SAYF(cCYA "afl-llvm-dict2file" VERSION cRST " by Marc \"vanHauser\" Heuse \n"); - } else + } else { be_quiet = 1; + } + + if (getenv("AFL_LLVM_DICT2FILE_NO_MAIN")) { handle_main = 0; } + scanForDangerousFunctions(&M); ptr = getenv("AFL_LLVM_DICT2FILE"); @@ -210,7 +214,14 @@ bool AFLdict2filePass::runOnModule(Module &M) { for (auto &F : M) { - if (isIgnoreFunction(&F)) continue; + if (!handle_main && + (!F.getName().compare("main") || !F.getName().compare("_main"))) { + + continue; + + } + + if (isIgnoreFunction(&F)) { continue; } if (!isInInstrumentList(&F, MNAME) || !F.size()) { continue; } /* Some implementation notes. diff --git a/src/afl-cc.c b/src/afl-cc.c index 7c3682fb..7b059d40 100644 --- a/src/afl-cc.c +++ b/src/afl-cc.c @@ -2041,6 +2041,8 @@ int main(int argc, char **argv, char **envp) { " AFL_LLVM_DICT2FILE: generate an afl dictionary based on found " "comparisons\n" + " AFL_LLVM_DICT2FILE_NO_MAIN: skip parsing main() for the " + "dictionary\n" " AFL_LLVM_LAF_ALL: enables all LAF splits/transforms\n" " AFL_LLVM_LAF_SPLIT_COMPARES: enable cascaded comparisons\n" " AFL_LLVM_LAF_SPLIT_COMPARES_BITW: size limit (default 8)\n" @@ -2128,7 +2130,8 @@ int main(int argc, char **argv, char **envp) { "defaults.\n" "Recommended is afl-clang-lto with AFL_LLVM_CMPLOG or afl-clang-fast " "with\n" - "AFL_LLVM_CMPLOG and AFL_LLVM_DICT2FILE.\n\n"); + "AFL_LLVM_CMPLOG and " + "AFL_LLVM_DICT2FILE+AFL_LLVM_DICT2FILE_NO_MAIN.\n\n"); exit(1); -- cgit 1.4.1 From ebaac23a514cd3950d4a6cb597bd921e13ab9baa Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 20 Feb 2023 11:42:40 +0100 Subject: clarify AFL_NO_STARTUP_CALIBRATION --- docs/env_variables.md | 3 ++- docs/fuzzing_in_depth.md | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) (limited to 'docs/fuzzing_in_depth.md') diff --git a/docs/env_variables.md b/docs/env_variables.md index 22a5c386..646db3f2 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -474,7 +474,8 @@ checks or alter some of the more exotic semantics of the tool: output from afl-fuzz is redirected to a file or to a pipe. - Setting `AFL_NO_STARTUP_CALIBRATION` will skip the initial calibration - of all starting seeds, and start fuzzing at once. + of all starting seeds, and start fuzzing at once. Use with care, this + degrades the fuzzing performance! - In QEMU mode (-Q) and FRIDA mode (-O), `AFL_PATH` will be searched for afl-qemu-trace and afl-frida-trace.so. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 87f31a58..2a088201 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -628,7 +628,8 @@ If you have a large corpus, a corpus from a previous run or are fuzzing in a CI, then also set `export AFL_CMPLOG_ONLY_NEW=1` and `export AFL_FAST_CAL=1`. If the queue in the CI is huge and/or the execution time is slow then you can also add `AFL_NO_STARTUP_CALIBRATION=1` to skip the initial queue calibration -phase and start fuzzing at once. +phase and start fuzzing at once - but only do this if the calibration phase +would be too long for your fuzz run time. You can also use different fuzzers. If you are using AFL spinoffs or AFL conforming fuzzers, then just use the same -o directory and give it a unique @@ -914,7 +915,8 @@ normal fuzzing campaigns as these are much shorter runnings. If the queue in the CI is huge and/or the execution time is slow then you can also add `AFL_NO_STARTUP_CALIBRATION=1` to skip the initial queue calibration -phase and start fuzzing at once. +phase and start fuzzing at once. But only do that if the calibration time is +too long for your overall available fuzz run time. 1. Always: * LTO has a much longer compile time which is diametrical to short fuzzing - -- cgit 1.4.1