From b1d71136b0a5eddc064ec03e19a3aaaaa579ec88 Mon Sep 17 00:00:00 2001 From: Ashis Kumar Singh <49238464+ashiscs@users.noreply.github.com> Date: Wed, 25 Mar 2020 01:21:02 +0530 Subject: fixed typos in README.md --- README.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 5b2995d8..88e2b3c6 100644 --- a/README.md +++ b/README.md @@ -54,7 +54,7 @@ * Win32 PE binary-only fuzzing with QEMU and Wine - * Radamsa mutator (enable with `-R` to add or `-RR` to run it exclusivly). + * Radamsa mutator (enable with `-R` to add or `-RR` to run it exclusively). * QBDI mode to fuzz android native libraries via QBDI framework @@ -85,7 +85,7 @@ (4) Only for LLVM >= 9 and not all targets compile - (5) upcoming, development in branch + (5) upcoming, development in the branch (6) not compatible with LTO and InsTrim modes @@ -118,13 +118,13 @@ We are happy to be part of [Google Summer of Code 2020](https://summerofcode.withgoogle.com/organizations/5100744400699392/)! :-) We have several ideas we would like to see in AFL++ to make it even better. -However we already work on so many things that we do not have the time for +However, we already work on so many things that we do not have the time for all the big ideas. This can be your way to support and contribute to AFL++ - extend it to something cool -We have an idea list in [docs/ideas.md](docs/ideas.md) +We have an idealist in [docs/ideas.md](docs/ideas.md) For everyone who wants to contribute (and send pull requests) please read [CONTRIBUTING.md](CONTRIBUTING.md) before your submit. @@ -205,9 +205,9 @@ There have been numerous attempts to solve this problem. One of the early approaches - pioneered by Tavis Ormandy - is corpus distillation. The method relies on coverage signals to select a subset of interesting seeds from a massive, high-quality corpus of candidate files, and then fuzz them by -traditional means. The approach works exceptionally well, but requires such +traditional means. The approach works exceptionally well but requires such a corpus to be readily available. In addition, block coverage measurements -provide only a very simplistic understanding of program state, and are less +provide only a very simplistic understanding of the program state and are less useful for guiding the fuzzing effort in the long haul. Other, more sophisticated research has focused on techniques such as program @@ -228,7 +228,7 @@ Simplifying a bit, the overall algorithm can be summed up as: 1) Load user-supplied initial test cases into the queue, - 2) Take next input file from the queue, + 2) Take the next input file from the queue, 3) Attempt to trim the test case to the smallest size that doesn't alter the measured behavior of the program, @@ -295,7 +295,7 @@ Using partial instrumentation is also recommended, see When testing libraries, you need to find or write a simple program that reads data from stdin or from a file and passes it to the tested library. In such a case, it is essential to link this executable against a static version of the -instrumented library, or to make sure that the correct .so file is loaded at +instrumented library or to make sure that the correct .so file is loaded at runtime (usually by setting `LD_LIBRARY_PATH`). The simplest option is a static build, usually possible via: @@ -316,7 +316,7 @@ file for important caveats. When source code is *NOT* available, the fuzzer offers experimental support for fast, on-the-fly instrumentation of black-box binaries. This is accomplished -with a version of QEMU running in the lesser-known "user space emulation" mode. +with a version of QEMU running in the lesser-known "userspace emulation" mode. QEMU is a project separate from AFL, but you can conveniently build the feature by doing: @@ -342,7 +342,7 @@ A more comprehensive description of these and other options can be found in ## Good examples and writeups -Here are some good writeups to show how to effectibly use AFL++: +Here are some good writeups to show how to effectively use AFL++: * [https://aflplus.plus/docs/tutorials/libxml2_tutorial/](https://aflplus.plus/docs/tutorials/libxml2_tutorial/) * [https://bananamafia.dev/post/gb-fuzz/](https://bananamafia.dev/post/gb-fuzz/) @@ -372,7 +372,7 @@ The available schedules are: - mmopt (experimental) - rare (experimental) -In parallel mode (-M/-S, several instances with shared queue), we suggest to +In parallel mode (-M/-S, several instances with the shared queue), we suggest to run the master using the explore or fast schedule (-p explore) and the slaves with a combination of cut-off-exponential (-p coe), exponential (-p fast), explore (-p explore) and mmopt (-p mmopt) schedules. If a schedule does @@ -380,7 +380,7 @@ not perform well for a target, restart the slave with a different schedule. In single mode, using -p fast is usually slightly more beneficial than the default explore mode. -(We don't want to change the default behaviour of afl, so "fast" has not been +(We don't want to change the default behavior of afl, so "fast" has not been made the default mode). More details can be found in the paper published at the 23rd ACM Conference on @@ -450,12 +450,12 @@ See the [docs/status_screen.md](docs/status_screen.md) file for information on how to interpret the displayed stats and monitor the health of the process. Be sure to consult this file especially if any UI elements are highlighted in red. -The fuzzing process will continue until you press Ctrl-C. At minimum, you want +The fuzzing process will continue until you press Ctrl-C. At a minimum, you want to allow the fuzzer to complete one queue cycle, which may take anywhere from a couple of hours to a week or so. There are three subdirectories created within the output directory and updated -in real time: +in real-time: - queue/ - test cases for every distinctive execution path, plus all the starting files given by the user. This is the synthesized corpus @@ -480,7 +480,7 @@ involve any state transitions not seen in previously-recorded faults. If a single bug can be reached in multiple ways, there will be some count inflation early in the process, but this should quickly taper off. -The file names for crashes and hangs are correlated with parent, non-faulting +The file names for crashes and hangs are correlated with the parent, non-faulting queue entries. This should help with debugging. When you can't reproduce a crash found by afl-fuzz, the most likely cause is @@ -544,10 +544,10 @@ instrumentation feedback alone. This actually works in practice, say: PS. Even when no explicit dictionary is given, afl-fuzz will try to extract existing syntax tokens in the input corpus by watching the instrumentation very closely during deterministic byte flips. This works for some types of -parsers and grammars, but isn't nearly as good as the -x mode. +parsers and grammars but isn't nearly as good as the -x mode. If a dictionary is really hard to come by, another option is to let AFL run -for a while, and then use the token capture library that comes as a companion +for a while and then use the token capture library that comes as a companion utility with AFL. For that, see [libtokencap/README.md](libtokencap/README.tokencap.md). ## Crash triage @@ -562,7 +562,7 @@ difficult to quickly evaluate for exploitability without a lot of debugging and code analysis work. To assist with this task, afl-fuzz supports a very unique "crash exploration" mode enabled with the -C flag. -In this mode, the fuzzer takes one or more crashing test cases as the input, +In this mode, the fuzzer takes one or more crashing test cases as the input and uses its feedback-driven fuzzing strategies to very quickly enumerate all code paths that can be reached in the program while keeping it in the crashing state. @@ -601,7 +601,7 @@ near the end of [docs/technical_details.md](docs/technical_details.md). Fuzzing is a wonderful and underutilized technique for discovering non-crashing design and implementation errors, too. Quite a few interesting bugs have been -found by modifying the target programs to call abort() when, say: +found by modifying the target programs to call abort() when say: - Two bignum libraries produce different outputs when given the same fuzzer-generated input, @@ -623,7 +623,7 @@ shared with libfuzzer) or `#ifdef __AFL_COMPILER` (this one is just for AFL). ## Common-sense risks Please keep in mind that, similarly to many other computationally-intensive -tasks, fuzzing may put strain on your hardware and on the OS. In particular: +tasks, fuzzing may put a strain on your hardware and on the OS. In particular: - Your CPU will run hot and will need adequate cooling. In most cases, if cooling is insufficient or stops working properly, CPU speeds will be @@ -656,7 +656,7 @@ Here are some of the most important caveats for AFL: - AFL detects faults by checking for the first spawned process dying due to a signal (SIGSEGV, SIGABRT, etc). Programs that install custom handlers for these signals may need to have the relevant code commented out. In the same - vein, faults in child processed spawned by the fuzzed target may evade + vein, faults in child process spawned by the fuzzed target may evade detection unless you manually add some code to catch that. - As with any other brute-force tool, the fuzzer offers limited coverage if -- cgit 1.4.1 From 3c3a5aa503a137c7f9a487ab82e93c638e699c03 Mon Sep 17 00:00:00 2001 From: Ashis Kumar Singh <49238464+ashiscs@users.noreply.github.com> Date: Wed, 25 Mar 2020 19:35:38 +0530 Subject: updated with required changes --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 88e2b3c6..781c8b49 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ all the big ideas. This can be your way to support and contribute to AFL++ - extend it to something cool -We have an idealist in [docs/ideas.md](docs/ideas.md) +We have an idea list in [docs/ideas.md](docs/ideas.md) For everyone who wants to contribute (and send pull requests) please read [CONTRIBUTING.md](CONTRIBUTING.md) before your submit. @@ -316,7 +316,7 @@ file for important caveats. When source code is *NOT* available, the fuzzer offers experimental support for fast, on-the-fly instrumentation of black-box binaries. This is accomplished -with a version of QEMU running in the lesser-known "userspace emulation" mode. +with a version of QEMU running in the lesser-known "user space emulation" mode. QEMU is a project separate from AFL, but you can conveniently build the feature by doing: @@ -656,7 +656,7 @@ Here are some of the most important caveats for AFL: - AFL detects faults by checking for the first spawned process dying due to a signal (SIGSEGV, SIGABRT, etc). Programs that install custom handlers for these signals may need to have the relevant code commented out. In the same - vein, faults in child process spawned by the fuzzed target may evade + vein, faults in child processes spawned by the fuzzed target may evade detection unless you manually add some code to catch that. - As with any other brute-force tool, the fuzzer offers limited coverage if -- cgit 1.4.1 From 34c090a31db7939558bf0047f0f1693bbde76c1f Mon Sep 17 00:00:00 2001 From: van Hauser Date: Fri, 27 Mar 2020 12:09:06 +0100 Subject: add CFI sanitizer --- README.md | 2 +- docs/Changelog.md | 1 + docs/env_variables.md | 4 +++- docs/notes_for_asan.md | 2 +- llvm_mode/LLVMInsTrim.so.cc | 1 + llvm_mode/afl-clang-fast.c | 21 +++++++++++++++++++-- llvm_mode/afl-llvm-lto-instrumentation.so.cc | 1 + llvm_mode/afl-llvm-pass.so.cc | 1 + src/afl-common.c | 2 +- 9 files changed, 29 insertions(+), 6 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 781c8b49..827a7aec 100644 --- a/README.md +++ b/README.md @@ -259,7 +259,7 @@ superior to blind fuzzing or coverage-only tools. ## Instrumenting programs for use with AFL PLEASE NOTE: llvm_mode compilation with afl-clang-fast/afl-clang-fast++ -instead of afl-gcc/afl-g++ is much faster and has a few cool features. +instead of afl-gcc/afl-g++ is much faster and has many cool features. See llvm_mode/ - however few code does not compile with llvm. We support llvm versions 3.8.0 to 11. diff --git a/docs/Changelog.md b/docs/Changelog.md index 6af269ce..198909d1 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -31,6 +31,7 @@ sending a mail to . the last 5 queue entries - rare: puts focus on queue entries that hits rare branches, also ignores runtime + - llvm_mode: added Control Flow Integrity sanatizer (AFL_USE_CFISAN) - LTO collision free instrumented added in llvm_mode with afl-clang-lto - note that this mode is amazing, but quite some targets won't compile - Added llvm_mode NGRAM prev_loc coverage by Adrean Herrera diff --git a/docs/env_variables.md b/docs/env_variables.md index 98f27bdf..ae283b1c 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -31,7 +31,9 @@ tools make fairly broad use of environmental variables: (You can also enable MSAN via AFL_USE_MSAN; ASAN and MSAN come with the same gotchas; the modes are mutually exclusive. UBSAN can be enabled - similarly by setting the environment variable AFL_USE_UBSAN=1) + similarly by setting the environment variable AFL_USE_UBSAN=1. Finally + there is the Control Flow Integrity sanitizer that can be activated by + AFL_USE_CFISAN=1) - Setting AFL_CC, AFL_CXX, and AFL_AS lets you use alternate downstream compilation tools, rather than the default 'clang', 'gcc', or 'as' binaries diff --git a/docs/notes_for_asan.md b/docs/notes_for_asan.md index b65873be..6a4806c0 100644 --- a/docs/notes_for_asan.md +++ b/docs/notes_for_asan.md @@ -29,7 +29,7 @@ Note that ASAN is incompatible with -static, so be mindful of that. (You can also use AFL_USE_MSAN=1 to enable MSAN instead.) NOTE: if you run several slaves only one should run the target compiled with -ASAN (and UBSAN), the others should run the target with no sanitizers +ASAN (and UBSAN, CFISAN), the others should run the target with no sanitizers compiled in. There is also the option of generating a corpus using a non-ASAN binary, and diff --git a/llvm_mode/LLVMInsTrim.so.cc b/llvm_mode/LLVMInsTrim.so.cc index a94eb907..c4033523 100644 --- a/llvm_mode/LLVMInsTrim.so.cc +++ b/llvm_mode/LLVMInsTrim.so.cc @@ -513,6 +513,7 @@ struct InsTrim : public ModulePass { getenv("AFL_HARDEN") ? "hardened" : "non-hardened", getenv("AFL_USE_ASAN") ? ", ASAN" : "", getenv("AFL_USE_MSAN") ? ", MSAN" : "", + getenv("AFL_USE_CFISAN") ? ", CFISAN" : "", getenv("AFL_USE_UBSAN") ? ", UBSAN" : ""); OKF("Instrumented %u locations (%llu, %llu) (%s mode)\n", total_instr, diff --git a/llvm_mode/afl-clang-fast.c b/llvm_mode/afl-clang-fast.c index 7050e22d..c45c8799 100644 --- a/llvm_mode/afl-clang-fast.c +++ b/llvm_mode/afl-clang-fast.c @@ -135,7 +135,7 @@ static void find_obj(u8 *argv0) { /* Copy argv to cc_params, making the necessary edits. */ -static void edit_params(u32 argc, char **argv) { +static void edit_params(u32 argc, char **argv, char **envp) { u8 fortify_set = 0, asan_set = 0, x_set = 0, maybe_linking = 1, bit_mode = 0; u8 has_llvm_config = 0; @@ -395,6 +395,22 @@ static void edit_params(u32 argc, char **argv) { } + if (getenv("AFL_USE_CFISAN")) { + + if (!lto_mode) { + + uint32_t i = 0, found = 0; + while (envp[i] != NULL && !found) + if (strncmp("-flto", envp[i++], 5) == 0) + found = 1; + if (!found) cc_params[cc_par_cnt++] = "-flto"; + + } + cc_params[cc_par_cnt++] = "-fsanitize=cfi"; + cc_params[cc_par_cnt++] = "-fvisibility=hidden"; + + } + #ifdef USE_TRACE_PC if (getenv("USE_TRACE_PC") || getenv("AFL_USE_TRACE_PC") || @@ -596,6 +612,7 @@ int main(int argc, char **argv, char **envp) { "AFL_USE_ASAN: activate address sanitizer\n" "AFL_USE_MSAN: activate memory sanitizer\n" "AFL_USE_UBSAN: activate undefined behaviour sanitizer\n" + "AFL_USE_CFISAN: activate control flow sanitizer\n" "AFL_LLVM_WHITELIST: enable whitelisting (selective " "instrumentation)\n" "AFL_LLVM_NOT_ZERO: use cycling trace counters that skip zero\n" @@ -685,7 +702,7 @@ int main(int argc, char **argv, char **envp) { find_obj(argv[0]); #endif - edit_params(argc, argv); + edit_params(argc, argv, envp); if (debug) { diff --git a/llvm_mode/afl-llvm-lto-instrumentation.so.cc b/llvm_mode/afl-llvm-lto-instrumentation.so.cc index febb8950..8bf485af 100644 --- a/llvm_mode/afl-llvm-lto-instrumentation.so.cc +++ b/llvm_mode/afl-llvm-lto-instrumentation.so.cc @@ -396,6 +396,7 @@ bool AFLLTOPass::runOnModule(Module &M) { getenv("AFL_HARDEN") ? "hardened" : "non-hardened", getenv("AFL_USE_ASAN") ? ", ASAN" : "", getenv("AFL_USE_MSAN") ? ", MSAN" : "", + getenv("AFL_USE_CFISAN") ? ", CFISAN" : "", getenv("AFL_USE_UBSAN") ? ", UBSAN" : ""); OKF("Instrumented %u locations with no collisions (on average %llu " "collisions would be in afl-gcc/afl-clang-fast) (%s mode).", diff --git a/llvm_mode/afl-llvm-pass.so.cc b/llvm_mode/afl-llvm-pass.so.cc index fefd9edd..e8f449b1 100644 --- a/llvm_mode/afl-llvm-pass.so.cc +++ b/llvm_mode/afl-llvm-pass.so.cc @@ -572,6 +572,7 @@ bool AFLCoverage::runOnModule(Module &M) { getenv("AFL_HARDEN") ? "hardened" : "non-hardened", getenv("AFL_USE_ASAN") ? ", ASAN" : "", getenv("AFL_USE_MSAN") ? ", MSAN" : "", + getenv("AFL_USE_CFISAN") ? ", CFISAN" : "", getenv("AFL_USE_UBSAN") ? ", UBSAN" : ""); OKF("Instrumented %u locations (%s mode, ratio %u%%).", inst_blocks, modeline, inst_ratio); diff --git a/src/afl-common.c b/src/afl-common.c index 8c4d53e8..e10de6b3 100644 --- a/src/afl-common.c +++ b/src/afl-common.c @@ -73,7 +73,7 @@ char * afl_environment_variables[] = { "AFL_SHUFFLE_QUEUE", "AFL_SKIP_BIN_CHECK", "AFL_SKIP_CPUFREQ", "AFL_SKIP_CRASHES", "AFL_TMIN_EXACT", "AFL_TMPDIR", "AFL_TOKEN_FILE", "AFL_TRACE_PC", "AFL_USE_ASAN", "AFL_USE_MSAN", "AFL_USE_TRACE_PC", - "AFL_USE_UBSAN", "AFL_WINE_PATH", NULL}; + "AFL_USE_UBSAN", "AFL_USE_CFISAN", "AFL_WINE_PATH", NULL}; void detect_file_args(char **argv, u8 *prog_in, u8 *use_stdin) { -- cgit 1.4.1 From 514036898fd4535c7f23fce2f7b368a84939d6a0 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Mon, 30 Mar 2020 17:45:17 +0200 Subject: added unit target to make help and README --- Makefile | 1 + README.md | 1 + 2 files changed, 2 insertions(+) (limited to 'README.md') diff --git a/Makefile b/Makefile index 2e4a6570..f10aadc9 100644 --- a/Makefile +++ b/Makefile @@ -206,6 +206,7 @@ help: @echo "clean: cleans everything. for qemu_mode it means it deletes all downloads as well" @echo "code-format: format the code, do this before you commit and send a PR please!" @echo "tests: this runs the test framework. It is more catered for the developers, but if you run into problems this helps pinpointing the problem" + @echo "unit: perform unit tests (based on cmocka)" @echo "document: creates afl-fuzz-document which will only do one run and save all manipulated inputs into out/queue/mutations" @echo "help: shows these build options :-)" @echo "==========================================" diff --git a/README.md b/README.md index 827a7aec..e1ca5949 100644 --- a/README.md +++ b/README.md @@ -160,6 +160,7 @@ These build targets exist: * clean: cleans everything. for qemu_mode and unicorn_mode it means it deletes all downloads as well * code-format: format the code, do this before you commit and send a PR please! * tests: runs test cases to ensure that all features are still working as they should +* unit: perform unit tests (based on cmocka) * help: shows these build options [Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html) you can also build statically linked versions of the -- cgit 1.4.1