diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/Changelog.md | 28 | ||||
| -rw-r--r-- | docs/PATCHES.md | 43 | ||||
| -rw-r--r-- | docs/custom_mutators.md | 2 | ||||
| -rw-r--r-- | docs/env_variables.md | 67 | ||||
| -rw-r--r-- | docs/ideas.md | 43 | ||||
| -rw-r--r-- | docs/notes_for_asan.md | 7 | ||||
| -rw-r--r-- | docs/perf_tips.md | 3 | ||||
| -rw-r--r-- | docs/status_screen.md | 11 |
8 files changed, 90 insertions, 114 deletions
diff --git a/docs/Changelog.md b/docs/Changelog.md index 5b7d6ab6..9c9a3976 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -8,6 +8,34 @@ Want to stay in the loop on major new features? Join our mailing list by sending a mail to <afl-users+subscribe@googlegroups.com>. +### Version ++3.13a (development) + - frida_mode - new mode that uses frida to fuzz binary-only targets, + thanks to @WorksButNotTested! + - create a fuzzing dictionary with the help of CodeQL thanks to + @microsvuln! see utils/autodict_ql + - afl-fuzz: + - added patch by @realmadsci to support @@ as part of command line + options, e.g. `afl-fuzz ... -- ./target --infile=@@` + - add recording of previous fuzz attempts for persistent mode + to allow replay of non-reproducable crashes, see + AFL_PERSISTENT_RECORD in config.h and docs/envs.h + - default cmplog level (-l) is now 2, better efficiency. + - cmplog level 3 (-l 3) now performs redqueen on everything. + use with care. + - better fuzzing strategy yields for enabled options + - ensure one fuzzer sync per cycle + - fix afl_custom_queue_new_entry original file name when syncing + from fuzzers + - added AFL_EXIT_ON_SEED_ISSUES env that will exit if a seed in + -i dir crashes the target or results in a timeout. By default + afl++ ignores these and uses them for splicing instead. + - afl-cc: + - Leak Sanitizer (AFL_USE_LSAN) added by Joshua Rogers, thanks! + - Removed InsTrim instrumentation as it is not as good as PCGUARD + - Removed automatic linking with -lc++ for LTO mode + - utils/aflpp_driver/aflpp_qemu_driver_hook fixed to work with qemu mode + - add -d (add dead fuzzer stats) to afl-whatsup + ### Version ++3.12c (release) - afl-fuzz: - added AFL_TARGET_ENV variable to pass extra env vars to the target diff --git a/docs/PATCHES.md b/docs/PATCHES.md deleted file mode 100644 index b2cff43a..00000000 --- a/docs/PATCHES.md +++ /dev/null @@ -1,43 +0,0 @@ -# Applied Patches - -The following patches from https://github.com/vanhauser-thc/afl-patches -have been installed or not installed: - - -## INSTALLED -``` -afl-llvm-fix.diff by kcwu(at)csie(dot)org -afl-sort-all_uniq-fix.diff by legarrec(dot)vincent(at)gmail(dot)com -laf-intel.diff by heiko(dot)eissfeldt(at)hexco(dot)de -afl-llvm-optimize.diff by mh(at)mh-sec(dot)de -afl-fuzz-tmpdir.diff by mh(at)mh-sec(dot)de -afl-fuzz-79x24.diff by heiko(dot)eissfeldt(at)hexco(dot)de -afl-fuzz-fileextensionopt.diff tbd -afl-as-AFL_INST_RATIO.diff by legarrec(dot)vincent(at)gmail(dot)com -afl-qemu-ppc64.diff by william(dot)barsse(at)airbus(dot)com -afl-qemu-optimize-entrypoint.diff by mh(at)mh-sec(dot)de -afl-qemu-speed.diff by abiondo on github -afl-qemu-optimize-map.diff by mh(at)mh-sec(dot)de -``` - -+ llvm_mode ngram prev_loc coverage (github.com/adrianherrera/afl-ngram-pass) -+ Custom mutator (native library) (by kyakdan) -+ unicorn_mode (modernized and updated by domenukk) -+ instrim (https://github.com/csienslab/instrim) was integrated -+ MOpt (github.com/puppet-meteor/MOpt-AFL) was imported -+ AFLfast additions (github.com/mboehme/aflfast) were incorporated. -+ Qemu 3.1 upgrade with enhancement patches (github.com/andreafioraldi/afl) -+ Python mutator modules support (github.com/choller/afl) -+ Instrument file list in LLVM mode (github.com/choller/afl) -+ forkserver patch for afl-tmin (github.com/nccgroup/TriforceAFL) - - -## NOT INSTALLED - -``` -afl-fuzz-context_sensitive.diff - changes too much of the behaviour -afl-tmpfs.diff - same as afl-fuzz-tmpdir.diff but more complex -afl-cmin-reduce-dataset.diff - unsure of the impact -afl-llvm-fix2.diff - not needed with the other patches -``` - diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 61d711e4..62e01f83 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -89,10 +89,10 @@ def queue_get(filename): def queue_new_entry(filename_new_queue, filename_orig_queue): pass -``` def introspection(): return string +``` ### Custom Mutation diff --git a/docs/env_variables.md b/docs/env_variables.md index 409425f1..0100ffac 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -55,7 +55,7 @@ make fairly broad use of environmental variables instead: overridden. - Setting `AFL_USE_ASAN` automatically enables ASAN, provided that your - compiler supports that. Note that fuzzing with ASAN is mildly challenging + compiler supports it. Note that fuzzing with ASAN is mildly challenging - see [notes_for_asan.md](notes_for_asan.md). (You can also enable MSAN via `AFL_USE_MSAN`; ASAN and MSAN come with the @@ -64,6 +64,13 @@ make fairly broad use of environmental variables instead: there is the Control Flow Integrity sanitizer that can be activated by `AFL_USE_CFISAN=1`) + - Setting `AFL_USE_LSAN` automatically enables Leak-Sanitizer, provided + that your compiler supports it. To perform a leak check within your + program at a certain point (such as at the end of an __AFL_LOOP), + you can run the macro __AFL_LEAK_CHECK(); which will cause + an abort if any memory is leaked (you can combine this with the + LSAN_OPTIONS=suppressions option to supress some known leaks). + - Setting `AFL_CC`, `AFL_CXX`, and `AFL_AS` lets you use alternate downstream compilation tools, rather than the default 'clang', 'gcc', or 'as' binaries in your `$PATH`. @@ -130,16 +137,15 @@ Then there are a few specific features that are only available in instrumentatio PCGUARD - our own pcgard based instrumentation (default) NATIVE - clang's original pcguard based instrumentation CLASSIC - classic AFL (map[cur_loc ^ prev_loc >> 1]++) (default) - CFG - InsTrim instrumentation (see below) LTO - LTO instrumentation (see below) CTX - context sensitive instrumentation (see below) NGRAM-x - deeper previous location coverage (from NGRAM-2 up to NGRAM-16) GCC - outdated gcc instrumentation CLANG - outdated clang instrumentation - In CLASSIC (default) and CFG/INSTRIM you can also specify CTX and/or - NGRAM, seperate the options with a comma "," then, e.g.: - `AFL_LLVM_INSTRUMENT=CFG,CTX,NGRAM-4` - Not that this is a good idea to use both CTX and NGRAM :) + In CLASSIC you can also specify CTX and/or NGRAM, seperate the options + with a comma "," then, e.g.: + `AFL_LLVM_INSTRUMENT=CLASSIC,CTX,NGRAM-4` + Note that this is actually not a good idea to use both CTX and NGRAM :) ### LTO @@ -173,24 +179,6 @@ Then there are a few specific features that are only available in instrumentatio See [instrumentation/README.lto.md](../instrumentation/README.lto.md) for more information. -### INSTRIM - - This feature increases the speed by ~15% without any disadvantages to the - classic instrumentation. - - Note that there is also an LTO version (if you have llvm 11 or higher) - - that is the best instrumentation we have. Use `afl-clang-lto` to activate. - The InsTrim LTO version additionally has all the options and features of - LTO (see above). - - - Setting `AFL_LLVM_INSTRIM` or `AFL_LLVM_INSTRUMENT=CFG` activates this mode - - - Setting `AFL_LLVM_INSTRIM_LOOPHEAD=1` expands on INSTRIM to optimize loops. - afl-fuzz will only be able to see the path the loop took, but not how - many times it was called (unless it is a complex loop). - - See [instrumentation/README.instrim.md](../instrumentation/README.instrim.md) - ### NGRAM - Setting `AFL_LLVM_NGRAM_SIZE` or `AFL_LLVM_INSTRUMENT=NGRAM-{value}` @@ -296,6 +284,9 @@ checks or alter some of the more exotic semantics of the tool: normally indicated by the cycle counter in the UI turning green. May be convenient for some types of automated jobs. + - `AFL_EXIT_ON_SEED_ISSUES` will restore the vanilla afl-fuzz behaviour + which does not allow crashes or timeout seeds in the initial -i corpus. + - `AFL_MAP_SIZE` sets the size of the shared map that afl-fuzz, afl-showmap, afl-tmin and afl-analyze create to gather instrumentation data from the target. This must be equal or larger than the size the target was @@ -391,7 +382,8 @@ checks or alter some of the more exotic semantics of the tool: may complain of high load prematurely, especially on systems with low core counts. To avoid the alarming red color, you can set `AFL_NO_CPU_RED`. - - In QEMU mode (-Q), `AFL_PATH` will be searched for afl-qemu-trace. + - In QEMU mode (-Q) and Frida mode (-O), `AFL_PATH` will + be searched for afl-qemu-trace and afl-frida-trace.so. - In QEMU mode (-Q), setting `AFL_QEMU_CUSTOM_BIN` cause afl-fuzz to skip prepending `afl-qemu-trace` to your command line. Use this if you wish to use a @@ -424,6 +416,16 @@ checks or alter some of the more exotic semantics of the tool: - Setting `AFL_FORCE_UI` will force painting the UI on the screen even if no valid terminal was detected (for virtual consoles) + - If you are using persistent mode (you should, see [instrumentation/README.persistent_mode.md](instrumentation/README.persistent_mode.md)) + some targets keep inherent state due which a detected crash testcase does + not crash the target again when the testcase is given. To be able to still + re-trigger these crashes you can use the `AFL_PERSISTENT_RECORD` variable + with a value of how many previous fuzz cases to keep prio a crash. + if set to e.g. 10, then the 9 previous inputs are written to + out/default/crashes as RECORD:000000,cnt:000000 to RECORD:000000,cnt:000008 + and RECORD:000000,cnt:000009 being the crash case. + NOTE: This option needs to be enabled in config.h first! + - If you are Jakub, you may need `AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES`. Others need not apply, unless they also want to disable the `/proc/sys/kernel/core_pattern` check. @@ -614,7 +616,7 @@ optimal values if not already present in the environment: override this by setting `LD_BIND_LAZY` beforehand, but it is almost certainly pointless. - - By default, `ASAN_OPTIONS` are set to: + - By default, `ASAN_OPTIONS` are set to (among others): ``` abort_on_error=1 detect_leaks=0 @@ -635,7 +637,14 @@ optimal values if not already present in the environment: msan_track_origins=0 allocator_may_return_null=1 ``` - Be sure to include the first one when customizing anything, since some - MSAN versions don't call `abort()` on error, and we need a way to detect - faults. + - Similarly, the default `LSAN_OPTIONS` are set to: +``` + exit_code=23 + fast_unwind_on_malloc=0 + symbolize=0 + print_suppressions=0 +``` + Be sure to include the first ones for LSAN and MSAN when customizing + anything, since some MSAN and LSAN versions don't call `abort()` on + error, and we need a way to detect faults. diff --git a/docs/ideas.md b/docs/ideas.md index 0130cf61..e25d3ba6 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,42 +3,6 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. -# GSoC 2021 - -All GSoC 2021 projects will be in the Rust development language! - -## UI for libaflrs - -Write a user interface to libaflrs, the upcoming backend of afl++. -This might look like the afl-fuzz UI, but you can improve on it - and should! - -## Schedulers for libaflrs - -Schedulers is a mechanism that selects items from the fuzzing corpus based -on strategy and randomness. One scheduler might focus on long paths, -another on rarity of edges disocvered, still another on a combination on -things. Some of the schedulers in afl++ have to be ported, but you are free -to come up with your own if you want to - and see how it performs. - -## Forkserver support for libaflrs - -The current libaflrs implementation fuzzes in-memory, however obviously we -want to support afl instrumented binaries as well. -Hence a forkserver support needs to be implemented - forking off the target -and talking to the target via a socketpair and the communication protocol -within. - -## More Observers for libaflrs - -An observer is measuring functionality that looks at the target being fuzzed -and documents something about it. In traditional fuzzing this is the coverage -in the target, however we want to add various more observers, e.g. stack depth, -heap usage, etc. - this is a topic for an experienced Rust developer. - -# Generic ideas and wishlist - NOT PART OF GSoC 2021 ! - -The below list is not part of GSoC 2021. - ## Analysis software Currently analysis is done by using afl-plot, which is rather outdated. @@ -65,6 +29,13 @@ the current Unicorn instrumentation. Mentor: any +## Support other programming languages + +Other programming languages also use llvm hence they could (easily?) supported +for fuzzing, e.g. mono, swift, go, kotlin native, fortran, ... + +Mentor: vanhauser-thc + ## Machine Learning Something with machine learning, better than [NEUZZ](https://github.com/dongdongshe/neuzz) :-) diff --git a/docs/notes_for_asan.md b/docs/notes_for_asan.md index 2b3bc028..f55aeaf2 100644 --- a/docs/notes_for_asan.md +++ b/docs/notes_for_asan.md @@ -28,6 +28,13 @@ Note that ASAN is incompatible with -static, so be mindful of that. (You can also use AFL_USE_MSAN=1 to enable MSAN instead.) +When compiling with AFL_USE_LSAN, the leak sanitizer will normally run +when the program exits. In order to utilize this check at different times, +such as at the end of a loop, you may use the macro __AFL_LEAK_CHECK();. +This macro will report a crash in afl-fuzz if any memory is left leaking +at this stage. You can also use LSAN_OPTIONS and a supressions file +for more fine-tuned checking, however make sure you keep exitcode=23. + NOTE: if you run several secondary instances, only one should run the target compiled with ASAN (and UBSAN, CFISAN), the others should run the target with no sanitizers compiled in. diff --git a/docs/perf_tips.md b/docs/perf_tips.md index fbcb4d8d..c5968206 100644 --- a/docs/perf_tips.md +++ b/docs/perf_tips.md @@ -69,9 +69,6 @@ If you are only interested in specific parts of the code being fuzzed, you can instrument_files the files that are actually relevant. This improves the speed and accuracy of afl. See instrumentation/README.instrument_list.md -Also use the InsTrim mode on larger binaries, this improves performance and -coverage a lot. - ## 4. Profile and optimize the binary Check for any parameters or settings that obviously improve performance. For diff --git a/docs/status_screen.md b/docs/status_screen.md index 0329d960..e3abcc5f 100644 --- a/docs/status_screen.md +++ b/docs/status_screen.md @@ -251,8 +251,9 @@ exceed it by a margin sufficient to be classified as hangs. | arithmetics : 53/2.54M, 0/537k, 0/55.2k | | known ints : 8/322k, 12/1.32M, 10/1.70M | | dictionary : 9/52k, 1/53k, 1/24k | - | havoc : 1903/20.0M, 0/0 | - | trim : 20.31%/9201, 17.05% | + |havoc/splice : 1903/20.0M, 0/0 | + |py/custom/rq : unused, 53/2.54M, unused | + | trim/eff : 20.31%/9201, 17.05% | +-----------------------------------------------------+ ``` @@ -268,6 +269,12 @@ goal. Finally, the third number shows the proportion of bytes that, although not possible to remove, were deemed to have no effect and were excluded from some of the more expensive deterministic fuzzing steps. +Note that when deterministic mutation mode is off (which is the default +because it is not very efficient) the first five lines display +"disabled (default, enable with -D)". + +Only what is activated will have counter shown. + ### Path geometry ``` |