From e30b2c6af6e369844c92c00a20ebdd53473a747c Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 5 Sep 2020 13:18:28 +0200 Subject: final changes for pre-3.0 --- docs/ideas.md | 57 --------------------------------------------------------- 1 file changed, 57 deletions(-) (limited to 'docs/ideas.md') diff --git a/docs/ideas.md b/docs/ideas.md index 65e2e8e6..a5d40963 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,49 +3,6 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. -For GSOC2020 interested students please see -[https://github.com/AFLplusplus/AFLplusplus/issues/208](https://github.com/AFLplusplus/AFLplusplus/issues/208) - -## Flexible Grammar Mutator (currently in development) - -Currently, AFL++'s mutation does not have deeper knowledge about the fuzzed -binary, apart from feedback, even though the developer may have insights -about the target. - -A developer may choose to provide dictionaries and implement own mutations -in python or C, but an easy mutator that behaves according to a given grammar, -does not exist. - -State-of-the-art research on grammar fuzzing has some problems in their -implementations like code quality, scalability, or ease of use and other -common issues of the academic code. - -We aim to develop a pluggable grammar mutator for afl++ that combines -various results. - -Mentor: andreafioraldi - -## perf-fuzz Linux Kernel Module - -Expand on [snapshot LKM](https://github.com/AFLplusplus/AFL-Snapshot-LKM) -To make it thread safe, can snapshot several processes at once and increase -overall performance. - -Mentor: any - -## QEMU 5-based Instrumentation - -First tests to use QEMU 4 for binary-only AFL++ showed that caching behavior -changed, which vastly decreases fuzzing speeds. - -In this task test if QEMU 5 performs better and port the afl++ QEMU 3.1 -patches to QEMU 5. - -Understanding the current instrumentation and fixing the current caching -issues will be needed. - -Mentor: andreafioraldi - ## WASM Instrumentation Currently, AFL++ can be used for source code fuzzing and traditional binaries. @@ -66,20 +23,6 @@ Either improve a single mutator thorugh learning of many different bugs Mentor: domenukk -## Reengineer `afl-fuzz` as Thread Safe, Embeddable Library (currently in development) - -Right now, afl-fuzz is single threaded, cannot safely be embedded in tools, -and not multi-threaded. It makes use of a large number of globals, must always -be the parent process and exec child processes. -Instead, afl-fuzz could be refactored to contain no global state and globals. -This allows for different use cases that could be implemented during this -project. -Note that in the mean time a lot has happened here already, but e.g. making -it all work and implement multithreading in afl-fuzz ... there is still quite -some work to do. - -Mentor: hexcoder- or vanhauser-thc - ## Collision-free Binary-Only Maps AFL++ supports collison-free maps using an LTO (link-time-optimization) pass. -- cgit 1.4.1 From 7bcbfd48e54eba5a99d05b04f4f3d6bea29cde80 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sun, 6 Sep 2020 15:12:23 +0200 Subject: update ideas --- docs/ideas.md | 13 +++++++++++++ src/README.md | 33 +++++++++++++++++++-------------- 2 files changed, 32 insertions(+), 14 deletions(-) (limited to 'docs/ideas.md') diff --git a/docs/ideas.md b/docs/ideas.md index a5d40963..aaa3eed1 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,6 +3,19 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. +## Analysis software + +Currently analysis is done by using afl-plot, which is rather outdated. +A GTK or browser tool to create run-time analysis based on fuzzer_stats, +queue/id* information and plot_data that allows for zooming in and out, +changing min/max display values etc. and doing that for a single run, +different runs and campaigns vs campaigns. +Interesting values are execs, and execs/s, edges discovered (total, when +each edge was discovered and which other fuzzer share finding that edge), +test cases executed. +It should be clickable which value is X and Y axis, zoom factor, log scaling +on-off, etc. + ## WASM Instrumentation Currently, AFL++ can be used for source code fuzzing and traditional binaries. diff --git a/src/README.md b/src/README.md index 6da534c3..35af6ab9 100644 --- a/src/README.md +++ b/src/README.md @@ -2,23 +2,28 @@ Quick explanation about the files here: -- `afl-analyze.c` - afl-analyze binary tool +- `afl-analyze.c` - afl-analyze binary tool - `afl-as.c` - afl-as binary tool -- `afl-gotcpu.c` - afl-gotcpu binary tool -- `afl-showmap.c` - afl-showmap binary tool -- `afl-tmin.c` - afl-tmin binary tool -- `afl-fuzz.c` - afl-fuzz binary tool (just main() and usage()) +- `afl-cc.c` - afl-cc binary tool +- `afl-common.c` - common functions, used by afl-analyze, afl-fuzz, afl-showmap and afl-tmin +- `afl-forkserver.c` - forkserver implementation, used by afl-fuzz afl-showmap, afl-tmin - `afl-fuzz-bitmap.c` - afl-fuzz bitmap handling +- `afl-fuzz.c` - afl-fuzz binary tool (just main() and usage()) +- `afl-fuzz-cmplog.c` - afl-fuzz cmplog functions - `afl-fuzz-extras.c` - afl-fuzz the *extra* function calls -- `afl-fuzz-state.c` - afl-fuzz state and globals -- `afl-fuzz-init.c` - afl-fuzz initialization -- `afl-fuzz-misc.c` - afl-fuzz misc functions -- `afl-fuzz-one.c` - afl-fuzz fuzzer_one big loop, this is where the mutation is happening +- `afl-fuzz-init.c` - afl-fuzz initialization +- `afl-fuzz-misc.c` - afl-fuzz misc functions +- `afl-fuzz-mutators.c` - afl-fuzz custom mutator and python support +- `afl-fuzz-one.c` - afl-fuzz fuzzer_one big loop, this is where the mutation is happening +- `afl-fuzz-performance.c` - hash64 and rand functions - `afl-fuzz-python.c` - afl-fuzz the python mutator extension - `afl-fuzz-queue.c` - afl-fuzz handling the queue -- `afl-fuzz-run.c` - afl-fuzz running the target +- `afl-fuzz-redqueen.c` - afl-fuzz redqueen implemention +- `afl-fuzz-run.c` - afl-fuzz running the target +- `afl-fuzz-state.c` - afl-fuzz state and globals - `afl-fuzz-stats.c` - afl-fuzz writing the statistics file -- `afl-gcc.c` - afl-gcc binary tool (deprecated) -- `afl-common.c` - common functions, used by afl-analyze, afl-fuzz, afl-showmap and afl-tmin -- `afl-forkserver.c` - forkserver implementation, used by afl-fuzz and afl-tmin -afl-sharedmem.c - sharedmem implementation, used by afl-fuzz and afl-tmin +- `afl-gotcpu.c` - afl-gotcpu binary tool +- `afl-ld-lto.c` - LTO linker helper +- `afl-sharedmem.c` - sharedmem implementation, used by afl-fuzz, afl-showmap, afl-tmin +- `afl-showmap.c` - afl-showmap binary tool +- `afl-tmin.c` - afl-tmin binary tool -- cgit 1.4.1 From 893cd47d9cdbfa44e43d03e7d40a56a0c2ad7936 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sun, 31 Jan 2021 13:03:00 +0100 Subject: disable trimming for -M --- README.md | 1 + docs/ideas.md | 15 ++------------- src/afl-fuzz.c | 3 ++- 3 files changed, 5 insertions(+), 14 deletions(-) (limited to 'docs/ideas.md') diff --git a/README.md b/README.md index 2806b734..8c4aab93 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ behaviours and defaults: * a caching of testcases can now be performed and can be modified by editing config.h for TESTCASE_CACHE or by specifying the env variable `AFL_TESTCACHE_SIZE` (in MB). Good values are between 50-500 (default: 50). + * -M mains do not perform trimming * examples/ got renamed to utils/ * libtokencap/ libdislocator/ and qdbi_mode/ were moved to utils/ * afl-cmin/afl-cmin.bash now search first in PATH and last in AFL_PATH diff --git a/docs/ideas.md b/docs/ideas.md index aaa3eed1..7cbe60a5 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -16,6 +16,8 @@ test cases executed. It should be clickable which value is X and Y axis, zoom factor, log scaling on-off, etc. +Mentor: vanhauser-thc + ## WASM Instrumentation Currently, AFL++ can be used for source code fuzzing and traditional binaries. @@ -36,19 +38,6 @@ Either improve a single mutator thorugh learning of many different bugs Mentor: domenukk -## Collision-free Binary-Only Maps - -AFL++ supports collison-free maps using an LTO (link-time-optimization) pass. -This should be possible to implement for QEMU and Unicorn instrumentations. -As the forkserver parent caches just in time translated translation blocks, -adding a simple counter between jumps should be doable. - -Note: this is already in development for qemu by Andrea, so for people who -want to contribute it might make more sense to port his solution to unicorn. - -Mentor: andreafioraldi or domenukk -Issue/idea tracker: [https://github.com/AFLplusplus/AFLplusplus/issues/237](https://github.com/AFLplusplus/AFLplusplus/issues/237) - ## Your idea! Finally, we are open to proposals! diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index 9d9b0434..647a665e 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -145,7 +145,7 @@ static void usage(u8 *argv0, int more_help) { "Other stuff:\n" " -M/-S id - distributed mode (see docs/parallel_fuzzing.md)\n" - " -M auto-sets -D and -Z (use -d to disable -D)\n" + " -M auto-sets -D, -Z (use -d to disable -D) and no trimming\n" " -F path - sync to a foreign fuzzer queue directory (requires " "-M, can\n" " be specified up to %u times)\n" @@ -502,6 +502,7 @@ int main(int argc, char **argv_orig, char **envp) { afl->sync_id = ck_strdup(optarg); afl->skip_deterministic = 0; // force deterministic fuzzing afl->old_seed_selection = 1; // force old queue walking seed selection + afl->disable_trim = 1; // disable trimming if ((c = strchr(afl->sync_id, ':'))) { -- cgit 1.4.1 From 5b2634f711e95b48b6105d3cac659e51706ff4e9 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 10 Feb 2021 17:56:27 +0100 Subject: update changelog + ideas --- docs/Changelog.md | 12 ++++++------ docs/ideas.md | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 6 deletions(-) (limited to 'docs/ideas.md') diff --git a/docs/Changelog.md b/docs/Changelog.md index f2041917..56137eec 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -16,8 +16,8 @@ sending a mail to . to be placed in the source code. Check out instrumentation/README.instrument_list.md - afl-fuzz - - Making AFL_MAP_SIZE obsolete - afl-fuzz now learns on start the - target map size + - Making AFL_MAP_SIZE (mostly) obsolete - afl-fuzz now learns on start + the target map size - upgraded cmplog/redqueen: solving for floating point, solving transformations (e.g. toupper, tolower, to/from hex, xor, arithmetics, etc.). this is costly hence new command line option @@ -27,7 +27,7 @@ sending a mail to . - fix crash for very, very fast targets+systems (thanks to mhlakhani for reporting) - on restarts (-i)/autoresume (AFL_AUTORESUME) the stats are now - reloaded and used, thanks to Vimal Joseph for this PR! + reloaded and used, thanks to Vimal Joseph for this patch! - if determinstic mode is active (-D, or -M without -d) then we sync after every queue entry as this can take very long time otherwise - better detection if a target needs a large shared map @@ -47,8 +47,10 @@ sending a mail to . - added AFL_LLVM_INSTRUMENT option NATIVE for native clang pc-guard support (less performant than our own), GCC for old afl-gcc and CLANG for old afl-clang + - fixed a potential crash in the LAF feature - qemuafl - - ported QASan to qemuafl! see qemu_mode/libqasan/README.md + - QASan (address sanitizer for Qemu) ported to qemuafl! + See qemu_mode/libqasan/README.md - solved some persistent mode bugs (thanks Dil4rd) - solved an issue when dumping the memory maps (thanks wizche) - Android support for QASan @@ -58,8 +60,6 @@ sending a mail to . - Added a new example harness to compare python, c, and rust bindings - changed default: no memory limit for afl-cmin and afl-cmin.bash - warn on any _AFL and __AFL env vars - - LLVM mode is now compiled with -j4, unicorn with all cores. qemu was - already building with all cores, the gcc plugin needs only one. - added dummy Makefile to instrumentation/ - Updated utils/afl_frida to be 5% faster, 7% on x86_x64 - Added AFL_KILL_SIGNAL env variable (thanks @v-p-b) diff --git a/docs/ideas.md b/docs/ideas.md index 7cbe60a5..08cb16ef 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,6 +3,40 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. +# GSoC 2021 + +All GSoC 2021 projects will be in the Rust development language! + +## UI for libaflrs + +Write a user interface to libaflrs, the upcoming backend of afl++. +This might look like the afl-fuzz UI, but you can improve on it - and should! + +## Schedulers for libaflrs + +Schedulers is a mechanism that selects items from the fuzzing corpus based +on strategy and randomness. One scheduler might focus on long paths, +another on rarity of edges disocvered, still another on a combination on +things. Some of the schedulers in afl++ have to be ported, but you are free +to come up with your own if you want to - and see how it performs. + +## Forkserver support for libaflrs + +The current libaflrs implementation fuzzes in-memory, however obviously we +want to support afl instrumented binaries as well. +Hence a forkserver support needs to be implemented - forking off the target +and talking to the target via a socketpair and the communication protocol +within. + +## More Observers for libaflrs + +An observer is measuring functionality that looks at the target being fuzzed +and documents something about it. In traditional fuzzing this is the coverage +in the target, however we want to add various more observers, e.g. stack depth, +heap usage, etc. - this is a topic for an experienced Rust developer. + +# Generic ideas and wishlist + ## Analysis software Currently analysis is done by using afl-plot, which is rather outdated. -- cgit 1.4.1 From 6cc59a38be02d812e1f5bbb46b6a94c5a31b460a Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 10 Mar 2021 15:37:28 +0100 Subject: gsoc2021 clarification --- docs/ideas.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'docs/ideas.md') diff --git a/docs/ideas.md b/docs/ideas.md index 08cb16ef..0130cf61 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -35,7 +35,9 @@ and documents something about it. In traditional fuzzing this is the coverage in the target, however we want to add various more observers, e.g. stack depth, heap usage, etc. - this is a topic for an experienced Rust developer. -# Generic ideas and wishlist +# Generic ideas and wishlist - NOT PART OF GSoC 2021 ! + +The below list is not part of GSoC 2021. ## Analysis software -- cgit 1.4.1