From 26a5bd625ccbd8de4fbc9b5eea263d092bd405e5 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Fri, 18 Nov 2022 12:23:18 +0100 Subject: write queue statistics --- docs/Changelog.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index c5eb6be3..4df47645 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -9,6 +9,9 @@ send fuzz data to the target as you need, e.g. via IPC. - cmplog mode now has -l R option for random colorization, thanks to guyf2010 for the PR! + - queue statistics are written every 30 minutes to + out/NAME/queue_data - likely this will be moved to a debug flag + in the future. - afl-showmap/afl-cmin - -t none now translates to -t 120000 (120 seconds) - unicorn_mode updated -- cgit 1.4.1 From b7c87350cf3481416b782fe19bc56467090ff220 Mon Sep 17 00:00:00 2001 From: Dominik Maier Date: Fri, 18 Nov 2022 17:04:16 +0100 Subject: Make env description extra clear --- docs/env_variables.md | 8 ++++---- unicorn_mode/unicornafl | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index d1c13e15..22a5c386 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -378,10 +378,10 @@ checks or alter some of the more exotic semantics of the tool: valid terminal was detected (for virtual consoles). - Setting `AFL_FORKSRV_INIT_TMOUT` allows you to specify a different timeout - to wait for the forkserver to spin up. The default is the `-t` value times - `FORK_WAIT_MULT` from `config.h` (usually 10), so for a `-t 100`, the - default would wait for `1000` milliseconds. Setting a different time here is - useful if the target has a very slow startup time, for example, when doing + to wait for the forkserver to spin up. The specified value is the new timeout, in milliseconds. + The default is the `-t` value times `FORK_WAIT_MULT` from `config.h` (usually 10), so for a `-t 100`, the default would wait for `1000` milliseconds. + The `AFL_FORKSRV_INIT_TMOUT` value does not get multiplied. It overwrites the initial timeout afl-fuzz waits for the target to come up with a constant time. + Setting a different time here is useful if the target has a very slow startup time, for example, when doing full-system fuzzing or emulation, but you don't want the actual runs to wait too long for timeouts. diff --git a/unicorn_mode/unicornafl b/unicorn_mode/unicornafl index 6e00ceac..0a31c2b2 160000 --- a/unicorn_mode/unicornafl +++ b/unicorn_mode/unicornafl @@ -1 +1 @@ -Subproject commit 6e00ceac6fd5627e42e1858c543c84f2fbdaedda +Subproject commit 0a31c2b28bf7037fe8b0ff376521fdbdf28a9efe -- cgit 1.4.1 From ef0921d858be0d54f3ebfe88e361ba54fb9ba69d Mon Sep 17 00:00:00 2001 From: fedotoff Date: Mon, 21 Nov 2022 12:59:23 +0300 Subject: Add CASR as third party tool in docs. --- docs/third_party_tools.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'docs') diff --git a/docs/third_party_tools.md b/docs/third_party_tools.md index 1175d9e5..97f2c362 100644 --- a/docs/third_party_tools.md +++ b/docs/third_party_tools.md @@ -62,3 +62,5 @@ generates builds of debian packages suitable for AFL. * [afl-fid](https://github.com/FoRTE-Research/afl-fid) - a set of tools for working with input data. +* [CASR](https://github.com/ispras/casr) - a set of tools for crash triage and + analysis. -- cgit 1.4.1 From 2c39c51263fd38de50ef41ff30075c1282997e14 Mon Sep 17 00:00:00 2001 From: fedotoff Date: Mon, 21 Nov 2022 15:18:19 +0300 Subject: casr-afl short description in fuzzing_in_depth. --- docs/fuzzing_in_depth.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'docs') diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index a0bf1566..1645ba5c 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -900,6 +900,32 @@ then color-codes the input based on which sections appear to be critical and which are not; while not bulletproof, it can often offer quick insights into complex file formats. +`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides a +straightforward CASR integration with AFL++. While walking through afl +instances, `casr-afl` generates crash reports depending on target binary. For +binary with ASAN `casr-san` is used, otherwise `casr-gdb`. On the next step +report deduplication is done by `casr-cluster`. Finally, reports are triaged +into clusters. Crash reports contain many useful information: severity +(like [exploitable](https://github.com/jfoote/exploitable)), OS and package +versions, command line, stack trace, register values, disassembly, and even +source code fragment where crash appeared. + +**NOTE:** `casr-gdb` and `casr-san` should be in PATH to make `casr-afl` work. +Before using casr-afl, please, follow the installation +[guide](https://github.com/ispras/casr#getting-started). Using `casr-afl` is +very simple: + +```shell +casr-afl -i /path/to/afl/out/dir -o /path/to/casr/out/dir +``` + +Output directory contains subdirectories (cl1...clN) with report clusters. To +view reports you could use `casr-cli` tool: + +```shell +casr-cli /path/to/casr/out/dir/cl1/report.casrep +``` + ## 5. CI fuzzing Some notes on continuous integration (CI) fuzzing - this fuzzing is different to -- cgit 1.4.1 From a16726039f167548da86ce51d0cf4bd1b04e5374 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 21 Nov 2022 13:28:07 +0100 Subject: shorten text --- docs/fuzzing_in_depth.md | 25 +++---------------------- unicorn_mode/unicornafl | 2 +- 2 files changed, 4 insertions(+), 23 deletions(-) (limited to 'docs') diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 1645ba5c..87f31a58 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -900,32 +900,13 @@ then color-codes the input based on which sections appear to be critical and which are not; while not bulletproof, it can often offer quick insights into complex file formats. -`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides a -straightforward CASR integration with AFL++. While walking through afl -instances, `casr-afl` generates crash reports depending on target binary. For -binary with ASAN `casr-san` is used, otherwise `casr-gdb`. On the next step -report deduplication is done by `casr-cluster`. Finally, reports are triaged -into clusters. Crash reports contain many useful information: severity -(like [exploitable](https://github.com/jfoote/exploitable)), OS and package -versions, command line, stack trace, register values, disassembly, and even -source code fragment where crash appeared. - -**NOTE:** `casr-gdb` and `casr-san` should be in PATH to make `casr-afl` work. -Before using casr-afl, please, follow the installation -[guide](https://github.com/ispras/casr#getting-started). Using `casr-afl` is -very simple: - +`casr-afl` from [CASR](https://github.com/ispras/casr) tools provides +comfortable triaging for crashes found by AFL++. Reports are clustered and +contain severity and other information. ```shell casr-afl -i /path/to/afl/out/dir -o /path/to/casr/out/dir ``` -Output directory contains subdirectories (cl1...clN) with report clusters. To -view reports you could use `casr-cli` tool: - -```shell -casr-cli /path/to/casr/out/dir/cl1/report.casrep -``` - ## 5. CI fuzzing Some notes on continuous integration (CI) fuzzing - this fuzzing is different to diff --git a/unicorn_mode/unicornafl b/unicorn_mode/unicornafl index 0a31c2b2..6e00ceac 160000 --- a/unicorn_mode/unicornafl +++ b/unicorn_mode/unicornafl @@ -1 +1 @@ -Subproject commit 0a31c2b28bf7037fe8b0ff376521fdbdf28a9efe +Subproject commit 6e00ceac6fd5627e42e1858c543c84f2fbdaedda -- cgit 1.4.1 From 342081d5ee367f473df3fc34c55edb5df7e42d0f Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Fri, 23 Dec 2022 16:32:53 +0100 Subject: make CI green --- GNUmakefile | 6 ++++-- docs/Changelog.md | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/GNUmakefile b/GNUmakefile index 6b55635f..43f96ffe 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -628,9 +628,9 @@ distrib: all -$(MAKE) -j$(nproc) -f GNUmakefile.llvm ifneq "$(SYS)" "Darwin" -$(MAKE) -f GNUmakefile.gcc_plugin -endif -$(MAKE) -C utils/libdislocator -$(MAKE) -C utils/libtokencap +endif -$(MAKE) -C utils/afl_network_proxy -$(MAKE) -C utils/socket_fuzzing -$(MAKE) -C utils/argv_fuzzing @@ -659,8 +659,10 @@ endif .PHONY: binary-only binary-only: test_shm test_python ready $(PROGS) +ifneq "$(SYS)" "Darwin" -$(MAKE) -C utils/libdislocator -$(MAKE) -C utils/libtokencap +#endif -$(MAKE) -C utils/afl_network_proxy -$(MAKE) -C utils/socket_fuzzing -$(MAKE) -C utils/argv_fuzzing @@ -717,9 +719,9 @@ source-only: all -$(MAKE) -j$(nproc) -f GNUmakefile.llvm ifneq "$(SYS)" "Darwin" -$(MAKE) -f GNUmakefile.gcc_plugin -endif -$(MAKE) -C utils/libdislocator -$(MAKE) -C utils/libtokencap +endif # -$(MAKE) -C utils/plot_ui ifeq "$(SYS)" "Linux" ifndef NO_NYX diff --git a/docs/Changelog.md b/docs/Changelog.md index 4df47645..6bfb314d 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -4,6 +4,10 @@ release of the tool. See README.md for the general instruction manual. ### Version ++4.05a (dev) + - MacOS: libdislocator, libtokencap etc. do not work with modern + MacOS anymore, but could be patched to work, see this issue if you + want to make the effort and send a PR: + https://github.com/AFLplusplus/AFLplusplus/issues/1594 - afl-fuzz: - added afl_custom_fuzz_send custom mutator feature. Now your can send fuzz data to the target as you need, e.g. via IPC. -- cgit 1.4.1 From 885a6fc106757b3968b86f9a4314f662bb04de43 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 5 Jan 2023 12:43:53 +0100 Subject: 4.05c release --- README.md | 4 ++-- docs/Changelog.md | 11 ++++++----- include/config.h | 2 +- 3 files changed, 9 insertions(+), 8 deletions(-) (limited to 'docs') diff --git a/README.md b/README.md index 4ff8c514..eeab7aa1 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ AFL++ logo -Release version: [4.04c](https://github.com/AFLplusplus/AFLplusplus/releases) +Release version: [4.05c](https://github.com/AFLplusplus/AFLplusplus/releases) -GitHub version: 4.05a +GitHub version: 4.06a Repository: [https://github.com/AFLplusplus/AFLplusplus](https://github.com/AFLplusplus/AFLplusplus) diff --git a/docs/Changelog.md b/docs/Changelog.md index 6bfb314d..b9376711 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -3,7 +3,7 @@ This is the list of all noteworthy changes made in every public release of the tool. See README.md for the general instruction manual. -### Version ++4.05a (dev) +### Version ++4.05c (release) - MacOS: libdislocator, libtokencap etc. do not work with modern MacOS anymore, but could be patched to work, see this issue if you want to make the effort and send a PR: @@ -11,14 +11,15 @@ - afl-fuzz: - added afl_custom_fuzz_send custom mutator feature. Now your can send fuzz data to the target as you need, e.g. via IPC. - - cmplog mode now has -l R option for random colorization, thanks + - cmplog mode now has a -l R option for random colorization, thanks to guyf2010 for the PR! - queue statistics are written every 30 minutes to - out/NAME/queue_data - likely this will be moved to a debug flag - in the future. + out/NAME/queue_data if compiled with INTROSPECTION + - new env: AFL_FORK_SERVER_KILL_SIGNAL - afl-showmap/afl-cmin - - -t none now translates to -t 120000 (120 seconds) + - `-t none` now translates to `-t 120000` (120 seconds) - unicorn_mode updated + - several minor bugfixes ### Version ++4.04c (release) - fix gramatron and grammar_mutator build scripts diff --git a/include/config.h b/include/config.h index b3310270..67b9f932 100644 --- a/include/config.h +++ b/include/config.h @@ -26,7 +26,7 @@ /* Version string: */ // c = release, a = volatile github dev, e = experimental branch -#define VERSION "++4.05a" +#define VERSION "++4.05c" /****************************************************** * * -- cgit 1.4.1 From 57e7408774a5276baaecd9dd5a6c73f1cbbf866b Mon Sep 17 00:00:00 2001 From: Dominik Maier Date: Thu, 5 Jan 2023 12:15:54 +0000 Subject: add update info to changelog --- docs/Changelog.md | 1 + 1 file changed, 1 insertion(+) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index b9376711..7a9b74c5 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -19,6 +19,7 @@ - afl-showmap/afl-cmin - `-t none` now translates to `-t 120000` (120 seconds) - unicorn_mode updated + - updated rust custom mutator dependencies and LibAFL custom mutator - several minor bugfixes ### Version ++4.04c (release) -- cgit 1.4.1 From 8fe5e29104fc514551bbc926c5142dac68562b43 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 18 Jan 2023 14:56:26 +0100 Subject: ignore timeout env option --- docs/env_variables.md | 3 +++ include/afl-fuzz.h | 2 +- include/envs.h | 1 + src/afl-fuzz-bitmap.c | 6 ++++++ src/afl-fuzz-state.c | 7 +++++++ src/afl-fuzz.c | 3 ++- 6 files changed, 20 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index 22a5c386..0a57d190 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -354,6 +354,9 @@ checks or alter some of the more exotic semantics of the tool: - Setting `AFL_KEEP_TIMEOUTS` will keep longer running inputs if they reach new coverage + - On the contrary, if you are not interested in any timeouts, you can set + `AFL_IGNORE_TIMEOUTS` to get a bit of speed instead. + - `AFL_EXIT_ON_SEED_ISSUES` will restore the vanilla afl-fuzz behavior which does not allow crashes or timeout seeds in the initial -i corpus. diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index edef9207..69fea579 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -398,7 +398,7 @@ typedef struct afl_env_vars { afl_cycle_schedules, afl_expand_havoc, afl_statsd, afl_cmplog_only_new, afl_exit_on_seed_issues, afl_try_affinity, afl_ignore_problems, afl_keep_timeouts, afl_pizza_mode, afl_no_crash_readme, - afl_no_startup_calibration; + afl_ignore_timeouts, afl_no_startup_calibration; u8 *afl_tmpdir, *afl_custom_mutator_library, *afl_python_module, *afl_path, *afl_hang_tmout, *afl_forksrv_init_tmout, *afl_preload, diff --git a/include/envs.h b/include/envs.h index f4cdf390..0770f94d 100644 --- a/include/envs.h +++ b/include/envs.h @@ -103,6 +103,7 @@ static char *afl_environment_variables[] = { "AFL_HARDEN", "AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES", "AFL_IGNORE_PROBLEMS", + "AFL_IGNORE_TIMEOUTS", "AFL_IGNORE_UNKNOWN_ENVS", "AFL_IMPORT_FIRST", "AFL_INPUT_LEN_MIN", diff --git a/src/afl-fuzz-bitmap.c b/src/afl-fuzz-bitmap.c index 485b82db..b4e9537e 100644 --- a/src/afl-fuzz-bitmap.c +++ b/src/afl-fuzz-bitmap.c @@ -457,6 +457,12 @@ save_if_interesting(afl_state_t *afl, void *mem, u32 len, u8 fault) { if (unlikely(len == 0)) { return 0; } + if (unlikely(fault == FSRV_RUN_TMOUT && afl->afl_env.afl_ignore_timeouts)) { + + return 0; + + } + u8 fn[PATH_MAX]; u8 *queue_fn = ""; u8 new_bits = 0, keeping = 0, res, classified = 0, is_timeout = 0; diff --git a/src/afl-fuzz-state.c b/src/afl-fuzz-state.c index 896b5f71..104b1e4b 100644 --- a/src/afl-fuzz-state.c +++ b/src/afl-fuzz-state.c @@ -292,6 +292,13 @@ void read_afl_environment(afl_state_t *afl, char **envp) { afl->afl_env.afl_ignore_problems = get_afl_env(afl_environment_variables[i]) ? 1 : 0; + } else if (!strncmp(env, "AFL_IGNORE_TIMEOUTS", + + afl_environment_variable_len)) { + + afl->afl_env.afl_ignore_timeouts = + get_afl_env(afl_environment_variables[i]) ? 1 : 0; + } else if (!strncmp(env, "AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES", afl_environment_variable_len)) { diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index 5e0ecd1e..4db55b5e 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -258,8 +258,9 @@ static void usage(u8 *argv0, int more_help) { "AFL_FORKSRV_INIT_TMOUT: time spent waiting for forkserver during startup (in ms)\n" "AFL_HANG_TMOUT: override timeout value (in milliseconds)\n" "AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES: don't warn about core dump handlers\n" - "AFL_IGNORE_UNKNOWN_ENVS: don't warn on unknown env vars\n" "AFL_IGNORE_PROBLEMS: do not abort fuzzing if an incorrect setup is detected\n" + "AFL_IGNORE_TIMEOUTS: do not process or save any timeouts\n" + "AFL_IGNORE_UNKNOWN_ENVS: don't warn on unknown env vars\n" "AFL_IMPORT_FIRST: sync and import test cases from other fuzzer instances first\n" "AFL_INPUT_LEN_MIN/AFL_INPUT_LEN_MAX: like -g/-G set min/max fuzz length produced\n" "AFL_PIZZA_MODE: 1 - enforce pizza mode, 0 - disable for April 1st\n" -- cgit 1.4.1 From 14d8eb9e40a6329abcb2f153174b543349c68c13 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 18 Jan 2023 22:17:14 +0100 Subject: autotoken: splicing; splice_optout --- custom_mutators/autotokens/Makefile | 6 +- custom_mutators/autotokens/autotokens.cpp | 103 +++++++++++++++++++++++++++--- docs/custom_mutators.md | 11 ++++ include/afl-fuzz.h | 14 ++++ src/afl-fuzz-mutators.c | 13 ++++ src/afl-fuzz-one.c | 3 +- src/afl-fuzz-python.c | 16 +++++ 7 files changed, 155 insertions(+), 11 deletions(-) (limited to 'docs') diff --git a/custom_mutators/autotokens/Makefile b/custom_mutators/autotokens/Makefile index 8af63635..ab1da4b6 100644 --- a/custom_mutators/autotokens/Makefile +++ b/custom_mutators/autotokens/Makefile @@ -1,5 +1,9 @@ ifdef debug - CFLAGS += "-fsanitize=address -Wall" + CFLAGS += -fsanitize=address -Wall + CXX := clang++ +endif +ifdef DEBUG + CFLAGS += -fsanitize=address -Wall CXX := clang++ endif diff --git a/custom_mutators/autotokens/autotokens.cpp b/custom_mutators/autotokens/autotokens.cpp index 7aecb010..c9ec4352 100644 --- a/custom_mutators/autotokens/autotokens.cpp +++ b/custom_mutators/autotokens/autotokens.cpp @@ -19,6 +19,13 @@ extern "C" { #define AUTOTOKENS_ALTERNATIVE_TOKENIZE 0 #define AUTOTOKENS_CHANGE_MIN 8 #define AUTOTOKENS_WHITESPACE " " +#define AUTOTOKENS_SIZE_MIN 8 +#define AUTOTOKENS_SPLICE_MIN 4 +#define AUTOTOKENS_SPLICE_MAX 64 + +#if AUTOTOKENS_SPLICE_MIN >= AUTOTOKENS_SIZE_MIN + #error SPLICE_MIN must be lower than SIZE_MIN +#endif using namespace std; @@ -42,6 +49,7 @@ static u32 extras_cnt, a_extras_cnt; static u64 all_spaces, all_tabs, all_lf, all_ws; static u64 all_structure_items; static unordered_map *> file_mapping; +static unordered_map *> id_mapping; static unordered_map token_to_id; static unordered_map id_to_token; static string whitespace = AUTOTOKENS_WHITESPACE; @@ -76,6 +84,8 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, u8 **out_buf, u8 *add_buf, size_t add_buf_size, size_t max_size) { + (void)(data); + if (s == NULL) { *out_buf = NULL; @@ -92,14 +102,14 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, afl_ptr->havoc_div / 256)); // DEBUG(stderr, "structure size: %lu, rounds: %u \n", m.size(), rounds); - u32 max_rand = 7; + u32 max_rand = 14; for (i = 0; i < rounds; ++i) { switch (rand_below(afl_ptr, max_rand)) { /* CHANGE */ - case 0 ... 3: /* fall through */ + case 0 ... 7: /* fall through */ { u32 pos = rand_below(afl_ptr, m_size); @@ -122,18 +132,19 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, } /* INSERT (m_size +1 so we insert also after last place) */ - case 4 ... 5: { + case 8 ... 9: { u32 new_item; do { new_item = rand_below(afl_ptr, current_id); - } while (!alternative_tokenize && new_item >= whitespace_ids); + } while (unlikely(!alternative_tokenize && new_item >= whitespace_ids)); u32 pos = rand_below(afl_ptr, m_size + 1); m.insert(m.begin() + pos, new_item); ++m_size; + DEBUG(stderr, "INS: %u at %u\n", new_item, pos); if (likely(!alternative_tokenize)) { @@ -168,8 +179,63 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, } + /* SPLICING */ + case 10 ... 11: { + + u32 strategy = rand_below(afl_ptr, 4), dst_off, n; + auto src = id_mapping[rand_below(afl_ptr, valid_structures)]; + u32 src_size = src->size(); + u32 src_off = rand_below(afl_ptr, src_size - AUTOTOKENS_SPLICE_MIN); + u32 rand_r = 1 + MAX(AUTOTOKENS_SPLICE_MIN, + MIN(AUTOTOKENS_SPLICE_MAX, src_size - src_off)); + + switch (strategy) { + + // insert + case 0: { + + dst_off = rand_below(afl_ptr, m_size); + n = AUTOTOKENS_SPLICE_MIN + + rand_below(afl_ptr, MIN(AUTOTOKENS_SPLICE_MAX, + rand_r - AUTOTOKENS_SPLICE_MIN)); + m.insert(m.begin() + dst_off, src->begin() + src_off, + src->begin() + src_off + n); + m_size += n; + DEBUG(stderr, "SPLICE-INS: %u at %u\n", n, dst_off); + break; + + } + + // overwrite + default: { + + dst_off = rand_below(afl_ptr, m_size - AUTOTOKENS_SPLICE_MIN); + n = AUTOTOKENS_SPLICE_MIN + + rand_below( + afl_ptr, + MIN(AUTOTOKENS_SPLICE_MAX - AUTOTOKENS_SPLICE_MIN, + MIN(m_size - dst_off - AUTOTOKENS_SPLICE_MIN, + src_size - src_off - AUTOTOKENS_SPLICE_MIN))); + + for (u32 i = 0; i < n; ++i) { + + m[dst_off + i] = (*src)[src_off + i]; + + } + + DEBUG(stderr, "SPLICE-MUT: %u at %u\n", n, dst_off); + break; + + } + + } + + break; + + } + /* ERASE - only if large enough */ - case 6: { + case 12 ... 13: { if (m_size > 8) { @@ -178,7 +244,7 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, } else { - max_rand = 6; + max_rand = 12; } @@ -236,12 +302,15 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, extern "C" unsigned char afl_custom_queue_get(void *data, const unsigned char *filename) { + (void)(data); + if (likely(!debug)) { if ((afl_ptr->shm.cmplog_mode && !afl_ptr->queue_cur->is_ascii) || (only_fav && !afl_ptr->queue_cur->favored)) { s = NULL; + DEBUG(stderr, "cmplog not ascii or only_fav and not favorite\n"); return 0; } @@ -334,8 +403,8 @@ extern "C" unsigned char afl_custom_queue_get(void *data, fclose(fp); file_mapping[fn] = structure; // NULL ptr so we don't read the file again - DEBUG(stderr, "Too short (%lu) %s\n", len, filename); s = NULL; + DEBUG(stderr, "Too short (%lu) %s\n", len, filename); return 0; } @@ -362,8 +431,8 @@ extern "C" unsigned char afl_custom_queue_get(void *data, if (((len * AFL_TXT_MIN_PERCENT) / 100) > valid_chars) { file_mapping[fn] = NULL; - DEBUG(stderr, "Not text (%lu) %s\n", len, filename); s = NULL; + DEBUG(stderr, "Not text (%lu) %s\n", len, filename); return 0; } @@ -766,6 +835,15 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } + if (tokens.size() < AUTOTOKENS_SIZE_MIN) { + + file_mapping[fn] = NULL; + s = NULL; + DEBUG(stderr, "too few tokens\n"); + return 0; + + } + /* Now we transform the tokens into an ID list and saved that */ structure = new vector(); @@ -791,8 +869,9 @@ extern "C" unsigned char afl_custom_queue_get(void *data, // save the token structure to the file mapping file_mapping[fn] = structure; - s = structure; + id_mapping[valid_structures] = structure; ++valid_structures; + s = structure; all_structure_items += structure->size(); // we are done! @@ -897,6 +976,12 @@ extern "C" my_mutator_t *afl_custom_init(afl_state *afl, unsigned int seed) { } +extern "C" void afl_custom_splice_optout(my_mutator_t *data) { + + (void)(data); + +} + extern "C" void afl_custom_deinit(my_mutator_t *data) { /* we use this to print statistics at exit :-) diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 4ffeda7a..322caa5b 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -48,6 +48,7 @@ C/C++: ```c void *afl_custom_init(afl_state_t *afl, unsigned int seed); unsigned int afl_custom_fuzz_count(void *data, const unsigned char *buf, size_t buf_size); +void afl_custom_splice_optout(void *data); size_t afl_custom_fuzz(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf, unsigned char *add_buf, size_t add_buf_size, size_t max_size); const char *afl_custom_describe(void *data, size_t max_description_len); size_t afl_custom_post_process(void *data, unsigned char *buf, size_t buf_size, unsigned char **out_buf); @@ -72,6 +73,9 @@ def init(seed): def fuzz_count(buf): return cnt +def splice_optout() + pass + def fuzz(buf, add_buf, max_size): return mutated_out @@ -132,6 +136,13 @@ def deinit(): # optional for Python for a specific queue entry, use this function. This function is most useful if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. +- `splice_optout` (optional): + + If this function is present, no splicing target is passed to the `fuzz` + function. This saves time if splicing data is not needed by the custom + fuzzing function. + This function is never called, just needs to be present to activate. + - `fuzz` (optional): This method performs custom mutations on a given input. It also accepts an diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 69fea579..1e8d085d 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -344,6 +344,7 @@ enum { /* 12 */ PY_FUNC_INTROSPECTION, /* 13 */ PY_FUNC_DESCRIBE, /* 14 */ PY_FUNC_FUZZ_SEND, + /* 15 */ PY_FUNC_SPLICE_OPTOUT, PY_FUNC_COUNT }; @@ -495,6 +496,7 @@ typedef struct afl_state { no_unlink, /* do not unlink cur_input */ debug, /* Debug mode */ custom_only, /* Custom mutator only mode */ + custom_splice_optout, /* Custom mutator no splice buffer */ is_main_node, /* if this is the main node */ is_secondary_node, /* if this is a secondary instance */ pizza_is_served; /* pizza mode */ @@ -828,6 +830,17 @@ struct custom_mutator { */ u32 (*afl_custom_fuzz_count)(void *data, const u8 *buf, size_t buf_size); + /** + * Opt-out of a splicing input for the fuzz mutator + * + * Empty dummy function. It's presence tells afl-fuzz not to pass a + * splice data pointer and len. + * + * @param data pointer returned in afl_custom_init by this custom mutator + * @noreturn + */ + void (*afl_custom_splice_optout)(void *data); + /** * Perform custom mutations on a given input * @@ -1057,6 +1070,7 @@ u8 havoc_mutation_probability_py(void *); u8 queue_get_py(void *, const u8 *); const char *introspection_py(void *); u8 queue_new_entry_py(void *, const u8 *, const u8 *); +void splice_optout(void *); void deinit_py(void *); #endif diff --git a/src/afl-fuzz-mutators.c b/src/afl-fuzz-mutators.c index 22e5262e..ce43064a 100644 --- a/src/afl-fuzz-mutators.c +++ b/src/afl-fuzz-mutators.c @@ -358,6 +358,19 @@ struct custom_mutator *load_custom_mutator(afl_state_t *afl, const char *fn) { } + /* "afl_custom_splice_optout", optional, never called */ + mutator->afl_custom_splice_optout = dlsym(dh, "afl_custom_splice_optout"); + if (!mutator->afl_custom_splice_optout) { + + ACTF("optional symbol 'afl_custom_splice_optout' not found."); + + } else { + + OKF("Found 'afl_custom_splice_optout'."); + afl->custom_splice_optout = 1; + + } + /* "afl_custom_fuzz_send", optional */ mutator->afl_custom_fuzz_send = dlsym(dh, "afl_custom_fuzz_send"); if (!mutator->afl_custom_fuzz_send) { diff --git a/src/afl-fuzz-one.c b/src/afl-fuzz-one.c index eaf65987..5e352dcb 100644 --- a/src/afl-fuzz-one.c +++ b/src/afl-fuzz-one.c @@ -1954,7 +1954,8 @@ custom_mutator_stage: u32 target_len = 0; /* check if splicing makes sense yet (enough entries) */ - if (likely(afl->ready_for_splicing_count > 1)) { + if (likely(!afl->custom_splice_optout && + afl->ready_for_splicing_count > 1)) { /* Pick a random other queue entry for passing to external API that has the necessary length */ diff --git a/src/afl-fuzz-python.c b/src/afl-fuzz-python.c index b509b936..69c305f7 100644 --- a/src/afl-fuzz-python.c +++ b/src/afl-fuzz-python.c @@ -248,6 +248,8 @@ static py_mutator_t *init_py_module(afl_state_t *afl, u8 *module_name) { PyObject_GetAttrString(py_module, "queue_get"); py_functions[PY_FUNC_FUZZ_SEND] = PyObject_GetAttrString(py_module, "fuzz_send"); + py_functions[PY_FUNC_SPLICE_OPTOUT] = + PyObject_GetAttrString(py_module, "splice_optout"); py_functions[PY_FUNC_QUEUE_NEW_ENTRY] = PyObject_GetAttrString(py_module, "queue_new_entry"); py_functions[PY_FUNC_INTROSPECTION] = @@ -394,6 +396,13 @@ void deinit_py(void *py_mutator) { } +void splice_optout_py(void *py_mutator) { + + // this is never called + (void)(py_mutator); + +} + struct custom_mutator *load_custom_mutator_py(afl_state_t *afl, char *module_name) { @@ -474,6 +483,13 @@ struct custom_mutator *load_custom_mutator_py(afl_state_t *afl, } + if (py_functions[PY_FUNC_SPLICE_OPTOUT]) { + + mutator->afl_custom_splice_optout = splice_optout_py; + afl->custom_splice_optout = 1; + + } + if (py_functions[PY_FUNC_QUEUE_NEW_ENTRY]) { mutator->afl_custom_queue_new_entry = queue_new_entry_py; -- cgit 1.4.1 From eeca3a0b2939c605497e9b3a615ee4a466f4a3f2 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 19 Jan 2023 11:52:19 +0100 Subject: lots of fixes --- custom_mutators/autotokens/TODO | 2 +- custom_mutators/autotokens/autotokens.cpp | 424 +++++++++++++++++++----------- docs/custom_mutators.md | 1 + include/afl-fuzz.h | 11 +- src/afl-fuzz-one.c | 3 +- 5 files changed, 279 insertions(+), 162 deletions(-) (limited to 'docs') diff --git a/custom_mutators/autotokens/TODO b/custom_mutators/autotokens/TODO index 95b79373..2e39511c 100644 --- a/custom_mutators/autotokens/TODO +++ b/custom_mutators/autotokens/TODO @@ -3,4 +3,4 @@ cmplog: only add tokens that were found to fit? create from thin air if no good seed after a cycle and dict large enough? (static u32 no_of_struct_inputs;) -splicing -> check if whitespace/token is needed \ No newline at end of file +splicing -> check if whitespace/token is needed diff --git a/custom_mutators/autotokens/autotokens.cpp b/custom_mutators/autotokens/autotokens.cpp index 4f3289c9..102bea0f 100644 --- a/custom_mutators/autotokens/autotokens.cpp +++ b/custom_mutators/autotokens/autotokens.cpp @@ -38,8 +38,10 @@ typedef struct my_mutator { } my_mutator_t; -#define DEBUG \ +#undef DEBUGF +#define DEBUGF \ if (unlikely(debug)) fprintf +#define IFDEBUG if (unlikely(debug)) static afl_state *afl_ptr; static int debug = AUTOTOKENS_DEBUG; @@ -57,12 +59,12 @@ static unordered_map *> id_mapping; static unordered_map token_to_id; static unordered_map id_to_token; static string whitespace = AUTOTOKENS_WHITESPACE; +static string output; static regex *regex_comment_custom; -static regex regex_comment_star("/\\*([:print:]|\n)*?\\*/", - regex::multiline | regex::optimize); -static regex regex_word("[A-Za-z0-9_$.-]+", regex::optimize); -static regex regex_whitespace(R"([ \t]+)", regex::optimize); -static regex regex_string("\"[[:print:]]*?\"|'[[:print:]]*?'", regex::optimize); +static regex regex_comment_star("/\\*([:print:]|\n)*?\\*/", + regex::multiline | regex::optimize); +static regex regex_word("[A-Za-z0-9_$.-]+", regex::optimize); +static regex regex_whitespace(R"([ \t]+)", regex::optimize); static vector *s; // the structure of the currently selected input u32 good_whitespace_or_singleval() { @@ -104,7 +106,7 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, MAX(AUTOTOKENS_CHANGE_MIN, MIN(m_size >> 3, HAVOC_CYCLES * afl_ptr->queue_cur->perf_score * afl_ptr->havoc_div / 256)); - // DEBUG(stderr, "structure size: %lu, rounds: %u \n", m.size(), rounds); + // DEBUGF(stderr, "structure size: %lu, rounds: %u \n", m.size(), rounds); #if AUTOTOKENS_SPLICE_DISABLE == 1 #define AUTOTOKENS_MUT_MAX 12 @@ -112,7 +114,7 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, #define AUTOTOKENS_MUT_MAX 14 #endif - u32 max_rand = AUTOTOKENS_MUT_MAX; + u32 max_rand = AUTOTOKENS_MUT_MAX, new_item, pos; for (i = 0; i < rounds; ++i) { @@ -122,8 +124,8 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, case 0 ... 7: /* fall through */ { - u32 pos = rand_below(afl_ptr, m_size); - u32 cur_item = m[pos], new_item; + pos = rand_below(afl_ptr, m_size); + u32 cur_item = m[pos]; do { new_item = rand_below(afl_ptr, current_id); @@ -135,7 +137,7 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, ((whitespace_ids < new_item && whitespace_ids >= cur_item) || (whitespace_ids >= new_item && whitespace_ids < cur_item))))); - DEBUG(stderr, "MUT: %u -> %u\n", cur_item, new_item); + DEBUGF(stderr, "MUT: %u -> %u\n", cur_item, new_item); m[pos] = new_item; break; @@ -144,7 +146,6 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, /* INSERT (m_size +1 so we insert also after last place) */ case 8 ... 9: { - u32 new_item; do { new_item = rand_below(afl_ptr, current_id); @@ -154,7 +155,7 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, u32 pos = rand_below(afl_ptr, m_size + 1); m.insert(m.begin() + pos, new_item); ++m_size; - DEBUG(stderr, "INS: %u at %u\n", new_item, pos); + DEBUGF(stderr, "INS: %u at %u\n", new_item, pos); if (likely(!alternative_tokenize)) { @@ -212,7 +213,8 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, m.insert(m.begin() + dst_off, src->begin() + src_off, src->begin() + src_off + n); m_size += n; - DEBUG(stderr, "SPLICE-INS: %u at %u\n", n, dst_off); + DEBUGF(stderr, "SPLICE-INS: %u at %u\n", n, dst_off); + break; } @@ -231,13 +233,36 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, copy(src->begin() + src_off, src->begin() + src_off + n, m.begin() + dst_off); - DEBUG(stderr, "SPLICE-MUT: %u at %u\n", n, dst_off); + DEBUGF(stderr, "SPLICE-MUT: %u at %u\n", n, dst_off); break; } } + if (likely(!alternative_tokenize)) { + + // do we need a whitespace/token at the beginning? + if (dst_off && id_to_token[m[dst_off - 1]].size() > 1 && + id_to_token[m[dst_off]].size() > 1) { + + m.insert(m.begin() + dst_off, good_whitespace_or_singleval()); + ++m_size; + + } + + // do we need a whitespace/token at the end? + if (dst_off + n < m_size && + id_to_token[m[dst_off + n - 1]].size() > 1 && + id_to_token[m[dst_off + n]].size() > 1) { + + m.insert(m.begin() + dst_off + n, good_whitespace_or_singleval()); + ++m_size; + + } + + } + break; } @@ -249,11 +274,32 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, if (m_size > 8) { - m.erase(m.begin() + rand_below(afl_ptr, m_size)); - --m_size; + do { + + pos = rand_below(afl_ptr, m_size); + + } while (unlikely(pos < whitespace_ids)); + + // if what we delete will result in a missing whitespace/token, + // instead of deleting we switch the item to a whitespace or token. + if (likely(!alternative_tokenize) && pos && pos < m_size && + id_to_token[m[pos - 1]].size() > 1 && + id_to_token[m[pos + 1]].size() > 1) { + + m[pos] = good_whitespace_or_singleval(); + + } else { + + m.erase(m.begin() + pos); + --m_size; + + } } else { + // if the data is already too small do not try to make it smaller + // again this run. + max_rand = AUTOTOKENS_MUT_MAX - 2; } @@ -262,14 +308,12 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, } - // TODO: add full line insert splice, replace splace, delete - } } - string output; - u32 m_size_1 = m_size - 1; + u32 m_size_1 = m_size - 1; + output = ""; for (i = 0; i < m_size; ++i) { @@ -282,31 +326,108 @@ extern "C" size_t afl_custom_fuzz(my_mutator_t *data, u8 *buf, size_t buf_size, } - u32 mutated_size = output.size(); - u8 *mutated_out = (u8 *)afl_realloc((void **)out_buf, mutated_size); + u32 mutated_size = (u32)output.size(); + u8 *mutated_out = (u8 *)output.data(); - if (unlikely(!mutated_out)) { + if (unlikely(mutated_size > max_size)) { mutated_size = max_size; } - *out_buf = NULL; - return 0; - - } - - if (unlikely(debug)) { + IFDEBUG { - DEBUG(stderr, "MUTATED to %u bytes:\n", mutated_size); + DEBUGF(stderr, "MUTATED to %u bytes:\n", mutated_size); fwrite(output.data(), 1, mutated_size, stderr); - DEBUG(stderr, "\n---\n"); + DEBUGF(stderr, "\n---\n"); } - memcpy(mutated_out, output.data(), mutated_size); *out_buf = mutated_out; ++fuzz_count; return mutated_size; } +/* I get f*cking stack overflow using C++ regex with a regex of + "\"[[:print:]]*?\"" if this matches a long string even with regex::optimize + enabled :-( */ +u8 my_search_string(string::const_iterator cur, string::const_iterator ende, + string::const_iterator *match_begin, + string::const_iterator *match_end) { + + string::const_iterator start = cur, found_begin; + u8 quote_type = 0; + + while (cur < ende) { + + switch (*cur) { + + case '"': { + + if (cur == start || *(cur - 1) != '\\') { + + if (!quote_type) { + + found_begin = cur; + quote_type = 1; + + } else if (quote_type == 1) { + + *match_begin = found_begin; + *match_end = cur + 1; + return 1; + + } + + } + + break; + + } + + case '\'': { + + if (cur == start || *(cur - 1) != '\\') { + + if (!quote_type) { + + found_begin = cur; + quote_type = 2; + + } else if (quote_type == 2) { + + *match_begin = found_begin; + *match_end = cur + 1; + return 1; + + } + + } + + break; + + } + + case '\n': + case '\r': + case 0: { + + quote_type = 0; + break; + + } + + default: + if (unlikely(quote_type && !isprint(*cur))) { quote_type = 0; } + break; + + } + + ++cur; + + } + + return 0; + +} + /* We are not using afl_custom_queue_new_entry() because not every corpus entry will be necessarily fuzzed. so we use afl_custom_queue_get() instead */ @@ -321,7 +442,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, (only_fav && !afl_ptr->queue_cur->favored)) { s = NULL; - DEBUG(stderr, "cmplog not ascii or only_fav and not favorite\n"); + DEBUGF(stderr, "cmplog not ascii or only_fav and not favorite\n"); return 0; } @@ -356,7 +477,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } ++extras_cnt; - DEBUG(stderr, "Added from dictionary: \"%s\"\n", ptr); + DEBUGF(stderr, "Added from dictionary: \"%s\"\n", ptr); } @@ -385,7 +506,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } ++a_extras_cnt; - DEBUG(stderr, "Added from auto dictionary: \"%s\"\n", ptr); + DEBUGF(stderr, "Added from auto dictionary: \"%s\"\n", ptr); } @@ -415,7 +536,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, fclose(fp); file_mapping[fn] = structure; // NULL ptr so we don't read the file again s = NULL; - DEBUG(stderr, "Too short (%lu) %s\n", len, filename); + DEBUGF(stderr, "Too short (%lu) %s\n", len, filename); return 0; } @@ -443,14 +564,14 @@ extern "C" unsigned char afl_custom_queue_get(void *data, file_mapping[fn] = NULL; s = NULL; - DEBUG(stderr, "Not text (%lu) %s\n", len, filename); + DEBUGF(stderr, "Not text (%lu) %s\n", len, filename); return 0; } } - // DEBUG(stderr, "Read %lu bytes for %s\nBefore comment trim:\n%s\n", + // DEBUGF(stderr, "Read %lu bytes for %s\nBefore comment trim:\n%s\n", // input.size(), filename, input.c_str()); if (regex_comment_custom) { @@ -463,15 +584,15 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - DEBUG(stderr, "After replace %lu bytes for %s\n%s\n", input.size(), - filename, input.c_str()); + DEBUGF(stderr, "After replace %lu bytes for %s\n%s\n", input.size(), + filename, input.c_str()); u32 spaces = count(input.begin(), input.end(), ' '); u32 tabs = count(input.begin(), input.end(), '\t'); u32 linefeeds = count(input.begin(), input.end(), '\n'); bool ends_with_linefeed = input[input.length() - 1] == '\n'; - DEBUG(stderr, "spaces=%u tabs=%u linefeeds=%u ends=%u\n", spaces, tabs, - linefeeds, ends_with_linefeed); + DEBUGF(stderr, "spaces=%u tabs=%u linefeeds=%u ends=%u\n", spaces, tabs, + linefeeds, ends_with_linefeed); all_spaces += spaces; all_tabs += tabs; all_lf += linefeeds; @@ -479,25 +600,28 @@ extern "C" unsigned char afl_custom_queue_get(void *data, // now extract all tokens vector tokens; - smatch match; - string::const_iterator cur = input.begin(), ende = input.end(), found, prev; + string::const_iterator cur = input.begin(), ende = input.end(), found, prev, + match_begin, match_end; - DEBUG(stderr, "START!\n"); + DEBUGF(stderr, "START!\n"); if (likely(!alternative_tokenize)) { - while (regex_search(cur, ende, match, regex_string, - regex_constants::match_any | - regex_constants::match_not_null | - regex_constants::match_continuous)) { + while (my_search_string(cur, ende, &match_begin, &match_end)) { prev = cur; - found = match[0].first; - cur = match[0].second; - DEBUG(stderr, - "string %s found at start %lu offset %lu continue at %lu\n", - match[0].str().c_str(), prev - input.begin(), match.position(), - cur - input.begin()); + found = match_begin; + cur = match_end; + + IFDEBUG { + + string foo(match_begin, match_end); + DEBUGF(stderr, + "string %s found at start %lu offset %lu continue at %lu\n", + foo.c_str(), prev - input.begin(), found - prev, + cur - input.begin()); + + } if (prev < found) { // there are items between search start and find while (prev < found) { @@ -512,8 +636,8 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } tokens.push_back(std::string(start, prev)); - DEBUG(stderr, "WHITESPACE %ld \"%s\"\n", prev - start, - tokens[tokens.size() - 1].c_str()); + DEBUGF(stderr, "WHITESPACE %ld \"%s\"\n", prev - start, + tokens[tokens.size() - 1].c_str()); } else if (isalnum(*prev) || *prev == '$' || *prev == '_') { @@ -525,14 +649,14 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - tokens.push_back(std::string(start, prev)); - DEBUG(stderr, "IDENTIFIER %ld \"%s\"\n", prev - start, - tokens[tokens.size() - 1].c_str()); + tokens.push_back(string(start, prev)); + DEBUGF(stderr, "IDENTIFIER %ld \"%s\"\n", prev - start, + tokens[tokens.size() - 1].c_str()); } else { - tokens.push_back(std::string(prev, prev + 1)); - DEBUG(stderr, "OTHER \"%c\"\n", *prev); + tokens.push_back(string(prev, prev + 1)); + DEBUGF(stderr, "OTHER \"%c\"\n", *prev); ++prev; } @@ -541,11 +665,12 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - if (match[0].length() > 0) { tokens.push_back(match[0]); } + tokens.push_back(string(match_begin, match_end)); + DEBUGF(stderr, "TOK: %s\n", tokens[tokens.size() - 1].c_str()); } - DEBUG(stderr, "AFTER all strings\n"); + DEBUGF(stderr, "AFTER all strings\n"); if (cur < ende) { @@ -561,8 +686,8 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } tokens.push_back(std::string(start, cur)); - DEBUG(stderr, "WHITESPACE %ld \"%s\"\n", cur - start, - tokens[tokens.size() - 1].c_str()); + DEBUGF(stderr, "WHITESPACE %ld \"%s\"\n", cur - start, + tokens[tokens.size() - 1].c_str()); } else if (isalnum(*cur) || *cur == '$' || *cur == '_') { @@ -575,13 +700,13 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } tokens.push_back(std::string(start, cur)); - DEBUG(stderr, "IDENTIFIER %ld \"%s\"\n", cur - start, - tokens[tokens.size() - 1].c_str()); + DEBUGF(stderr, "IDENTIFIER %ld \"%s\"\n", cur - start, + tokens[tokens.size() - 1].c_str()); } else { tokens.push_back(std::string(cur, cur + 1)); - DEBUG(stderr, "OTHER \"%c\"\n", *cur); + DEBUGF(stderr, "OTHER \"%c\"\n", *cur); ++cur; } @@ -593,19 +718,21 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } else { // alternative tokenize - - while (regex_search(cur, ende, match, regex_string, - regex_constants::match_any | - regex_constants::match_not_null | - regex_constants::match_continuous)) { + while (my_search_string(cur, ende, &match_begin, &match_end)) { prev = cur; - found = match[0].first; - cur = match[0].second; - DEBUG(stderr, - "string %s found at start %lu offset %lu continue at %lu\n", - match[0].str().c_str(), prev - input.begin(), match.position(), - cur - input.begin()); + found = match_begin; + cur = match_end; + IFDEBUG { + + string foo(match_begin, match_end); + DEBUGF(stderr, + "string %s found at start %lu offset %lu continue at %lu\n", + foo.c_str(), prev - input.begin(), found - prev, + cur - input.begin()); + + } + if (prev < found) { // there are items between search start and find sregex_token_iterator it{prev, found, regex_whitespace, -1}; vector tokenized{it, {}}; @@ -619,10 +746,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, tokenized.end()); tokens.reserve(tokens.size() + tokenized.size() * 2 + 1); - if (unlikely(debug)) { + IFDEBUG { - DEBUG(stderr, "tokens: %lu input size: %lu\n", tokenized.size(), - input.size()); + DEBUGF(stderr, "tokens1: %lu input size: %lu\n", tokenized.size(), + input.size()); for (auto x : tokenized) { cerr << x << endl; @@ -636,10 +763,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, string::const_iterator c = token.begin(), e = token.end(), f, p; smatch m; - while (regex_search(c, e, m, regex_word, - regex_constants::match_any | - regex_constants::match_not_null | - regex_constants::match_continuous)) { + while (regex_search(c, e, m, regex_word)) { p = c; f = m[0].first; @@ -649,10 +773,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, // there are items between search start and find while (p < f) { - if (unlikely(debug)) { + IFDEBUG { string foo(p, p + 1); - DEBUG(stderr, "before string: \"%s\"\n", foo.c_str()); + DEBUGF(stderr, "before string: \"%s\"\n", foo.c_str()); } @@ -661,20 +785,21 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - /* - string foo(p, f); - DEBUG(stderr, "before string: \"%s\"\n", - foo.c_str()); tokens.push_back(std::string(p, f)); - */ + IFDEBUG { + + string foo(p, f); + DEBUGF(stderr, "before string: \"%s\"\n", foo.c_str()); + tokens.push_back(std::string(p, f)); + + } } - DEBUG( - stderr, - "SUBstring \"%s\" found at start %lu offset %lu continue at " - "%lu\n", - m[0].str().c_str(), p - input.begin(), m.position(), - c - token.begin()); + DEBUGF(stderr, + "SUBstring \"%s\" found at start %lu offset %lu continue " + "at %lu\n", + m[0].str().c_str(), p - input.begin(), m.position(), + c - token.begin()); tokens.push_back(m[0].str()); } @@ -683,10 +808,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, while (c < e) { - if (unlikely(debug)) { + IFDEBUG { string foo(c, c + 1); - DEBUG(stderr, "after string: \"%s\"\n", foo.c_str()); + DEBUGF(stderr, "after string: \"%s\"\n", foo.c_str()); } @@ -695,17 +820,14 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - /* - if (unlikely(debug)) { + IFDEBUG { - string foo(c, e); - DEBUG(stderr, "after string: \"%s\"\n", - foo.c_str()); + string foo(c, e); + DEBUGF(stderr, "after string: \"%s\"\n", foo.c_str()); - } + } - tokens.push_back(std::string(c, e)); - */ + tokens.push_back(std::string(c, e)); } @@ -713,7 +835,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - if (match[0].length() > 0) { tokens.push_back(match[0]); } + tokens.push_back(string(match_begin, match_end)); } @@ -727,10 +849,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, tokenized.end()); tokens.reserve(tokens.size() + tokenized.size() * 2 + 1); - if (unlikely(debug)) { + IFDEBUG { - DEBUG(stderr, "tokens: %lu input size: %lu\n", tokenized.size(), - input.size()); + DEBUGF(stderr, "tokens2: %lu input size: %lu\n", tokenized.size(), + input.size()); for (auto x : tokenized) { cerr << x << endl; @@ -744,10 +866,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, string::const_iterator c = token.begin(), e = token.end(), f, p; smatch m; - while (regex_search(c, e, m, regex_word, - regex_constants::match_any | - regex_constants::match_not_null | - regex_constants::match_continuous)) { + while (regex_search(c, e, m, regex_word)) { p = c; f = m[0].first; @@ -757,10 +876,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, // there are items between search start and find while (p < f) { - if (unlikely(debug)) { + IFDEBUG { string foo(p, p + 1); - DEBUG(stderr, "before string: \"%s\"\n", foo.c_str()); + DEBUGF(stderr, "before string: \"%s\"\n", foo.c_str()); } @@ -769,25 +888,22 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - /* - if (unlikely(debug)) { + IFDEBUG { - string foo(p, f); - DEBUG(stderr, "before string: \"%s\"\n", - foo.c_str()); + string foo(p, f); + DEBUGF(stderr, "before string: \"%s\"\n", foo.c_str()); - } + } - tokens.push_back(std::string(p, f)); - */ + tokens.push_back(std::string(p, f)); } - DEBUG(stderr, - "SUB2string \"%s\" found at start %lu offset %lu continue at " - "%lu\n", - m[0].str().c_str(), p - input.begin(), m.position(), - c - token.begin()); + DEBUGF(stderr, + "SUB2string \"%s\" found at start %lu offset %lu continue " + "at %lu\n", + m[0].str().c_str(), p - input.begin(), m.position(), + c - token.begin()); tokens.push_back(m[0].str()); } @@ -796,10 +912,10 @@ extern "C" unsigned char afl_custom_queue_get(void *data, while (c < e) { - if (unlikely(debug)) { + IFDEBUG { string foo(c, c + 1); - DEBUG(stderr, "after string: \"%s\"\n", foo.c_str()); + DEBUGF(stderr, "after string: \"%s\"\n", foo.c_str()); } @@ -808,16 +924,14 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - /* - if (unlikely(debug)) { + IFDEBUG { - string foo(c, e); - DEBUG(stderr, "after string: \"%s\"\n", foo.c_str()); + string foo(c, e); + DEBUGF(stderr, "after string: \"%s\"\n", foo.c_str()); - } + } - tokens.push_back(std::string(c, e)); - */ + tokens.push_back(std::string(c, e)); } @@ -827,22 +941,22 @@ extern "C" unsigned char afl_custom_queue_get(void *data, } - if (unlikely(debug)) { + IFDEBUG { - DEBUG(stderr, "DUMPING TOKENS:\n"); + DEBUGF(stderr, "DUMPING TOKENS:\n"); u32 size_1 = tokens.size() - 1; for (u32 i = 0; i < tokens.size(); ++i) { - DEBUG(stderr, "%s", tokens[i].c_str()); + DEBUGF(stderr, "%s", tokens[i].c_str()); if (unlikely(alternative_tokenize && i < size_1)) { - DEBUG(stderr, "%s", whitespace.c_str()); + DEBUGF(stderr, "%s", whitespace.c_str()); } } - DEBUG(stderr, "---------------------------\n"); + DEBUGF(stderr, "---------------------------\n"); } @@ -850,7 +964,7 @@ extern "C" unsigned char afl_custom_queue_get(void *data, file_mapping[fn] = NULL; s = NULL; - DEBUG(stderr, "too few tokens\n"); + DEBUGF(stderr, "too few tokens\n"); return 0; } @@ -886,21 +1000,23 @@ extern "C" unsigned char afl_custom_queue_get(void *data, all_structure_items += structure->size(); // we are done! - DEBUG(stderr, "DONE! We have %lu tokens in the structure\n", - structure->size()); + DEBUGF(stderr, "DONE! We have %lu tokens in the structure\n", + structure->size()); + + } - } else { + else { if (entry->second == NULL) { - DEBUG(stderr, "Skipping %s\n", filename); + DEBUGF(stderr, "Skipping %s\n", filename); s = NULL; return 0; } s = entry->second; - DEBUG(stderr, "OK %s\n", filename); + DEBUGF(stderr, "OK %s\n", filename); } diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 322caa5b..82131c92 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -150,6 +150,7 @@ def deinit(): # optional for Python sense to use it. You would only skip this if `post_process` is used to fix checksums etc. so if you are using it, e.g., as a post processing library. Note that a length > 0 *must* be returned! + The returned output buffer is under **your** memory management! - `describe` (optional): diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 1e8d085d..229bc025 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -844,15 +844,16 @@ struct custom_mutator { /** * Perform custom mutations on a given input * - * (Optional for now. Required in the future) + * (Optional) * - * @param data pointer returned in afl_custom_init by this custom mutator + * Getting an add_buf can be skipped by using afl_custom_splice_optout(). + * + * @param[in] data Pointer returned in afl_custom_init by this custom mutator * @param[in] buf Pointer to the input data to be mutated and the mutated * output * @param[in] buf_size Size of the input/output data - * @param[out] out_buf the new buffer. We may reuse *buf if large enough. - * *out_buf = NULL is treated as FATAL. - * @param[in] add_buf Buffer containing the additional test case + * @param[out] out_buf The new buffer, under your memory mgmt. + * @param[in] add_buf Buffer containing an additional test case (splicing) * @param[in] add_buf_size Size of the additional test case * @param[in] max_size Maximum size of the mutated output. The mutation must * not produce data larger than max_size. diff --git a/src/afl-fuzz-one.c b/src/afl-fuzz-one.c index 5e352dcb..bd482562 100644 --- a/src/afl-fuzz-one.c +++ b/src/afl-fuzz-one.c @@ -564,8 +564,7 @@ u8 fuzz_one_original(afl_state_t *afl) { if (afl->cmplog_lvl == 3 || (afl->cmplog_lvl == 2 && afl->queue_cur->tc_ref) || afl->queue_cur->favored || - !(afl->fsrv.total_execs % afl->queued_items) || - get_cur_time() - afl->last_find_time > 300000) { // 300 seconds + get_cur_time() - afl->last_find_time > 600000) { // 600 seconds if (input_to_state_stage(afl, in_buf, out_buf, len)) { -- cgit 1.4.1 From 4063a3eb4c4099e37aef4f1d96e8b80d58d65fe2 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 23 Jan 2023 12:50:57 +0100 Subject: nit --- docs/Changelog.md | 4 ++++ src/afl-cc.c | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 7a9b74c5..fb573c73 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -3,6 +3,10 @@ This is the list of all noteworthy changes made in every public release of the tool. See README.md for the general instruction manual. +### Version ++4.06a (dev) + - afl-cc: + - add CFI sanitizer variant to gcc targets + ### Version ++4.05c (release) - MacOS: libdislocator, libtokencap etc. do not work with modern MacOS anymore, but could be patched to work, see this issue if you diff --git a/src/afl-cc.c b/src/afl-cc.c index cbf57047..7c3682fb 100644 --- a/src/afl-cc.c +++ b/src/afl-cc.c @@ -1052,7 +1052,7 @@ static void edit_params(u32 argc, char **argv, char **envp) { if (compiler_mode == GCC_PLUGIN || compiler_mode == GCC) { - cc_params[cc_par_cnt++] = "-fcf-protection"; + cc_params[cc_par_cnt++] = "-fcf-protection=full"; } else { -- cgit 1.4.1 From f4a13585a1a205798093291fd04659a4158b4d50 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 26 Jan 2023 12:21:47 +0100 Subject: better asan defaults everwhere --- docs/Changelog.md | 1 + include/common.h | 1 + src/afl-analyze.c | 84 +--------------------------- src/afl-common.c | 57 +++++++++++++++++++ src/afl-forkserver.c | 54 +----------------- src/afl-showmap.c | 45 +-------------- src/afl-tmin.c | 83 +-------------------------- utils/afl_network_proxy/afl-network-server.c | 17 +----- 8 files changed, 68 insertions(+), 274 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index fb573c73..434bc101 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -24,6 +24,7 @@ - `-t none` now translates to `-t 120000` (120 seconds) - unicorn_mode updated - updated rust custom mutator dependencies and LibAFL custom mutator + - overall better sanitizer default setting handling - several minor bugfixes ### Version ++4.04c (release) diff --git a/include/common.h b/include/common.h index b5dbc6de..c5a32cdb 100644 --- a/include/common.h +++ b/include/common.h @@ -43,6 +43,7 @@ u32 check_binary_signatures(u8 *fn); void detect_file_args(char **argv, u8 *prog_in, bool *use_stdin); void print_suggested_envs(char *mispelled_env); void check_environment_vars(char **env); +void set_sanitizer_defaults(); char **argv_cpy_dup(int argc, char **argv); void argv_cpy_free(char **argv); diff --git a/src/afl-analyze.c b/src/afl-analyze.c index da1def3b..d4a9aa91 100644 --- a/src/afl-analyze.c +++ b/src/afl-analyze.c @@ -656,28 +656,6 @@ static void set_up_environment(char **argv) { if (fsrv.out_fd < 0) { PFATAL("Unable to create '%s'", fsrv.out_file); } /* Set sane defaults... */ - - x = get_afl_env("ASAN_OPTIONS"); - - if (x) { - - if (!strstr(x, "abort_on_error=1")) { - - FATAL("Custom ASAN_OPTIONS set without abort_on_error=1 - please fix!"); - - } - -#ifndef ASAN_BUILD - if (!getenv("AFL_DEBUG") && !strstr(x, "symbolize=0")) { - - FATAL("Custom ASAN_OPTIONS set without symbolize=0 - please fix!"); - - } - -#endif - - } - x = get_afl_env("MSAN_OPTIONS"); if (x) { @@ -689,69 +667,9 @@ static void set_up_environment(char **argv) { } - if (!strstr(x, "symbolize=0")) { - - FATAL("Custom MSAN_OPTIONS set without symbolize=0 - please fix!"); - - } - - } - - x = get_afl_env("LSAN_OPTIONS"); - - if (x) { - - if (!strstr(x, "symbolize=0")) { - - FATAL("Custom LSAN_OPTIONS set without symbolize=0 - please fix!"); - - } - } - setenv("ASAN_OPTIONS", - "abort_on_error=1:" - "detect_leaks=0:" - "allocator_may_return_null=1:" - "detect_odr_violation=0:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("UBSAN_OPTIONS", - "halt_on_error=1:" - "abort_on_error=1:" - "malloc_context_size=0:" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("MSAN_OPTIONS", "exit_code=" STRINGIFY(MSAN_ERROR) ":" - "abort_on_error=1:" - "msan_track_origins=0" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", 0); - - setenv("LSAN_OPTIONS", - "exitcode=" STRINGIFY(LSAN_ERROR) ":" - "fast_unwind_on_malloc=0:" - "symbolize=0:" - "print_suppressions=0", - 0); + set_sanitizer_defaults(); if (get_afl_env("AFL_PRELOAD")) { diff --git a/src/afl-common.c b/src/afl-common.c index 211d5bf2..d83130b4 100644 --- a/src/afl-common.c +++ b/src/afl-common.c @@ -58,6 +58,63 @@ u8 last_intr = 0; #define AFL_PATH "/usr/local/lib/afl/" #endif +void set_sanitizer_defaults() { + + /* Set sane defaults for ASAN if nothing else is specified. */ + u8 *have_asan_options = getenv("ASAN_OPTIONS"); + u8 *have_ubsan_options = getenv("UBSAN_OPTIONS"); + u8 *have_msan_options = getenv("MSAN_OPTIONS"); + u8 *have_lsan_options = getenv("LSAN_OPTIONS"); + u8 have_san_options = 0; + if (have_asan_options || have_ubsan_options || have_msan_options || + have_lsan_options) + have_san_options = 1; + u8 default_options[1024] = + "detect_odr_violation=0:abort_on_error=1:symbolize=0:malloc_context_" + "size=0:allocator_may_return_null=1:handle_segv=0:handle_sigbus=0:" + "handle_abort=0:handle_sigfpe=0:handle_sigill=0:"; + + if (!have_lsan_options) strcat(default_options, "detect_leaks=0:"); + + /* Set sane defaults for ASAN if nothing else is specified. */ + + if (!have_san_options) setenv("ASAN_OPTIONS", default_options, 1); + + /* Set sane defaults for UBSAN if nothing else is specified. */ + + if (!have_san_options) setenv("UBSAN_OPTIONS", default_options, 1); + + /* MSAN is tricky, because it doesn't support abort_on_error=1 at this + point. So, we do this in a very hacky way. */ + + if (!have_msan_options) { + + u8 buf[2048] = ""; + if (!have_san_options) strcpy(buf, default_options); + strcat(buf, "exit_code=" STRINGIFY(MSAN_ERROR) ":msan_track_origins=0:"); + setenv("MSAN_OPTIONS", buf, 1); + + } + + /* LSAN, too, does not support abort_on_error=1. (is this still true??) */ + + if (!have_lsan_options) { + + u8 buf[2048] = ""; + if (!have_san_options) strcpy(buf, default_options); + strcat(buf, + "exitcode=" STRINGIFY( + LSAN_ERROR) ":fast_unwind_on_malloc=0:print_suppressions=0:"); + setenv("LSAN_OPTIONS", buf, 1); + + } + + /* Envs for QASan */ + setenv("QASAN_MAX_CALL_STACK", "0", 0); + setenv("QASAN_SYMBOLIZE", "0", 0); + +} + u32 check_binary_signatures(u8 *fn) { int ret = 0, fd = open(fn, O_RDONLY); diff --git a/src/afl-forkserver.c b/src/afl-forkserver.c index ef2fa904..89d01460 100644 --- a/src/afl-forkserver.c +++ b/src/afl-forkserver.c @@ -688,58 +688,8 @@ void afl_fsrv_start(afl_forkserver_t *fsrv, char **argv, if (!getenv("LD_BIND_LAZY")) { setenv("LD_BIND_NOW", "1", 1); } - /* Set sane defaults for ASAN if nothing else is specified. */ - u8 *have_asan_options = getenv("ASAN_OPTIONS"); - u8 *have_ubsan_options = getenv("UBSAN_OPTIONS"); - u8 *have_msan_options = getenv("MSAN_OPTIONS"); - u8 *have_lsan_options = getenv("LSAN_OPTIONS"); - u8 have_san_options = 0; - if (have_asan_options || have_ubsan_options || have_msan_options || - have_lsan_options) - have_san_options = 1; - u8 default_options[1024] = - "detect_odr_violation=0:abort_on_error=1:symbolize=0:malloc_context_" - "size=0:allocator_may_return_null=1:handle_segv=0:handle_sigbus=0:" - "handle_abort=0:handle_sigfpe=0:handle_sigill=0:"; - - if (!have_lsan_options) strcat(default_options, "detect_leaks=0:"); - - /* Set sane defaults for ASAN if nothing else is specified. */ - - if (!have_san_options) setenv("ASAN_OPTIONS", default_options, 1); - - /* Set sane defaults for UBSAN if nothing else is specified. */ - - if (!have_san_options) setenv("UBSAN_OPTIONS", default_options, 1); - - /* MSAN is tricky, because it doesn't support abort_on_error=1 at this - point. So, we do this in a very hacky way. */ - - if (!have_msan_options) { - - u8 buf[2048] = ""; - if (!have_san_options) strcpy(buf, default_options); - strcat(buf, "exit_code=" STRINGIFY(MSAN_ERROR) ":msan_track_origins=0:"); - setenv("MSAN_OPTIONS", buf, 1); - - } - - /* LSAN, too, does not support abort_on_error=1. (is this still true??) */ - - if (!have_lsan_options) { - - u8 buf[2048] = ""; - if (!have_san_options) strcpy(buf, default_options); - strcat(buf, - "exitcode=" STRINGIFY( - LSAN_ERROR) ":fast_unwind_on_malloc=0:print_suppressions=0:"); - setenv("LSAN_OPTIONS", buf, 1); - - } - - /* Envs for QASan */ - setenv("QASAN_MAX_CALL_STACK", "0", 0); - setenv("QASAN_SYMBOLIZE", "0", 0); + /* Set sane defaults for sanitizers */ + set_sanitizer_defaults(); fsrv->init_child_func(fsrv, argv); diff --git a/src/afl-showmap.c b/src/afl-showmap.c index 4e019794..1e281d08 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -597,49 +597,8 @@ static void set_up_environment(afl_forkserver_t *fsrv, char **argv) { char *afl_preload; char *frida_afl_preload = NULL; - setenv("ASAN_OPTIONS", - "abort_on_error=1:" - "detect_leaks=0:" - "allocator_may_return_null=1:" - "symbolize=0:" - "detect_odr_violation=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("LSAN_OPTIONS", - "exitcode=" STRINGIFY(LSAN_ERROR) ":" - "fast_unwind_on_malloc=0:" - "symbolize=0:" - "print_suppressions=0", - 0); - - setenv("UBSAN_OPTIONS", - "halt_on_error=1:" - "abort_on_error=1:" - "malloc_context_size=0:" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("MSAN_OPTIONS", "exit_code=" STRINGIFY(MSAN_ERROR) ":" - "abort_on_error=1:" - "msan_track_origins=0" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", 0); + + set_sanitizer_defaults(); if (get_afl_env("AFL_PRELOAD")) { diff --git a/src/afl-tmin.c b/src/afl-tmin.c index 687bb0e7..12c5e0c9 100644 --- a/src/afl-tmin.c +++ b/src/afl-tmin.c @@ -674,27 +674,6 @@ static void set_up_environment(afl_forkserver_t *fsrv, char **argv) { /* Set sane defaults... */ - x = get_afl_env("ASAN_OPTIONS"); - - if (x) { - - if (!strstr(x, "abort_on_error=1")) { - - FATAL("Custom ASAN_OPTIONS set without abort_on_error=1 - please fix!"); - - } - -#ifndef ASAN_BUILD - if (!getenv("AFL_DEBUG") && !strstr(x, "symbolize=0")) { - - FATAL("Custom ASAN_OPTIONS set without symbolize=0 - please fix!"); - - } - -#endif - - } - x = get_afl_env("MSAN_OPTIONS"); if (x) { @@ -706,69 +685,9 @@ static void set_up_environment(afl_forkserver_t *fsrv, char **argv) { } - if (!strstr(x, "symbolize=0")) { - - FATAL("Custom MSAN_OPTIONS set without symbolize=0 - please fix!"); - - } - - } - - x = get_afl_env("LSAN_OPTIONS"); - - if (x) { - - if (!strstr(x, "symbolize=0")) { - - FATAL("Custom LSAN_OPTIONS set without symbolize=0 - please fix!"); - - } - } - setenv("ASAN_OPTIONS", - "abort_on_error=1:" - "detect_leaks=0:" - "allocator_may_return_null=1:" - "symbolize=0:" - "detect_odr_violation=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("UBSAN_OPTIONS", - "halt_on_error=1:" - "abort_on_error=1:" - "malloc_context_size=0:" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", - 0); - - setenv("MSAN_OPTIONS", "exit_code=" STRINGIFY(MSAN_ERROR) ":" - "abort_on_error=1:" - "msan_track_origins=0" - "allocator_may_return_null=1:" - "symbolize=0:" - "handle_segv=0:" - "handle_sigbus=0:" - "handle_abort=0:" - "handle_sigfpe=0:" - "handle_sigill=0", 0); - - setenv("LSAN_OPTIONS", - "exitcode=" STRINGIFY(LSAN_ERROR) ":" - "fast_unwind_on_malloc=0:" - "symbolize=0:" - "print_suppressions=0", - 0); + set_sanitizer_defaults(); if (get_afl_env("AFL_PRELOAD")) { diff --git a/utils/afl_network_proxy/afl-network-server.c b/utils/afl_network_proxy/afl-network-server.c index 2ae4c165..04309ada 100644 --- a/utils/afl_network_proxy/afl-network-server.c +++ b/utils/afl_network_proxy/afl-network-server.c @@ -194,7 +194,7 @@ static void set_up_environment(afl_forkserver_t *fsrv) { } - if (!strstr(x, "symbolize=0")) { + if (!getenv("AFL_DEBUG") && !strstr(x, "symbolize=0")) { FATAL("Custom ASAN_OPTIONS set without symbolize=0 - please fix!"); @@ -213,7 +213,7 @@ static void set_up_environment(afl_forkserver_t *fsrv) { } - if (!strstr(x, "symbolize=0")) { + if (!getenv("AFL_DEBUG") && !strstr(x, "symbolize=0")) { FATAL("Custom MSAN_OPTIONS set without symbolize=0 - please fix!"); @@ -221,18 +221,7 @@ static void set_up_environment(afl_forkserver_t *fsrv) { } - setenv("ASAN_OPTIONS", - "abort_on_error=1:" - "detect_leaks=0:" - "symbolize=0:" - "allocator_may_return_null=1", - 0); - - setenv("MSAN_OPTIONS", "exit_code=" STRINGIFY(MSAN_ERROR) ":" - "symbolize=0:" - "abort_on_error=1:" - "allocator_may_return_null=1:" - "msan_track_origins=0", 0); + set_sanitizer_defaults(); if (get_afl_env("AFL_PRELOAD")) { -- cgit 1.4.1 From 33eba1fc5652060e8d877b02135fce2325813d0c Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Fri, 27 Jan 2023 10:17:16 +0100 Subject: update changelog --- docs/Changelog.md | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 434bc101..eee88a51 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -4,8 +4,16 @@ release of the tool. See README.md for the general instruction manual. ### Version ++4.06a (dev) + - afl-fuzz: + - ensure temporary file descriptor is closed when not used - afl-cc: - add CFI sanitizer variant to gcc targets + - llvm 16 support (thanks to @devnexen!) + - support llvm 15 native pcguard changes + - better sanitizer default options support for all tools + - unicorn_mode: updated and minor issues fixed + - frida_mode: fix issue on MacOS + - more minor fixes ### Version ++4.05c (release) - MacOS: libdislocator, libtokencap etc. do not work with modern -- cgit 1.4.1 From 80eabd6e8a30c2ffc0f084ab34df8b9d582419c3 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 13 Feb 2023 11:34:14 +0100 Subject: AFL_LLVM_DICT2FILE_NO_MAIN support --- TODO.md | 3 +-- docs/Changelog.md | 1 + docs/env_variables.md | 3 +++ docs/fuzzing_in_depth.md | 4 +++- include/envs.h | 1 + instrumentation/README.llvm.md | 4 ++++ instrumentation/SanitizerCoverageLTO.so.cc | 11 ++++++++++- instrumentation/afl-llvm-dict2file.so.cc | 17 ++++++++++++++--- src/afl-cc.c | 5 ++++- 9 files changed, 41 insertions(+), 8 deletions(-) (limited to 'docs') diff --git a/TODO.md b/TODO.md index 862224f0..187fa191 100644 --- a/TODO.md +++ b/TODO.md @@ -9,13 +9,12 @@ - afl-plot to support multiple plot_data - parallel builds for source-only targets - get rid of check_binary, replace with more forkserver communication - - first fuzzer should be a main automatically + - first fuzzer should be a main automatically? not sure. ## Maybe - forkserver tells afl-fuzz if cmplog is supported and if so enable it by default, with AFL_CMPLOG_NO=1 (?) set to skip? - - afl_custom_fuzz_splice_optin() - afl_custom_splice() - cmdline option from-to range for mutations diff --git a/docs/Changelog.md b/docs/Changelog.md index eee88a51..89c37912 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -10,6 +10,7 @@ - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) - support llvm 15 native pcguard changes + - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support - better sanitizer default options support for all tools - unicorn_mode: updated and minor issues fixed - frida_mode: fix issue on MacOS diff --git a/docs/env_variables.md b/docs/env_variables.md index 0a57d190..61fb1e2b 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -129,6 +129,9 @@ subset of the settings discussed in section 1, with the exception of: write all constant string comparisons to this file to be used later with afl-fuzz' `-x` option. + - An option to `AFL_LLVM_DICT2FILE` is `AFL_LLVM_DICT2FILE_NO_MAIN=1` which + skill not parse `main()`. + - `TMPDIR` and `AFL_KEEP_ASSEMBLY`, since no temporary assembly files are created. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 87f31a58..efab0633 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -534,6 +534,8 @@ dictionaries/FORMAT.dict`. * With `afl-clang-fast`, you can set `AFL_LLVM_DICT2FILE=/full/path/to/new/file.dic` to automatically generate a dictionary during target compilation. + Adding `AFL_LLVM_DICT2FILE_NO_MAIN=1` to not parse main (usually command line + parameter parsing) is often a good idea too. * You also have the option to generate a dictionary yourself during an independent run of the target, see [utils/libtokencap/README.md](../utils/libtokencap/README.md). @@ -935,7 +937,7 @@ phase and start fuzzing at once. 3. Also randomize the afl-fuzz runtime options, e.g.: * 65% for `AFL_DISABLE_TRIM` * 50% for `AFL_KEEP_TIMEOUTS` - * 50% use a dictionary generated by `AFL_LLVM_DICT2FILE` + * 50% use a dictionary generated by `AFL_LLVM_DICT2FILE` + `AFL_LLVM_DICT2FILE_NO_MAIN=1` * 40% use MOpt (`-L 0`) * 40% for `AFL_EXPAND_HAVOC_NOW` * 20% for old queue processing (`-Z`) diff --git a/include/envs.h b/include/envs.h index 0770f94d..5018b0f8 100644 --- a/include/envs.h +++ b/include/envs.h @@ -133,6 +133,7 @@ static char *afl_environment_variables[] = { "AFL_LLVM_CTX", "AFL_LLVM_CTX_K", "AFL_LLVM_DICT2FILE", + "AFL_LLVM_DICT2FILE_NO_MAIN", "AFL_LLVM_DOCUMENT_IDS", "AFL_LLVM_INSTRIM_LOOPHEAD", "AFL_LLVM_INSTRUMENT", diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index 9da1b0f6..c0677474 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -167,6 +167,10 @@ Just specify `AFL_LLVM_DICT2FILE=/absolute/path/file.txt` and during compilation all constant string compare parameters will be written to this file to be used with afl-fuzz' `-x` option. +Adding `AFL_LLVM_DICT2FILE_NO_MAIN=1` will skip parsing `main()` which often +does command line parsing which has string comparisons that are not helpful +for fuzzing. + ## 6) AFL++ Context Sensitive Branch Coverage ### What is this? diff --git a/instrumentation/SanitizerCoverageLTO.so.cc b/instrumentation/SanitizerCoverageLTO.so.cc index 231151f5..f82224ed 100644 --- a/instrumentation/SanitizerCoverageLTO.so.cc +++ b/instrumentation/SanitizerCoverageLTO.so.cc @@ -236,6 +236,7 @@ class ModuleSanitizerCoverageLTO // const SpecialCaseList * Allowlist; // const SpecialCaseList * Blocklist; uint32_t autodictionary = 1; + uint32_t autodictionary_no_main = 0; uint32_t inst = 0; uint32_t afl_global_id = 0; uint32_t unhandled = 0; @@ -411,7 +412,8 @@ bool ModuleSanitizerCoverageLTO::instrumentModule( /* Show a banner */ setvbuf(stdout, NULL, _IONBF, 0); - if (getenv("AFL_DEBUG")) debug = 1; + if (getenv("AFL_DEBUG")) { debug = 1; } + if (getenv("AFL_LLVM_DICT2FILE_NO_MAIN")) { autodictionary_no_main = 1; } if ((isatty(2) && !getenv("AFL_QUIET")) || debug) { @@ -503,6 +505,13 @@ bool ModuleSanitizerCoverageLTO::instrumentModule( if (!isInInstrumentList(&F, MNAME) || !F.size()) { continue; } + if (autodictionary_no_main && + (!F.getName().compare("main") || !F.getName().compare("_main"))) { + + continue; + + } + for (auto &BB : F) { for (auto &IN : BB) { diff --git a/instrumentation/afl-llvm-dict2file.so.cc b/instrumentation/afl-llvm-dict2file.so.cc index bbbbe32c..97f1d47f 100644 --- a/instrumentation/afl-llvm-dict2file.so.cc +++ b/instrumentation/afl-llvm-dict2file.so.cc @@ -182,7 +182,7 @@ bool AFLdict2filePass::runOnModule(Module &M) { DenseMap valueMap; char *ptr; - int found = 0; + int found = 0, handle_main = 1; /* Show a banner */ setvbuf(stdout, NULL, _IONBF, 0); @@ -192,10 +192,14 @@ bool AFLdict2filePass::runOnModule(Module &M) { SAYF(cCYA "afl-llvm-dict2file" VERSION cRST " by Marc \"vanHauser\" Heuse \n"); - } else + } else { be_quiet = 1; + } + + if (getenv("AFL_LLVM_DICT2FILE_NO_MAIN")) { handle_main = 0; } + scanForDangerousFunctions(&M); ptr = getenv("AFL_LLVM_DICT2FILE"); @@ -210,7 +214,14 @@ bool AFLdict2filePass::runOnModule(Module &M) { for (auto &F : M) { - if (isIgnoreFunction(&F)) continue; + if (!handle_main && + (!F.getName().compare("main") || !F.getName().compare("_main"))) { + + continue; + + } + + if (isIgnoreFunction(&F)) { continue; } if (!isInInstrumentList(&F, MNAME) || !F.size()) { continue; } /* Some implementation notes. diff --git a/src/afl-cc.c b/src/afl-cc.c index 7c3682fb..7b059d40 100644 --- a/src/afl-cc.c +++ b/src/afl-cc.c @@ -2041,6 +2041,8 @@ int main(int argc, char **argv, char **envp) { " AFL_LLVM_DICT2FILE: generate an afl dictionary based on found " "comparisons\n" + " AFL_LLVM_DICT2FILE_NO_MAIN: skip parsing main() for the " + "dictionary\n" " AFL_LLVM_LAF_ALL: enables all LAF splits/transforms\n" " AFL_LLVM_LAF_SPLIT_COMPARES: enable cascaded comparisons\n" " AFL_LLVM_LAF_SPLIT_COMPARES_BITW: size limit (default 8)\n" @@ -2128,7 +2130,8 @@ int main(int argc, char **argv, char **envp) { "defaults.\n" "Recommended is afl-clang-lto with AFL_LLVM_CMPLOG or afl-clang-fast " "with\n" - "AFL_LLVM_CMPLOG and AFL_LLVM_DICT2FILE.\n\n"); + "AFL_LLVM_CMPLOG and " + "AFL_LLVM_DICT2FILE+AFL_LLVM_DICT2FILE_NO_MAIN.\n\n"); exit(1); -- cgit 1.4.1 From 668f5e1fa9c126bb8c751a6e4ef038ae60a442fa Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 15 Feb 2023 09:32:32 +0100 Subject: debug output --- custom_mutators/autotokens/Makefile | 8 ++++++-- custom_mutators/autotokens/autotokens.cpp | 17 ++++++++++++++++- docs/Changelog.md | 1 + docs/env_variables.md | 2 ++ include/afl-fuzz.h | 2 +- include/envs.h | 1 + src/afl-fuzz-init.c | 2 +- src/afl-fuzz-one.c | 2 +- src/afl-fuzz-run.c | 2 +- src/afl-fuzz-state.c | 7 +++++++ 10 files changed, 37 insertions(+), 7 deletions(-) (limited to 'docs') diff --git a/custom_mutators/autotokens/Makefile b/custom_mutators/autotokens/Makefile index 6ee7d324..0daba17d 100644 --- a/custom_mutators/autotokens/Makefile +++ b/custom_mutators/autotokens/Makefile @@ -13,10 +13,14 @@ endif all: autotokens.so -autotokens.so: autotokens.cpp +afl-fuzz-queue.o: ../../src/afl-fuzz-queue.c $(CC) -D_STANDALONE_MODULE=1 -I../../include -g -O3 $(CPPFLAGS) -fPIC -c -o ./afl-fuzz-queue.o ../../src/afl-fuzz-queue.c + +afl-common.o: ../../src/afl-common.c $(CC) -I../../include -g -O3 $(CPPFLAGS) -DBIN_PATH=\"dummy\" -Wno-pointer-sign -fPIC -c -o ./afl-common.o ../../src/afl-common.c + +autotokens.so: afl-fuzz-queue.o afl-common.o autotokens.cpp $(CXX) -Wno-deprecated -g -O3 $(CXXFLAGS) $(CPPFLAGS) -shared -fPIC -o autotokens.so -I../../include autotokens.cpp ./afl-fuzz-queue.o ../../src/afl-performance.o ./afl-common.o clean: - rm -f autotokens.so *~ core + rm -f autotokens.so *.o *~ core diff --git a/custom_mutators/autotokens/autotokens.cpp b/custom_mutators/autotokens/autotokens.cpp index cda90a38..043d9588 100644 --- a/custom_mutators/autotokens/autotokens.cpp +++ b/custom_mutators/autotokens/autotokens.cpp @@ -145,6 +145,9 @@ static void first_run(void *data) { if ((valid * 100) / afl_ptr->extras_cnt < 95) { module_disabled = 1; } + DEBUGF(stderr, "DICT: valid %u, total %u, %u < 95 == disable\n", valid, + afl_ptr->extras_cnt, (u32)((valid * 100) / afl_ptr->extras_cnt)); + } else { module_disabled = 1; @@ -190,6 +193,10 @@ static void first_run(void *data) { if ((is_ascii * 100) / valid < 70) { module_disabled = 1; } + DEBUGF(stderr, "seeds: total %u, valid %u, ascii %u, %u < 70 == disabled\n", + afl_ptr->active_items, valid, is_ascii, + (u32)((is_ascii * 100) / valid)); + } static u32 good_whitespace_or_singleval() { @@ -538,7 +545,15 @@ extern "C" unsigned char afl_custom_queue_get(void *data, is_first_run = 0; first_run(data); - if (module_disabled) { WARNF("Autotokens custom module is disabled."); } + if (module_disabled) { + + WARNF("Autotokens custom module is disabled."); + + } else if (auto_disable) { + + OKF("Autotokens custom module is enabled."); + + } } diff --git a/docs/Changelog.md b/docs/Changelog.md index 89c37912..5f253064 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -6,6 +6,7 @@ ### Version ++4.06a (dev) - afl-fuzz: - ensure temporary file descriptor is closed when not used + - added `AFL_NO_WARN_INSTABILITY` - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) diff --git a/docs/env_variables.md b/docs/env_variables.md index 61fb1e2b..7a574e59 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -482,6 +482,8 @@ checks or alter some of the more exotic semantics of the tool: - Setting `AFL_NO_STARTUP_CALIBRATION` will skip the initial calibration of all starting seeds, and start fuzzing at once. + - Setting `AFL_NO_WARN_INSTABILITY` will suppress instability warnings. + - In QEMU mode (-Q) and FRIDA mode (-O), `AFL_PATH` will be searched for afl-qemu-trace and afl-frida-trace.so. diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 229bc025..9bf91faf 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -399,7 +399,7 @@ typedef struct afl_env_vars { afl_cycle_schedules, afl_expand_havoc, afl_statsd, afl_cmplog_only_new, afl_exit_on_seed_issues, afl_try_affinity, afl_ignore_problems, afl_keep_timeouts, afl_pizza_mode, afl_no_crash_readme, - afl_ignore_timeouts, afl_no_startup_calibration; + afl_ignore_timeouts, afl_no_startup_calibration, afl_no_warn_instability; u8 *afl_tmpdir, *afl_custom_mutator_library, *afl_python_module, *afl_path, *afl_hang_tmout, *afl_forksrv_init_tmout, *afl_preload, diff --git a/include/envs.h b/include/envs.h index 5018b0f8..56675eda 100644 --- a/include/envs.h +++ b/include/envs.h @@ -172,6 +172,7 @@ static char *afl_environment_variables[] = { "AFL_NO_UI", "AFL_NO_PYTHON", "AFL_NO_STARTUP_CALIBRATION", + "AFL_NO_WARN_INSTABILITY", "AFL_UNTRACER_FILE", "AFL_LLVM_USE_TRACE_PC", "AFL_MAP_SIZE", diff --git a/src/afl-fuzz-init.c b/src/afl-fuzz-init.c index 1182bd41..c20965b4 100644 --- a/src/afl-fuzz-init.c +++ b/src/afl-fuzz-init.c @@ -1120,7 +1120,7 @@ void perform_dry_run(afl_state_t *afl) { } - if (q->var_behavior) { + if (unlikely(q->var_behavior && !afl->afl_env.afl_no_warn_instability)) { WARNF("Instrumentation output varies across runs."); diff --git a/src/afl-fuzz-one.c b/src/afl-fuzz-one.c index 2f016217..e97db273 100644 --- a/src/afl-fuzz-one.c +++ b/src/afl-fuzz-one.c @@ -1988,7 +1988,7 @@ custom_mutator_stage: if (unlikely(!mutated_buf)) { - //FATAL("Error in custom_fuzz. Size returned: %zu", mutated_size); + // FATAL("Error in custom_fuzz. Size returned: %zu", mutated_size); break; } diff --git a/src/afl-fuzz-run.c b/src/afl-fuzz-run.c index 7dd83150..f5425011 100644 --- a/src/afl-fuzz-run.c +++ b/src/afl-fuzz-run.c @@ -523,7 +523,7 @@ u8 calibrate_case(afl_state_t *afl, struct queue_entry *q, u8 *use_mem, } - if (unlikely(!var_detected)) { + if (unlikely(!var_detected && !afl->afl_env.afl_no_warn_instability)) { // note: from_queue seems to only be set during initialization if (afl->afl_env.afl_no_ui || from_queue) { diff --git a/src/afl-fuzz-state.c b/src/afl-fuzz-state.c index 104b1e4b..6d8c8758 100644 --- a/src/afl-fuzz-state.c +++ b/src/afl-fuzz-state.c @@ -204,6 +204,13 @@ void read_afl_environment(afl_state_t *afl, char **envp) { afl->afl_env.afl_no_affinity = get_afl_env(afl_environment_variables[i]) ? 1 : 0; + } else if (!strncmp(env, "AFL_NO_WARN_INSTABILITY", + + afl_environment_variable_len)) { + + afl->afl_env.afl_no_warn_instability = + get_afl_env(afl_environment_variables[i]) ? 1 : 0; + } else if (!strncmp(env, "AFL_TRY_AFFINITY", afl_environment_variable_len)) { -- cgit 1.4.1 From ebaac23a514cd3950d4a6cb597bd921e13ab9baa Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 20 Feb 2023 11:42:40 +0100 Subject: clarify AFL_NO_STARTUP_CALIBRATION --- docs/env_variables.md | 3 ++- docs/fuzzing_in_depth.md | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index 22a5c386..646db3f2 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -474,7 +474,8 @@ checks or alter some of the more exotic semantics of the tool: output from afl-fuzz is redirected to a file or to a pipe. - Setting `AFL_NO_STARTUP_CALIBRATION` will skip the initial calibration - of all starting seeds, and start fuzzing at once. + of all starting seeds, and start fuzzing at once. Use with care, this + degrades the fuzzing performance! - In QEMU mode (-Q) and FRIDA mode (-O), `AFL_PATH` will be searched for afl-qemu-trace and afl-frida-trace.so. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 87f31a58..2a088201 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -628,7 +628,8 @@ If you have a large corpus, a corpus from a previous run or are fuzzing in a CI, then also set `export AFL_CMPLOG_ONLY_NEW=1` and `export AFL_FAST_CAL=1`. If the queue in the CI is huge and/or the execution time is slow then you can also add `AFL_NO_STARTUP_CALIBRATION=1` to skip the initial queue calibration -phase and start fuzzing at once. +phase and start fuzzing at once - but only do this if the calibration phase +would be too long for your fuzz run time. You can also use different fuzzers. If you are using AFL spinoffs or AFL conforming fuzzers, then just use the same -o directory and give it a unique @@ -914,7 +915,8 @@ normal fuzzing campaigns as these are much shorter runnings. If the queue in the CI is huge and/or the execution time is slow then you can also add `AFL_NO_STARTUP_CALIBRATION=1` to skip the initial queue calibration -phase and start fuzzing at once. +phase and start fuzzing at once. But only do that if the calibration time is +too long for your overall available fuzz run time. 1. Always: * LTO has a much longer compile time which is diametrical to short fuzzing - -- cgit 1.4.1 From eeccb2da69d7e6f32ee74c431e7c5053e8379dff Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 23 Feb 2023 11:45:26 +0100 Subject: nits --- docs/Changelog.md | 1 + qemu_mode/qemuafl | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 5f253064..8f71fd83 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -11,6 +11,7 @@ - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) - support llvm 15 native pcguard changes + - new custom module: autotoken, grammar free fuzzer for text inputs - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support - better sanitizer default options support for all tools - unicorn_mode: updated and minor issues fixed diff --git a/qemu_mode/qemuafl b/qemu_mode/qemuafl index a120c3fe..a8af9cbd 160000 --- a/qemu_mode/qemuafl +++ b/qemu_mode/qemuafl @@ -1 +1 @@ -Subproject commit a120c3feb573d4cade292cdeb7c1f6b1ce109efe +Subproject commit a8af9cbde71e333ce72a46f15e655d0b82ed0939 -- cgit 1.4.1 From d8ba0caab3661ef045f6bc6d99481ab14d165262 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Tue, 28 Feb 2023 11:03:09 +0100 Subject: update docs --- TODO.md | 1 + docs/ideas.md | 13 ++----------- 2 files changed, 3 insertions(+), 11 deletions(-) (limited to 'docs') diff --git a/TODO.md b/TODO.md index 187fa191..e7789cf6 100644 --- a/TODO.md +++ b/TODO.md @@ -2,6 +2,7 @@ ## Should + - splicing selection weighted? - support afl_custom_{send,post_process}, persistent and deferred fork server in afl-showmap - better autodetection of shifting runtime timeout values diff --git a/docs/ideas.md b/docs/ideas.md index b5de637f..4e419b80 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -3,6 +3,8 @@ In the following, we describe a variety of ideas that could be implemented for future AFL++ versions. +**NOTE:** Our GSoC participation is concerning [libafl](https://github.com/AFLplusplus/libafl), not AFL++. + ## Analysis software Currently analysis is done by using afl-plot, which is rather outdated. A GTK or @@ -16,17 +18,6 @@ and Y axis, zoom factor, log scaling on-off, etc. Mentor: vanhauser-thc -## WASM Instrumentation - -Currently, AFL++ can be used for source code fuzzing and traditional binaries. -With the rise of WASM as a compile target, however, a novel way of -instrumentation needs to be implemented for binaries compiled to Webassembly. -This can either be done by inserting instrumentation directly into the WASM AST, -or by patching feedback into a WASM VM of choice, similar to the current Unicorn -instrumentation. - -Mentor: any - ## Support other programming languages Other programming languages also use llvm hence they could be (easily?) -- cgit 1.4.1 From e9e440d7f33a61793c63f90f9555ff3c0f45b3b4 Mon Sep 17 00:00:00 2001 From: Amit Elkabetz <12958411+amitelka@users.noreply.github.com> Date: Sun, 5 Mar 2023 20:25:39 +0200 Subject: Fixed according to PR comment, moved cli flag to an env variable --- docs/env_variables.md | 6 ++++++ include/envs.h | 1 + src/afl-fuzz-state.c | 20 ++++++++++++++++++++ src/afl-fuzz.c | 26 ++++++++------------------ 4 files changed, 35 insertions(+), 18 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index 6cd4104b..c9dc1bbd 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -584,6 +584,12 @@ checks or alter some of the more exotic semantics of the tool: - Set `AFL_PIZZA_MODE` to 1 to enable the April 1st stats menu, set to 0 to disable although it is 1st of April. + - If you need a specific interval to update fuzzer_stats file, you can + set `AFL_FUZZER_STATS_UPDATE_INTERVAL` to the interval in seconds you'd + the file to be updated. + Note that will not be exact and with slow targets it can take seconds + until there is a slice for the time test. + ## 5) Settings for afl-qemu-trace The QEMU wrapper used to instrument binary-only code supports several settings: diff --git a/include/envs.h b/include/envs.h index cf069a00..066921b9 100644 --- a/include/envs.h +++ b/include/envs.h @@ -91,6 +91,7 @@ static char *afl_environment_variables[] = { "AFL_FRIDA_TRACEABLE", "AFL_FRIDA_VERBOSE", "AFL_FUZZER_ARGS", // oss-fuzz + "AFL_FUZZER_STATS_UPDATE_INTERVAL", "AFL_GDB", "AFL_GCC_ALLOWLIST", "AFL_GCC_DENYLIST", diff --git a/src/afl-fuzz-state.c b/src/afl-fuzz-state.c index e319c512..8964f38e 100644 --- a/src/afl-fuzz-state.c +++ b/src/afl-fuzz-state.c @@ -24,6 +24,7 @@ */ #include +#include #include "afl-fuzz.h" #include "envs.h" @@ -566,6 +567,25 @@ void read_afl_environment(afl_state_t *afl, char **envp) { } + } else if (!strncmp(env, "AFL_FUZZER_STATS_UPDATE_INTERVAL", + + afl_environment_variable_len)) { + + u64 stats_update_freq_sec = + strtoull(get_afl_env(afl_environment_variables[i]), NULL, 0); + if (ULLONG_MAX == stats_update_freq_sec || + 0 == stats_update_freq_sec) { + + WARNF( + "Incorrect value given to AFL_FUZZER_STATS_UPDATE_INTERVAL, " + "using default of 60 seconds\n"); + + } else { + + afl->stats_file_update_freq_msecs = stats_update_freq_sec * 1000; + + } + } } else { diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index 78d9da71..d7708fdf 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -211,8 +211,6 @@ static void usage(u8 *argv0, int more_help) { "(0-...)\n" " -e ext - file extension for the fuzz test input file (if " "needed)\n" - " -u - interval to update fuzzer_stats file in seconds, " - "defaults to 60 sec, minimum interval: 1 sec\n" "\n", argv0, EXEC_TIMEOUT, MEM_LIMIT, MAX_FILE, FOREIGN_SYNCS_MAX); @@ -315,6 +313,8 @@ static void usage(u8 *argv0, int more_help) { " afl-clang-lto/afl-gcc-fast target\n" "AFL_PERSISTENT: enforce persistent mode (if __AFL_LOOP is in a shared lib\n" "AFL_DEFER_FORKSRV: enforced deferred forkserver (__AFL_INIT is in a .so)\n" + "AFL_FUZZER_STATS_UPDATE_INTERVAL: interval to update fuzzer_stats file in seconds, " + "(default: 60, minimum: 1)\n" "\n" ); @@ -504,7 +504,7 @@ fail: int main(int argc, char **argv_orig, char **envp) { s32 opt, auto_sync = 0 /*, user_set_cache = 0*/; - u64 prev_queued = 0, stats_update_freq_sec = 0; + u64 prev_queued = 0; u32 sync_interval_cnt = 0, seek_to = 0, show_help = 0, default_output = 1, map_size = get_map_size(); u8 *extras_dir[4]; @@ -553,9 +553,11 @@ int main(int argc, char **argv_orig, char **envp) { afl->shmem_testcase_mode = 1; // we always try to perform shmem fuzzing - while ((opt = getopt(argc, argv, - "+Ab:B:c:CdDe:E:hi:I:f:F:g:G:l:L:m:M:nNOo:p:RQs:S:t:T:u:" - "UV:WXx:YZ")) > 0) { + while ( + (opt = getopt( + argc, argv, + "+Ab:B:c:CdDe:E:hi:I:f:F:g:G:l:L:m:M:nNOo:p:RQs:S:t:T:UV:WXx:YZ")) > + 0) { switch (opt) { @@ -666,18 +668,6 @@ int main(int argc, char **argv_orig, char **envp) { break; - case 'u': - if (sscanf(optarg, "%llu", &stats_update_freq_sec) < 1) { - - FATAL("Bad syntax used for -u"); - - } - - if (stats_update_freq_sec < 1) { FATAL("-u interval must be >= 1"); } - - afl->stats_file_update_freq_msecs = stats_update_freq_sec * 1000; - break; - case 'i': /* input dir */ if (afl->in_dir) { FATAL("Multiple -i options not supported"); } -- cgit 1.4.1 From a30664c5639a924fd2bfd40bd9570f11ef09fd10 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 6 Mar 2023 09:15:05 +0100 Subject: fix ci --- .github/workflows/ci.yml | 2 +- docs/fuzzing_binary-only_targets.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) (limited to 'docs') diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 04cbaca8..b7d8b3b7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,7 +23,7 @@ jobs: - name: debug run: apt-cache search plugin-dev | grep gcc-; echo; apt-cache search clang-format- | grep clang-format- - name: update - run: sudo apt-get update && sudo apt-get upgrade -y + run: sudo apt-get purge -y "grub*" && sudo apt-get update && sudo apt-get upgrade -y - name: install packages run: sudo apt-get install -y -m -f --install-suggests build-essential git libtool libtool-bin automake bison libglib2.0-0 clang llvm-dev libc++-dev findutils libcmocka-dev python3-dev python3-setuptools ninja-build - name: compiler installed diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index 266920e6..9d9d6bb6 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -201,10 +201,10 @@ afl-clang-fast's. ### RetroWrite RetroWrite is a static binary rewriter that can be combined with AFL++. If you -have an x86_64 binary that still has its symbols (i.e., not stripped binary), is -compiled with position independent code (PIC/PIE), and does not contain C++ -exceptions, then the RetroWrite solution might be for you. It decompiles to ASM -files which can then be instrumented with afl-gcc. +have an x86_64 or arm64 binary that does not contain C++ exceptions and - if +x86_64 - still has it's symbols and compiled with position independent code +(PIC/PIE), then the RetroWrite solution might be for you. +It decompiles to ASM files which can then be instrumented with afl-gcc. Binaries that are statically instrumented for fuzzing using RetroWrite are close in performance to compiler-instrumented binaries and outperform the QEMU-based -- cgit 1.4.1 From e6a05382b83817b245da51bcba16be5df56eb283 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 6 Mar 2023 09:59:52 +0100 Subject: fix IGNORE_PROBLEMS and update qemuafl --- docs/Changelog.md | 2 ++ instrumentation/afl-compiler-rt.o.c | 39 ++++++++++++++++++++++++------------- instrumentation/afl-llvm-common.h | 8 ++++---- qemu_mode/QEMUAFL_VERSION | 2 +- qemu_mode/qemuafl | 2 +- src/afl-fuzz-stats.c | 22 ++++++++++----------- 6 files changed, 44 insertions(+), 31 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 8f71fd83..f4fa4382 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -11,6 +11,8 @@ - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) - support llvm 15 native pcguard changes + - qemu_mode: + - fix _RANGES envs to allow hyphens in the filenames - new custom module: autotoken, grammar free fuzzer for text inputs - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support - better sanitizer default options support for all tools diff --git a/instrumentation/afl-compiler-rt.o.c b/instrumentation/afl-compiler-rt.o.c index 9871d7f4..94022a65 100644 --- a/instrumentation/afl-compiler-rt.o.c +++ b/instrumentation/afl-compiler-rt.o.c @@ -1539,12 +1539,16 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) { if (start == stop || *start) return; x = getenv("AFL_INST_RATIO"); - if (x) { inst_ratio = (u32)atoi(x); } + if (x) { - if (!inst_ratio || inst_ratio > 100) { + inst_ratio = (u32)atoi(x); - fprintf(stderr, "[-] ERROR: Invalid AFL_INST_RATIO (must be 1-100).\n"); - abort(); + if (!inst_ratio || inst_ratio > 100) { + + fprintf(stderr, "[-] ERROR: Invalid AFL_INST_RATIO (must be 1-100).\n"); + abort(); + + } } @@ -1568,10 +1572,16 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) { while (start < stop) { - if (likely(inst_ratio == 100) || R(100) < inst_ratio) - *start = offset; - else - *start = 0; // write to map[0] + if (likely(inst_ratio == 100) || R(100) < inst_ratio) { + + *(start++) = offset; + + } else { + + *(start++) = 0; // write to map[0] + + } + if (unlikely(++offset >= __afl_final_loc)) { offset = 4; } } @@ -1592,12 +1602,15 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) { while (start < stop) { - if (likely(inst_ratio == 100) || R(100) < inst_ratio) - *start = ++__afl_final_loc; - else - *start = 0; // write to map[0] + if (likely(inst_ratio == 100) || R(100) < inst_ratio) { + + *(start++) = ++__afl_final_loc; - start++; + } else { + + *(start++) = 0; // write to map[0] + + } } diff --git a/instrumentation/afl-llvm-common.h b/instrumentation/afl-llvm-common.h index 0112c325..16a13da5 100644 --- a/instrumentation/afl-llvm-common.h +++ b/instrumentation/afl-llvm-common.h @@ -37,10 +37,10 @@ typedef long double max_align_t; #define MNAME M.getSourceFileName() #define FMNAME F.getParent()->getSourceFileName() #if LLVM_VERSION_MAJOR >= 16 - // None becomes deprecated - // the standard std::nullopt_t is recommended instead - // from C++17 and onwards. - constexpr std::nullopt_t None = std::nullopt; +// None becomes deprecated +// the standard std::nullopt_t is recommended instead +// from C++17 and onwards. +constexpr std::nullopt_t None = std::nullopt; #endif #else #define MNAME std::string("") diff --git a/qemu_mode/QEMUAFL_VERSION b/qemu_mode/QEMUAFL_VERSION index 9c68f02c..39e41f79 100644 --- a/qemu_mode/QEMUAFL_VERSION +++ b/qemu_mode/QEMUAFL_VERSION @@ -1 +1 @@ -a8af9cbde7 +74c583b11a diff --git a/qemu_mode/qemuafl b/qemu_mode/qemuafl index a8af9cbd..74c583b1 160000 --- a/qemu_mode/qemuafl +++ b/qemu_mode/qemuafl @@ -1 +1 @@ -Subproject commit a8af9cbde71e333ce72a46f15e655d0b82ed0939 +Subproject commit 74c583b11ac508b90660723da7ee9ff7ff77ee92 diff --git a/src/afl-fuzz-stats.c b/src/afl-fuzz-stats.c index 26e1a50e..53ab8c77 100644 --- a/src/afl-fuzz-stats.c +++ b/src/afl-fuzz-stats.c @@ -672,12 +672,11 @@ void show_stats_normal(afl_state_t *afl) { /* If no coverage was found yet, check whether run time is greater than * exit_on_time. */ - if (unlikely( - !afl->non_instrumented_mode && afl->afl_env.afl_exit_on_time && - ((afl->last_find_time && - (cur_ms - afl->last_find_time) > afl->exit_on_time) || - (!afl->last_find_time && (cur_ms - afl->start_time) - > afl->exit_on_time)))) { + if (unlikely(!afl->non_instrumented_mode && afl->afl_env.afl_exit_on_time && + ((afl->last_find_time && + (cur_ms - afl->last_find_time) > afl->exit_on_time) || + (!afl->last_find_time && + (cur_ms - afl->start_time) > afl->exit_on_time)))) { afl->stop_soon = 2; @@ -1476,12 +1475,11 @@ void show_stats_pizza(afl_state_t *afl) { /* If no coverage was found yet, check whether run time is greater than * exit_on_time. */ - if (unlikely( - !afl->non_instrumented_mode && afl->afl_env.afl_exit_on_time && - ((afl->last_find_time && - (cur_ms - afl->last_find_time) > afl->exit_on_time) || - (!afl->last_find_time && (cur_ms - afl->start_time) - > afl->exit_on_time)))) { + if (unlikely(!afl->non_instrumented_mode && afl->afl_env.afl_exit_on_time && + ((afl->last_find_time && + (cur_ms - afl->last_find_time) > afl->exit_on_time) || + (!afl->last_find_time && + (cur_ms - afl->start_time) > afl->exit_on_time)))) { afl->stop_soon = 2; -- cgit 1.4.1 From e0866f51c7984c28866e7acdb153b5304c5cf7da Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 9 Mar 2023 13:57:03 +0100 Subject: support LLVMFuzzerTestOneInput -1 return --- docs/Changelog.md | 2 ++ test/test-cmplog.c | 2 +- utils/aflpp_driver/aflpp_driver.c | 13 ++++++++++++- utils/aflpp_driver/aflpp_driver_test.c | 13 ++++++++----- 4 files changed, 23 insertions(+), 7 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index f4fa4382..5287d038 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -7,10 +7,12 @@ - afl-fuzz: - ensure temporary file descriptor is closed when not used - added `AFL_NO_WARN_INSTABILITY` + - added `AFL_FRIDA_STATS_INTERVAL` - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) - support llvm 15 native pcguard changes + - support for LLVMFuzzerTestOneInput -1 return - qemu_mode: - fix _RANGES envs to allow hyphens in the filenames - new custom module: autotoken, grammar free fuzzer for text inputs diff --git a/test/test-cmplog.c b/test/test-cmplog.c index d724ecaf..bd1b73e3 100644 --- a/test/test-cmplog.c +++ b/test/test-cmplog.c @@ -8,7 +8,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *buf, size_t i) { - if (i < 30) return 0; + if (i < 30) return -1; if (buf[0] != 'A') return 0; if (buf[1] != 'B') return 0; if (buf[2] != 'C') return 0; diff --git a/utils/aflpp_driver/aflpp_driver.c b/utils/aflpp_driver/aflpp_driver.c index 03376b6a..f08c9864 100644 --- a/utils/aflpp_driver/aflpp_driver.c +++ b/utils/aflpp_driver/aflpp_driver.c @@ -58,10 +58,15 @@ $AFL_HOME/afl-fuzz -i IN -o OUT ./a.out #include "hash.h" #endif +// AFL++ shared memory fuzz cases int __afl_sharedmem_fuzzing = 1; extern unsigned int *__afl_fuzz_len; extern unsigned char *__afl_fuzz_ptr; +// AFL++ coverage map +extern unsigned char *__afl_area_ptr; +extern unsigned int __afl_map_size; + // libFuzzer interface is thin, so we don't include any libFuzzer headers. __attribute__((weak)) int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size); @@ -375,7 +380,13 @@ int LLVMFuzzerRunDriver(int *argcp, char ***argvp, } prev_length = length; - (void)callback(__afl_fuzz_ptr, length); + + if (unlikely(callback(__afl_fuzz_ptr, length) == -1)) { + + memset(__afl_area_ptr, 0, __afl_map_size); + __afl_area_ptr[0] = 1; + + } } diff --git a/utils/aflpp_driver/aflpp_driver_test.c b/utils/aflpp_driver/aflpp_driver_test.c index 527ba57b..7cffa4a1 100644 --- a/utils/aflpp_driver/aflpp_driver_test.c +++ b/utils/aflpp_driver/aflpp_driver_test.c @@ -2,9 +2,9 @@ #include #include -void __attribute__((noinline)) crashme(const uint8_t *Data, size_t Size) { +int __attribute__((noinline)) crashme(const uint8_t *Data, size_t Size) { - if (Size < 5) return; + if (Size < 5) return -1; if (Data[0] == 'F') if (Data[1] == 'A') @@ -12,13 +12,16 @@ void __attribute__((noinline)) crashme(const uint8_t *Data, size_t Size) { if (Data[3] == '$') if (Data[4] == '$') abort(); + return 0; + } int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { - if (Size) crashme(Data, Size); - - return 0; + if (Size) + return crashme(Data, Size); + else + return -1; } -- cgit 1.4.1 From 5221938945cc5ff15af04b727c6a7e0085005044 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 9 Mar 2023 17:36:13 +0100 Subject: various fixes --- docs/Changelog.md | 2 +- include/common.h | 3 +++ instrumentation/afl-compiler-rt.o.c | 18 ++++++++++++------ src/afl-analyze.c | 4 ++++ src/afl-common.c | 37 ++++++++++++++++++++++++++++++------- src/afl-fuzz-init.c | 19 ++++++++++++------- src/afl-fuzz-stats.c | 2 +- src/afl-showmap.c | 4 ++++ src/afl-tmin.c | 4 ++++ 9 files changed, 71 insertions(+), 22 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 5287d038..25c1f6bc 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -20,7 +20,7 @@ - better sanitizer default options support for all tools - unicorn_mode: updated and minor issues fixed - frida_mode: fix issue on MacOS - - more minor fixes + - more minor fixes and cross-platform support ### Version ++4.05c (release) - MacOS: libdislocator, libtokencap etc. do not work with modern diff --git a/include/common.h b/include/common.h index c5a32cdb..5d198468 100644 --- a/include/common.h +++ b/include/common.h @@ -143,5 +143,8 @@ FILE *create_ffile(u8 *fn); /* create a file */ s32 create_file(u8 *fn); +/* memmem implementation as not all platforms support this */ +void *afl_memmem(const void *haystack, size_t haystacklen, const void *needle, size_t needlelen); + #endif diff --git a/instrumentation/afl-compiler-rt.o.c b/instrumentation/afl-compiler-rt.o.c index 94022a65..a88396d4 100644 --- a/instrumentation/afl-compiler-rt.o.c +++ b/instrumentation/afl-compiler-rt.o.c @@ -1622,17 +1622,23 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) { } - if (__afl_already_initialized_shm && __afl_final_loc > __afl_map_size) { + if (__afl_already_initialized_shm) { - if (__afl_debug) { + if (__afl_final_loc > __afl_map_size) { + + if (__afl_debug) { + + fprintf(stderr, "Reinit shm necessary (+%u)\n", + __afl_final_loc - __afl_map_size); + + } - fprintf(stderr, "Reinit shm necessary (+%u)\n", - __afl_final_loc - __afl_map_size); + __afl_unmap_shm(); + __afl_map_shm(); } - __afl_unmap_shm(); - __afl_map_shm(); + __afl_map_size = __afl_final_loc + 1; } diff --git a/src/afl-analyze.c b/src/afl-analyze.c index d4a9aa91..9734f75c 100644 --- a/src/afl-analyze.c +++ b/src/afl-analyze.c @@ -725,7 +725,11 @@ static void setup_signal_handlers(void) { struct sigaction sa; sa.sa_handler = NULL; + #ifdef SA_RESTART sa.sa_flags = SA_RESTART; + #else + sa.sa_flags = 0; + #endif sa.sa_sigaction = NULL; sigemptyset(&sa.sa_mask); diff --git a/src/afl-common.c b/src/afl-common.c index b0df1994..86226c9f 100644 --- a/src/afl-common.c +++ b/src/afl-common.c @@ -58,6 +58,25 @@ u8 last_intr = 0; #define AFL_PATH "/usr/local/lib/afl/" #endif +void *afl_memmem(const void *haystack, size_t haystacklen, const void *needle, + size_t needlelen) { + + if (unlikely(needlelen > haystacklen)) { return NULL; } + + for (u32 i = 0; i <= haystacklen - needlelen; ++i) { + + if (unlikely(memcmp(haystack + i, needle, needlelen) == 0)) { + + return (void *)(haystack + i); + + } + + } + + return (void *)NULL; + +} + void set_sanitizer_defaults() { /* Set sane defaults for ASAN if nothing else is specified. */ @@ -67,9 +86,9 @@ void set_sanitizer_defaults() { u8 *have_lsan_options = getenv("LSAN_OPTIONS"); u8 have_san_options = 0; u8 default_options[1024] = - "detect_odr_violation=0:abort_on_error=1:symbolize=0:malloc_context_" - "size=0:allocator_may_return_null=1:handle_segv=0:handle_sigbus=0:" - "handle_abort=0:handle_sigfpe=0:handle_sigill=0:"; + "detect_odr_violation=0:abort_on_error=1:symbolize=0:allocator_may_" + "return_null=1:handle_segv=0:handle_sigbus=0:handle_abort=0:handle_" + "sigfpe=0:handle_sigill=0:"; if (have_asan_options || have_ubsan_options || have_msan_options || have_lsan_options) { @@ -84,14 +103,18 @@ void set_sanitizer_defaults() { u8 buf[2048] = ""; if (!have_san_options) { strcpy(buf, default_options); } - strcat(buf, "exitcode=" STRINGIFY(LSAN_ERROR) ":fast_unwind_on_malloc=0:print_suppressions=0:detect_leaks=1:"); + strcat(buf, "exitcode=" STRINGIFY(LSAN_ERROR) ":fast_unwind_on_malloc=0:print_suppressions=0:detect_leaks=1:malloc_context_size=30:"); setenv("LSAN_OPTIONS", buf, 1); } /* for everything not LSAN we disable detect_leaks */ - if (!have_lsan_options) { strcat(default_options, "detect_leaks=0:"); } + if (!have_lsan_options) { + + strcat(default_options, "detect_leaks=0:malloc_context_size=0:"); + + } /* Set sane defaults for ASAN if nothing else is specified. */ @@ -130,7 +153,7 @@ u32 check_binary_signatures(u8 *fn) { if (f_data == MAP_FAILED) { PFATAL("Unable to mmap file '%s'", fn); } close(fd); - if (memmem(f_data, f_len, PERSIST_SIG, strlen(PERSIST_SIG) + 1)) { + if (afl_memmem(f_data, f_len, PERSIST_SIG, strlen(PERSIST_SIG) + 1)) { if (!be_quiet) { OKF(cPIN "Persistent mode binary detected."); } setenv(PERSIST_ENV_VAR, "1", 1); @@ -155,7 +178,7 @@ u32 check_binary_signatures(u8 *fn) { } - if (memmem(f_data, f_len, DEFER_SIG, strlen(DEFER_SIG) + 1)) { + if (afl_memmem(f_data, f_len, DEFER_SIG, strlen(DEFER_SIG) + 1)) { if (!be_quiet) { OKF(cPIN "Deferred forkserver binary detected."); } setenv(DEFER_ENV_VAR, "1", 1); diff --git a/src/afl-fuzz-init.c b/src/afl-fuzz-init.c index c20965b4..3b441eee 100644 --- a/src/afl-fuzz-init.c +++ b/src/afl-fuzz-init.c @@ -24,7 +24,9 @@ */ #include "afl-fuzz.h" +#include "common.h" #include +#include #include "cmplog.h" #ifdef HAVE_AFFINITY @@ -2786,7 +2788,7 @@ void check_binary(afl_state_t *afl, u8 *fname) { !afl->fsrv.nyx_mode && #endif !afl->fsrv.cs_mode && !afl->non_instrumented_mode && - !memmem(f_data, f_len, SHM_ENV_VAR, strlen(SHM_ENV_VAR) + 1)) { + !afl_memmem(f_data, f_len, SHM_ENV_VAR, strlen(SHM_ENV_VAR) + 1)) { SAYF("\n" cLRD "[-] " cRST "Looks like the target binary is not instrumented! The fuzzer depends " @@ -2817,7 +2819,7 @@ void check_binary(afl_state_t *afl, u8 *fname) { } if ((afl->fsrv.cs_mode || afl->fsrv.qemu_mode || afl->fsrv.frida_mode) && - memmem(f_data, f_len, SHM_ENV_VAR, strlen(SHM_ENV_VAR) + 1)) { + afl_memmem(f_data, f_len, SHM_ENV_VAR, strlen(SHM_ENV_VAR) + 1)) { SAYF("\n" cLRD "[-] " cRST "This program appears to be instrumented with afl-gcc, but is being " @@ -2830,9 +2832,9 @@ void check_binary(afl_state_t *afl, u8 *fname) { } - if (memmem(f_data, f_len, "__asan_init", 11) || - memmem(f_data, f_len, "__msan_init", 11) || - memmem(f_data, f_len, "__lsan_init", 11)) { + if (afl_memmem(f_data, f_len, "__asan_init", 11) || + afl_memmem(f_data, f_len, "__msan_init", 11) || + afl_memmem(f_data, f_len, "__lsan_init", 11)) { afl->fsrv.uses_asan = 1; @@ -2840,7 +2842,7 @@ void check_binary(afl_state_t *afl, u8 *fname) { /* Detect persistent & deferred init signatures in the binary. */ - if (memmem(f_data, f_len, PERSIST_SIG, strlen(PERSIST_SIG) + 1)) { + if (afl_memmem(f_data, f_len, PERSIST_SIG, strlen(PERSIST_SIG) + 1)) { OKF(cPIN "Persistent mode binary detected."); setenv(PERSIST_ENV_VAR, "1", 1); @@ -2867,7 +2869,7 @@ void check_binary(afl_state_t *afl, u8 *fname) { } if (afl->fsrv.frida_mode || - memmem(f_data, f_len, DEFER_SIG, strlen(DEFER_SIG) + 1)) { + afl_memmem(f_data, f_len, DEFER_SIG, strlen(DEFER_SIG) + 1)) { OKF(cPIN "Deferred forkserver binary detected."); setenv(DEFER_ENV_VAR, "1", 1); @@ -2923,8 +2925,11 @@ void setup_signal_handlers(void) { struct sigaction sa; + memset((void*)&sa, 0, sizeof(sa)); sa.sa_handler = NULL; +#ifdef SA_RESTART sa.sa_flags = SA_RESTART; +#endif sa.sa_sigaction = NULL; sigemptyset(&sa.sa_mask); diff --git a/src/afl-fuzz-stats.c b/src/afl-fuzz-stats.c index 65caf5ee..f53fd610 100644 --- a/src/afl-fuzz-stats.c +++ b/src/afl-fuzz-stats.c @@ -62,7 +62,7 @@ void write_setup_file(afl_state_t *afl, u32 argc, char **argv) { if (memchr(argv[i], '\'', strlen(argv[i]))) { #else - if (index(argv[i], '\'')) { + if (strchr(argv[i], '\'')) { #endif diff --git a/src/afl-showmap.c b/src/afl-showmap.c index 1e281d08..32dd1c20 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -654,7 +654,11 @@ static void setup_signal_handlers(void) { struct sigaction sa; sa.sa_handler = NULL; + #ifdef SA_RESTART sa.sa_flags = SA_RESTART; + #else + sa.sa_flags = 0; + #endif sa.sa_sigaction = NULL; sigemptyset(&sa.sa_mask); diff --git a/src/afl-tmin.c b/src/afl-tmin.c index 12c5e0c9..530578d9 100644 --- a/src/afl-tmin.c +++ b/src/afl-tmin.c @@ -743,7 +743,11 @@ static void setup_signal_handlers(void) { struct sigaction sa; sa.sa_handler = NULL; + #ifdef SA_RESTART sa.sa_flags = SA_RESTART; + #else + sa.sa_flags = 0; + #endif sa.sa_sigaction = NULL; sigemptyset(&sa.sa_mask); -- cgit 1.4.1 From b1bfc1ae76bacbf48b81c81480a9f32e0c8919f6 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 27 Mar 2023 17:51:53 +0200 Subject: time_wo_finds in fuzzer_stats --- docs/Changelog.md | 1 + docs/afl-fuzz_approach.md | 1 + include/afl-fuzz.h | 1 + src/afl-fuzz-queue.c | 19 ++++++++++++++++++- src/afl-fuzz-stats.c | 6 ++++++ 5 files changed, 27 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 25c1f6bc..3c06a785 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -8,6 +8,7 @@ - ensure temporary file descriptor is closed when not used - added `AFL_NO_WARN_INSTABILITY` - added `AFL_FRIDA_STATS_INTERVAL` + - added time_wo_finds to fuzzer_stats - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index 6af39769..cb173f10 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -483,6 +483,7 @@ directory. This includes: - `fuzzer_pid` - PID of the fuzzer process - `cycles_done` - queue cycles completed so far - `cycles_wo_finds` - number of cycles without any new paths found +- `time_wo_finds` - longest time in seconds no new path was found - `execs_done` - number of execve() calls attempted - `execs_per_sec` - overall number of execs per second - `corpus_count` - total number of entries in the queue diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 6a8e8b5d..58d02af5 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -591,6 +591,7 @@ typedef struct afl_state { last_find_time, /* Time for most recent path (ms) */ last_crash_time, /* Time for most recent crash (ms) */ last_hang_time, /* Time for most recent hang (ms) */ + longest_find_time, /* Longest time taken for a find */ exit_on_time, /* Delay to exit if no new paths */ sync_time; /* Sync time (ms) */ diff --git a/src/afl-fuzz-queue.c b/src/afl-fuzz-queue.c index 4eb55bb3..40184645 100644 --- a/src/afl-fuzz-queue.c +++ b/src/afl-fuzz-queue.c @@ -577,7 +577,24 @@ void add_to_queue(afl_state_t *afl, u8 *fname, u32 len, u8 passed_det) { queue_buf[afl->queued_items - 1] = q; q->id = afl->queued_items - 1; - afl->last_find_time = get_cur_time(); + u64 cur_time = get_cur_time(); + + if (likely(afl->start_time) && + unlikely(afl->longest_find_time < cur_time - afl->last_find_time)) { + + if (unlikely(!afl->last_find_time)) { + + afl->longest_find_time = cur_time - afl->start_time; + + } else { + + afl->longest_find_time = cur_time - afl->last_find_time; + + } + + } + + afl->last_find_time = cur_time; if (afl->custom_mutators_count) { diff --git a/src/afl-fuzz-stats.c b/src/afl-fuzz-stats.c index f53fd610..25ebe987 100644 --- a/src/afl-fuzz-stats.c +++ b/src/afl-fuzz-stats.c @@ -251,6 +251,7 @@ void write_stats_file(afl_state_t *afl, u32 t_bytes, double bitmap_cvg, "fuzzer_pid : %u\n" "cycles_done : %llu\n" "cycles_wo_finds : %llu\n" + "time_wo_finds : %llu\n" "execs_done : %llu\n" "execs_per_sec : %0.02f\n" "execs_ps_last_min : %0.02f\n" @@ -291,6 +292,11 @@ void write_stats_file(afl_state_t *afl, u32 t_bytes, double bitmap_cvg, (afl->start_time - afl->prev_run_time) / 1000, cur_time / 1000, (afl->prev_run_time + cur_time - afl->start_time) / 1000, (u32)getpid(), afl->queue_cycle ? (afl->queue_cycle - 1) : 0, afl->cycles_wo_finds, + afl->longest_find_time > cur_time - afl->last_find_time + ? afl->longest_find_time / 1000 + : ((afl->start_time == 0 || afl->last_find_time == 0) + ? 0 + : (cur_time - afl->last_find_time) / 1000), afl->fsrv.total_execs, afl->fsrv.total_execs / ((double)(afl->prev_run_time + get_cur_time() - afl->start_time) / -- cgit 1.4.1 From e46e0bce44f0799731f5e7724ba3dfacafd4c41a Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sun, 2 Apr 2023 12:03:45 +0200 Subject: allow pizza mode to be disabled --- docs/Changelog.md | 3 +++ docs/env_variables.md | 2 +- include/afl-fuzz.h | 6 ++++-- src/afl-fuzz-state.c | 10 +++++++++- 4 files changed, 17 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 3c06a785..fbf50137 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -9,6 +9,9 @@ - added `AFL_NO_WARN_INSTABILITY` - added `AFL_FRIDA_STATS_INTERVAL` - added time_wo_finds to fuzzer_stats + - fixed a crash in pizza (1st april easter egg) mode. Sorry for + everyone who was affected! + - allow pizza mode to be disabled when AFL_PIZZA_MODE is set to -1 - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) diff --git a/docs/env_variables.md b/docs/env_variables.md index c9dc1bbd..a6a0ae44 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -581,7 +581,7 @@ checks or alter some of the more exotic semantics of the tool: constructors in your target, you can set `AFL_EARLY_FORKSERVER`. Note that this is not a compile time option but a runtime option :-) - - Set `AFL_PIZZA_MODE` to 1 to enable the April 1st stats menu, set to 0 + - Set `AFL_PIZZA_MODE` to 1 to enable the April 1st stats menu, set to -1 to disable although it is 1st of April. - If you need a specific interval to update fuzzer_stats file, you can diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 58d02af5..5fd393dd 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -398,8 +398,8 @@ typedef struct afl_env_vars { afl_bench_until_crash, afl_debug_child, afl_autoresume, afl_cal_fast, afl_cycle_schedules, afl_expand_havoc, afl_statsd, afl_cmplog_only_new, afl_exit_on_seed_issues, afl_try_affinity, afl_ignore_problems, - afl_keep_timeouts, afl_pizza_mode, afl_no_crash_readme, - afl_ignore_timeouts, afl_no_startup_calibration, afl_no_warn_instability; + afl_keep_timeouts, afl_no_crash_readme, afl_ignore_timeouts, + afl_no_startup_calibration, afl_no_warn_instability; u8 *afl_tmpdir, *afl_custom_mutator_library, *afl_python_module, *afl_path, *afl_hang_tmout, *afl_forksrv_init_tmout, *afl_preload, @@ -408,6 +408,8 @@ typedef struct afl_env_vars { *afl_testcache_entries, *afl_child_kill_signal, *afl_fsrv_kill_signal, *afl_target_env, *afl_persistent_record, *afl_exit_on_time; + s32 afl_pizza_mode; + } afl_env_vars_t; struct afl_pass_stat { diff --git a/src/afl-fuzz-state.c b/src/afl-fuzz-state.c index f9aa5cfe..46b67def 100644 --- a/src/afl-fuzz-state.c +++ b/src/afl-fuzz-state.c @@ -648,7 +648,15 @@ void read_afl_environment(afl_state_t *afl, char **envp) { } - if (afl->afl_env.afl_pizza_mode) { afl->pizza_is_served = 1; } + if (afl->afl_env.afl_pizza_mode > 0) { + + afl->pizza_is_served = 1; + + } else if (afl->afl_env.afl_pizza_mode < 0) { + + OKF("Pizza easter egg mode is now disabled."); + + } if (issue_detected) { sleep(2); } -- cgit 1.4.1 From 36127fb1970746f53fec44f9394061f57a4e94c3 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 5 Apr 2023 12:59:20 +0200 Subject: add -z switch --- docs/Changelog.md | 1 + include/afl-fuzz.h | 3 ++- src/afl-fuzz-queue.c | 7 ++++++- src/afl-fuzz.c | 5 +++++ 4 files changed, 14 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index fbf50137..8127e594 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -12,6 +12,7 @@ - fixed a crash in pizza (1st april easter egg) mode. Sorry for everyone who was affected! - allow pizza mode to be disabled when AFL_PIZZA_MODE is set to -1 + - add -z switch to prefer new coverage findings in seed selection - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 5fd393dd..7ff3315b 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -501,7 +501,8 @@ typedef struct afl_state { custom_splice_optout, /* Custom mutator no splice buffer */ is_main_node, /* if this is the main node */ is_secondary_node, /* if this is a secondary instance */ - pizza_is_served; /* pizza mode */ + pizza_is_served, /* pizza mode */ + prefer_new; /* prefer new queue entries */ u32 stats_update_freq; /* Stats update frequency (execs) */ diff --git a/src/afl-fuzz-queue.c b/src/afl-fuzz-queue.c index 40184645..1cdc8b54 100644 --- a/src/afl-fuzz-queue.c +++ b/src/afl-fuzz-queue.c @@ -74,9 +74,14 @@ double compute_weight(afl_state_t *afl, struct queue_entry *q, if (likely(afl->schedule < RARE)) { weight *= (avg_exec_us / q->exec_us); } weight *= (log(q->bitmap_size) / avg_bitmap_size); weight *= (1 + (q->tc_ref / avg_top_size)); - if (unlikely(weight < 1.0)) { weight = 1.0; } + if (unlikely(weight < 0.1)) { weight = 0.1; } if (unlikely(q->favored)) { weight *= 5; } if (unlikely(!q->was_fuzzed)) { weight *= 2; } + if (unlikely(afl->prefer_new)) { + + weight *= (2.0 * (q->id / (afl->queued_items - 1))); + + } return weight; diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index 3380fd90..0f01360e 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -132,6 +132,7 @@ static void usage(u8 *argv0, int more_help) { " fast(default), explore, exploit, seek, rare, mmopt, " "coe, lin\n" " quad -- see docs/FAQ.md for more information\n" + " -z - prefer new coverage findings when fuzzing\n" " -f file - location read by the fuzzed program (default: stdin " "or @@)\n" " -t msec - timeout for each run (auto-scaled, default %u ms). " @@ -569,6 +570,10 @@ int main(int argc, char **argv_orig, char **envp) { afl->max_length = atoi(optarg); break; + case 'z': + afl->prefer_new = 1; + break; + case 'Z': afl->old_seed_selection = 1; break; -- cgit 1.4.1 From 75d7a094691550afe86519a1d669def0d698b5ce Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 8 Apr 2023 13:48:07 +0200 Subject: show custom mutator name in UI --- docs/Changelog.md | 1 + src/afl-fuzz-mutators.c | 16 ++++++++++++---- src/afl-fuzz-one.c | 1 + src/afl-fuzz-python.c | 13 ++++++++++++- 4 files changed, 26 insertions(+), 5 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 8127e594..40c328ec 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -13,6 +13,7 @@ everyone who was affected! - allow pizza mode to be disabled when AFL_PIZZA_MODE is set to -1 - add -z switch to prefer new coverage findings in seed selection + - print name of custom mutator in UI - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 support (thanks to @devnexen!) diff --git a/src/afl-fuzz-mutators.c b/src/afl-fuzz-mutators.c index 9ea46e7a..64dbe7c6 100644 --- a/src/afl-fuzz-mutators.c +++ b/src/afl-fuzz-mutators.c @@ -179,11 +179,19 @@ struct custom_mutator *load_custom_mutator(afl_state_t *afl, const char *fn) { void *dh; struct custom_mutator *mutator = ck_alloc(sizeof(struct custom_mutator)); - mutator->name = fn; - if (memchr(fn, '/', strlen(fn))) - mutator->name_short = strrchr(fn, '/') + 1; - else + if (memchr(fn, '/', strlen(fn))) { + + mutator->name_short = strdup(strrchr(fn, '/') + 1); + + } else { + mutator->name_short = strdup(fn); + + } + + if (strlen(mutator->name_short) > 22) { mutator->name_short[21] = 0; } + + mutator->name = fn; ACTF("Loading custom mutator library from '%s'...", fn); dh = dlopen(fn, RTLD_NOW); diff --git a/src/afl-fuzz-one.c b/src/afl-fuzz-one.c index eec5e4b5..ee562f96 100644 --- a/src/afl-fuzz-one.c +++ b/src/afl-fuzz-one.c @@ -1931,6 +1931,7 @@ custom_mutator_stage: if (el->afl_custom_fuzz) { afl->current_custom_fuzz = el; + afl->stage_name = el->name_short; if (el->afl_custom_fuzz_count) { diff --git a/src/afl-fuzz-python.c b/src/afl-fuzz-python.c index 2799268b..673e5a6c 100644 --- a/src/afl-fuzz-python.c +++ b/src/afl-fuzz-python.c @@ -414,10 +414,21 @@ struct custom_mutator *load_custom_mutator_py(afl_state_t *afl, struct custom_mutator *mutator; mutator = ck_alloc(sizeof(struct custom_mutator)); - mutator->name = module_name; ACTF("Loading Python mutator library from '%s'...", module_name); + if (memchr(module_name, '/', strlen(module_name))) { + + mutator->name_short = strdup(strrchr(module_name, '/') + 1); + + } else { + + mutator->name_short = strdup(module_name); + + } + + if (strlen(mutator->name_short) > 22) { mutator->name_short[21] = 0; } + py_mutator_t *py_mutator; py_mutator = init_py_module(afl, module_name); mutator->data = py_mutator; -- cgit 1.4.1 From 824385f52ce3133ecd033e587aa1a3b324adf76c Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 12 Apr 2023 14:03:29 +0200 Subject: make llvm 17 work --- docs/Changelog.md | 2 +- instrumentation/SanitizerCoverageLTO.so.cc | 2 ++ instrumentation/SanitizerCoveragePCGUARD.so.cc | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 40c328ec..736deb30 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -16,7 +16,7 @@ - print name of custom mutator in UI - afl-cc: - add CFI sanitizer variant to gcc targets - - llvm 16 support (thanks to @devnexen!) + - llvm 16 + 17 support (thanks to @devnexen!) - support llvm 15 native pcguard changes - support for LLVMFuzzerTestOneInput -1 return - qemu_mode: diff --git a/instrumentation/SanitizerCoverageLTO.so.cc b/instrumentation/SanitizerCoverageLTO.so.cc index b024179a..5603c455 100644 --- a/instrumentation/SanitizerCoverageLTO.so.cc +++ b/instrumentation/SanitizerCoverageLTO.so.cc @@ -17,7 +17,9 @@ #include "llvm/Transforms/Instrumentation/SanitizerCoverage.h" #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" +#if LLVM_VERSION_MAJOR < 17 #include "llvm/ADT/Triple.h" +#endif #include "llvm/Analysis/EHPersonalities.h" #include "llvm/Analysis/PostDominators.h" #include "llvm/Analysis/ValueTracking.h" diff --git a/instrumentation/SanitizerCoveragePCGUARD.so.cc b/instrumentation/SanitizerCoveragePCGUARD.so.cc index c4a564f7..5f23698b 100644 --- a/instrumentation/SanitizerCoveragePCGUARD.so.cc +++ b/instrumentation/SanitizerCoveragePCGUARD.so.cc @@ -13,7 +13,9 @@ #include "llvm/Transforms/Instrumentation/SanitizerCoverage.h" #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" +#if LLVM_VERSION_MAJOR < 17 #include "llvm/ADT/Triple.h" +#endif #include "llvm/Analysis/EHPersonalities.h" #include "llvm/Analysis/PostDominators.h" #include "llvm/IR/CFG.h" -- cgit 1.4.1 From 6cc8d607fb24e060591ece4b42d83fc06de68fc6 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 13 Apr 2023 11:44:39 +0200 Subject: remove -z option, use -p mmopt instead --- GNUmakefile | 2 +- docs/Changelog.md | 2 +- include/afl-fuzz.h | 3 +-- instrumentation/SanitizerCoverageLTO.so.cc | 2 +- instrumentation/SanitizerCoveragePCGUARD.so.cc | 2 +- src/afl-fuzz-queue.c | 14 ++++---------- src/afl-fuzz.c | 7 +------ 7 files changed, 10 insertions(+), 22 deletions(-) (limited to 'docs') diff --git a/GNUmakefile b/GNUmakefile index 208e965b..85f164f5 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -546,7 +546,7 @@ ifndef AFL_NO_X86 test_build: afl-cc afl-gcc afl-as afl-showmap @echo "[*] Testing the CC wrapper afl-cc and its instrumentation output..." @unset AFL_MAP_SIZE AFL_USE_UBSAN AFL_USE_CFISAN AFL_USE_LSAN AFL_USE_ASAN AFL_USE_MSAN; ASAN_OPTIONS=detect_leaks=0 AFL_INST_RATIO=100 AFL_PATH=. ./afl-cc test-instr.c $(LDFLAGS) -o test-instr 2>&1 || (echo "Oops, afl-cc failed"; exit 1 ) - - ASAN_OPTIONS=detect_leaks=0 ./afl-showmap -m none -o .test-instr0 ./test-instr < /dev/null + -ASAN_OPTIONS=detect_leaks=0 ./afl-showmap -q -m none -o .test-instr0 ./test-instr < /dev/null -echo 1 | ASAN_OPTIONS=detect_leaks=0 ./afl-showmap -m none -q -o .test-instr1 ./test-instr @rm -f test-instr @cmp -s .test-instr0 .test-instr1; DR="$$?"; rm -f .test-instr0 .test-instr1; if [ "$$DR" = "0" ]; then echo; echo "Oops, the instrumentation of afl-cc does not seem to be behaving correctly!"; echo; echo "Please post to https://github.com/AFLplusplus/AFLplusplus/issues to troubleshoot the issue."; echo; exit 1; fi diff --git a/docs/Changelog.md b/docs/Changelog.md index 736deb30..501300b1 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -12,7 +12,7 @@ - fixed a crash in pizza (1st april easter egg) mode. Sorry for everyone who was affected! - allow pizza mode to be disabled when AFL_PIZZA_MODE is set to -1 - - add -z switch to prefer new coverage findings in seed selection + - option `-p mmopt` now also selects new queue items more often - print name of custom mutator in UI - afl-cc: - add CFI sanitizer variant to gcc targets diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 7ff3315b..5fd393dd 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -501,8 +501,7 @@ typedef struct afl_state { custom_splice_optout, /* Custom mutator no splice buffer */ is_main_node, /* if this is the main node */ is_secondary_node, /* if this is a secondary instance */ - pizza_is_served, /* pizza mode */ - prefer_new; /* prefer new queue entries */ + pizza_is_served; /* pizza mode */ u32 stats_update_freq; /* Stats update frequency (execs) */ diff --git a/instrumentation/SanitizerCoverageLTO.so.cc b/instrumentation/SanitizerCoverageLTO.so.cc index 5603c455..e41f19b6 100644 --- a/instrumentation/SanitizerCoverageLTO.so.cc +++ b/instrumentation/SanitizerCoverageLTO.so.cc @@ -18,7 +18,7 @@ #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" #if LLVM_VERSION_MAJOR < 17 -#include "llvm/ADT/Triple.h" + #include "llvm/ADT/Triple.h" #endif #include "llvm/Analysis/EHPersonalities.h" #include "llvm/Analysis/PostDominators.h" diff --git a/instrumentation/SanitizerCoveragePCGUARD.so.cc b/instrumentation/SanitizerCoveragePCGUARD.so.cc index 5f23698b..85b1ddd5 100644 --- a/instrumentation/SanitizerCoveragePCGUARD.so.cc +++ b/instrumentation/SanitizerCoveragePCGUARD.so.cc @@ -14,7 +14,7 @@ #include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/SmallVector.h" #if LLVM_VERSION_MAJOR < 17 -#include "llvm/ADT/Triple.h" + #include "llvm/ADT/Triple.h" #endif #include "llvm/Analysis/EHPersonalities.h" #include "llvm/Analysis/PostDominators.h" diff --git a/src/afl-fuzz-queue.c b/src/afl-fuzz-queue.c index 6fc3c743..8ad7cd97 100644 --- a/src/afl-fuzz-queue.c +++ b/src/afl-fuzz-queue.c @@ -149,21 +149,15 @@ void create_alias_table(afl_state_t *afl) { } - if (unlikely(afl->prefer_new) && afl->queued_discovered) { + if (unlikely(afl->schedule == MMOPT) && afl->queued_discovered) { - double avg_weight = sum / active; + u32 cnt = afl->queued_discovered >= 5 ? 5 : afl->queued_discovered; - for (i = n - afl->queued_discovered; i < n; i++) { + for (i = n - cnt; i < n; i++) { struct queue_entry *q = afl->queue_buf[i]; - if (likely(!q->disabled) && q->weight > avg_weight) { - - double prev_weight = q->weight; - q->weight *= (2.0 * (i / n)); - sum += (q->weight - prev_weight); - - } + if (likely(!q->disabled)) { q->weight *= 2.0; } } diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index a0c322da..5ba54d0b 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -132,7 +132,6 @@ static void usage(u8 *argv0, int more_help) { " fast(default), explore, exploit, seek, rare, mmopt, " "coe, lin\n" " quad -- see docs/FAQ.md for more information\n" - " -z - prefer new coverage findings when fuzzing\n" " -f file - location read by the fuzzed program (default: stdin " "or @@)\n" " -t msec - timeout for each run (auto-scaled, default %u ms). " @@ -556,7 +555,7 @@ int main(int argc, char **argv_orig, char **envp) { while ( (opt = getopt( argc, argv, - "+Ab:B:c:CdDe:E:hi:I:f:F:g:G:l:L:m:M:nNOo:p:RQs:S:t:T:UV:WXx:YzZ")) > + "+Ab:B:c:CdDe:E:hi:I:f:F:g:G:l:L:m:M:nNOo:p:RQs:S:t:T:UV:WXx:YZ")) > 0) { switch (opt) { @@ -569,10 +568,6 @@ int main(int argc, char **argv_orig, char **envp) { afl->max_length = atoi(optarg); break; - case 'z': - afl->prefer_new = 1; - break; - case 'Z': afl->old_seed_selection = 1; break; -- cgit 1.4.1 From f756734ad2782c3ed56feadb4b7b23fc82a7a968 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 13 Apr 2023 12:07:27 +0200 Subject: fix attempt at post_process implementation --- docs/Changelog.md | 1 + docs/custom_mutators.md | 9 +++++++-- include/afl-fuzz.h | 9 +++++++-- src/afl-fuzz-python.c | 49 +++++++++---------------------------------------- src/afl-fuzz-run.c | 7 ++++++- 5 files changed, 30 insertions(+), 45 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 501300b1..9436fc9f 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -13,6 +13,7 @@ everyone who was affected! - allow pizza mode to be disabled when AFL_PIZZA_MODE is set to -1 - option `-p mmopt` now also selects new queue items more often + - fix bug in post_process custom mutator implementation - print name of custom mutator in UI - afl-cc: - add CFI sanitizer variant to gcc targets diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 82131c92..a1de479e 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -118,7 +118,7 @@ def deinit(): # optional for Python ### Custom Mutation -- `init`: +- `init` (optional in Python): This method is called when AFL++ starts up and is used to seed RNG and set up buffers and state. @@ -184,6 +184,11 @@ def deinit(): # optional for Python to the target, e.g. if it is too short, too corrupted, etc. If so, return a NULL buffer and zero length (or a 0 length string in Python). + NOTE: Do not make any random changes to the data in this function! + + PERFORMANCE for C/C++: If possible make the changes in-place (so modify + the `*data` directly, and return it as `*outbuf = data`. + - `fuzz_send` (optional): This method can be used if you want to send data to the target yourself, @@ -202,7 +207,7 @@ def deinit(): # optional for Python discovered if compiled with INTROSPECTION. The custom mutator can then return a string (const char *) that reports the exact mutations used. -- `deinit`: +- `deinit` (optional in Python): The last method to be called, deinitializing the state. diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 5fd393dd..8b6502b4 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -885,14 +885,19 @@ struct custom_mutator { * A post-processing function to use right before AFL writes the test case to * disk in order to execute the target. * - * (Optional) If this functionality is not needed, simply don't define this + * NOTE: Do not do any random changes to the data in this function! + * + * PERFORMANCE: If you can modify the data in-place you will have a better + * performance. Modify *data and set `*out_buf = data`. + * + * (Optional) If this functionality is not needed, simply do not define this * function. * * @param[in] data pointer returned in afl_custom_init by this custom mutator * @param[in] buf Buffer containing the test case to be executed * @param[in] buf_size Size of the test case * @param[out] out_buf Pointer to the buffer storing the test case after - * processing. External library should allocate memory for out_buf. + * processing. The external library should allocate memory for out_buf. * It can chose to alter buf in-place, if the space is large enough. * @return Size of the output buffer. */ diff --git a/src/afl-fuzz-python.c b/src/afl-fuzz-python.c index 673e5a6c..7dad0770 100644 --- a/src/afl-fuzz-python.c +++ b/src/afl-fuzz-python.c @@ -219,11 +219,14 @@ static py_mutator_t *init_py_module(afl_state_t *afl, u8 *module_name) { if (py_module != NULL) { - u8 py_notrim = 0, py_idx; - /* init, required */ + u8 py_notrim = 0; py_functions[PY_FUNC_INIT] = PyObject_GetAttrString(py_module, "init"); - if (!py_functions[PY_FUNC_INIT]) - FATAL("init function not found in python module"); + if (!py_functions[PY_FUNC_INIT]) { + + WARNF("init function not found in python module"); + + } + py_functions[PY_FUNC_FUZZ] = PyObject_GetAttrString(py_module, "fuzz"); if (!py_functions[PY_FUNC_FUZZ]) py_functions[PY_FUNC_FUZZ] = PyObject_GetAttrString(py_module, "mutate"); @@ -231,12 +234,6 @@ static py_mutator_t *init_py_module(afl_state_t *afl, u8 *module_name) { PyObject_GetAttrString(py_module, "describe"); py_functions[PY_FUNC_FUZZ_COUNT] = PyObject_GetAttrString(py_module, "fuzz_count"); - if (!py_functions[PY_FUNC_FUZZ]) { - - WARNF("fuzz function not found in python module"); - - } - py_functions[PY_FUNC_POST_PROCESS] = PyObject_GetAttrString(py_module, "post_process"); py_functions[PY_FUNC_INIT_TRIM] = @@ -263,36 +260,6 @@ static py_mutator_t *init_py_module(afl_state_t *afl, u8 *module_name) { if (!py_functions[PY_FUNC_DEINIT]) WARNF("deinit function not found in python module"); - for (py_idx = 0; py_idx < PY_FUNC_COUNT; ++py_idx) { - - if (!py_functions[py_idx] || !PyCallable_Check(py_functions[py_idx])) { - - if (py_idx >= PY_FUNC_INIT_TRIM && py_idx <= PY_FUNC_TRIM) { - - // Implementing the trim API is optional for now - if (PyErr_Occurred()) { PyErr_Print(); } - py_notrim = 1; - - } else if (py_idx >= PY_OPTIONAL) { - - // Only _init and _deinit are not optional currently - - if (PyErr_Occurred()) { PyErr_Print(); } - - } else { - - fprintf(stderr, - "Cannot find/call function with index %d in external " - "Python module.\n", - py_idx); - return NULL; - - } - - } - - } - if (py_notrim) { py_functions[PY_FUNC_INIT_TRIM] = NULL; @@ -345,6 +312,8 @@ static void init_py(afl_state_t *afl, py_mutator_t *py_mutator, (void)afl; + if (py_mutator->py_functions[PY_FUNC_INIT] == NULL) { return; } + PyObject *py_args, *py_value; /* Provide the init function a seed for the Python RNG */ diff --git a/src/afl-fuzz-run.c b/src/afl-fuzz-run.c index f5425011..26e8549d 100644 --- a/src/afl-fuzz-run.c +++ b/src/afl-fuzz-run.c @@ -133,7 +133,12 @@ write_to_testcase(afl_state_t *afl, void **mem, u32 len, u32 fix) { } - if (new_mem != *mem) { *mem = new_mem; } + if (new_mem != *mem && new_mem != NULL && new_size > 0) { + + *mem = afl_realloc((void **)mem, new_size); + memmove(*mem, new_mem, new_size); + + } if (unlikely(afl->custom_mutators_count)) { -- cgit 1.4.1 From b5f7f42cd0a1bce83c8c6d4af9846e7c5da3cdd1 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Fri, 14 Apr 2023 10:22:00 +0200 Subject: update qemuafl, info in afl-plot --- afl-plot | 4 ++-- docs/Changelog.md | 14 +++++++++----- qemu_mode/QEMUAFL_VERSION | 2 +- qemu_mode/qemuafl | 2 +- 4 files changed, 13 insertions(+), 9 deletions(-) (limited to 'docs') diff --git a/afl-plot b/afl-plot index 90a46d24..230d3bfe 100755 --- a/afl-plot +++ b/afl-plot @@ -287,9 +287,9 @@ $PLOT_EG _EOF_ -) | gnuplot +) | gnuplot || echo "Note: if you see errors concerning 'unknown or ambiguous terminal type' then you need to use a gnuplot that has png support compiled in." -echo "[?] You can also use -g flag to view the plots in an GUI window, and interact with the plots (if you have built afl-plot-ui). Run \"afl-plot-h\" to know more." +echo "[?] You can also use -g flag to view the plots in an GUI window, and interact with the plots (if you have built afl-plot-ui). Run \"afl-plot -h\" to know more." fi diff --git a/docs/Changelog.md b/docs/Changelog.md index 9436fc9f..9f4a8653 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -7,7 +7,6 @@ - afl-fuzz: - ensure temporary file descriptor is closed when not used - added `AFL_NO_WARN_INSTABILITY` - - added `AFL_FRIDA_STATS_INTERVAL` - added time_wo_finds to fuzzer_stats - fixed a crash in pizza (1st april easter egg) mode. Sorry for everyone who was affected! @@ -15,18 +14,23 @@ - option `-p mmopt` now also selects new queue items more often - fix bug in post_process custom mutator implementation - print name of custom mutator in UI + - slight changes that improve fuzzer performance - afl-cc: - add CFI sanitizer variant to gcc targets - llvm 16 + 17 support (thanks to @devnexen!) - support llvm 15 native pcguard changes - support for LLVMFuzzerTestOneInput -1 return + - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support - qemu_mode: - fix _RANGES envs to allow hyphens in the filenames - - new custom module: autotoken, grammar free fuzzer for text inputs - - LTO autoken and llvm_mode: added AFL_LLVM_DICT2FILE_NO_MAIN support + - basic riscv support + - frida_mode: + - added `AFL_FRIDA_STATS_INTERVAL` + - fix issue on MacOS + - unicorn_mode: + - updated and minor issues fixed + - new custom module: autotoken, a grammar free fuzzer for text inputs - better sanitizer default options support for all tools - - unicorn_mode: updated and minor issues fixed - - frida_mode: fix issue on MacOS - more minor fixes and cross-platform support ### Version ++4.05c (release) diff --git a/qemu_mode/QEMUAFL_VERSION b/qemu_mode/QEMUAFL_VERSION index 43dc832b..fa44d173 100644 --- a/qemu_mode/QEMUAFL_VERSION +++ b/qemu_mode/QEMUAFL_VERSION @@ -1 +1 @@ -249bf0c872 +0569eff8a1 diff --git a/qemu_mode/qemuafl b/qemu_mode/qemuafl index 249bf0c8..0569eff8 160000 --- a/qemu_mode/qemuafl +++ b/qemu_mode/qemuafl @@ -1 +1 @@ -Subproject commit 249bf0c8723671a1eebe400a9631d9e69306ff4c +Subproject commit 0569eff8a12dec73642b96757f6b5b51a618a03a -- cgit 1.4.1 From e12acaa20367f335549c2db97b88ac5c8ffbeab7 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 15 Apr 2023 10:12:20 +0200 Subject: fix custom mutator C examples --- custom_mutators/examples/custom_mutator_helpers.h | 342 ---------------------- custom_mutators/examples/custom_send.c | 8 +- custom_mutators/examples/example.c | 116 +++----- custom_mutators/examples/post_library_gif.so.c | 53 +--- custom_mutators/examples/post_library_png.so.c | 24 +- custom_mutators/examples/simple_example.c | 28 +- docs/Changelog.md | 1 + 7 files changed, 73 insertions(+), 499 deletions(-) delete mode 100644 custom_mutators/examples/custom_mutator_helpers.h (limited to 'docs') diff --git a/custom_mutators/examples/custom_mutator_helpers.h b/custom_mutators/examples/custom_mutator_helpers.h deleted file mode 100644 index 62e6efba..00000000 --- a/custom_mutators/examples/custom_mutator_helpers.h +++ /dev/null @@ -1,342 +0,0 @@ -#ifndef CUSTOM_MUTATOR_HELPERS -#define CUSTOM_MUTATOR_HELPERS - -#include "config.h" -#include "types.h" -#include - -#define INITIAL_GROWTH_SIZE (64) - -#define RAND_BELOW(limit) (rand() % (limit)) - -/* Use in a struct: creates a name_buf and a name_size variable. */ -#define BUF_VAR(type, name) \ - type * name##_buf; \ - size_t name##_size; -/* this fills in `&structptr->something_buf, &structptr->something_size`. */ -#define BUF_PARAMS(struct, name) \ - (void **)&struct->name##_buf, &struct->name##_size - -typedef struct { - -} afl_t; - -static void surgical_havoc_mutate(u8 *out_buf, s32 begin, s32 end) { - - static s8 interesting_8[] = {INTERESTING_8}; - static s16 interesting_16[] = {INTERESTING_8, INTERESTING_16}; - static s32 interesting_32[] = {INTERESTING_8, INTERESTING_16, INTERESTING_32}; - - switch (RAND_BELOW(12)) { - - case 0: { - - /* Flip a single bit somewhere. Spooky! */ - - s32 bit_idx = ((RAND_BELOW(end - begin) + begin) << 3) + RAND_BELOW(8); - - out_buf[bit_idx >> 3] ^= 128 >> (bit_idx & 7); - - break; - - } - - case 1: { - - /* Set byte to interesting value. */ - - u8 val = interesting_8[RAND_BELOW(sizeof(interesting_8))]; - out_buf[(RAND_BELOW(end - begin) + begin)] = val; - - break; - - } - - case 2: { - - /* Set word to interesting value, randomly choosing endian. */ - - if (end - begin < 2) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 1) break; - - switch (RAND_BELOW(2)) { - - case 0: - *(u16 *)(out_buf + byte_idx) = - interesting_16[RAND_BELOW(sizeof(interesting_16) >> 1)]; - break; - case 1: - *(u16 *)(out_buf + byte_idx) = - SWAP16(interesting_16[RAND_BELOW(sizeof(interesting_16) >> 1)]); - break; - - } - - break; - - } - - case 3: { - - /* Set dword to interesting value, randomly choosing endian. */ - - if (end - begin < 4) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 3) break; - - switch (RAND_BELOW(2)) { - - case 0: - *(u32 *)(out_buf + byte_idx) = - interesting_32[RAND_BELOW(sizeof(interesting_32) >> 2)]; - break; - case 1: - *(u32 *)(out_buf + byte_idx) = - SWAP32(interesting_32[RAND_BELOW(sizeof(interesting_32) >> 2)]); - break; - - } - - break; - - } - - case 4: { - - /* Set qword to interesting value, randomly choosing endian. */ - - if (end - begin < 8) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 7) break; - - switch (RAND_BELOW(2)) { - - case 0: - *(u64 *)(out_buf + byte_idx) = - (s64)interesting_32[RAND_BELOW(sizeof(interesting_32) >> 2)]; - break; - case 1: - *(u64 *)(out_buf + byte_idx) = SWAP64( - (s64)interesting_32[RAND_BELOW(sizeof(interesting_32) >> 2)]); - break; - - } - - break; - - } - - case 5: { - - /* Randomly subtract from byte. */ - - out_buf[(RAND_BELOW(end - begin) + begin)] -= 1 + RAND_BELOW(ARITH_MAX); - - break; - - } - - case 6: { - - /* Randomly add to byte. */ - - out_buf[(RAND_BELOW(end - begin) + begin)] += 1 + RAND_BELOW(ARITH_MAX); - - break; - - } - - case 7: { - - /* Randomly subtract from word, random endian. */ - - if (end - begin < 2) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 1) break; - - if (RAND_BELOW(2)) { - - *(u16 *)(out_buf + byte_idx) -= 1 + RAND_BELOW(ARITH_MAX); - - } else { - - u16 num = 1 + RAND_BELOW(ARITH_MAX); - - *(u16 *)(out_buf + byte_idx) = - SWAP16(SWAP16(*(u16 *)(out_buf + byte_idx)) - num); - - } - - break; - - } - - case 8: { - - /* Randomly add to word, random endian. */ - - if (end - begin < 2) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 1) break; - - if (RAND_BELOW(2)) { - - *(u16 *)(out_buf + byte_idx) += 1 + RAND_BELOW(ARITH_MAX); - - } else { - - u16 num = 1 + RAND_BELOW(ARITH_MAX); - - *(u16 *)(out_buf + byte_idx) = - SWAP16(SWAP16(*(u16 *)(out_buf + byte_idx)) + num); - - } - - break; - - } - - case 9: { - - /* Randomly subtract from dword, random endian. */ - - if (end - begin < 4) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 3) break; - - if (RAND_BELOW(2)) { - - *(u32 *)(out_buf + byte_idx) -= 1 + RAND_BELOW(ARITH_MAX); - - } else { - - u32 num = 1 + RAND_BELOW(ARITH_MAX); - - *(u32 *)(out_buf + byte_idx) = - SWAP32(SWAP32(*(u32 *)(out_buf + byte_idx)) - num); - - } - - break; - - } - - case 10: { - - /* Randomly add to dword, random endian. */ - - if (end - begin < 4) break; - - s32 byte_idx = (RAND_BELOW(end - begin) + begin); - - if (byte_idx >= end - 3) break; - - if (RAND_BELOW(2)) { - - *(u32 *)(out_buf + byte_idx) += 1 + RAND_BELOW(ARITH_MAX); - - } else { - - u32 num = 1 + RAND_BELOW(ARITH_MAX); - - *(u32 *)(out_buf + byte_idx) = - SWAP32(SWAP32(*(u32 *)(out_buf + byte_idx)) + num); - - } - - break; - - } - - case 11: { - - /* Just set a random byte to a random value. Because, - why not. We use XOR with 1-255 to eliminate the - possibility of a no-op. */ - - out_buf[(RAND_BELOW(end - begin) + begin)] ^= 1 + RAND_BELOW(255); - - break; - - } - - } - -} - -/* This function calculates the next power of 2 greater or equal its argument. - @return The rounded up power of 2 (if no overflow) or 0 on overflow. -*/ -static inline size_t next_pow2(size_t in) { - - if (in == 0 || in > (size_t)-1) - return 0; /* avoid undefined behaviour under-/overflow */ - size_t out = in - 1; - out |= out >> 1; - out |= out >> 2; - out |= out >> 4; - out |= out >> 8; - out |= out >> 16; - return out + 1; - -} - -/* This function makes sure *size is > size_needed after call. - It will realloc *buf otherwise. - *size will grow exponentially as per: - https://blog.mozilla.org/nnethercote/2014/11/04/please-grow-your-buffers-exponentially/ - Will return NULL and free *buf if size_needed is <1 or realloc failed. - @return For convenience, this function returns *buf. - */ -static inline void *maybe_grow(void **buf, size_t *size, size_t size_needed) { - - /* No need to realloc */ - if (likely(size_needed && *size >= size_needed)) return *buf; - - /* No initial size was set */ - if (size_needed < INITIAL_GROWTH_SIZE) size_needed = INITIAL_GROWTH_SIZE; - - /* grow exponentially */ - size_t next_size = next_pow2(size_needed); - - /* handle overflow */ - if (!next_size) { next_size = size_needed; } - - /* alloc */ - *buf = realloc(*buf, next_size); - *size = *buf ? next_size : 0; - - return *buf; - -} - -/* Swaps buf1 ptr and buf2 ptr, as well as their sizes */ -static inline void afl_swap_bufs(void **buf1, size_t *size1, void **buf2, - size_t *size2) { - - void * scratch_buf = *buf1; - size_t scratch_size = *size1; - *buf1 = *buf2; - *size1 = *size2; - *buf2 = scratch_buf; - *size2 = scratch_size; - -} - -#undef INITIAL_GROWTH_SIZE - -#endif - diff --git a/custom_mutators/examples/custom_send.c b/custom_mutators/examples/custom_send.c index 7de72819..9cc4b160 100644 --- a/custom_mutators/examples/custom_send.c +++ b/custom_mutators/examples/custom_send.c @@ -10,21 +10,21 @@ // afl-fuzz -i in -o out -- ./test-instr -f /tmp/foo // -#include "custom_mutator_helpers.h" - #include #include #include #include #include +#include "afl-fuzz.h" + typedef struct my_mutator { - afl_t *afl; + afl_state_t *afl; } my_mutator_t; -my_mutator_t *afl_custom_init(afl_t *afl, unsigned int seed) { +my_mutator_t *afl_custom_init(afl_state_t *afl, unsigned int seed) { my_mutator_t *data = calloc(1, sizeof(my_mutator_t)); if (!data) { diff --git a/custom_mutators/examples/example.c b/custom_mutators/examples/example.c index e680ec8e..42c7469c 100644 --- a/custom_mutators/examples/example.c +++ b/custom_mutators/examples/example.c @@ -7,7 +7,7 @@ */ // You need to use -I/path/to/AFLplusplus/include -I. -#include "custom_mutator_helpers.h" +#include "afl-fuzz.h" #include #include @@ -26,19 +26,14 @@ static const char *commands[] = { typedef struct my_mutator { - afl_t *afl; + afl_state_t *afl; // any additional data here! size_t trim_size_current; int trimmming_steps; int cur_step; - // Reused buffers: - BUF_VAR(u8, fuzz); - BUF_VAR(u8, data); - BUF_VAR(u8, havoc); - BUF_VAR(u8, trim); - BUF_VAR(u8, post_process); + u8 *mutated_out, *post_process_buf, *trim_buf; } my_mutator_t; @@ -53,7 +48,7 @@ typedef struct my_mutator { * There may be multiple instances of this mutator in one afl-fuzz run! * Return NULL on error. */ -my_mutator_t *afl_custom_init(afl_t *afl, unsigned int seed) { +my_mutator_t *afl_custom_init(afl_state_t *afl, unsigned int seed) { srand(seed); // needed also by surgical_havoc_mutate() @@ -65,6 +60,27 @@ my_mutator_t *afl_custom_init(afl_t *afl, unsigned int seed) { } + if ((data->mutated_out = (u8 *)malloc(MAX_FILE)) == NULL) { + + perror("afl_custom_init malloc"); + return NULL; + + } + + if ((data->post_process_buf = (u8 *)malloc(MAX_FILE)) == NULL) { + + perror("afl_custom_init malloc"); + return NULL; + + } + + if ((data->trim_buf = (u8 *)malloc(MAX_FILE)) == NULL) { + + perror("afl_custom_init malloc"); + return NULL; + + } + data->afl = afl; return data; @@ -96,31 +112,14 @@ size_t afl_custom_fuzz(my_mutator_t *data, uint8_t *buf, size_t buf_size, // the fuzzer size_t mutated_size = DATA_SIZE <= max_size ? DATA_SIZE : max_size; - // maybe_grow is optimized to be quick for reused buffers. - u8 *mutated_out = maybe_grow(BUF_PARAMS(data, fuzz), mutated_size); - if (!mutated_out) { - - *out_buf = NULL; - perror("custom mutator allocation (maybe_grow)"); - return 0; /* afl-fuzz will very likely error out after this. */ - - } + memcpy(data->mutated_out, buf, buf_size); // Randomly select a command string to add as a header to the packet - memcpy(mutated_out, commands[rand() % 3], 3); + memcpy(data->mutated_out, commands[rand() % 3], 3); - // Mutate the payload of the packet - int i; - for (i = 0; i < 8; ++i) { + if (mutated_size > max_size) { mutated_size = max_size; } - // Randomly perform one of the (no len modification) havoc mutations - surgical_havoc_mutate(mutated_out, 3, mutated_size); - - } - - if (max_size > mutated_size) { mutated_size = max_size; } - - *out_buf = mutated_out; + *out_buf = data->mutated_out; return mutated_size; } @@ -144,24 +143,16 @@ size_t afl_custom_fuzz(my_mutator_t *data, uint8_t *buf, size_t buf_size, size_t afl_custom_post_process(my_mutator_t *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf) { - uint8_t *post_process_buf = - maybe_grow(BUF_PARAMS(data, post_process), buf_size + 5); - if (!post_process_buf) { + if (buf_size + 5 > MAX_FILE) { buf_size = MAX_FILE - 5; } - perror("custom mutator realloc failed."); - *out_buf = NULL; - return 0; + memcpy(data->post_process_buf + 5, buf, buf_size); + data->post_process_buf[0] = 'A'; + data->post_process_buf[1] = 'F'; + data->post_process_buf[2] = 'L'; + data->post_process_buf[3] = '+'; + data->post_process_buf[4] = '+'; - } - - memcpy(post_process_buf + 5, buf, buf_size); - post_process_buf[0] = 'A'; - post_process_buf[1] = 'F'; - post_process_buf[2] = 'L'; - post_process_buf[3] = '+'; - post_process_buf[4] = '+'; - - *out_buf = post_process_buf; + *out_buf = data->post_process_buf; return buf_size + 5; @@ -197,13 +188,6 @@ int32_t afl_custom_init_trim(my_mutator_t *data, uint8_t *buf, data->cur_step = 0; - if (!maybe_grow(BUF_PARAMS(data, trim), buf_size)) { - - perror("init_trim grow"); - return -1; - - } - memcpy(data->trim_buf, buf, buf_size); data->trim_size_current = buf_size; @@ -284,27 +268,11 @@ int32_t afl_custom_post_trim(my_mutator_t *data, int success) { size_t afl_custom_havoc_mutation(my_mutator_t *data, u8 *buf, size_t buf_size, u8 **out_buf, size_t max_size) { - if (buf_size == 0) { - - *out_buf = maybe_grow(BUF_PARAMS(data, havoc), 1); - if (!*out_buf) { - - perror("custom havoc: maybe_grow"); - return 0; - - } + *out_buf = buf; // in-place mutation - **out_buf = rand() % 256; - buf_size = 1; - - } else { - - // We reuse buf here. It's legal and faster. - *out_buf = buf; - - } + if (buf_size <= sizeof(size_t)) { return buf_size; } - size_t victim = rand() % buf_size; + size_t victim = rand() % (buf_size - sizeof(size_t)); (*out_buf)[victim] += rand() % 10; return buf_size; @@ -371,9 +339,7 @@ uint8_t afl_custom_queue_new_entry(my_mutator_t *data, void afl_custom_deinit(my_mutator_t *data) { free(data->post_process_buf); - free(data->havoc_buf); - free(data->data_buf); - free(data->fuzz_buf); + free(data->mutated_out); free(data->trim_buf); free(data); diff --git a/custom_mutators/examples/post_library_gif.so.c b/custom_mutators/examples/post_library_gif.so.c index 3cb018a6..6737c627 100644 --- a/custom_mutators/examples/post_library_gif.so.c +++ b/custom_mutators/examples/post_library_gif.so.c @@ -45,9 +45,8 @@ 1) If you don't want to modify the test case, simply set `*out_buf = in_buf` and return the original `len`. - NOTE: the following is currently NOT true, we abort in this case! 2) If you want to skip this test case altogether and have AFL generate a - new one, return 0 or set `*out_buf = NULL`. + new one, return 0. Use this sparingly - it's faster than running the target program with patently useless inputs, but still wastes CPU time. @@ -59,8 +58,6 @@ Note that the buffer will *not* be freed for you. To avoid memory leaks, you need to free it or reuse it on subsequent calls (as shown below). - *** Feel free to reuse the original 'in_buf' BUFFER and return it. *** - Alright. The example below shows a simple postprocessor that tries to make sure that all input files start with "GIF89a". @@ -72,7 +69,7 @@ #include #include #include -#include "alloc-inl.h" +#include "afl-fuzz.h" /* Header that must be present at the beginning of every test case: */ @@ -80,8 +77,7 @@ typedef struct post_state { - unsigned char *buf; - size_t size; + size_t size; } post_state_t; @@ -95,15 +91,6 @@ void *afl_custom_init(void *afl) { } - state->buf = calloc(sizeof(unsigned char), 4096); - if (!state->buf) { - - free(state); - perror("calloc"); - return NULL; - - } - return state; } @@ -113,6 +100,10 @@ void *afl_custom_init(void *afl) { size_t afl_custom_post_process(post_state_t *data, unsigned char *in_buf, unsigned int len, unsigned char **out_buf) { + /* we do in-place modification as we do not increase the size */ + + *out_buf = in_buf; + /* Skip execution altogether for buffers shorter than 6 bytes (just to show how it's done). We can trust len to be sane. */ @@ -120,34 +111,7 @@ size_t afl_custom_post_process(post_state_t *data, unsigned char *in_buf, /* Do nothing for buffers that already start with the expected header. */ - if (!memcmp(in_buf, HEADER, strlen(HEADER))) { - - *out_buf = in_buf; - return len; - - } - - /* Allocate memory for new buffer, reusing previous allocation if - possible. Note we have to use afl-fuzz's own realloc! - We use afl_realloc because it is effective. - You can also work within in_buf, and assign it to *out_buf. */ - - *out_buf = afl_realloc(out_buf, len); - - /* If we're out of memory, the most graceful thing to do is to return the - original buffer and give up on modifying it. Let AFL handle OOM on its - own later on. */ - - if (!*out_buf) { - - *out_buf = in_buf; - return len; - - } - - if (len > strlen(HEADER)) - memcpy(*out_buf + strlen(HEADER), in_buf + strlen(HEADER), - len - strlen(HEADER)); + if (!memcmp(in_buf, HEADER, strlen(HEADER))) { return len; } /* Insert the new header. */ @@ -162,7 +126,6 @@ size_t afl_custom_post_process(post_state_t *data, unsigned char *in_buf, /* Gets called afterwards */ void afl_custom_deinit(post_state_t *data) { - free(data->buf); free(data); } diff --git a/custom_mutators/examples/post_library_png.so.c b/custom_mutators/examples/post_library_png.so.c index cd65b1bc..652da497 100644 --- a/custom_mutators/examples/post_library_png.so.c +++ b/custom_mutators/examples/post_library_png.so.c @@ -30,7 +30,7 @@ #include #include #include -#include "alloc-inl.h" +#include "afl-fuzz.h" /* A macro to round an integer up to 4 kB. */ @@ -53,7 +53,7 @@ void *afl_custom_init(void *afl) { } - state->buf = calloc(sizeof(unsigned char), 4096); + state->buf = calloc(sizeof(unsigned char), MAX_FILE); if (!state->buf) { free(state); @@ -80,21 +80,7 @@ size_t afl_custom_post_process(post_state_t *data, const unsigned char *in_buf, } - /* This is not a good way to do it, if you do not need to grow the buffer - then just work with in_buf instead for speed reasons. - But we want to show how to grow a buffer, so this is how it's done: */ - - unsigned int pos = 8; - unsigned char *new_buf = afl_realloc(out_buf, UP4K(len)); - - if (!new_buf) { - - *out_buf = in_buf; - return len; - - } - - memcpy(new_buf, in_buf, len); + unsigned int pos = 8; /* Minimum size of a zero-length PNG chunk is 12 bytes; if we don't have that, we can bail out. */ @@ -124,7 +110,7 @@ size_t afl_custom_post_process(post_state_t *data, const unsigned char *in_buf, if (real_cksum != file_cksum) { - *(uint32_t *)(new_buf + pos + 8 + chunk_len) = real_cksum; + *(uint32_t *)(data->buf + pos + 8 + chunk_len) = real_cksum; } @@ -134,7 +120,7 @@ size_t afl_custom_post_process(post_state_t *data, const unsigned char *in_buf, } - *out_buf = new_buf; + *out_buf = data->buf; return len; } diff --git a/custom_mutators/examples/simple_example.c b/custom_mutators/examples/simple_example.c index d888ec1f..2c0abe29 100644 --- a/custom_mutators/examples/simple_example.c +++ b/custom_mutators/examples/simple_example.c @@ -1,6 +1,6 @@ // This simple example just creates random buffer <= 100 filled with 'A' // needs -I /path/to/AFLplusplus/include -#include "custom_mutator_helpers.h" +#include "afl-fuzz.h" #include #include @@ -13,14 +13,14 @@ typedef struct my_mutator { - afl_t *afl; + afl_state_t *afl; // Reused buffers: - BUF_VAR(u8, fuzz); + u8 *fuzz_buf; } my_mutator_t; -my_mutator_t *afl_custom_init(afl_t *afl, unsigned int seed) { +my_mutator_t *afl_custom_init(afl_state_t *afl, unsigned int seed) { srand(seed); my_mutator_t *data = calloc(1, sizeof(my_mutator_t)); @@ -31,6 +31,14 @@ my_mutator_t *afl_custom_init(afl_t *afl, unsigned int seed) { } + data->fuzz_buf = (u8 *)malloc(MAX_FILE); + if (!data->fuzz_buf) { + + perror("afl_custom_init malloc"); + return NULL; + + } + data->afl = afl; return data; @@ -44,18 +52,10 @@ size_t afl_custom_fuzz(my_mutator_t *data, uint8_t *buf, size_t buf_size, int size = (rand() % 100) + 1; if (size > max_size) size = max_size; - u8 *mutated_out = maybe_grow(BUF_PARAMS(data, fuzz), size); - if (!mutated_out) { - - *out_buf = NULL; - perror("custom mutator allocation (maybe_grow)"); - return 0; /* afl-fuzz will very likely error out after this. */ - - } - memset(mutated_out, _FIXED_CHAR, size); + memset(data->fuzz_buf, _FIXED_CHAR, size); - *out_buf = mutated_out; + *out_buf = data->fuzz_buf; return size; } diff --git a/docs/Changelog.md b/docs/Changelog.md index 9f4a8653..3337feb9 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -30,6 +30,7 @@ - unicorn_mode: - updated and minor issues fixed - new custom module: autotoken, a grammar free fuzzer for text inputs + - fixed custom mutator C examples - better sanitizer default options support for all tools - more minor fixes and cross-platform support -- cgit 1.4.1 From 2f6242d3f8b09bfbf9e2b172acd0c67015e6ef2b Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 15 Apr 2023 12:28:39 +0200 Subject: update docs --- docs/third_party_tools.md | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'docs') diff --git a/docs/third_party_tools.md b/docs/third_party_tools.md index 97f2c362..02a40ce5 100644 --- a/docs/third_party_tools.md +++ b/docs/third_party_tools.md @@ -1,5 +1,10 @@ # Tools that help fuzzing with AFL++ +## AFL++ and other development languages + +* [afl-rs](https://github.com/rust-fuzz/afl.rs) - AFL++ for RUST +* [WASM](https://github.com/fgsect/WAFL) - AFL++ for WASM + ## Speeding up fuzzing * [libfiowrapper](https://github.com/marekzmyslowski/libfiowrapper) - if the -- cgit 1.4.1 From 7f734c96d187312868178e8ead95dc103c557c1f Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 17 Apr 2023 10:25:10 +0200 Subject: v4.06c release --- README.md | 4 ++-- docs/Changelog.md | 5 +++-- include/config.h | 2 +- 3 files changed, 6 insertions(+), 5 deletions(-) (limited to 'docs') diff --git a/README.md b/README.md index 821b8cb7..c012c400 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ AFL++ logo -Release version: [4.05c](https://github.com/AFLplusplus/AFLplusplus/releases) +Release version: [4.06c](https://github.com/AFLplusplus/AFLplusplus/releases) -GitHub version: 4.06a +GitHub version: 4.07a Repository: [https://github.com/AFLplusplus/AFLplusplus](https://github.com/AFLplusplus/AFLplusplus) diff --git a/docs/Changelog.md b/docs/Changelog.md index 3337feb9..587fb64c 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -3,7 +3,7 @@ This is the list of all noteworthy changes made in every public release of the tool. See README.md for the general instruction manual. -### Version ++4.06a (dev) +### Version ++4.06c (release) - afl-fuzz: - ensure temporary file descriptor is closed when not used - added `AFL_NO_WARN_INSTABILITY` @@ -29,9 +29,10 @@ - fix issue on MacOS - unicorn_mode: - updated and minor issues fixed + - nyx_mode support for all tools + - better sanitizer default options support for all tools - new custom module: autotoken, a grammar free fuzzer for text inputs - fixed custom mutator C examples - - better sanitizer default options support for all tools - more minor fixes and cross-platform support ### Version ++4.05c (release) diff --git a/include/config.h b/include/config.h index e46f515a..b6249a0f 100644 --- a/include/config.h +++ b/include/config.h @@ -26,7 +26,7 @@ /* Version string: */ // c = release, a = volatile github dev, e = experimental branch -#define VERSION "++4.06a" +#define VERSION "++4.06c" /****************************************************** * * -- cgit 1.4.1 From 02b9e583f2a5dd7d83bd4c02af8d2081532689ed Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 17 Apr 2023 14:41:05 +0200 Subject: v4.07a init --- docs/Changelog.md | 4 ++++ include/config.h | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index 587fb64c..30e76f2c 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -3,6 +3,10 @@ This is the list of all noteworthy changes made in every public release of the tool. See README.md for the general instruction manual. +### Version ++4.07a (dev) + - soon :) + + ### Version ++4.06c (release) - afl-fuzz: - ensure temporary file descriptor is closed when not used diff --git a/include/config.h b/include/config.h index b6249a0f..764c29dc 100644 --- a/include/config.h +++ b/include/config.h @@ -26,7 +26,7 @@ /* Version string: */ // c = release, a = volatile github dev, e = experimental branch -#define VERSION "++4.06c" +#define VERSION "++4.07a" /****************************************************** * * -- cgit 1.4.1 From 4e5f42cab6b8c501eeaf76ec7ca920089f6e0f3a Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 20 Apr 2023 10:39:23 +0200 Subject: afl-showmap custom mutator support --- GNUmakefile | 2 +- TODO.md | 4 +-- afl-cmin | 2 ++ afl-cmin.bash | 2 ++ docs/Changelog.md | 3 +- include/afl-fuzz.h | 8 +++-- src/afl-showmap.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 7 files changed, 107 insertions(+), 7 deletions(-) (limited to 'docs') diff --git a/GNUmakefile b/GNUmakefile index 5bc3f9d5..0f890308 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -453,7 +453,7 @@ afl-fuzz: $(COMM_HDR) include/afl-fuzz.h $(AFL_FUZZ_FILES) src/afl-common.o src/ $(CC) $(CFLAGS) $(COMPILE_STATIC) $(CFLAGS_FLTO) $(AFL_FUZZ_FILES) src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o -o $@ $(PYFLAGS) $(LDFLAGS) -lm afl-showmap: src/afl-showmap.c src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o $(COMM_HDR) | test_x86 - $(CC) $(CFLAGS) $(COMPILE_STATIC) $(CFLAGS_FLTO) src/$@.c src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o -o $@ $(LDFLAGS) + $(CC) $(CFLAGS) $(COMPILE_STATIC) $(CFLAGS_FLTO) src/$@.c src/afl-fuzz-mutators.c src/afl-fuzz-python.c src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o -o $@ $(PYFLAGS) $(LDFLAGS) afl-tmin: src/afl-tmin.c src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o $(COMM_HDR) | test_x86 $(CC) $(CFLAGS) $(COMPILE_STATIC) $(CFLAGS_FLTO) src/$@.c src/afl-common.o src/afl-sharedmem.o src/afl-forkserver.o src/afl-performance.o -o $@ $(LDFLAGS) diff --git a/TODO.md b/TODO.md index e7789cf6..dba75070 100644 --- a/TODO.md +++ b/TODO.md @@ -3,14 +3,14 @@ ## Should - splicing selection weighted? - - support afl_custom_{send,post_process}, persistent and deferred fork - server in afl-showmap + - support persistent and deferred fork server in afl-showmap? - better autodetection of shifting runtime timeout values - Update afl->pending_not_fuzzed for MOpt - afl-plot to support multiple plot_data - parallel builds for source-only targets - get rid of check_binary, replace with more forkserver communication - first fuzzer should be a main automatically? not sure. + - reload fuzz binary on signal ## Maybe diff --git a/afl-cmin b/afl-cmin index c5e64410..e2c26d91 100755 --- a/afl-cmin +++ b/afl-cmin @@ -133,6 +133,8 @@ function usage() { "AFL_PRINT_FILENAMES: If set, the filename currently processed will be " \ "printed to stdout\n" \ "AFL_SKIP_BIN_CHECK: skip afl instrumentation checks for target binary\n" +"AFL_CUSTOM_MUTATOR_LIBRARY: custom mutator library (post_process and send)\n" +"AFL_PYTHON_MODULE: custom mutator library (post_process and send)\n" exit 1 } diff --git a/afl-cmin.bash b/afl-cmin.bash index bcf62eba..5258758e 100755 --- a/afl-cmin.bash +++ b/afl-cmin.bash @@ -151,6 +151,8 @@ AFL_KEEP_TRACES: leave the temporary \.traces directory AFL_NO_FORKSRV: run target via execve instead of using the forkserver AFL_PATH: last resort location to find the afl-showmap binary AFL_SKIP_BIN_CHECK: skip check for target binary +AFL_CUSTOM_MUTATOR_LIBRARY: custom mutator library (post_process and send) +AFL_PYTHON_MODULE: custom mutator library (post_process and send) _EOF_ exit 1 fi diff --git a/docs/Changelog.md b/docs/Changelog.md index 30e76f2c..5ed5ef2b 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -4,7 +4,8 @@ release of the tool. See README.md for the general instruction manual. ### Version ++4.07a (dev) - - soon :) + - afl-showmap: + - added custom mutator post_process and send support ### Version ++4.06c (release) diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h index 8b6502b4..ec69ba17 100644 --- a/include/afl-fuzz.h +++ b/include/afl-fuzz.h @@ -1103,7 +1103,6 @@ u32 count_bits(afl_state_t *, u8 *); u32 count_bytes(afl_state_t *, u8 *); u32 count_non_255_bytes(afl_state_t *, u8 *); void simplify_trace(afl_state_t *, u8 *); -void classify_counts(afl_forkserver_t *); #ifdef WORD_SIZE_64 void discover_word(u8 *ret, u64 *current, u64 *virgin); #else @@ -1117,6 +1116,9 @@ u8 *describe_op(afl_state_t *, u8, size_t); u8 save_if_interesting(afl_state_t *, void *, u32, u8); u8 has_new_bits(afl_state_t *, u8 *); u8 has_new_bits_unclassified(afl_state_t *, u8 *); +#ifndef AFL_SHOWMAP +void classify_counts(afl_forkserver_t *); +#endif /* Extras */ @@ -1192,11 +1194,13 @@ void fix_up_sync(afl_state_t *); void check_asan_opts(afl_state_t *); void check_binary(afl_state_t *, u8 *); void check_if_tty(afl_state_t *); -void setup_signal_handlers(void); void save_cmdline(afl_state_t *, u32, char **); void read_foreign_testcases(afl_state_t *, int); void write_crash_readme(afl_state_t *afl); u8 check_if_text_buf(u8 *buf, u32 len); +#ifndef AFL_SHOWMAP +void setup_signal_handlers(void); +#endif /* CmpLog */ diff --git a/src/afl-showmap.c b/src/afl-showmap.c index df030672..b5a61de5 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -30,8 +30,10 @@ */ #define AFL_MAIN +#define AFL_SHOWMAP #include "config.h" +#include "afl-fuzz.h" #include "types.h" #include "debug.h" #include "alloc-inl.h" @@ -62,6 +64,8 @@ #include #include +static afl_state_t *afl; + static char *stdin_file; /* stdin file */ static u8 *in_dir = NULL, /* input folder */ @@ -308,12 +312,73 @@ static u32 write_results_to_file(afl_forkserver_t *fsrv, u8 *outfile) { } +void pre_afl_fsrv_write_to_testcase(afl_forkserver_t *fsrv, u8 *mem, u32 len) { + + static u8 buf[MAX_FILE]; + u32 sent = 0; + + if (unlikely(afl->custom_mutators_count)) { + + ssize_t new_size = len; + u8 *new_mem = mem; + u8 *new_buf = NULL; + + LIST_FOREACH(&afl->custom_mutator_list, struct custom_mutator, { + + if (el->afl_custom_post_process) { + + new_size = + el->afl_custom_post_process(el->data, new_mem, new_size, &new_buf); + + if (unlikely(!new_buf || new_size <= 0)) { + + return; + + } else { + + new_mem = new_buf; + len = new_size; + + } + + } + + }); + + if (new_mem != mem && new_mem != NULL) { + + mem = buf; + memcpy(mem, new_mem, new_size); + + } + + if (unlikely(afl->custom_mutators_count)) { + + LIST_FOREACH(&afl->custom_mutator_list, struct custom_mutator, { + + if (el->afl_custom_fuzz_send) { + + el->afl_custom_fuzz_send(el->data, mem, len); + sent = 1; + + } + + }); + + } + + } + + if (likely(!sent)) { afl_fsrv_write_to_testcase(fsrv, mem, len); } + +} + /* Execute target application. */ static void showmap_run_target_forkserver(afl_forkserver_t *fsrv, u8 *mem, u32 len) { - afl_fsrv_write_to_testcase(fsrv, mem, len); + pre_afl_fsrv_write_to_testcase(fsrv, mem, len); if (!quiet_mode) { SAYF("-- Program output begins --\n" cRST); } @@ -835,6 +900,10 @@ static void usage(u8 *argv0) { "This tool displays raw tuple data captured by AFL instrumentation.\n" "For additional help, consult %s/README.md.\n\n" + "If you use -i mode, then custom mutator post_process send send " + "functionality\n" + "is supported.\n\n" + "Environment variables used:\n" "LD_BIND_LAZY: do not set LD_BIND_NOW env var for target\n" "AFL_CMIN_CRASHES_ONLY: (cmin_mode) only write tuples for crashing " @@ -1266,6 +1335,8 @@ int main(int argc, char **argv_orig, char **envp) { } + afl = calloc(1, sizeof(afl_state_t)); + if (getenv("AFL_FORKSRV_INIT_TMOUT")) { s32 forksrv_init_tmout = atoi(getenv("AFL_FORKSRV_INIT_TMOUT")); @@ -1380,6 +1451,26 @@ int main(int argc, char **argv_orig, char **envp) { } + if (in_dir) { + + afl->fsrv.dev_urandom_fd = open("/dev/urandom", O_RDONLY); + afl->afl_env.afl_custom_mutator_library = + getenv("AFL_CUSTOM_MUTATOR_LIBRARY"); + afl->afl_env.afl_python_module = getenv("AFL_PYTHON_MODULE"); + setup_custom_mutators(afl); + + } else { + + if (getenv("AFL_CUSTOM_MUTATOR_LIBRARY") || getenv("AFL_PYTHON_MODULE")) { + + WARNF( + "Custom mutator environment detected, this is only supported in -i " + "mode!\n"); + + } + + } + if (in_dir) { DIR *dir_in, *dir_out = NULL; -- cgit 1.4.1 From 30495e6bfe4119c9be6597ad0def01e7e0cb8a67 Mon Sep 17 00:00:00 2001 From: eleguevel Date: Fri, 21 Apr 2023 12:00:56 +0200 Subject: frida mode: add dynamic loaded code exclusion Add the AFL_FRIDA_INST_NO_DYNAMIC_LOAD environment variable and its associated JS function setInstrumentNoDynamicLoad to prevent the instrumentation of late dynamic loaded code. Resolve #1708 --- docs/env_variables.md | 2 ++ frida_mode/README.md | 4 +++- frida_mode/Scripting.md | 6 ++++++ frida_mode/frida.map | 1 + frida_mode/include/ranges.h | 1 + frida_mode/src/js/api.js | 7 +++++++ frida_mode/src/js/js_api.c | 7 +++++++ frida_mode/src/ranges.c | 39 ++++++++++++++++++++++++++++++++++++--- frida_mode/ts/lib/afl.ts | 12 ++++++++++++ include/envs.h | 1 + 10 files changed, 76 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index a6a0ae44..c5995d13 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -677,6 +677,8 @@ support. * `AFL_FRIDA_INST_JIT` - Enable the instrumentation of Just-In-Time compiled code. Code is considered to be JIT if the executable segment is not backed by a file. +* `AFL_FRIDA_INST_NO_DYNAMIC_LOAD` - Don't instrument the code loaded late at + runtime. Strictly limits instrumentation to what has been included. * `AFL_FRIDA_INST_NO_OPTIMIZE` - Don't use optimized inline assembly coverage instrumentation (the default where available). Required to use `AFL_FRIDA_INST_TRACE`. diff --git a/frida_mode/README.md b/frida_mode/README.md index aac13153..49a1fe38 100644 --- a/frida_mode/README.md +++ b/frida_mode/README.md @@ -178,11 +178,13 @@ Default is 256Mb. * `AFL_FRIDA_INST_JIT` - Enable the instrumentation of Just-In-Time compiled code. Code is considered to be JIT if the executable segment is not backed by a file. +* `AFL_FRIDA_INST_NO_DYNAMIC_LOAD` - Don't instrument the code loaded late at + runtime. Strictly limits instrumentation to what has been included. * `AFL_FRIDA_INST_NO_OPTIMIZE` - Don't use optimized inline assembly coverage instrumentation (the default where available). Required to use + `AFL_FRIDA_INST_TRACE`. * `AFL_FRIDA_INST_REGS_FILE` - File to write raw register contents at the start of each block. - `AFL_FRIDA_INST_TRACE`. * `AFL_FRIDA_INST_NO_CACHE` - Don't use a look-up table to cache real to instrumented address block translations. * `AFL_FRIDA_INST_NO_PREFETCH` - Disable prefetching. By default, the child will diff --git a/frida_mode/Scripting.md b/frida_mode/Scripting.md index 023e4a19..dfd09e7b 100644 --- a/frida_mode/Scripting.md +++ b/frida_mode/Scripting.md @@ -844,6 +844,12 @@ class Afl { static setInstrumentLibraries() { Afl.jsApiSetInstrumentLibraries(); } + /** + * See `AFL_FRIDA_INST_NO_DYNAMIC_LOAD` + */ + static setInstrumentNoDynamicLoad() { + Afl.jsApiSetInstrumentNoDynamicLoad(); + } /** * See `AFL_FRIDA_INST_NO_OPTIMIZE` */ diff --git a/frida_mode/frida.map b/frida_mode/frida.map index baf067ab..a98c2096 100644 --- a/frida_mode/frida.map +++ b/frida_mode/frida.map @@ -19,6 +19,7 @@ js_api_set_instrument_jit; js_api_set_instrument_libraries; js_api_set_instrument_instructions; + js_api_set_instrument_no_dynamic_load; js_api_set_instrument_no_optimize; js_api_set_instrument_regs_file; js_api_set_instrument_seed; diff --git a/frida_mode/include/ranges.h b/frida_mode/include/ranges.h index 3bd9eaa6..ca28acd9 100644 --- a/frida_mode/include/ranges.h +++ b/frida_mode/include/ranges.h @@ -6,6 +6,7 @@ extern gboolean ranges_debug_maps; extern gboolean ranges_inst_libs; extern gboolean ranges_inst_jit; +extern gboolean ranges_inst_dynamic_load; void ranges_config(void); void ranges_init(void); diff --git a/frida_mode/src/js/api.js b/frida_mode/src/js/api.js index f9ea1ffb..a65d32df 100644 --- a/frida_mode/src/js/api.js +++ b/frida_mode/src/js/api.js @@ -150,6 +150,12 @@ class Afl { static setInstrumentLibraries() { Afl.jsApiSetInstrumentLibraries(); } + /** + * See `AFL_FRIDA_INST_NO_DYNAMIC_LOAD` + */ + static setInstrumentNoDynamicLoad() { + Afl.jsApiSetInstrumentNoDynamicLoad(); + } /** * See `AFL_FRIDA_INST_NO_OPTIMIZE` */ @@ -342,6 +348,7 @@ Afl.jsApiSetInstrumentDebugFile = Afl.jsApiGetFunction("js_api_set_instrument_de Afl.jsApiSetInstrumentInstructions = Afl.jsApiGetFunction("js_api_set_instrument_instructions", "void", []); Afl.jsApiSetInstrumentJit = Afl.jsApiGetFunction("js_api_set_instrument_jit", "void", []); Afl.jsApiSetInstrumentLibraries = Afl.jsApiGetFunction("js_api_set_instrument_libraries", "void", []); +Afl.jsApiSetInstrumentNoDynamicLoad = Afl.jsApiGetFunction("js_api_set_instrument_no_dynamic_load", "void", []); Afl.jsApiSetInstrumentNoOptimize = Afl.jsApiGetFunction("js_api_set_instrument_no_optimize", "void", []); Afl.jsApiSetInstrumentRegsFile = Afl.jsApiGetFunction("js_api_set_instrument_regs_file", "void", ["pointer"]); Afl.jsApiSetInstrumentSeed = Afl.jsApiGetFunction("js_api_set_instrument_seed", "void", ["uint64"]); diff --git a/frida_mode/src/js/js_api.c b/frida_mode/src/js/js_api.c index 2e996c1c..00278082 100644 --- a/frida_mode/src/js/js_api.c +++ b/frida_mode/src/js/js_api.c @@ -156,6 +156,13 @@ __attribute__((visibility("default"))) void js_api_set_instrument_instructions( } +__attribute__((visibility("default"))) void js_api_set_instrument_no_dynamic_load( + void) { + + ranges_inst_dynamic_load = FALSE; + +} + __attribute__((visibility("default"))) void js_api_set_instrument_no_optimize( void) { diff --git a/frida_mode/src/ranges.c b/frida_mode/src/ranges.c index 72cb9730..e9fc3b4e 100644 --- a/frida_mode/src/ranges.c +++ b/frida_mode/src/ranges.c @@ -18,6 +18,7 @@ typedef struct { gboolean ranges_debug_maps = FALSE; gboolean ranges_inst_libs = FALSE; gboolean ranges_inst_jit = FALSE; +gboolean ranges_inst_dynamic_load = TRUE; static GArray *module_ranges = NULL; static GArray *libs_ranges = NULL; @@ -25,6 +26,7 @@ static GArray *jit_ranges = NULL; static GArray *include_ranges = NULL; static GArray *exclude_ranges = NULL; static GArray *ranges = NULL; +static GArray *whole_memory_ranges = NULL; static void convert_address_token(gchar *token, GumMemoryRange *range) { @@ -387,6 +389,21 @@ static GArray *collect_jit_ranges(void) { } +static GArray *collect_whole_mem_ranges(void) { + + GArray *result; + GumMemoryRange range; + result = g_array_new(false, false, sizeof(GumMemoryRange)); + + range.base_address = 0; + range.size = G_MAXULONG; + + g_array_append_val(result, range); + + return result; + +} + static gboolean intersect_range(GumMemoryRange *rr, GumMemoryRange *ra, GumMemoryRange *rb) { @@ -574,11 +591,17 @@ void ranges_config(void) { if (getenv("AFL_FRIDA_DEBUG_MAPS") != NULL) { ranges_debug_maps = TRUE; } if (getenv("AFL_INST_LIBS") != NULL) { ranges_inst_libs = TRUE; } if (getenv("AFL_FRIDA_INST_JIT") != NULL) { ranges_inst_jit = TRUE; } + if (getenv("AFL_FRIDA_INST_NO_DYNAMIC_LOAD") != NULL) { + + ranges_inst_dynamic_load = FALSE; + + } if (ranges_debug_maps) { ranges_print_debug_maps(); } include_ranges = collect_ranges("AFL_FRIDA_INST_RANGES"); exclude_ranges = collect_ranges("AFL_FRIDA_EXCLUDE_RANGES"); + whole_memory_ranges = collect_whole_mem_ranges(); } @@ -628,10 +651,20 @@ void ranges_init(void) { print_ranges("step4", step4); /* - * After step4, we have the total ranges to be instrumented, we now subtract - * that from the original ranges of the modules to configure stalker. + * After step 4 we have the total ranges to be instrumented, we now subtract + * that either from the original ranges of the modules or from the whole + * memory if AFL_INST_NO_DYNAMIC_LOAD to configure the stalker. */ - step5 = subtract_ranges(module_ranges, step4); + if (ranges_inst_dynamic_load) { + + step5 = subtract_ranges(module_ranges, step4); + + } else { + + step5 = subtract_ranges(whole_memory_ranges, step4); + + } + print_ranges("step5", step5); ranges = merge_ranges(step5); diff --git a/frida_mode/ts/lib/afl.ts b/frida_mode/ts/lib/afl.ts index 6a2350e7..7d1fac6b 100644 --- a/frida_mode/ts/lib/afl.ts +++ b/frida_mode/ts/lib/afl.ts @@ -178,6 +178,13 @@ class Afl { Afl.jsApiSetInstrumentLibraries(); } + /** + * See `AFL_FRIDA_INST_NO_DYNAMIC_LOAD` + */ + public static setInstrumentNoDynamicLoad(): void { + Afl.jsApiSetInstrumentNoDynamicLoad(); + } + /** * See `AFL_FRIDA_INST_NO_OPTIMIZE` */ @@ -443,6 +450,11 @@ class Afl { "void", []); + private static readonly jsApiSetInstrumentNoDynamicLoad = Afl.jsApiGetFunction( + "js_api_set_instrument_no_dynamic_load", + "void", + []); + private static readonly jsApiSetInstrumentNoOptimize = Afl.jsApiGetFunction( "js_api_set_instrument_no_optimize", "void", diff --git a/include/envs.h b/include/envs.h index 066921b9..41eabf60 100644 --- a/include/envs.h +++ b/include/envs.h @@ -65,6 +65,7 @@ static char *afl_environment_variables[] = { "AFL_FRIDA_INST_INSN", "AFL_FRIDA_INST_JIT", "AFL_FRIDA_INST_NO_CACHE", + "AFL_FRIDA_INST_NO_DYNAMIC_LOAD", "AFL_FRIDA_INST_NO_OPTIMIZE", "AFL_FRIDA_INST_NO_PREFETCH", "AFL_FRIDA_INST_NO_PREFETCH_BACKPATCH", -- cgit 1.4.1 From 6bd48a48cbed1f923ff0999ea24af1f548c2e2bc Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Sat, 22 Apr 2023 11:39:44 +0200 Subject: code format --- custom_mutators/README.md | 33 +- custom_mutators/atnwalk/atnwalk.c | 681 ++++++++++++++++++++++---------------- docs/Changelog.md | 4 + frida_mode/src/js/js_api.c | 4 +- 4 files changed, 423 insertions(+), 299 deletions(-) (limited to 'docs') diff --git a/custom_mutators/README.md b/custom_mutators/README.md index 8d01856f..a5a572c0 100644 --- a/custom_mutators/README.md +++ b/custom_mutators/README.md @@ -11,7 +11,20 @@ The `./examples` folder contains examples for custom mutators in python and C. In `./rust`, you will find rust bindings, including a simple example in `./rust/example` and an example for structured fuzzing, based on lain, in`./rust/example_lain`. -## The AFL++ grammar agnostic grammar mutator +## Production-Ready Custom Mutators + +This directory holds ready to use custom mutators. +Just type "make" in the individual subdirectories. + +Use with e.g. + +`AFL_CUSTOM_MUTATOR_LIBRARY=custom_mutators/radamsa/radamsa-mutator.so afl-fuzz ....` + +and add `AFL_CUSTOM_MUTATOR_ONLY=1` if you only want to use the custom mutator. + +Multiple custom mutators can be used by separating their paths with `:` in the environment variable. + +### The AFL++ grammar agnostic grammar mutator In `./autotokens` you find a token-level fuzzer that does not need to know anything about the grammar of an input as long as it is in ascii and allows @@ -21,7 +34,7 @@ It is very fast and effective. If you are looking for an example of how to effectively create a custom mutator take a look at this one. -## The AFL++ Grammar Mutator +### The AFL++ Grammar Mutator If you use git to clone AFL++, then the following will incorporate our excellent grammar custom mutator: @@ -34,18 +47,18 @@ Read the README in the [Grammar-Mutator] repository on how to use it. [Grammar-Mutator]: https://github.com/AFLplusplus/Grammar-Mutator -## Production-Ready Custom Mutators - -This directory holds ready to use custom mutators. -Just type "make" in the individual subdirectories. +Note that this custom mutator is not very good though! -Use with e.g. +### Other Mutators -`AFL_CUSTOM_MUTATOR_LIBRARY=custom_mutators/radamsa/radamsa-mutator.so afl-fuzz ....` +atnwalk and gramatron are grammar custom mutators. Example grammars are +provided. -and add `AFL_CUSTOM_MUTATOR_ONLY=1` if you only want to use the custom mutator. +honggfuzz, libfuzzer and libafl are partial implementations based on the +mutator implementations of the respective fuzzers. +More for playing than serious usage. -Multiple custom mutators can be used by separating their paths with `:` in the environment variable. +radamsa is slow and not very good. ## 3rd Party Custom Mutators diff --git a/custom_mutators/atnwalk/atnwalk.c b/custom_mutators/atnwalk/atnwalk.c index 901b8a9e..c3a2cd95 100644 --- a/custom_mutators/atnwalk/atnwalk.c +++ b/custom_mutators/atnwalk/atnwalk.c @@ -11,10 +11,12 @@ #define BUF_SIZE_INIT 4096 #define SOCKET_NAME "./atnwalk.socket" -// how many errors (e.g. timeouts) to tolerate until moving on to the next queue entry +// how many errors (e.g. timeouts) to tolerate until moving on to the next queue +// entry #define ATNWALK_ERRORS_MAX 1 -// how many execution timeouts to tolerate until moving on to the next queue entry +// how many execution timeouts to tolerate until moving on to the next queue +// entry #define EXEC_TIMEOUT_MAX 2 // handshake constants @@ -27,80 +29,87 @@ const uint8_t SERVER_MUTATE_BIT = 0b00000010; const uint8_t SERVER_DECODE_BIT = 0b00000100; const uint8_t SERVER_ENCODE_BIT = 0b00001000; - typedef struct atnwalk_mutator { - afl_state_t *afl; - uint8_t atnwalk_error_count; - uint64_t prev_timeouts; - uint32_t prev_hits; - uint32_t stage_havoc_cur; - uint32_t stage_havoc_max; - uint32_t stage_splice_cur; - uint32_t stage_splice_max; - uint8_t *fuzz_buf; - size_t fuzz_size; - uint8_t *post_process_buf; - size_t post_process_size; -} atnwalk_mutator_t; + afl_state_t *afl; + uint8_t atnwalk_error_count; + uint64_t prev_timeouts; + uint32_t prev_hits; + uint32_t stage_havoc_cur; + uint32_t stage_havoc_max; + uint32_t stage_splice_cur; + uint32_t stage_splice_max; + uint8_t *fuzz_buf; + size_t fuzz_size; + uint8_t *post_process_buf; + size_t post_process_size; + +} atnwalk_mutator_t; int read_all(int fd, uint8_t *buf, size_t buf_size) { - int n; - size_t offset = 0; - while (offset < buf_size) { - n = read(fd, buf + offset, buf_size - offset); - if (n == -1) { - return 0; - } - offset += n; - } - return 1; -} + int n; + size_t offset = 0; + while (offset < buf_size) { + + n = read(fd, buf + offset, buf_size - offset); + if (n == -1) { return 0; } + offset += n; + + } + + return 1; -int write_all(int fd, uint8_t *buf, size_t buf_size) { - int n; - size_t offset = 0; - while (offset < buf_size) { - n = write(fd, buf + offset, buf_size - offset); - if (n == -1) { - return 0; - } - offset += n; - } - return 1; } +int write_all(int fd, uint8_t *buf, size_t buf_size) { + + int n; + size_t offset = 0; + while (offset < buf_size) { + + n = write(fd, buf + offset, buf_size - offset); + if (n == -1) { return 0; } + offset += n; + + } + + return 1; -void put_uint32(uint8_t *buf, uint32_t val) { - buf[0] = (uint8_t) (val >> 24); - buf[1] = (uint8_t) ((val & 0x00ff0000) >> 16); - buf[2] = (uint8_t) ((val & 0x0000ff00) >> 8); - buf[3] = (uint8_t) (val & 0x000000ff); } +void put_uint32(uint8_t *buf, uint32_t val) { + + buf[0] = (uint8_t)(val >> 24); + buf[1] = (uint8_t)((val & 0x00ff0000) >> 16); + buf[2] = (uint8_t)((val & 0x0000ff00) >> 8); + buf[3] = (uint8_t)(val & 0x000000ff); -uint32_t to_uint32(uint8_t *buf) { - uint32_t val = 0; - val |= (((uint32_t) buf[0]) << 24); - val |= (((uint32_t) buf[1]) << 16); - val |= (((uint32_t) buf[2]) << 8); - val |= ((uint32_t) buf[3]); - return val; } +uint32_t to_uint32(uint8_t *buf) { + + uint32_t val = 0; + val |= (((uint32_t)buf[0]) << 24); + val |= (((uint32_t)buf[1]) << 16); + val |= (((uint32_t)buf[2]) << 8); + val |= ((uint32_t)buf[3]); + return val; -void put_uint64(uint8_t *buf, uint64_t val) { - buf[0] = (uint8_t) (val >> 56); - buf[1] = (uint8_t) ((val & 0x00ff000000000000) >> 48); - buf[2] = (uint8_t) ((val & 0x0000ff0000000000) >> 40); - buf[3] = (uint8_t) ((val & 0x000000ff00000000) >> 32); - buf[4] = (uint8_t) ((val & 0x00000000ff000000) >> 24); - buf[5] = (uint8_t) ((val & 0x0000000000ff0000) >> 16); - buf[6] = (uint8_t) ((val & 0x000000000000ff00) >> 8); - buf[7] = (uint8_t) (val & 0x00000000000000ff); } +void put_uint64(uint8_t *buf, uint64_t val) { + + buf[0] = (uint8_t)(val >> 56); + buf[1] = (uint8_t)((val & 0x00ff000000000000) >> 48); + buf[2] = (uint8_t)((val & 0x0000ff0000000000) >> 40); + buf[3] = (uint8_t)((val & 0x000000ff00000000) >> 32); + buf[4] = (uint8_t)((val & 0x00000000ff000000) >> 24); + buf[5] = (uint8_t)((val & 0x0000000000ff0000) >> 16); + buf[6] = (uint8_t)((val & 0x000000000000ff00) >> 8); + buf[7] = (uint8_t)(val & 0x00000000000000ff); + +} /** * Initialize this custom mutator @@ -114,69 +123,82 @@ void put_uint64(uint8_t *buf, uint64_t val) { * Return NULL on error. */ atnwalk_mutator_t *afl_custom_init(afl_state_t *afl, unsigned int seed) { - srand(seed); - atnwalk_mutator_t *data = (atnwalk_mutator_t *) malloc(sizeof(atnwalk_mutator_t)); - if (!data) { - perror("afl_custom_init alloc"); - return NULL; - } - data->afl = afl; - data->prev_hits = 0; - data->fuzz_buf = (uint8_t *) malloc(BUF_SIZE_INIT); - data->fuzz_size = BUF_SIZE_INIT; - data->post_process_buf = (uint8_t *) malloc(BUF_SIZE_INIT); - data->post_process_size = BUF_SIZE_INIT; - return data; -} + srand(seed); + atnwalk_mutator_t *data = + (atnwalk_mutator_t *)malloc(sizeof(atnwalk_mutator_t)); + if (!data) { -unsigned int afl_custom_fuzz_count(atnwalk_mutator_t *data, const unsigned char *buf, size_t buf_size) { - // afl_custom_fuzz_count is called exactly once before entering the 'stage-loop' for the current queue entry - // thus, we use it to reset the error count and to initialize stage variables (somewhat not intended by the API, - // but still better than rewriting the whole thing to have a custom mutator stage) - data->atnwalk_error_count = 0; - data->prev_timeouts = data->afl->total_tmouts; + perror("afl_custom_init alloc"); + return NULL; + + } + + data->afl = afl; + data->prev_hits = 0; + data->fuzz_buf = (uint8_t *)malloc(BUF_SIZE_INIT); + data->fuzz_size = BUF_SIZE_INIT; + data->post_process_buf = (uint8_t *)malloc(BUF_SIZE_INIT); + data->post_process_size = BUF_SIZE_INIT; + return data; - // it might happen that on the last execution of the splice stage a new path is found - // we need to fix that here and count it - if (data->prev_hits) { - data->afl->stage_finds[STAGE_SPLICE] += data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; - } - data->prev_hits = data->afl->queued_items + data->afl->saved_crashes; - data->stage_havoc_cur = 0; - data->stage_splice_cur = 0; - - // 50% havoc, 50% splice - data->stage_havoc_max = data->afl->stage_max >> 1; - if (data->stage_havoc_max < HAVOC_MIN) { - data->stage_havoc_max = HAVOC_MIN; - } - data->stage_splice_max = data->stage_havoc_max; - return data->stage_havoc_max + data->stage_splice_max; } +unsigned int afl_custom_fuzz_count(atnwalk_mutator_t *data, + const unsigned char *buf, size_t buf_size) { + + // afl_custom_fuzz_count is called exactly once before entering the + // 'stage-loop' for the current queue entry thus, we use it to reset the error + // count and to initialize stage variables (somewhat not intended by the API, + // but still better than rewriting the whole thing to have a custom mutator + // stage) + data->atnwalk_error_count = 0; + data->prev_timeouts = data->afl->total_tmouts; + + // it might happen that on the last execution of the splice stage a new path + // is found we need to fix that here and count it + if (data->prev_hits) { + + data->afl->stage_finds[STAGE_SPLICE] += + data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; + + } + + data->prev_hits = data->afl->queued_items + data->afl->saved_crashes; + data->stage_havoc_cur = 0; + data->stage_splice_cur = 0; + + // 50% havoc, 50% splice + data->stage_havoc_max = data->afl->stage_max >> 1; + if (data->stage_havoc_max < HAVOC_MIN) { data->stage_havoc_max = HAVOC_MIN; } + data->stage_splice_max = data->stage_havoc_max; + return data->stage_havoc_max + data->stage_splice_max; -size_t fail_fatal(int fd_socket, uint8_t **out_buf) { - if (fd_socket != -1) { - close(fd_socket); - } - *out_buf = NULL; - return 0; } +size_t fail_fatal(int fd_socket, uint8_t **out_buf) { + + if (fd_socket != -1) { close(fd_socket); } + *out_buf = NULL; + return 0; -size_t fail_gracefully(int fd_socket, atnwalk_mutator_t *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf) { - if (fd_socket != -1) { - close(fd_socket); - } - data->atnwalk_error_count++; - if (data->atnwalk_error_count > ATNWALK_ERRORS_MAX) { - data->afl->stage_max = data->afl->stage_cur; - } - *out_buf = buf; - return buf_size; } +size_t fail_gracefully(int fd_socket, atnwalk_mutator_t *data, uint8_t *buf, + size_t buf_size, uint8_t **out_buf) { + + if (fd_socket != -1) { close(fd_socket); } + data->atnwalk_error_count++; + if (data->atnwalk_error_count > ATNWALK_ERRORS_MAX) { + + data->afl->stage_max = data->afl->stage_cur; + + } + + *out_buf = buf; + return buf_size; + +} /** * Perform custom mutations on a given input @@ -194,152 +216,214 @@ size_t fail_gracefully(int fd_socket, atnwalk_mutator_t *data, uint8_t *buf, siz * produce data larger than max_size. * @return Size of the mutated output. */ -size_t afl_custom_fuzz(atnwalk_mutator_t *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf, - uint8_t *add_buf, size_t add_buf_size, size_t max_size) { - struct sockaddr_un addr; - int fd_socket; - uint8_t ctrl_buf[8]; - uint8_t wanted; - - // let's display what's going on in a nice way - if (data->stage_havoc_cur == 0) { - data->afl->stage_name = (uint8_t *) "atnwalk - havoc"; - } - if (data->stage_havoc_cur == data->stage_havoc_max) { - data->afl->stage_name = (uint8_t *) "atnwalk - splice"; - } +size_t afl_custom_fuzz(atnwalk_mutator_t *data, uint8_t *buf, size_t buf_size, + uint8_t **out_buf, uint8_t *add_buf, size_t add_buf_size, + size_t max_size) { + + struct sockaddr_un addr; + int fd_socket; + uint8_t ctrl_buf[8]; + uint8_t wanted; + + // let's display what's going on in a nice way + if (data->stage_havoc_cur == 0) { + + data->afl->stage_name = (uint8_t *)"atnwalk - havoc"; + + } + + if (data->stage_havoc_cur == data->stage_havoc_max) { + + data->afl->stage_name = (uint8_t *)"atnwalk - splice"; + + } + + // increase the respective havoc or splice counters + if (data->stage_havoc_cur < data->stage_havoc_max) { + + data->stage_havoc_cur++; + data->afl->stage_cycles[STAGE_HAVOC]++; + + } else { + + // if there is nothing to splice, continue with havoc and skip splicing this + // time + if (data->afl->ready_for_splicing_count < 1) { + + data->stage_havoc_max = data->afl->stage_max; + data->stage_havoc_cur++; + data->afl->stage_cycles[STAGE_HAVOC]++; - // increase the respective havoc or splice counters - if (data->stage_havoc_cur < data->stage_havoc_max) { - data->stage_havoc_cur++; - data->afl->stage_cycles[STAGE_HAVOC]++; } else { - // if there is nothing to splice, continue with havoc and skip splicing this time - if (data->afl->ready_for_splicing_count < 1) { - data->stage_havoc_max = data->afl->stage_max; - data->stage_havoc_cur++; - data->afl->stage_cycles[STAGE_HAVOC]++; - } else { - data->stage_splice_cur++; - data->afl->stage_cycles[STAGE_SPLICE]++; - } - } - // keep track of found new corpus seeds per stage - if (data->afl->queued_items + data->afl->saved_crashes > data->prev_hits) { - if (data->stage_splice_cur <= 1) { - data->afl->stage_finds[STAGE_HAVOC] += data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; - } else { - data->afl->stage_finds[STAGE_SPLICE] += - data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; - } - } - data->prev_hits = data->afl->queued_items + data->afl->saved_crashes; + data->stage_splice_cur++; + data->afl->stage_cycles[STAGE_SPLICE]++; - // check whether this input produces a lot of timeouts, if it does then abandon this queue entry - if (data->afl->total_tmouts - data->prev_timeouts >= EXEC_TIMEOUT_MAX) { - data->afl->stage_max = data->afl->stage_cur; - return fail_gracefully(-1, data, buf, buf_size, out_buf); } - // initialize the socket - fd_socket = socket(AF_UNIX, SOCK_STREAM, 0); - if (fd_socket == -1) { return fail_fatal(fd_socket, out_buf); } - memset(&addr, 0, sizeof(addr)); - addr.sun_family = AF_UNIX; - strncpy(addr.sun_path, SOCKET_NAME, sizeof(addr.sun_path) - 1); - if (connect(fd_socket, (const struct sockaddr *) &addr, sizeof(addr)) == -1) { - return fail_fatal(fd_socket, out_buf); - } + } - // ask whether the server is alive - ctrl_buf[0] = SERVER_ARE_YOU_ALIVE; - if (!write_all(fd_socket, ctrl_buf, 1)) { - return fail_fatal(fd_socket, out_buf); - } + // keep track of found new corpus seeds per stage + if (data->afl->queued_items + data->afl->saved_crashes > data->prev_hits) { - // see whether the server replies as expected - if (!read_all(fd_socket, ctrl_buf, 1) || ctrl_buf[0] != SERVER_YES_I_AM_ALIVE) { - return fail_fatal(fd_socket, out_buf); - } + if (data->stage_splice_cur <= 1) { - // tell the server what we want to do - wanted = SERVER_MUTATE_BIT | SERVER_ENCODE_BIT; + data->afl->stage_finds[STAGE_HAVOC] += + data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; - // perform a crossover if we are splicing - if (data->stage_splice_cur > 0) { - wanted |= SERVER_CROSSOVER_BIT; - } + } else { + + data->afl->stage_finds[STAGE_SPLICE] += + data->afl->queued_items + data->afl->saved_crashes - data->prev_hits; - // tell the server what we want and how much data will be sent - ctrl_buf[0] = wanted; - put_uint32(ctrl_buf + 1, (uint32_t) buf_size); - if (!write_all(fd_socket, ctrl_buf, 5)) { - return fail_fatal(fd_socket, out_buf); } - // send the data to mutate and encode - if (!write_all(fd_socket, buf, buf_size)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + } + + data->prev_hits = data->afl->queued_items + data->afl->saved_crashes; + + // check whether this input produces a lot of timeouts, if it does then + // abandon this queue entry + if (data->afl->total_tmouts - data->prev_timeouts >= EXEC_TIMEOUT_MAX) { + + data->afl->stage_max = data->afl->stage_cur; + return fail_gracefully(-1, data, buf, buf_size, out_buf); + + } + + // initialize the socket + fd_socket = socket(AF_UNIX, SOCK_STREAM, 0); + if (fd_socket == -1) { return fail_fatal(fd_socket, out_buf); } + memset(&addr, 0, sizeof(addr)); + addr.sun_family = AF_UNIX; + strncpy(addr.sun_path, SOCKET_NAME, sizeof(addr.sun_path) - 1); + if (connect(fd_socket, (const struct sockaddr *)&addr, sizeof(addr)) == -1) { + + return fail_fatal(fd_socket, out_buf); + + } + + // ask whether the server is alive + ctrl_buf[0] = SERVER_ARE_YOU_ALIVE; + if (!write_all(fd_socket, ctrl_buf, 1)) { + + return fail_fatal(fd_socket, out_buf); + + } + + // see whether the server replies as expected + if (!read_all(fd_socket, ctrl_buf, 1) || + ctrl_buf[0] != SERVER_YES_I_AM_ALIVE) { + + return fail_fatal(fd_socket, out_buf); + + } + + // tell the server what we want to do + wanted = SERVER_MUTATE_BIT | SERVER_ENCODE_BIT; + + // perform a crossover if we are splicing + if (data->stage_splice_cur > 0) { wanted |= SERVER_CROSSOVER_BIT; } + + // tell the server what we want and how much data will be sent + ctrl_buf[0] = wanted; + put_uint32(ctrl_buf + 1, (uint32_t)buf_size); + if (!write_all(fd_socket, ctrl_buf, 5)) { + + return fail_fatal(fd_socket, out_buf); + + } + + // send the data to mutate and encode + if (!write_all(fd_socket, buf, buf_size)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + if (wanted & SERVER_CROSSOVER_BIT) { + + // since we requested crossover, we will first tell how much additional data + // is to be expected + put_uint32(ctrl_buf, (uint32_t)add_buf_size); + if (!write_all(fd_socket, ctrl_buf, 4)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + } - if (wanted & SERVER_CROSSOVER_BIT) { - // since we requested crossover, we will first tell how much additional data is to be expected - put_uint32(ctrl_buf, (uint32_t) add_buf_size); - if (!write_all(fd_socket, ctrl_buf, 4)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } - - // send the additional data for crossover - if (!write_all(fd_socket, add_buf, add_buf_size)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } - - // lastly, a seed is required for crossover so send one - put_uint64(ctrl_buf, (uint64_t) rand()); - if (!write_all(fd_socket, ctrl_buf, 8)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } + // send the additional data for crossover + if (!write_all(fd_socket, add_buf, add_buf_size)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + } - // since we requested mutation, we need to provide a seed for that - put_uint64(ctrl_buf, (uint64_t) rand()); + // lastly, a seed is required for crossover so send one + put_uint64(ctrl_buf, (uint64_t)rand()); if (!write_all(fd_socket, ctrl_buf, 8)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } - // obtain the required buffer size for the data that will be returned - if (!read_all(fd_socket, ctrl_buf, 4)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } - size_t new_size = (size_t) to_uint32(ctrl_buf); + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - // if the data is too large then we ignore this round - if (new_size > max_size) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); } - if (new_size > buf_size) { - // buf is too small, need to use data->fuzz_buf, let's see whether we need to reallocate - if (new_size > data->fuzz_size) { - data->fuzz_size = new_size << 1; - data->fuzz_buf = (uint8_t *) realloc(data->fuzz_buf, data->fuzz_size); - } - *out_buf = data->fuzz_buf; - } else { - // new_size fits into buf, so re-use it - *out_buf = buf; - } + } + + // since we requested mutation, we need to provide a seed for that + put_uint64(ctrl_buf, (uint64_t)rand()); + if (!write_all(fd_socket, ctrl_buf, 8)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + // obtain the required buffer size for the data that will be returned + if (!read_all(fd_socket, ctrl_buf, 4)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + size_t new_size = (size_t)to_uint32(ctrl_buf); + + // if the data is too large then we ignore this round + if (new_size > max_size) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + if (new_size > buf_size) { + + // buf is too small, need to use data->fuzz_buf, let's see whether we need + // to reallocate + if (new_size > data->fuzz_size) { + + data->fuzz_size = new_size << 1; + data->fuzz_buf = (uint8_t *)realloc(data->fuzz_buf, data->fuzz_size); - // obtain the encoded data - if (!read_all(fd_socket, *out_buf, new_size)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); } - close(fd_socket); - return new_size; -} + *out_buf = data->fuzz_buf; + + } else { + + // new_size fits into buf, so re-use it + *out_buf = buf; + } + + // obtain the encoded data + if (!read_all(fd_socket, *out_buf, new_size)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + close(fd_socket); + return new_size; + +} /** * A post-processing function to use right before AFL writes the test case to @@ -357,68 +441,88 @@ size_t afl_custom_fuzz(atnwalk_mutator_t *data, uint8_t *buf, size_t buf_size, u * @return Size of the output buffer after processing or the needed amount. * A return of 0 indicates an error. */ -size_t afl_custom_post_process(atnwalk_mutator_t *data, uint8_t *buf, size_t buf_size, uint8_t **out_buf) { - struct sockaddr_un addr; - int fd_socket; - uint8_t ctrl_buf[8]; - - // initialize the socket - fd_socket = socket(AF_UNIX, SOCK_STREAM, 0); - if (fd_socket == -1) { - return fail_fatal(fd_socket, out_buf); - } - memset(&addr, 0, sizeof(addr)); - addr.sun_family = AF_UNIX; - strncpy(addr.sun_path, SOCKET_NAME, sizeof(addr.sun_path) - 1); - if (connect(fd_socket, (const struct sockaddr *) &addr, sizeof(addr)) == -1) { - return fail_fatal(fd_socket, out_buf); - } +size_t afl_custom_post_process(atnwalk_mutator_t *data, uint8_t *buf, + size_t buf_size, uint8_t **out_buf) { - // ask whether the server is alive - ctrl_buf[0] = SERVER_ARE_YOU_ALIVE; - if (!write_all(fd_socket, ctrl_buf, 1)) { - return fail_fatal(fd_socket, out_buf); - } + struct sockaddr_un addr; + int fd_socket; + uint8_t ctrl_buf[8]; - // see whether the server replies as expected - if (!read_all(fd_socket, ctrl_buf, 1) || ctrl_buf[0] != SERVER_YES_I_AM_ALIVE) { - return fail_fatal(fd_socket, out_buf); - } + // initialize the socket + fd_socket = socket(AF_UNIX, SOCK_STREAM, 0); + if (fd_socket == -1) { return fail_fatal(fd_socket, out_buf); } + memset(&addr, 0, sizeof(addr)); + addr.sun_family = AF_UNIX; + strncpy(addr.sun_path, SOCKET_NAME, sizeof(addr.sun_path) - 1); + if (connect(fd_socket, (const struct sockaddr *)&addr, sizeof(addr)) == -1) { - // tell the server what we want and how much data will be sent - ctrl_buf[0] = SERVER_DECODE_BIT; - put_uint32(ctrl_buf + 1, (uint32_t) buf_size); - if (!write_all(fd_socket, ctrl_buf, 5)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } + return fail_fatal(fd_socket, out_buf); - // send the data to decode - if (!write_all(fd_socket, buf, buf_size)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } + } - // obtain the required buffer size for the data that will be returned - if (!read_all(fd_socket, ctrl_buf, 4)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } - size_t new_size = (size_t) to_uint32(ctrl_buf); + // ask whether the server is alive + ctrl_buf[0] = SERVER_ARE_YOU_ALIVE; + if (!write_all(fd_socket, ctrl_buf, 1)) { - // need to use data->post_process_buf, let's see whether we need to reallocate - if (new_size > data->post_process_size) { - data->post_process_size = new_size << 1; - data->post_process_buf = (uint8_t *) realloc(data->post_process_buf, data->post_process_size); - } - *out_buf = data->post_process_buf; + return fail_fatal(fd_socket, out_buf); - // obtain the decoded data - if (!read_all(fd_socket, *out_buf, new_size)) { - return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); - } + } - close(fd_socket); - return new_size; -} + // see whether the server replies as expected + if (!read_all(fd_socket, ctrl_buf, 1) || + ctrl_buf[0] != SERVER_YES_I_AM_ALIVE) { + + return fail_fatal(fd_socket, out_buf); + + } + + // tell the server what we want and how much data will be sent + ctrl_buf[0] = SERVER_DECODE_BIT; + put_uint32(ctrl_buf + 1, (uint32_t)buf_size); + if (!write_all(fd_socket, ctrl_buf, 5)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + } + + // send the data to decode + if (!write_all(fd_socket, buf, buf_size)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + // obtain the required buffer size for the data that will be returned + if (!read_all(fd_socket, ctrl_buf, 4)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + size_t new_size = (size_t)to_uint32(ctrl_buf); + + // need to use data->post_process_buf, let's see whether we need to reallocate + if (new_size > data->post_process_size) { + + data->post_process_size = new_size << 1; + data->post_process_buf = + (uint8_t *)realloc(data->post_process_buf, data->post_process_size); + + } + + *out_buf = data->post_process_buf; + + // obtain the decoded data + if (!read_all(fd_socket, *out_buf, new_size)) { + + return fail_gracefully(fd_socket, data, buf, buf_size, out_buf); + + } + + close(fd_socket); + return new_size; + +} /** * Deinitialize everything @@ -426,7 +530,10 @@ size_t afl_custom_post_process(atnwalk_mutator_t *data, uint8_t *buf, size_t buf * @param data The data ptr from afl_custom_init */ void afl_custom_deinit(atnwalk_mutator_t *data) { - free(data->fuzz_buf); - free(data->post_process_buf); - free(data); + + free(data->fuzz_buf); + free(data->post_process_buf); + free(data); + } + diff --git a/docs/Changelog.md b/docs/Changelog.md index 5ed5ef2b..f33acff9 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -4,8 +4,12 @@ release of the tool. See README.md for the general instruction manual. ### Version ++4.07a (dev) + - afl-fuzz: + - new env `AFL_POST_PROCESS_KEEP_ORIGINAL` to keep the orignal + data before post process on finds - afl-showmap: - added custom mutator post_process and send support + - a new grammar custom mutator atnwalk was submitted by @voidptr127 ! ### Version ++4.06c (release) diff --git a/frida_mode/src/js/js_api.c b/frida_mode/src/js/js_api.c index 00278082..288aec95 100644 --- a/frida_mode/src/js/js_api.c +++ b/frida_mode/src/js/js_api.c @@ -156,8 +156,8 @@ __attribute__((visibility("default"))) void js_api_set_instrument_instructions( } -__attribute__((visibility("default"))) void js_api_set_instrument_no_dynamic_load( - void) { +__attribute__((visibility("default"))) void +js_api_set_instrument_no_dynamic_load(void) { ranges_inst_dynamic_load = FALSE; -- cgit 1.4.1 From 8c228b0d23e303499dccf3df77c5d0b3a8b59b7b Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Mon, 24 Apr 2023 18:08:27 +0200 Subject: afl-showmap -I option --- afl-cmin | 2 +- docs/Changelog.md | 1 + src/afl-showmap.c | 175 +++++++++++++++++++++++++++++++++++++++++++++++------- 3 files changed, 156 insertions(+), 22 deletions(-) (limited to 'docs') diff --git a/afl-cmin b/afl-cmin index e2c26d91..6b36c261 100755 --- a/afl-cmin +++ b/afl-cmin @@ -234,7 +234,7 @@ BEGIN { } # while options if (!mem_limit) mem_limit = "none" - if (!timeout) timeout = "none" + if (!timeout) timeout = "5000" # get program args i = 0 diff --git a/docs/Changelog.md b/docs/Changelog.md index f33acff9..d4e68036 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -9,6 +9,7 @@ data before post process on finds - afl-showmap: - added custom mutator post_process and send support + - add `-I filelist` option, an alternative to `-i in_dir` - a new grammar custom mutator atnwalk was submitted by @voidptr127 ! diff --git a/src/afl-showmap.c b/src/afl-showmap.c index 0b9fc211..09a1d2dc 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -69,7 +69,9 @@ static afl_state_t *afl; static char *stdin_file; /* stdin file */ static u8 *in_dir = NULL, /* input folder */ - *out_file = NULL, *at_file = NULL; /* Substitution string for @@ */ + *out_file = NULL, /* output file or directory */ + *at_file = NULL, /* Substitution string for @@ */ + *in_filelist = NULL; /* input file list */ static u8 outfile[PATH_MAX]; @@ -878,6 +880,104 @@ u32 execute_testcases(u8 *dir) { } +u32 execute_testcases_filelist(u8 *fn) { + + u32 done = 0; + u8 buf[4096]; + u8 val_buf[2][STRINGIFY_VAL_SIZE_MAX]; + FILE *f; + + if (!be_quiet) { ACTF("Reading from '%s'...", fn); } + + if ((f = fopen(fn, "r")) == NULL) { FATAL("could not open '%s'", fn); } + + while (fgets(buf, sizeof(buf), f) != NULL) { + + struct stat st; + + u8 *fn2 = buf, *fn3; + ; + + while (*fn2 == ' ') { + + ++fn2; + + } + + if (*fn2) { + + while (fn2[strlen(fn2) - 1] == '\r' || fn2[strlen(fn2) - 1] == '\n' || + fn2[strlen(fn2) - 1] == ' ') { + + fn2[strlen(fn2) - 1] = 0; + + } + + } + + if (debug) { printf("Getting coverage for '%s'\n", fn2); } + + if (!*fn2) { continue; } + + if (lstat(fn2, &st) || access(fn2, R_OK)) { + + WARNF("Unable to access '%s'", fn2); + continue; + + } + + if (!S_ISREG(st.st_mode) || !st.st_size) { continue; } + + if ((fn3 = strrchr(fn2, '/'))) { + + ++fn3; + + } else { + + fn3 = fn2; + + } + + if (st.st_size > MAX_FILE && !be_quiet && !quiet_mode) { + + WARNF("Test case '%s' is too big (%s, limit is %s), partial reading", fn2, + stringify_mem_size(val_buf[0], sizeof(val_buf[0]), st.st_size), + stringify_mem_size(val_buf[1], sizeof(val_buf[1]), MAX_FILE)); + + } + + if (!collect_coverage) + snprintf(outfile, sizeof(outfile), "%s/%s", out_file, fn3); + + if (read_file(fn2)) { + + if (wait_for_gdb) { + + fprintf(stderr, "exec: gdb -p %d\n", fsrv->child_pid); + fprintf(stderr, "exec: kill -CONT %d\n", getpid()); + kill(0, SIGSTOP); + + } + + showmap_run_target_forkserver(fsrv, in_data, in_len); + ck_free(in_data); + ++done; + + if (child_crashed && debug) { WARNF("crashed: %s", fn2); } + + if (collect_coverage) + analyze_results(fsrv); + else + tcnt = write_results_to_file(fsrv, outfile); + + } + + } + + return done; + +} + /* Show banner. */ static void show_banner(void) { @@ -920,6 +1020,7 @@ static void usage(u8 *argv0) { " With -C, -o is a file, without -C it must be a " "directory\n" " and each bitmap will be written there individually.\n" + " -I filelist - alternatively to -i, -I is a list of files\n" " -C - collect coverage, writes all edges to -o and gives a " "summary\n" " Must be combined with -i.\n" @@ -932,7 +1033,7 @@ static void usage(u8 *argv0) { "This tool displays raw tuple data captured by AFL instrumentation.\n" "For additional help, consult %s/README.md.\n\n" - "If you use -i mode, then custom mutator post_process send send " + "If you use -i/-I mode, then custom mutator post_process send send " "functionality\n" "is supported.\n\n" @@ -994,7 +1095,7 @@ int main(int argc, char **argv_orig, char **envp) { if (getenv("AFL_QUIET") != NULL) { be_quiet = true; } - while ((opt = getopt(argc, argv, "+i:o:f:m:t:AeqCZOH:QUWbcrshXY")) > 0) { + while ((opt = getopt(argc, argv, "+i:I:o:f:m:t:AeqCZOH:QUWbcrshXY")) > 0) { switch (opt) { @@ -1012,6 +1113,11 @@ int main(int argc, char **argv_orig, char **envp) { in_dir = optarg; break; + case 'I': + if (in_filelist) { FATAL("Multiple -I options not supported"); } + in_filelist = optarg; + break; + case 'o': if (out_file) { FATAL("Multiple -o options not supported"); } @@ -1234,10 +1340,12 @@ int main(int argc, char **argv_orig, char **envp) { if (optind == argc || !out_file) { usage(argv[0]); } - if (in_dir) { + if (in_dir && in_filelist) { FATAL("you can only specify either -i or -I"); } + + if (in_dir || in_filelist) { if (!out_file && !collect_coverage) - FATAL("for -i you need to specify either -C and/or -o"); + FATAL("for -i/-I you need to specify either -C and/or -o"); } @@ -1294,7 +1402,7 @@ int main(int argc, char **argv_orig, char **envp) { } - if (in_dir) { + if (in_dir || in_filelist) { /* If we don't have a file name chosen yet, use a safe default. */ u8 *use_dir = "."; @@ -1400,7 +1508,7 @@ int main(int argc, char **argv_orig, char **envp) { } #ifdef __linux__ - if (!fsrv->nyx_mode && in_dir) { + if (!fsrv->nyx_mode && (in_dir || in_filelist)) { (void)check_binary_signatures(fsrv->target_path); @@ -1483,7 +1591,7 @@ int main(int argc, char **argv_orig, char **envp) { } - if (in_dir) { + if (in_dir || in_filelist) { afl->fsrv.dev_urandom_fd = open("/dev/urandom", O_RDONLY); afl->afl_env.afl_custom_mutator_library = @@ -1496,33 +1604,46 @@ int main(int argc, char **argv_orig, char **envp) { if (getenv("AFL_CUSTOM_MUTATOR_LIBRARY") || getenv("AFL_PYTHON_MODULE")) { WARNF( - "Custom mutator environment detected, this is only supported in -i " - "mode!\n"); + "Custom mutator environment detected, this is only supported in " + "-i/-I mode!\n"); } } - if (in_dir) { + if (in_dir || in_filelist) { DIR *dir_in, *dir_out = NULL; + u8 *dn = NULL; if (getenv("AFL_DEBUG_GDB")) wait_for_gdb = true; fsrv->dev_null_fd = open("/dev/null", O_RDWR); if (fsrv->dev_null_fd < 0) { PFATAL("Unable to open /dev/null"); } - // if a queue subdirectory exists switch to that - u8 *dn = alloc_printf("%s/queue", in_dir); - if ((dir_in = opendir(dn)) != NULL) { + if (in_filelist) { + + if (!be_quiet) ACTF("Reading from file list '%s'...", in_filelist); + + } else { + + // if a queue subdirectory exists switch to that + dn = alloc_printf("%s/queue", in_dir); + + if ((dir_in = opendir(dn)) != NULL) { + + closedir(dir_in); + in_dir = dn; + + } else { + + ck_free(dn); - closedir(dir_in); - in_dir = dn; + } - } else + if (!be_quiet) ACTF("Reading from directory '%s'...", in_dir); - ck_free(dn); - if (!be_quiet) ACTF("Reading from directory '%s'...", in_dir); + } if (!collect_coverage) { @@ -1576,9 +1697,21 @@ int main(int argc, char **argv_orig, char **envp) { if (fsrv->support_shmem_fuzz && !fsrv->use_shmem_fuzz) shm_fuzz = deinit_shmem(fsrv, shm_fuzz); - if (execute_testcases(in_dir) == 0) { + if (in_dir) { + + if (execute_testcases(in_dir) == 0) { + + FATAL("could not read input testcases from %s", in_dir); + + } + + } else { - FATAL("could not read input testcases from %s", in_dir); + if (execute_testcases_filelist(in_filelist) == 0) { + + FATAL("could not read input testcases from %s", in_filelist); + + } } -- cgit 1.4.1 From 7c3c0b26d1ae477fbae6944c0de18256621e1993 Mon Sep 17 00:00:00 2001 From: Keno Hassler <40292329+kenohassler@users.noreply.github.com> Date: Mon, 24 Apr 2023 20:21:54 +0200 Subject: document new env var --- docs/env_variables.md | 6 ++++-- src/afl-cc.c | 2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/env_variables.md b/docs/env_variables.md index c5995d13..087ccdb7 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -156,7 +156,7 @@ Available options: - LTO - LTO instrumentation - NATIVE - clang's original pcguard based instrumentation - NGRAM-x - deeper previous location coverage (from NGRAM-2 up to NGRAM-16) - - PCGUARD - our own pcgard based instrumentation (default) + - PCGUARD - our own pcguard based instrumentation (default) #### CMPLOG @@ -240,7 +240,9 @@ combined. the default `0x10000`. A value of 0 or empty sets the map address to be dynamic (the original AFL way, which is slower). - `AFL_LLVM_MAP_DYNAMIC` sets the shared memory address to be dynamic. - + - `AFL_LLVM_LTO_SKIPINIT` skips adding initialization code. Some global vars + (e.g. the highest location ID) are not injected. Needed to instrument with + [WAFL](https://github.com/fgsect/WAFL.git). For more information, see [instrumentation/README.lto.md](../instrumentation/README.lto.md). diff --git a/src/afl-cc.c b/src/afl-cc.c index 7f15ad76..d1001187 100644 --- a/src/afl-cc.c +++ b/src/afl-cc.c @@ -2099,6 +2099,8 @@ int main(int argc, char **argv, char **envp) { "bb\n" " AFL_REAL_LD: use this lld linker instead of the compiled in " "path\n" + " AFL_LLVM_LTO_SKIPINIT: don't inject initialization code " + "(used in WAFL mode)\n" "If anything fails - be sure to read README.lto.md!\n"); #endif -- cgit 1.4.1 From c0ecf7cf61fdca901b041d57e7e2bb78bc8fcf80 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Tue, 25 Apr 2023 08:33:51 +0200 Subject: only reverse reading the queue on restart --- docs/Changelog.md | 4 ++++ src/afl-fuzz-init.c | 30 ++++++++++++++++++++---------- 2 files changed, 24 insertions(+), 10 deletions(-) (limited to 'docs') diff --git a/docs/Changelog.md b/docs/Changelog.md index d4e68036..14323ae0 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -7,6 +7,10 @@ - afl-fuzz: - new env `AFL_POST_PROCESS_KEEP_ORIGINAL` to keep the orignal data before post process on finds + - reverse reading the seeds only on restarts + - afl-cc: + - new env `AFL_LLVM_LTO_SKIPINIT` to support the AFL++ based WASM + (https://github.com/fgsect/WAFL) project - afl-showmap: - added custom mutator post_process and send support - add `-I filelist` option, an alternative to `-i in_dir` diff --git a/src/afl-fuzz-init.c b/src/afl-fuzz-init.c index 01d1e82e..002a26f8 100644 --- a/src/afl-fuzz-init.c +++ b/src/afl-fuzz-init.c @@ -718,10 +718,21 @@ void read_testcases(afl_state_t *afl, u8 *directory) { if (nl_cnt) { - i = nl_cnt; + u32 done = 0; + + if (unlikely(afl->in_place_resume)) { + + i = nl_cnt; + + } else { + + i = 0; + + } + do { - --i; + if (unlikely(afl->in_place_resume)) { --i; } struct stat st; u8 dfn[PATH_MAX]; @@ -801,18 +812,17 @@ void read_testcases(afl_state_t *afl, u8 *directory) { } - /* - if (unlikely(afl->schedule >= FAST && afl->schedule <= RARE)) { + if (unlikely(afl->in_place_resume)) { - u64 cksum = hash64(afl->fsrv.trace_bits, afl->fsrv.map_size, - HASH_CONST); afl->queue_top->n_fuzz_entry = cksum % N_FUZZ_SIZE; - afl->n_fuzz[afl->queue_top->n_fuzz_entry] = 1; + if (unlikely(i == 0)) { done = 1; } - } + } else { + + if (unlikely(++i == (u32)nl_cnt)) { done = 1; } - */ + } - } while (i > 0); + } while (!done); } -- cgit 1.4.1 From 7b877e2c1d96efa7486ef4ba7860bec58dd1cd5b Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Tue, 25 Apr 2023 09:30:25 +0200 Subject: afl-cmin.bash -T support --- afl-cmin.bash | 107 +++++++++++++++++++++++++++++++++++++++++++++++------- docs/Changelog.md | 2 + 2 files changed, 96 insertions(+), 13 deletions(-) (limited to 'docs') diff --git a/afl-cmin.bash b/afl-cmin.bash index 5258758e..ba7083fa 100755 --- a/afl-cmin.bash +++ b/afl-cmin.bash @@ -7,6 +7,8 @@ # # Copyright 2014, 2015 Google Inc. All rights reserved. # +# Copyright 2019-2023 AFLplusplus +# # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -36,7 +38,7 @@ # array sizes. # -echo "corpus minimization tool for afl-fuzz by Michal Zalewski" +echo "corpus minimization tool for afl-fuzz" echo ######### @@ -46,14 +48,14 @@ echo # Process command-line options... MEM_LIMIT=none -TIMEOUT=none +TIMEOUT=5000 -unset IN_DIR OUT_DIR STDIN_FILE EXTRA_PAR MEM_LIMIT_GIVEN \ - AFL_CMIN_CRASHES_ONLY AFL_CMIN_ALLOW_ANY QEMU_MODE UNICORN_MODE +unset IN_DIR OUT_DIR STDIN_FILE EXTRA_PAR MEM_LIMIT_GIVEN F_ARG \ + AFL_CMIN_CRASHES_ONLY AFL_CMIN_ALLOW_ANY QEMU_MODE UNICORN_MODE T_ARG export AFL_QUIET=1 -while getopts "+i:o:f:m:t:eOQUAChXY" opt; do +while getopts "+i:o:f:m:t:T:eOQUAChXY" opt; do case "$opt" in @@ -69,6 +71,7 @@ while getopts "+i:o:f:m:t:eOQUAChXY" opt; do ;; "f") STDIN_FILE="$OPTARG" + F_ARG=1 ;; "m") MEM_LIMIT="$OPTARG" @@ -106,6 +109,9 @@ while getopts "+i:o:f:m:t:eOQUAChXY" opt; do EXTRA_PAR="$EXTRA_PAR -U" UNICORN_MODE=1 ;; + "T") + T_ARG="$OPTARG" + ;; "?") exit 1 ;; @@ -130,9 +136,10 @@ Required parameters: Execution control settings: - -f file - location read by the fuzzed program (stdin) - -m megs - memory limit for child process ($MEM_LIMIT MB) - -t msec - run time limit for child process (none) + -T tasks - how many parallel processes to create (default=1, "all"=nproc) + -f file - location read by the fuzzed program (default: stdin) + -m megs - memory limit for child process (default=$MEM_LIMIT MB) + -t msec - run time limit for child process (default: 5000ms) -O - use binary-only instrumentation (FRIDA mode) -Q - use binary-only instrumentation (QEMU mode) -U - use unicorn-based instrumentation (Unicorn mode) @@ -199,6 +206,11 @@ fi # Check for obvious errors. +if [ ! "$T_ARG" = "" -a ! "$F_ARG" = "" ]; then + echo "[-] Error: -T and -f can not be used together." 1>&2 + exit 1 +fi + if [ ! "$MEM_LIMIT" = "none" ]; then if [ "$MEM_LIMIT" -lt "5" ]; then @@ -233,7 +245,7 @@ if [ "$NYX_MODE" = "" ]; then fi -grep -aq AFL_DUMP_MAP_SIZE "./$TARGET_BIN" && { +grep -aq AFL_DUMP_MAP_SIZE "$TARGET_BIN" && { echo "[!] Trying to obtain the map size of the target ..." MAPSIZE=`AFL_DUMP_MAP_SIZE=1 "./$TARGET_BIN" 2>/dev/null` test -n "$MAPSIZE" && { @@ -299,14 +311,29 @@ if [ ! -x "$SHOWMAP" ]; then exit 1 fi +THREADS= +if [ ! "$T_ARG" = "" ]; then + if [ "$T_ARG" = "all" ]; then + THREADS=$(nproc) + else + if [ "$T_ARG" -gt 0 -a "$T_ARG" -le "$(nproc)" ]; then + THREADS=$T_ARG + else + echo "[-] Error: -T parameter must between 1 and $(nproc) or \"all\"." 1>&2 + fi + fi +fi + IN_COUNT=$((`ls -- "$IN_DIR" 2>/dev/null | wc -l`)) if [ "$IN_COUNT" = "0" ]; then - echo "[+] Hmm, no inputs in the target directory. Nothing to be done." + echo "[-] Hmm, no inputs in the target directory. Nothing to be done." rm -rf "$TRACE_DIR" exit 1 fi +echo "[+] Found $IN_COUNT files for minimizing." + FIRST_FILE=`ls "$IN_DIR" | head -1` # Make sure that we're not dealing with a directory. @@ -355,6 +382,18 @@ else fi +TMPFILE=$OUT_DIR/.list.$$ +if [ ! "$THREADS" = "" ]; then + ls -- "$IN_DIR" > $TMPFILE 2>/dev/null + IN_COUNT=$(cat $TMPFILE | wc -l) + SPLIT=$(($IN_COUNT / $THREADS)) + if [ "$(($IN_COUNT % $THREADS))" -gt 0 ]; then + SPLIT=$(($SPLIT + 1)) + fi + echo "[+] Splitting workload into $THREADS tasks with $SPLIT items on average each." + split -l $SPLIT $TMPFILE $TMPFILE. +fi + # Let's roll! ############################# @@ -363,6 +402,7 @@ fi echo "[*] Obtaining traces for input files in '$IN_DIR'..." +if [ "$THREADS" = "" ]; then ( CUR=0 @@ -386,17 +426,58 @@ echo "[*] Obtaining traces for input files in '$IN_DIR'..." printf "\\r Processing file $CUR/$IN_COUNT... " cp "$IN_DIR/$fn" "$STDIN_FILE" - "$SHOWMAP" -m "$MEM_LIMIT" -t "$TIMEOUT" -o "$TRACE_DIR/$fn" -Z $EXTRA_PAR -H "$STDIN_FILE" -- "$@" Date: Tue, 25 Apr 2023 13:13:43 +0200 Subject: afl-cmin -T support --- afl-cmin | 143 ++++++++++++++++++++++++++++++++++++++++++------------ docs/Changelog.md | 2 +- src/afl-showmap.c | 23 +++++---- 3 files changed, 124 insertions(+), 44 deletions(-) (limited to 'docs') diff --git a/afl-cmin b/afl-cmin index 6b36c261..c8bbd8d7 100755 --- a/afl-cmin +++ b/afl-cmin @@ -103,9 +103,10 @@ function usage() { " -o dir - output directory for minimized files\n" \ "\n" \ "Execution control settings:\n" \ +" -T tasks - how many parallel tasks to run (default: 1, all=nproc)\n" \ " -f file - location read by the fuzzed program (stdin)\n" \ " -m megs - memory limit for child process ("mem_limit" MB)\n" \ -" -t msec - run time limit for child process (default: none)\n" \ +" -t msec - run time limit for child process (default: 5000)\n" \ " -O - use binary-only instrumentation (FRIDA mode)\n" \ " -Q - use binary-only instrumentation (QEMU mode)\n" \ " -U - use unicorn-based instrumentation (unicorn mode)\n" \ @@ -119,7 +120,6 @@ function usage() { "For additional tips, please consult README.md\n" \ "\n" \ "Environment variables used:\n" \ -"AFL_ALLOW_TMP: allow unsafe use of input/output directories under {/var}/tmp\n" \ "AFL_CRASH_EXITCODE: optional child exit code to be interpreted as crash\n" \ "AFL_FORKSRV_INIT_TMOUT: time the fuzzer waits for the forkserver to come up\n" \ "AFL_KEEP_TRACES: leave the temporary /.traces directory\n" \ @@ -159,13 +159,19 @@ BEGIN { # process options Opterr = 1 # default is to diagnose Optind = 1 # skip ARGV[0] - while ((_go_c = getopt(ARGC, ARGV, "hi:o:f:m:t:eACOQUXY?")) != -1) { + while ((_go_c = getopt(ARGC, ARGV, "hi:o:f:m:t:eACOQUXYT:?")) != -1) { if (_go_c == "i") { if (!Optarg) usage() if (in_dir) { print "Option "_go_c" is only allowed once" > "/dev/stderr"} in_dir = Optarg continue } else + if (_go_c == "T") { + if (!Optarg) usage() + if (threads) { print "Option "_go_c" is only allowed once" > "/dev/stderr"} + threads = Optarg + continue + } else if (_go_c == "o") { if (!Optarg) usage() if (out_dir) { print "Option "_go_c" is only allowed once" > "/dev/stderr"} @@ -253,21 +259,30 @@ BEGIN { # Do a sanity check to discourage the use of /tmp, since we can't really # handle this safely from an awk script. - if (!ENVIRON["AFL_ALLOW_TMP"]) { - dirlist[0] = in_dir - dirlist[1] = target_bin - dirlist[2] = out_dir - dirlist[3] = stdin_file - "pwd" | getline dirlist[4] # current directory - for (dirind in dirlist) { - dir = dirlist[dirind] - - if (dir ~ /^(\/var)?\/tmp/) { - print "[-] Error: do not use this script in /tmp or /var/tmp." > "/dev/stderr" - exit 1 - } - } - delete dirlist + #if (!ENVIRON["AFL_ALLOW_TMP"]) { + # dirlist[0] = in_dir + # dirlist[1] = target_bin + # dirlist[2] = out_dir + # dirlist[3] = stdin_file + # "pwd" | getline dirlist[4] # current directory + # for (dirind in dirlist) { + # dir = dirlist[dirind] + # + # if (dir ~ /^(\/var)?\/tmp/) { + # print "[-] Error: do not use this script in /tmp or /var/tmp." > "/dev/stderr" + # exit 1 + # } + # } + # delete dirlist + #} + + if (threads && stdin_file) { + print "[-] Error: -T and -f cannot be used together." > "/dev/stderr" + exit 1 + } + + if (!threads && !stdin_file) { + print "[*] Are you aware of the '-T all' parallelize option that massively improves the speed for large corpuses?" } # If @@ is specified, but there's no -f, let's come up with a temporary input @@ -350,6 +365,18 @@ BEGIN { exit 1 } + if (threads) { + "nproc" | getline nproc + if (threads == "all") { + threads = nproc + } else { + if (!(threads > 1 && threads <= nproc)) { + print "[-] Error: -T option must be between 1 and "nproc" or \"all\"." > "/dev/stderr" + exit 1 + } + } + } + # Check for the more efficient way to copy files... if (0 != system("mkdir -p -m 0700 "trace_dir)) { print "[-] Error: Cannot create directory "trace_dir > "/dev/stderr" @@ -459,27 +486,81 @@ BEGIN { # STEP 1: Collecting traces # ############################# + if (threads) { + + inputsperfile = in_count / threads + if (in_count % threads) { + inputsperfile++; + } + + cnt = 0; + tmpfile=out_dir "/.filelist" + for (instance = 1; instance < threads; instance++) { + for (i = 0; i < inputsperfile; i++) { + print in_dir"/"infilesSmallToBigFull[cnt] >> tmpfile"."instance + cnt++ + } + } + for (; cnt < in_count; cnt++) { + print in_dir"/"infilesSmallToBigFull[cnt] >> tmpfile"."threads + } + + print "ls -l "tmpfile"*" + + } + print "[*] Obtaining traces for "in_count" input files in '"in_dir"'." cur = 0; - if (!stdin_file) { - print " Processing "in_count" files (forkserver mode)..." -# print AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -i \""in_dir"\" -- \""target_bin"\" "prog_args_string - retval = system(AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -i \""in_dir"\" -- \""target_bin"\" "prog_args_string) + + if (threads > 1) { + + print "[*] Creating " threads " parallel tasks with about " inputsperfile " each." + for (i = 1; i <= threads; i++) { + + if (!stdin_file) { +# print " { "AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -I \""tmpfile"."i"\" -- \""target_bin"\" "prog_args_string"; > "tmpfile"."i".done ; } &" + retval = system(" { "AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -I \""tmpfile"."i"\" -- \""target_bin"\" "prog_args_string"; > "tmpfile"."i".done ; } &") + } else { + stdin_file=tmpfile"."i".stdin" +# print " { "AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -I \""tmpfile"."i"\" -H \""stdin_file"\" -- \""target_bin"\" "prog_args_string" "tmpfile"."i".done ; } &" + retval = system(" { "AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -I \""tmpfile"."i"\" -H \""stdin_file"\" -- \""target_bin"\" "prog_args_string" "tmpfile"."i".done ; } &") + } + } + print "[*] Waiting for parallel tasks to complete ..." + # wait for all processes to finish + ok=0 + while (ok < threads) { + ok=0 + for (i = 1; i <= threads; i++) { + if (system("test -f "tmpfile"."i".done") == 0) { + ok++ + } + } + } + print "[*] Done!" + system("rm -f "tmpfile"*") } else { - print " Processing "in_count" files (forkserver mode)..." + if (!stdin_file) { + print " Processing "in_count" files (forkserver mode)..." +# print AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -i \""in_dir"\" -- \""target_bin"\" "prog_args_string + retval = system(AFL_MAP_SIZE AFL_CMIN_ALLOW_ANY AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -i \""in_dir"\" -- \""target_bin"\" "prog_args_string) + } else { + print " Processing "in_count" files (forkserver mode)..." # print AFL_CMIN_CRASHES_ONLY"\""showmap"\" -m "mem_limit" -t "timeout" -o \""trace_dir"\" -Z "extra_par" -i \""in_dir"\" -H \""stdin_file"\" -- \""target_bin"\" "prog_args_string" /dev/null") - system("rmdir "out_dir) + if (!ENVIRON["AFL_KEEP_TRACES"]) { + system("rm -rf "trace_dir" 2>/dev/null") + system("rmdir "out_dir) + } + exit retval } - exit retval + } ####################################################### diff --git a/docs/Changelog.md b/docs/Changelog.md index 816a864d..667fd634 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -14,7 +14,7 @@ - afl-showmap: - added custom mutator post_process and send support - add `-I filelist` option, an alternative to `-i in_dir` - - afl-cmin.bash: + - afl-cmin + afl-cmin.bash: - `-T threads` parallel task support, huge speedup! - a new grammar custom mutator atnwalk was submitted by @voidptr127 ! diff --git a/src/afl-showmap.c b/src/afl-showmap.c index 09a1d2dc..d0e01cb1 100644 --- a/src/afl-showmap.c +++ b/src/afl-showmap.c @@ -894,9 +894,7 @@ u32 execute_testcases_filelist(u8 *fn) { while (fgets(buf, sizeof(buf), f) != NULL) { struct stat st; - - u8 *fn2 = buf, *fn3; - ; + u8 *fn2 = buf, *fn3; while (*fn2 == ' ') { @@ -904,14 +902,11 @@ u32 execute_testcases_filelist(u8 *fn) { } - if (*fn2) { - - while (fn2[strlen(fn2) - 1] == '\r' || fn2[strlen(fn2) - 1] == '\n' || - fn2[strlen(fn2) - 1] == ' ') { - - fn2[strlen(fn2) - 1] = 0; + while (*fn2 && + (fn2[strlen(fn2) - 1] == '\r' || fn2[strlen(fn2) - 1] == '\n' || + fn2[strlen(fn2) - 1] == ' ')) { - } + fn2[strlen(fn2) - 1] = 0; } @@ -926,6 +921,8 @@ u32 execute_testcases_filelist(u8 *fn) { } + ++done; + if (!S_ISREG(st.st_mode) || !st.st_size) { continue; } if ((fn3 = strrchr(fn2, '/'))) { @@ -946,9 +943,12 @@ u32 execute_testcases_filelist(u8 *fn) { } - if (!collect_coverage) + if (!collect_coverage) { + snprintf(outfile, sizeof(outfile), "%s/%s", out_file, fn3); + } + if (read_file(fn2)) { if (wait_for_gdb) { @@ -961,7 +961,6 @@ u32 execute_testcases_filelist(u8 *fn) { showmap_run_target_forkserver(fsrv, in_data, in_len); ck_free(in_data); - ++done; if (child_crashed && debug) { WARNF("crashed: %s", fn2); } -- cgit 1.4.1 From 21865c622483d2e2285de3dfad4626c28ca27843 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Tue, 25 Apr 2023 16:47:37 +0200 Subject: rename env to AFL_IGNORE_PROBLEMS_COVERAGE --- docs/FAQ.md | 3 ++- docs/env_variables.md | 3 ++- include/envs.h | 1 + instrumentation/afl-compiler-rt.o.c | 4 ++-- src/afl-fuzz.c | 2 ++ 5 files changed, 9 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/docs/FAQ.md b/docs/FAQ.md index 4a9080f8..76350c79 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -229,7 +229,8 @@ If you find an interesting or important question missing, submit it via If this is not a viable option, you can set `AFL_IGNORE_PROBLEMS=1` but then the existing map will be used also for the newly loaded libraries, which allows it to work, however, the efficiency of the fuzzing will be partially - degraded. + degraded. Note that there is additionally `AFL_IGNORE_PROBLEMS_COVERAGE` to + additionally tell AFL++ to ignore any coverage from the late loaded libaries.

diff --git a/docs/env_variables.md b/docs/env_variables.md index 087ccdb7..b1f23159 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -406,7 +406,8 @@ checks or alter some of the more exotic semantics of the tool: - If afl-fuzz encounters an incorrect fuzzing setup during a fuzzing session (not at startup), it will terminate. If you do not want this, then you can - set `AFL_IGNORE_PROBLEMS`. + set `AFL_IGNORE_PROBLEMS`. If you additionally want to also ignore coverage + from late loaded libraries, you can set `AFL_IGNORE_PROBLEMS_COVERAGE`. - When running in the `-M` or `-S` mode, setting `AFL_IMPORT_FIRST` causes the fuzzer to import test cases from other instances before doing anything else. diff --git a/include/envs.h b/include/envs.h index 5e68c80b..fe5ee0e3 100644 --- a/include/envs.h +++ b/include/envs.h @@ -106,6 +106,7 @@ static char *afl_environment_variables[] = { "AFL_HARDEN", "AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES", "AFL_IGNORE_PROBLEMS", + "AFL_IGNORE_PROBLEMS_COVERAGE", "AFL_IGNORE_TIMEOUTS", "AFL_IGNORE_UNKNOWN_ENVS", "AFL_IMPORT_FIRST", diff --git a/instrumentation/afl-compiler-rt.o.c b/instrumentation/afl-compiler-rt.o.c index 74506e4c..0912e52b 100644 --- a/instrumentation/afl-compiler-rt.o.c +++ b/instrumentation/afl-compiler-rt.o.c @@ -1565,13 +1565,13 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) { "be able to fuzz them or LD_PRELOAD to run outside of afl-fuzz.\n" "To ignore this set AFL_IGNORE_PROBLEMS=1 but this will lead to " "ambiguous coverage data.\n" - "In addition, you can set AFL_LLVM_IGNORE_PROBLEMS_COVERAGE=1 to " + "In addition, you can set AFL_IGNORE_PROBLEMS_COVERAGE=1 to " "ignore the additional coverage instead (use with caution!).\n"); abort(); } else { - u8 ignore_dso_after_fs = !!getenv("AFL_LLVM_IGNORE_PROBLEMS_COVERAGE"); + u8 ignore_dso_after_fs = !!getenv("AFL_IGNORE_PROBLEMS_COVERAGE"); if (__afl_debug && ignore_dso_after_fs) { fprintf(stderr, "Ignoring coverage from dynamically loaded code\n"); diff --git a/src/afl-fuzz.c b/src/afl-fuzz.c index ebdbb3fa..c44144f5 100644 --- a/src/afl-fuzz.c +++ b/src/afl-fuzz.c @@ -259,6 +259,8 @@ static void usage(u8 *argv0, int more_help) { "AFL_HANG_TMOUT: override timeout value (in milliseconds)\n" "AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES: don't warn about core dump handlers\n" "AFL_IGNORE_PROBLEMS: do not abort fuzzing if an incorrect setup is detected\n" + "AFL_IGNORE_PROBLEMS_COVERAGE: if set in addition to AFL_IGNORE_PROBLEMS - also\n + " ignore those libs for coverage\n" "AFL_IGNORE_TIMEOUTS: do not process or save any timeouts\n" "AFL_IGNORE_UNKNOWN_ENVS: don't warn on unknown env vars\n" "AFL_IMPORT_FIRST: sync and import test cases from other fuzzer instances first\n" -- cgit 1.4.1 From b18bc7b98fa23ef805ed2ee3eec04dc1929afd49 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Wed, 26 Apr 2023 16:25:03 +0200 Subject: changelog updates --- TODO.md | 1 - docs/Changelog.md | 8 +++++--- 2 files changed, 5 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/TODO.md b/TODO.md index dba75070..2efcefea 100644 --- a/TODO.md +++ b/TODO.md @@ -10,7 +10,6 @@ - parallel builds for source-only targets - get rid of check_binary, replace with more forkserver communication - first fuzzer should be a main automatically? not sure. - - reload fuzz binary on signal ## Maybe diff --git a/docs/Changelog.md b/docs/Changelog.md index 667fd634..20b915fa 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -5,9 +5,11 @@ ### Version ++4.07a (dev) - afl-fuzz: + - reverse reading the seeds only on restarts (increases performance) - new env `AFL_POST_PROCESS_KEEP_ORIGINAL` to keep the orignal - data before post process on finds - - reverse reading the seeds only on restarts + data before post process on finds (for atnwalk custom mutator) + - new env `AFL_IGNORE_PROBLEMS_COVERAGE` to ignore coverage from + loaded libs after forkserver initialization (required by Mozilla) - afl-cc: - new env `AFL_LLVM_LTO_SKIPINIT` to support the AFL++ based WASM (https://github.com/fgsect/WAFL) project @@ -15,7 +17,7 @@ - added custom mutator post_process and send support - add `-I filelist` option, an alternative to `-i in_dir` - afl-cmin + afl-cmin.bash: - - `-T threads` parallel task support, huge speedup! + - `-T threads` parallel task support, can be a huge speedup! - a new grammar custom mutator atnwalk was submitted by @voidptr127 ! -- cgit 1.4.1