From d9e39be4e8dd0242877412d1a17589acc825292e Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 16:49:16 +0100 Subject: Remove the word "simply" --- utils/libtokencap/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'utils') diff --git a/utils/libtokencap/README.md b/utils/libtokencap/README.md index a39ed3a5..91ebebd1 100644 --- a/utils/libtokencap/README.md +++ b/utils/libtokencap/README.md @@ -40,10 +40,10 @@ when using afl-gcc. This setting specifically adds the following flags: -fno-builtin-strcasestr ``` -The next step is simply loading this library via LD_PRELOAD. The optimal usage -pattern is to allow afl-fuzz to fuzz normally for a while and build up a corpus, -and then fire off the target binary, with libtokencap.so loaded, on every file -found by AFL in that earlier run. This demonstrates the basic principle: +The next step is to load this library via LD_PRELOAD. The optimal usage pattern +is to allow afl-fuzz to fuzz normally for a while and build up a corpus, and +then fire off the target binary, with libtokencap.so loaded, on every file found +by AFL in that earlier run. This demonstrates the basic principle: ``` export AFL_TOKEN_FILE=$PWD/temp_output.txt -- cgit 1.4.1 From 0ae9b0dff4773c34913d89d8efcab4453552c140 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 17:01:45 +0100 Subject: Remove the word "please" --- README.md | 2 +- docs/custom_mutators.md | 9 +++++---- docs/env_variables.md | 2 +- docs/fuzzing_in_depth.md | 4 ++-- frida_mode/README.md | 4 ++-- instrumentation/README.laf-intel.md | 6 +++--- instrumentation/README.llvm.md | 8 ++++---- unicorn_mode/README.md | 38 ++++++++++++++++++------------------- utils/optimin/README.md | 8 ++++---- 9 files changed, 41 insertions(+), 40 deletions(-) (limited to 'utils') diff --git a/README.md b/README.md index e74c91e5..dbf49b20 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ terms of the Apache-2.0 License. See the [LICENSE](LICENSE) for details. Here is some information to get you started: -* For releases, please see the +* For releases, see the [Releases tab](https://github.com/AFLplusplus/AFLplusplus/releases) and [branches](#branches). Also take a look at the list of [important changes in AFL++](docs/important_changes.md). diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index b1dfd309..2caba560 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -5,7 +5,7 @@ For now, we support C/C++ library and Python module, collectivelly named as the custom mutator. There is also experimental support for Rust in `custom_mutators/rust`. -Please refer to that directory for documentation. +For documentation, refer to that directory. Run ```cargo doc -p custom_mutator --open``` in that directory to view the documentation in your web browser. @@ -25,7 +25,8 @@ Now AFL also supports multiple custom mutators which can be specified in the sam ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="full/path/to/mutator_first.so;full/path/to/mutator_second.so" ``` -Please see [APIs](#2-apis) and [Usage](#3-usage) for detail. + +For details, see [APIs](#2-apis) and [Usage](#3-usage). The custom mutation stage is set to be the first non-deterministic stage (right before the havoc stage). @@ -288,8 +289,8 @@ afl-fuzz /path/to/program ## 4) Example -Please see [example.c](../custom_mutators/examples/example.c) and -[example.py](../custom_mutators/examples/example.py) +See [example.c](../custom_mutators/examples/example.c) and +[example.py](../custom_mutators/examples/example.py). ## 5) Other Resources diff --git a/docs/env_variables.md b/docs/env_variables.md index 6f6110ae..bb7ea70b 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -320,7 +320,7 @@ checks or alter some of the more exotic semantics of the tool: additional mutations. If `AFL_CUSTOM_MUTATOR_ONLY` is also set, all mutations will solely be performed with the custom mutator. This feature allows to configure custom mutators which can be very helpful, e.g. fuzzing - XML or other highly flexible structured input. Please see + XML or other highly flexible structured input. For details, see [custom_mutators.md](custom_mutators.md). - Setting `AFL_CYCLE_SCHEDULES` will switch to a different schedule every time diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 7aabe090..c94590a4 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -1,7 +1,7 @@ # Fuzzing with AFL++ The following describes how to fuzz with a target if source code is available. -If you have a binary-only target, please go to +If you have a binary-only target, go to [fuzzing_binary-only_targets.md](fuzzing_binary-only_targets.md). Fuzzing source code is a three-step process: @@ -431,7 +431,7 @@ Run it like `screen -dmS afl-main -- afl-fuzz -M main-$HOSTNAME -i ...` and it will start away in a screen session. To enter this session, type `screen -r afl-main`. You see - it makes sense to name the screen session same as the afl-fuzz -M/-S naming :-) -For more information on screen or tmux please check their documentation. +For more information on screen or tmux, check their documentation. If you need to stop and re-start the fuzzing, use the same command line options (or even change them by selecting a different power schedule or another mutation diff --git a/frida_mode/README.md b/frida_mode/README.md index c2b98473..58519699 100644 --- a/frida_mode/README.md +++ b/frida_mode/README.md @@ -367,8 +367,8 @@ using `AFL_FRIDA_INST_RANGES` or similar. ## Debugging -Please refer to [DEBUGGING.md](DEBUGGING.md) for assistance should you encounter -problems with FRIDA mode. +Should you encounter problems with FRIDA mode, refer to +[DEBUGGING.md](DEBUGGING.md) for assistance. ## To do diff --git a/instrumentation/README.laf-intel.md b/instrumentation/README.laf-intel.md index 3cde10c3..06e653ea 100644 --- a/instrumentation/README.laf-intel.md +++ b/instrumentation/README.laf-intel.md @@ -42,9 +42,9 @@ comparisons. A new experimental feature is splitting floating point comparisons into a series of sign, exponent and mantissa comparisons followed by splitting each of them into 8 bit comparisons when necessary. It is activated with the -`AFL_LLVM_LAF_SPLIT_FLOATS` setting. Please note that full IEEE 754 -functionality is not preserved, that is values of nan and infinity will probably -behave differently. +`AFL_LLVM_LAF_SPLIT_FLOATS` setting. Note that full IEEE 754 functionality is +not preserved, that is values of nan and infinity will probably behave +differently. Note that setting this automatically activates `AFL_LLVM_LAF_SPLIT_COMPARES`. diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index 35f38261..8133cbe4 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -119,8 +119,8 @@ If you need just to instrument specific parts of the code, you can the instrument file list which C/C++ files to actually instrument. See [README.instrument_list.md](README.instrument_list.md) -For splitting memcmp, strncmp, etc. please see -[README.laf-intel.md](README.laf-intel.md) +For splitting memcmp, strncmp, etc., see +[README.laf-intel.md](README.laf-intel.md). Then there are different ways of instrumenting the target: @@ -157,8 +157,8 @@ nozero counter default for performance reasons. ## 4) deferred initialization, persistent mode, shared memory fuzzing -This is the most powerful and effective fuzzing you can do. Please see -[README.persistent_mode.md](README.persistent_mode.md) for a full explanation. +This is the most powerful and effective fuzzing you can do. For a full +explanation, see [README.persistent_mode.md](README.persistent_mode.md). ## 5) Bonus feature: 'dict2file' pass diff --git a/unicorn_mode/README.md b/unicorn_mode/README.md index d2b7d16f..ed85e687 100644 --- a/unicorn_mode/README.md +++ b/unicorn_mode/README.md @@ -10,8 +10,8 @@ The CompareCoverage and NeverZero counters features are by Andrea Fioraldi Date: Thu, 2 Dec 2021 17:13:12 +0100 Subject: Change the word "env var" to "environment variable" --- docs/env_variables.md | 2 +- docs/features.md | 4 ++-- docs/important_changes.md | 5 +++-- qemu_mode/README.md | 5 +++-- utils/qbdi_mode/README.md | 6 ++++-- 5 files changed, 13 insertions(+), 9 deletions(-) (limited to 'utils') diff --git a/docs/env_variables.md b/docs/env_variables.md index ec9b2b01..6c90e84c 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -564,7 +564,7 @@ The QEMU wrapper used to instrument binary-only code supports several settings: general purpose registers and restore them in each persistent cycle. - Another modality to execute the persistent loop is to specify also the - `AFL_QEMU_PERSISTENT_RET=end addr` env variable. With this variable + `AFL_QEMU_PERSISTENT_RET=end addr` environment variable. With this variable assigned, instead of patching the return address, the specified instruction is transformed to a jump towards `start addr`. diff --git a/docs/features.md b/docs/features.md index 35a869a9..fb688a64 100644 --- a/docs/features.md +++ b/docs/features.md @@ -19,8 +19,8 @@ QEMU 5.1 with laf-intel and redqueen, frida mode, unicorn mode, gcc plugin, full | Snapshot LKM Support | | (x)(8) | (x)(8) | | (x)(5) | | | | Shared Memory Test cases | | x | x | x86[_64]/arm64 | x | x | | -1. default for LLVM >= 9.0, env var for older version due an efficiency bug in - previous llvm versions +1. default for LLVM >= 9.0, environment variable for older version due an + efficiency bug in previous llvm versions 2. GCC creates non-performant code, hence it is disabled in gcc_plugin 3. with `AFL_LLVM_THREADSAFE_INST`, disables NeverZero 4. with pcguard mode and LTO mode for LLVM 11 and newer diff --git a/docs/important_changes.md b/docs/important_changes.md index 9d6bbbbe..726de64d 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -48,8 +48,9 @@ behaviors and defaults: * deterministic fuzzing is now disabled by default (unless using -M) and can be enabled with -D * a caching of test cases can now be performed and can be modified by - editing config.h for TESTCASE_CACHE or by specifying the env variable - `AFL_TESTCACHE_SIZE` (in MB). Good values are between 50-500 (default: 50). + editing config.h for TESTCASE_CACHE or by specifying the environment + variable `AFL_TESTCACHE_SIZE` (in MB). Good values are between 50-500 + (default: 50). * -M mains do not perform trimming * examples/ got renamed to utils/ * libtokencap/ libdislocator/ and qdbi_mode/ were moved to utils/ diff --git a/qemu_mode/README.md b/qemu_mode/README.md index b4b5e7bf..8e04cbf9 100644 --- a/qemu_mode/README.md +++ b/qemu_mode/README.md @@ -83,8 +83,9 @@ For more information, see [README.persistent.md](README.persistent.md). As an extension to persistent mode, qemuafl can snapshot and restore the memory state and brk(). For details, see [README.persistent.md](README.persistent.md). -The env var that enables the ready to use snapshot mode is `AFL_QEMU_SNAPSHOT` -and takes a hex address as a value that is the snapshot entry point. +The environment variable that enables the ready to use snapshot mode is +`AFL_QEMU_SNAPSHOT` and takes a hex address as a value that is the snapshot +entry point. Snapshot mode can work restoring all the writeable pages, that is typically slower than fork() mode but, on the other hand, it can scale better with diff --git a/utils/qbdi_mode/README.md b/utils/qbdi_mode/README.md index 8b768906..2e8b01d4 100755 --- a/utils/qbdi_mode/README.md +++ b/utils/qbdi_mode/README.md @@ -34,7 +34,9 @@ For x86 standalone-toolchain ./build/tools/make_standalone_toolchain.py --arch x86 --api 21 --install-dir ../android-standalone-toolchain-x86 ``` -In alternative you can also use the prebuilt toolchain, in that case make sure to set the proper CC and CXX env variables because there are many different compilers for each API version in the prebuilt toolchain. +In alternative you can also use the prebuilt toolchain, in that case make sure +to set the proper CC and CXX environment variables because there are many +different compilers for each API version in the prebuilt toolchain. For example: @@ -64,7 +66,7 @@ cd android-qbdi-sdk-x86_64/ tar xvf QBDI-0.7.0-android-X86_64.tar.gz ``` -Now set the `STANDALONE_TOOLCHAIN_PATH` to the path of standalone-toolchain +Now set the `STANDALONE_TOOLCHAIN_PATH` to the path of standalone-toolchain ``` export STANDALONE_TOOLCHAIN_PATH=/home/hac425/workspace/android-standalone-toolchain-x86_64 -- cgit 1.4.1 From b7395fa46710673602b8fb7257e502e5f129a56c Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 19:52:10 +0100 Subject: Change "AFL" to "AFL++" --- README.md | 2 +- coresight_mode/README.md | 7 +++++-- dictionaries/README.md | 2 +- docs/INSTALL.md | 6 +++--- docs/afl-fuzz_approach.md | 4 ++-- docs/custom_mutators.md | 3 ++- docs/env_variables.md | 4 ++-- docs/fuzzing_in_depth.md | 8 ++++---- testcases/README.md | 2 +- unicorn_mode/README.md | 10 +++++----- utils/argv_fuzzing/README.md | 4 ++-- utils/libdislocator/README.md | 8 ++++---- utils/libtokencap/README.md | 2 +- 13 files changed, 33 insertions(+), 29 deletions(-) (limited to 'utils') diff --git a/README.md b/README.md index dbf49b20..93c0dd10 100644 --- a/README.md +++ b/README.md @@ -99,7 +99,7 @@ Step-by-step quick start: To add a dictionary, add `-x /path/to/dictionary.txt` to afl-fuzz. If the program takes input from a file, you can put `@@` in the program's - command line; AFL will put an auto-generated file name in there for you. + command line; AFL++ will put an auto-generated file name in there for you. 4. Investigate anything shown in red in the fuzzer UI by promptly consulting [docs/afl-fuzz_approach.md#understanding-the-status-screen](docs/afl-fuzz_approach.md#understanding-the-status-screen). diff --git a/coresight_mode/README.md b/coresight_mode/README.md index cd1bccab..1a39d347 100644 --- a/coresight_mode/README.md +++ b/coresight_mode/README.md @@ -3,7 +3,7 @@ CoreSight mode enables binary-only fuzzing on ARM64 Linux using CoreSight (ARM's hardware tracing technology). NOTE: CoreSight mode is in the early development stage. Not applicable for production use. -Currently the following hardware boards are supported: +Currently the following hardware boards are supported: * NVIDIA Jetson TX2 (NVIDIA Parker) * NVIDIA Jetson Nano (NVIDIA Tegra X1) * GIGABYTE R181-T90 (Marvell ThunderX2 CN99XX) @@ -12,7 +12,10 @@ Currently the following hardware boards are supported: Please read the [RICSec/coresight-trace README](https://github.com/RICSecLab/coresight-trace/blob/master/README.md) and check the prerequisites (capstone) before getting started. -CoreSight mode supports the AFL fork server mode to reduce `exec` system call overhead. To support it for binary-only fuzzing, it needs to modify the target ELF binary to re-link to the patched glibc. We employ this design from [PTrix](https://github.com/junxzm1990/afl-pt). +CoreSight mode supports the AFL++ fork server mode to reduce `exec` system call +overhead. To support it for binary-only fuzzing, it needs to modify the target +ELF binary to re-link to the patched glibc. We employ this design from +[PTrix](https://github.com/junxzm1990/afl-pt). Check out all the git submodules in the `cs_mode` directory: diff --git a/dictionaries/README.md b/dictionaries/README.md index 7c587abb..2c0056f6 100644 --- a/dictionaries/README.md +++ b/dictionaries/README.md @@ -1,4 +1,4 @@ -# AFL dictionaries +# AFL++ dictionaries (See [../README.md](../README.md) for the general instruction manual.) diff --git a/docs/INSTALL.md b/docs/INSTALL.md index cfa20dea..ab6e735b 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -20,7 +20,7 @@ The easiest choice is to build and install everything: sudo apt-get update sudo apt-get install -y build-essential python3-dev automake git flex bison libglib2.0-dev libpixman-1-dev python3-setuptools # try to install llvm 11 and install the distro default if that fails -sudo apt-get install -y lld-11 llvm-11 llvm-11-dev clang-11 || sudo apt-get install -y lld llvm llvm-dev clang +sudo apt-get install -y lld-11 llvm-11 llvm-11-dev clang-11 || sudo apt-get install -y lld llvm llvm-dev clang sudo apt-get install -y gcc-$(gcc --version|head -n1|sed 's/.* //'|sed 's/\..*//')-plugin-dev libstdc++-$(gcc --version|head -n1|sed 's/.* //'|sed 's/\..*//')-dev sudo apt-get install -y ninja-build # for qemu_mode git clone https://github.com/AFLplusplus/AFLplusplus @@ -114,8 +114,8 @@ This means two things: - Fuzzing will be probably slower than on Linux. In fact, some folks report considerable performance gains by running the jobs inside a Linux VM on MacOS X. - - Some non-portable, platform-specific code may be incompatible with the - AFL forkserver. If you run into any problems, set `AFL_NO_FORKSRV=1` in the + - Some non-portable, platform-specific code may be incompatible with the AFL++ + forkserver. If you run into any problems, set `AFL_NO_FORKSRV=1` in the environment before starting afl-fuzz. User emulation mode of QEMU does not appear to be supported on MacOS X, so black-box instrumentation mode (`-Q`) will not work. diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index 4e8e5eaa..3e4faaec 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -348,7 +348,7 @@ That last bit is actually fairly interesting: it measures the consistency of observed traces. If a program always behaves the same for the same input data, it will earn a score of 100%. When the value is lower but still shown in purple, the fuzzing process is unlikely to be negatively affected. If it goes into red, -you may be in trouble, since AFL will have difficulty discerning between +you may be in trouble, since AFL++ will have difficulty discerning between meaningful and "phantom" effects of tweaking the input file. Now, most targets will just get a 100% score, but when you see lower figures, @@ -506,7 +506,7 @@ directory. This includes: - `edges_found` - how many edges have been found - `var_byte_count` - how many edges are non-deterministic - `afl_banner` - banner text (e.g. the target name) -- `afl_version` - the version of AFL used +- `afl_version` - the version of AFL++ used - `target_mode` - default, persistent, qemu, unicorn, non-instrumented - `command_line` - full command line used for the fuzzing session diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 2caba560..3a2ec3b2 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -21,7 +21,8 @@ fuzzing by using libraries that perform mutations according to a given grammar. The custom mutator is passed to `afl-fuzz` via the `AFL_CUSTOM_MUTATOR_LIBRARY` or `AFL_PYTHON_MODULE` environment variable, and must export a fuzz function. -Now AFL also supports multiple custom mutators which can be specified in the same `AFL_CUSTOM_MUTATOR_LIBRARY` environment variable like this. +Now AFL++ also supports multiple custom mutators which can be specified in the +same `AFL_CUSTOM_MUTATOR_LIBRARY` environment variable like this. ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="full/path/to/mutator_first.so;full/path/to/mutator_second.so" ``` diff --git a/docs/env_variables.md b/docs/env_variables.md index 6c90e84c..715a60cb 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -307,7 +307,7 @@ checks or alter some of the more exotic semantics of the tool: (`-i in`). This is an important feature to set when resuming a fuzzing session. - - Setting `AFL_CRASH_EXITCODE` sets the exit code AFL treats as crash. For + - Setting `AFL_CRASH_EXITCODE` sets the exit code AFL++ treats as crash. For example, if `AFL_CRASH_EXITCODE='-1'` is set, each input resulting in a `-1` return code (i.e. `exit(-1)` got called), will be treated as if a crash had occurred. This may be beneficial if you look for higher-level faulty @@ -493,7 +493,7 @@ checks or alter some of the more exotic semantics of the tool: This is especially useful when running multiple instances (`-M/-S` for example). Applied tags are `banner` and `afl_version`. `banner` corresponds to the name of the fuzzer provided through `-M/-S`. `afl_version` - corresponds to the currently running AFL version (e.g. `++3.0c`). Default + corresponds to the currently running AFL++ version (e.g. `++3.0c`). Default (empty/non present) will add no tags to the metrics. For more information, see [rpc_statsd.md](rpc_statsd.md). diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 8188a18e..4d2884f6 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -106,9 +106,9 @@ You can select the mode for the afl-cc compiler by: MODE can be one of: LTO (afl-clang-lto*), LLVM (afl-clang-fast*), GCC_PLUGIN (afl-g*-fast) or GCC (afl-gcc/afl-g++) or CLANG(afl-clang/afl-clang++). -Because no AFL specific command-line options are accepted (beside the --afl-MODE -command), the compile-time tools make fairly broad use of environment variables, -which can be listed with `afl-cc -hh` or by reading +Because no AFL++ specific command-line options are accepted (beside the +--afl-MODE command), the compile-time tools make fairly broad use of environment +variables, which can be listed with `afl-cc -hh` or by reading [env_variables.md](env_variables.md). ### b) Selecting instrumentation options @@ -213,7 +213,7 @@ is more effective). If the target has features that make fuzzing more difficult, e.g. checksums, HMAC, etc. then modify the source code so that checks for these values are removed. This can even be done safely for source code used in operational -products by eliminating these checks within these AFL specific blocks: +products by eliminating these checks within these AFL++ specific blocks: ``` #ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION diff --git a/testcases/README.md b/testcases/README.md index ef38d3c4..a2f74d68 100644 --- a/testcases/README.md +++ b/testcases/README.md @@ -1,4 +1,4 @@ -# AFL starting test cases +# AFL++ starting test cases (See [../README.md](../README.md) for the general instruction manual.) diff --git a/unicorn_mode/README.md b/unicorn_mode/README.md index ed85e687..4c95e8f3 100644 --- a/unicorn_mode/README.md +++ b/unicorn_mode/README.md @@ -8,7 +8,8 @@ The CompareCoverage and NeverZero counters features are by Andrea Fioraldi Date: Thu, 2 Dec 2021 20:14:54 +0100 Subject: Fix spelling of words related to "build" --- docs/custom_mutators.md | 2 +- instrumentation/README.lto.md | 2 +- utils/autodict_ql/readme.md | 6 +++--- utils/qbdi_mode/README.md | 6 +++--- 4 files changed, 8 insertions(+), 8 deletions(-) (limited to 'utils') diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 3a2ec3b2..7d362950 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -218,7 +218,7 @@ trimmed input. Here's a quick API description: to the maximum amount of steps you returned in `init_trim`). Omitting any of three trimming methods will cause the trimming to be disabled -and trigger a fallback to the builtin default trimming routine. +and trigger a fallback to the built-in default trimming routine. ### Environment Variables diff --git a/instrumentation/README.lto.md b/instrumentation/README.lto.md index 24e57b23..b97e5799 100644 --- a/instrumentation/README.lto.md +++ b/instrumentation/README.lto.md @@ -328,7 +328,7 @@ and `CXXFLAGS=-flto=full`). If this succeeds, then there is an issue with afl-clang-lto. Please report at [https://github.com/AFLplusplus/AFLplusplus/issues/226](https://github.com/AFLplusplus/AFLplusplus/issues/226). -Even some targets where clang-12 fails can be build if the fail is just in +Even some targets where clang-12 fails can be built if the fail is just in `./configure`, see `Solving difficult targets` above. ## History diff --git a/utils/autodict_ql/readme.md b/utils/autodict_ql/readme.md index 9170f552..a28f1725 100644 --- a/utils/autodict_ql/readme.md +++ b/utils/autodict_ql/readme.md @@ -43,7 +43,7 @@ Do the following : # chmod +x codeql-build.sh # ./codeql-build.sh # source ~/.bashrc -# codeql +# codeql ``` Then you should get: @@ -84,12 +84,12 @@ Commands: - `python3 autodict-ql.py [CURRECT_DIR] [CODEQL_DATABASE_PATH] [TOKEN_PATH]` - example : `python3 /home/user/AFLplusplus/utils/autodict_ql/autodict-ql.py $PWD /home/user/libxml/libxml-db tokens` - This will create the final `tokens` dir for you and you are done, then pass the tokens path to AFL++'s `-x` flag. -5. Done! +5. Done! ## More on dictionaries and tokens Core developer of the AFL++ project Marc Heuse also developed a similar tool named `dict2file` which is a LLVM pass which can automatically extract useful tokens, in addition with LTO instrumentation mode, this dict2file is automatically generates token extraction. `Autodict-QL` plugin gives you scripting capability and you can do whatever you want to extract from the Codebase and it's up to you. In addition it's independent from LLVM system. -On the other hand, you can also use Google dictionaries which have been made public in May 2020, but the problem of using Google dictionaries is that they are limited to specific file formats and specifications. For example, for testing binutils and ELF file format or AVI in FFMPEG, there are no prebuilt dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` features to automatically generate dictionaries based on the target. +On the other hand, you can also use Google dictionaries which have been made public in May 2020, but the problem of using Google dictionaries is that they are limited to specific file formats and specifications. For example, for testing binutils and ELF file format or AVI in FFMPEG, there are no pre-built dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` features to automatically generate dictionaries based on the target. I've personally prefered to use `Autodict-QL` or `dict2file` rather than Google dictionaries or any other manually generated dictionaries as `Autodict-QL` and `dict2file` are working based on the target. In overall, fuzzing with dictionaries and well-generated tokens will give better results. diff --git a/utils/qbdi_mode/README.md b/utils/qbdi_mode/README.md index 2e8b01d4..cd59fb9c 100755 --- a/utils/qbdi_mode/README.md +++ b/utils/qbdi_mode/README.md @@ -34,9 +34,9 @@ For x86 standalone-toolchain ./build/tools/make_standalone_toolchain.py --arch x86 --api 21 --install-dir ../android-standalone-toolchain-x86 ``` -In alternative you can also use the prebuilt toolchain, in that case make sure +In alternative you can also use the pre-built toolchain, in that case make sure to set the proper CC and CXX environment variables because there are many -different compilers for each API version in the prebuilt toolchain. +different compilers for each API version in the pre-built toolchain. For example: @@ -129,7 +129,7 @@ int target_func(char *buf, int size) { } ``` -This could be build to `libdemo.so`. +This could be built to `libdemo.so`. Then we should load the library in template.cpp and find the `target` function address. ```c -- cgit 1.4.1 From c85e0dc4f0e0a6deedfb7318292e1939503a10c2 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 20:37:21 +0100 Subject: Remove the word "we" --- TODO.md | 2 +- docs/afl-fuzz_approach.md | 27 +++++++++++++-------------- docs/custom_mutators.md | 4 ++-- docs/fuzzing_binary-only_targets.md | 8 ++++---- docs/fuzzing_in_depth.md | 2 +- frida_mode/DEBUGGING.md | 2 +- instrumentation/README.instrument_list.md | 2 +- unicorn_mode/samples/c/COMPILE.md | 2 +- unicorn_mode/samples/persistent/COMPILE.md | 4 ++-- unicorn_mode/samples/speedtest/README.md | 2 +- utils/aflpp_driver/README.md | 2 +- utils/autodict_ql/readme.md | 2 +- utils/qbdi_mode/README.md | 5 +++-- 13 files changed, 32 insertions(+), 32 deletions(-) (limited to 'utils') diff --git a/TODO.md b/TODO.md index 77fb080f..b8ac22ef 100644 --- a/TODO.md +++ b/TODO.md @@ -20,7 +20,7 @@ qemu_mode/frida_mode: - non colliding instrumentation - rename qemu specific envs to AFL_QEMU (AFL_ENTRYPOINT, AFL_CODE_START/END, AFL_COMPCOV_LEVEL?) - - add AFL_QEMU_EXITPOINT (maybe multiple?), maybe pointless as we have + - add AFL_QEMU_EXITPOINT (maybe multiple?), maybe pointless as there is persistent mode diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index 3e4faaec..242104f7 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -103,8 +103,8 @@ will be allowed to run for months. There's one important thing to watch out for: if the tool is not finding new paths within several minutes of starting, you're probably not invoking the -target binary correctly and it never gets to parse the input files we're -throwing at it; other possible explanations are that the default memory limit +target binary correctly and it never gets to parse the input files that are +thrown at it; other possible explanations are that the default memory limit (`-m`) is too restrictive and the program exits after failing to allocate a buffer very early on; or that the input files are patently invalid and always fail a basic header check. @@ -172,10 +172,9 @@ processed path is not "favored" (a property discussed later on). The section provides some trivia about the coverage observed by the instrumentation embedded in the target binary. -The first line in the box tells you how many branch tuples we have already hit, -in proportion to how much the bitmap can hold. The number on the left describes -the current input; the one on the right is the value for the entire input -corpus. +The first line in the box tells you how many branch tuples already were hit, in +proportion to how much the bitmap can hold. The number on the left describes the +current input; the one on the right is the value for the entire input corpus. Be wary of extremes: @@ -194,7 +193,7 @@ Be wary of extremes: The other line deals with the variability in tuple hit counts seen in the binary. In essence, if every taken branch is always taken a fixed number of -times for all the inputs we have tried, this will read `1.00`. As we manage to +times for all the inputs that were tried, this will read `1.00`. As we manage to trigger other hit counts for every branch, the needle will start to move toward `8.00` (every bit in the 8-bit map hit), but will probably never reach that extreme. @@ -295,9 +294,9 @@ exceed it by a margin sufficient to be classified as hangs. +-----------------------------------------------------+ ``` -This is just another nerd-targeted section keeping track of how many paths we -have netted, in proportion to the number of execs attempted, for each of the -fuzzing strategies discussed earlier on. This serves to convincingly validate +This is just another nerd-targeted section keeping track of how many paths were +netted, in proportion to the number of execs attempted, for each of the fuzzing +strategies discussed earlier on. This serves to convincingly validate assumptions about the usefulness of the various approaches taken by afl-fuzz. The trim strategy stats in this section are a bit different than the rest. The @@ -339,10 +338,10 @@ fuzzing yet. The same stat is also given for "favored" entries that the fuzzer really wants to get to in this queue cycle (the non-favored entries may have to wait a couple of cycles to get their chance). -Next, we have the number of new paths found during this fuzzing section and -imported from other fuzzer instances when doing parallelized fuzzing; and the -extent to which identical inputs appear to sometimes produce variable behavior -in the tested binary. +Next is the number of new paths found during this fuzzing section and imported +from other fuzzer instances when doing parallelized fuzzing; and the extent to +which identical inputs appear to sometimes produce variable behavior in the +tested binary. That last bit is actually fairly interesting: it measures the consistency of observed traces. If a program always behaves the same for the same input data, diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 7d362950..4018d633 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -204,8 +204,8 @@ trimmed input. Here's a quick API description: - `trim` (optional) This method is called for each trimming operation. It doesn't have any - arguments because we already have the initial buffer from `init_trim` and we - can memorize the current state in the data variables. This can also save + arguments because there is already the initial buffer from `init_trim` and + we can memorize the current state in the data variables. This can also save reparsing steps for each iteration. It should return the trimmed input buffer. diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index 2d57d0dc..c3204212 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -201,10 +201,10 @@ target at load time and then let it run - or save the binary with the changes. This is great for some things, e.g. fuzzing, and not so effective for others, e.g. malware analysis. -So, what we can do with Dyninst is taking every basic block and put AFL++'s -instrumentation code in there - and then save the binary. Afterwards, we can -just fuzz the newly saved target binary with afl-fuzz. Sounds great? It is. The -issue though - it is a non-trivial problem to insert instructions, which change +So, what you can do with Dyninst is taking every basic block and putting AFL++'s +instrumentation code in there - and then save the binary. Afterwards, just fuzz +the newly saved target binary with afl-fuzz. Sounds great? It is. The issue +though - it is a non-trivial problem to insert instructions, which change addresses in the process space, so that everything is still working afterwards. Hence, more often than not binaries crash when they are run. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 4d2884f6..92b3cf86 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -391,7 +391,7 @@ to be used in fuzzing! :-) ## 3. Fuzzing the target -In this final step we fuzz the target. There are not that many important options +In this final step, fuzz the target. There are not that many important options to run the target - unless you want to use many CPU cores/threads for the fuzzing, which will make the fuzzing much more useful. diff --git a/frida_mode/DEBUGGING.md b/frida_mode/DEBUGGING.md index b703ae43..207a48bf 100644 --- a/frida_mode/DEBUGGING.md +++ b/frida_mode/DEBUGGING.md @@ -95,7 +95,7 @@ gdb \ ``` Note: -- We have to manually set the `__AFL_PERSISTENT` environment variable which is +- You have to manually set the `__AFL_PERSISTENT` environment variable which is usually passed by `afl-fuzz`. - Setting breakpoints etc. is likely to interfere with FRIDA and cause spurious errors. diff --git a/instrumentation/README.instrument_list.md b/instrumentation/README.instrument_list.md index b412b600..3ed64807 100644 --- a/instrumentation/README.instrument_list.md +++ b/instrumentation/README.instrument_list.md @@ -128,4 +128,4 @@ Note that whitespace is ignored and comments (`# foo`) are supported. ### 3b) UNIX-style pattern matching You can add UNIX-style pattern matching in the "instrument file list" entries. -See `man fnmatch` for the syntax. We do not set any of the `fnmatch` flags. \ No newline at end of file +See `man fnmatch` for the syntax. Do not set any of the `fnmatch` flags. \ No newline at end of file diff --git a/unicorn_mode/samples/c/COMPILE.md b/unicorn_mode/samples/c/COMPILE.md index 7da140f7..4e3cf568 100644 --- a/unicorn_mode/samples/c/COMPILE.md +++ b/unicorn_mode/samples/c/COMPILE.md @@ -19,4 +19,4 @@ was built in case you want to rebuild it or recompile it for any reason. The pre-built binary (persistent_target_x86_64) was built using -g -O0 in gcc. -We then load the binary and execute the main function directly. +Then load the binary and execute the main function directly. diff --git a/unicorn_mode/samples/persistent/COMPILE.md b/unicorn_mode/samples/persistent/COMPILE.md index 9f2ae718..5e607aef 100644 --- a/unicorn_mode/samples/persistent/COMPILE.md +++ b/unicorn_mode/samples/persistent/COMPILE.md @@ -3,7 +3,7 @@ This shows a simple persistent harness for unicornafl in C. In contrast to the normal c harness, this harness manually resets the unicorn state on each new input. -Thanks to this, we can rerun the test case in unicorn multiple times, without +Thanks to this, you can rerun the test case in unicorn multiple times, without the need to fork again. ## Compiling sample.c @@ -25,4 +25,4 @@ was built in case you want to rebuild it or recompile it for any reason. The pre-built binary (persistent_target_x86_64.bin) was built using -g -O0 in gcc. -We then load the binary and we execute the main function directly. \ No newline at end of file +Then load the binary and execute the main function directly. \ No newline at end of file diff --git a/unicorn_mode/samples/speedtest/README.md b/unicorn_mode/samples/speedtest/README.md index 3c1184a2..496d75cd 100644 --- a/unicorn_mode/samples/speedtest/README.md +++ b/unicorn_mode/samples/speedtest/README.md @@ -44,7 +44,7 @@ was built in case you want to rebuild it or recompile it for any reason. The pre-built binary (simple_target_x86_64.bin) was built using -g -O0 in gcc. -We then load the binary and execute the main function directly. +Then load the binary and execute the main function directly. ## Addresses for the harness: To find the address (in hex) of main, run: diff --git a/utils/aflpp_driver/README.md b/utils/aflpp_driver/README.md index 4560be2b..d534cd7f 100644 --- a/utils/aflpp_driver/README.md +++ b/utils/aflpp_driver/README.md @@ -25,7 +25,7 @@ or `@@` as command line parameters. Note that you can use the driver too for frida_mode (`-O`). aflpp_qemu_driver is used for libfuzzer `LLVMFuzzerTestOneInput()` targets that -are to be fuzzed in qemu_mode. So we compile them with clang/clang++, without +are to be fuzzed in qemu_mode. So compile them with clang/clang++, without -fsantize=fuzzer or afl-clang-fast, and link in libAFLQemuDriver.a: `clang++ -o fuzz fuzzer_harness.cc libAFLQemuDriver.a [plus required linking]`. diff --git a/utils/autodict_ql/readme.md b/utils/autodict_ql/readme.md index a28f1725..491ec85b 100644 --- a/utils/autodict_ql/readme.md +++ b/utils/autodict_ql/readme.md @@ -37,7 +37,7 @@ sudo apt install build-essential libtool-bin python3-dev python3 automake git vi ``` The usage of Autodict-QL is pretty easy. But let's describe it as: -1. First of all, you need to have CodeQL installed on the system. we make this possible with `build-codeql.sh` bash script. This script will install CodeQL completety and will set the required environment variables for your system. +1. First of all, you need to have CodeQL installed on the system. We make this possible with `build-codeql.sh` bash script. This script will install CodeQL completety and will set the required environment variables for your system. Do the following : ```shell # chmod +x codeql-build.sh diff --git a/utils/qbdi_mode/README.md b/utils/qbdi_mode/README.md index cd59fb9c..c8d46fca 100755 --- a/utils/qbdi_mode/README.md +++ b/utils/qbdi_mode/README.md @@ -131,7 +131,8 @@ int target_func(char *buf, int size) { This could be built to `libdemo.so`. -Then we should load the library in template.cpp and find the `target` function address. +Then load the library in template.cpp and find the `target` function address: + ```c void *handle = dlopen(lib_path, RTLD_LAZY); .......................................... @@ -140,7 +141,7 @@ Then we should load the library in template.cpp and find the `target` function a p_target_func = (target_func)dlsym(handle, "target_func"); ``` -then we read the data from file and call the function in `fuzz_func` +Then read the data from file and call the function in `fuzz_func`: ```c QBDI_NOINLINE int fuzz_func() { -- cgit 1.4.1 From a7694e299a331bd8c4826b2402ee68cd6f83d8f9 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 20:45:48 +0100 Subject: Fix punctuation in connection with "however" --- docs/FAQ.md | 10 +++++++--- docs/best_practices.md | 27 ++++++++++++++++++--------- docs/custom_mutators.md | 10 +++++----- docs/fuzzing_in_depth.md | 10 +++++----- docs/important_changes.md | 4 ++-- instrumentation/README.llvm.md | 11 ++++++----- qemu_mode/README.persistent.md | 8 ++++---- qemu_mode/libcompcov/README.md | 8 ++++---- utils/afl_network_proxy/README.md | 13 +++++++------ utils/afl_untracer/README.md | 6 +++--- 10 files changed, 61 insertions(+), 46 deletions(-) (limited to 'utils') diff --git a/docs/FAQ.md b/docs/FAQ.md index ae4a77dc..49444999 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -12,7 +12,9 @@ If you find an interesting or important question missing, submit it via American Fuzzy Lop (AFL) was developed by MichaƂ "lcamtuf" Zalewski starting in 2013/2014, and when he left Google end of 2017 he stopped developing it. - At the end of 2019, the Google fuzzing team took over maintenance of AFL, however it is only accepting PRs from the community and is not developing enhancements anymore. + At the end of 2019, the Google fuzzing team took over maintenance of AFL, + however, it is only accepting PRs from the community and is not developing + enhancements anymore. In the second quarter of 2019, 1 1/2 years later, when no further development of AFL had happened and it became clear there would none be coming, AFL++ was born, where initially community patches were collected and applied for bug fixes and enhancements. Then from various AFL spin-offs - mostly academic research - features were integrated. @@ -121,8 +123,10 @@ If you find an interesting or important question missing, submit it via Sending the same input again and again should take the exact same path through the target every time. If that is the case, the stability is 100%. - If however randomness happens, e.g. a thread reading other external data, reaction to timing, etc., then in some of the re-executions with the same data the edge coverage result will be different accross runs. - Those edges that change are then flagged "unstable". + If, however, randomness happens, e.g. a thread reading other external data, + reaction to timing, etc., then in some of the re-executions with the same data + the edge coverage result will be different accross runs. Those edges that + change are then flagged "unstable". The more "unstable" edges, the more difficult for AFL++ to identify valid new paths. diff --git a/docs/best_practices.md b/docs/best_practices.md index 979849f4..15f8870c 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -54,9 +54,11 @@ to emulate the network. This is also much faster than the real network would be. See [utils/socket_fuzzing/](../utils/socket_fuzzing/). There is an outdated AFL++ branch that implements networking if you are -desperate though: [https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) - -however a better option is AFLnet ([https://github.com/aflnet/aflnet](https://github.com/aflnet/aflnet)) -which allows you to define network state with different type of data packets. +desperate though: +[https://github.com/AFLplusplus/AFLplusplus/tree/networking](https://github.com/AFLplusplus/AFLplusplus/tree/networking) +- however, a better option is AFLnet +([https://github.com/aflnet/aflnet](https://github.com/aflnet/aflnet)) which +allows you to define network state with different type of data packets. ## Improvements @@ -72,13 +74,16 @@ which allows you to define network state with different type of data packets. ### Improving stability -For fuzzing a 100% stable target that covers all edges is the best case. -A 90% stable target that covers all edges is however better than a 100% stable target that ignores 10% of the edges. +For fuzzing a 100% stable target that covers all edges is the best case. A 90% +stable target that covers all edges is, however, better than a 100% stable +target that ignores 10% of the edges. With instability, you basically have a partial coverage loss on an edge, with ignored functions you have a full loss on that edges. -There are functions that are unstable, but also provide value to coverage, e.g., init functions that use fuzz data as input. -If however a function that has nothing to do with the input data is the source of instability, e.g., checking jitter, or is a hash map function etc., then it should not be instrumented. +There are functions that are unstable, but also provide value to coverage, e.g., +init functions that use fuzz data as input. If, however, a function that has +nothing to do with the input data is the source of instability, e.g., checking +jitter, or is a hash map function etc., then it should not be instrumented. To be able to exclude these functions (based on AFL++'s measured stability), the following process will allow to identify functions with variable edges. @@ -116,8 +121,12 @@ Four steps are required to do this and it also requires quite some knowledge of If `PCGUARD` is used, then you need to follow this guide (needs llvm 12+!): [https://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation](https://clang.llvm.org/docs/SanitizerCoverage.html#partially-disabling-instrumentation) - Only exclude those functions from instrumentation that provide no value for coverage - that is if it does not process any fuzz data directly or indirectly (e.g. hash maps, thread management etc.). - If however a function directly or indirectly handles fuzz data, then you should not put the function in a deny instrumentation list and rather live with the instability it comes with. + Only exclude those functions from instrumentation that provide no value for + coverage - that is if it does not process any fuzz data directly or + indirectly (e.g. hash maps, thread management etc.). If, however, a + function directly or indirectly handles fuzz data, then you should not put + the function in a deny instrumentation list and rather live with the + instability it comes with. 4. Recompile the target diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 4018d633..fc5ecbf9 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -112,11 +112,11 @@ def deinit(): # optional for Python - `fuzz_count` (optional): - When a queue entry is selected to be fuzzed, afl-fuzz selects the number - of fuzzing attempts with this input based on a few factors. - If however the custom mutator wants to set this number instead on how often - it is called for a specific queue entry, use this function. - This function is most useful if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. + When a queue entry is selected to be fuzzed, afl-fuzz selects the number of + fuzzing attempts with this input based on a few factors. If, however, the + custom mutator wants to set this number instead on how often it is called + for a specific queue entry, use this function. This function is most useful + if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. - `fuzz` (optional): diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 92b3cf86..96e709ab 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -131,8 +131,8 @@ The following options are available when you instrument with LTO mode have to compile the target twice, once specifically with/for this mode by setting `AFL_LLVM_CMPLOG=1`, and pass this binary to afl-fuzz via the `-c` parameter. Note that you can compile also just a cmplog binary and use that - for both however there will be a performance penality. You can read more about - this in + for both, however, there will be a performance penality. You can read more + about this in [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md). If you use LTO, LLVM or GCC_PLUGIN mode @@ -151,7 +151,7 @@ only instrument parts of the target that you are interested in: inlined and then would not match! See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) -There are many more options and modes available however these are most of the +There are many more options and modes available, however, these are most of the time less effective. See: * [instrumentation/README.ctx.md](../instrumentation/README.ctx.md) * [instrumentation/README.ngram.md](../instrumentation/README.ngram.md) @@ -369,8 +369,8 @@ This step is highly recommended! ### c) Minimizing all corpus files The shorter the input files that still traverse the same path within the target, -the better the fuzzing will be. This minimization is done with `afl-tmin` -however it is a long process as this has to be done for every file: +the better the fuzzing will be. This minimization is done with `afl-tmin`, +however, it is a long process as this has to be done for every file: ``` mkdir input diff --git a/docs/important_changes.md b/docs/important_changes.md index 726de64d..6cd00791 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -15,8 +15,8 @@ With AFL++ 3.15 we introduced the following changes from previous behaviors: With AFL++ 3.14 we introduced the following changes from previous behaviors: * afl-fuzz: deterministic fuzzing it not a default for -M main anymore - * afl-cmin/afl-showmap -i now descends into subdirectories (afl-cmin.bash - however does not) + * afl-cmin/afl-showmap -i now descends into subdirectories (afl-cmin.bash, + however, does not) With AFL++ 3.10 we introduced the following changes from previous behaviors: * The '+' feature of the '-t' option now means to auto-calculate the timeout diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index 8133cbe4..d16049fa 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -106,9 +106,10 @@ either setting `AFL_CC_COMPILER=LLVM` or pass the parameter `--afl-llvm` via CFLAGS/CXXFLAGS/CPPFLAGS. The tool honors roughly the same environmental variables as afl-gcc (see -[docs/env_variables.md](../docs/env_variables.md)). This includes AFL_USE_ASAN, -AFL_HARDEN, and AFL_DONT_OPTIMIZE. However AFL_INST_RATIO is not honored as it -does not serve a good purpose with the more effective PCGUARD analysis. +[docs/env_variables.md](../docs/env_variables.md)). This includes +`AFL_USE_ASAN`, `AFL_HARDEN`, and `AFL_DONT_OPTIMIZE`. However, `AFL_INST_RATIO` +is not honored as it does not serve a good purpose with the more effective +PCGUARD analysis. ## 3) Options @@ -125,8 +126,8 @@ For splitting memcmp, strncmp, etc., see Then there are different ways of instrumenting the target: 1. An better instrumentation strategy uses LTO and link time instrumentation. - Note that not all targets can compile in this mode, however if it works it is - the best option you can use. To go with this option, use + Note that not all targets can compile in this mode, however, if it works it + is the best option you can use. To go with this option, use afl-clang-lto/afl-clang-lto++. See [README.lto.md](README.lto.md). 2. Alternatively you can choose a completely different coverage method: diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md index 7210a8cc..ab45860d 100644 --- a/qemu_mode/README.persistent.md +++ b/qemu_mode/README.persistent.md @@ -17,10 +17,10 @@ The start of the persistent loop has to be set with environment variable `AFL_QEMU_PERSISTENT_ADDR`. This address can be the address of whatever instruction. Setting this address to -the start of a function makes the usage simple. If the address is however within -a function, either RET, OFFSET, or EXITS (see below in 2.2, 2.3, 2.6) have to be -set. This address (as well as the RET address, see below) has to be defined in -hexadecimal with the 0x prefix or as a decimal value. +the start of a function makes the usage simple. If the address is, however, +within a function, either RET, OFFSET, or EXITS (see below in 2.2, 2.3, 2.6) +have to be set. This address (as well as the RET address, see below) has to be +defined in hexadecimal with the 0x prefix or as a decimal value. If both RET and EXITS are not set, QEMU will assume that START points to a function and will patch the return address (on stack or in the link register) to diff --git a/qemu_mode/libcompcov/README.md b/qemu_mode/libcompcov/README.md index 6a72f5ff..50f0d802 100644 --- a/qemu_mode/libcompcov/README.md +++ b/qemu_mode/libcompcov/README.md @@ -18,7 +18,7 @@ and this module is not capable to log the coverage in this case. If you have the source code of the fuzzing target you should nto use this library and QEMU but build it with afl-clang-fast and the laf-intel options. -To use this library make sure to preload it with AFL_PRELOAD. +To use this library, make sure to preload it with AFL_PRELOAD. ``` export AFL_PRELOAD=/path/to/libcompcov.so @@ -32,6 +32,6 @@ Level 1 logs just comparison with immediates / read-only memory and level 2 logs all the comparisons. The library make use of https://github.com/ouadev/proc_maps_parser and so it is -Linux specific. However this is not a strict dependency, other UNIX operating -systems can be supported by replacing the code related to the -/proc/self/maps parsing. \ No newline at end of file +Linux specific. However, this is not a strict dependency, other UNIX operating +systems can be supported by replacing the code related to the /proc/self/maps +parsing. \ No newline at end of file diff --git a/utils/afl_network_proxy/README.md b/utils/afl_network_proxy/README.md index 05659c45..d2c00be2 100644 --- a/utils/afl_network_proxy/README.md +++ b/utils/afl_network_proxy/README.md @@ -34,16 +34,17 @@ afl-network-server -i 1111 -m 25M -t 1000 -- /bin/target -f @@ ### on the (afl-fuzz) main node -Just run afl-fuzz with your normal options, however the target should be +Just run afl-fuzz with your normal options, however, the target should be `afl-network-client` with the IP and PORT of the `afl-network-server` and increase the -t value: + ``` afl-fuzz -i in -o out -t 2000+ -- afl-network-client TARGET-IP 1111 ``` -Note the '+' on the -t parameter value. The afl-network-server will take -care of proper timeouts hence afl-fuzz should not. The '+' increases the -timeout and the value itself should be 500-1000 higher than the one on -afl-network-server. + +Note the '+' on the -t parameter value. The afl-network-server will take care of +proper timeouts hence afl-fuzz should not. The '+' increases the timeout and the +value itself should be 500-1000 higher than the one on afl-network-server. ### networking @@ -53,7 +54,7 @@ either. Note that also the outgoing interface can be specified with a '%' for Also make sure your default TCP window size is larger than your MAP_SIZE (130kb is a good value). -On Linux that is the middle value of `/proc/sys/net/ipv4/tcp_rmem` +On Linux that is the middle value of `/proc/sys/net/ipv4/tcp_rmem` ## how to compile and install diff --git a/utils/afl_untracer/README.md b/utils/afl_untracer/README.md index ada0c916..9f41618f 100644 --- a/utils/afl_untracer/README.md +++ b/utils/afl_untracer/README.md @@ -5,9 +5,9 @@ afl-untracer is an example skeleton file which can easily be used to fuzz a closed source library. -It requires less memory and is x3-5 faster than qemu_mode however it is way -more course grained and does not provide interesting features like compcov -or cmplog. +It requires less memory and is x3-5 faster than qemu_mode, however, it is way +more course grained and does not provide interesting features like compcov or +cmplog. Supported is so far Intel (i386/x86_64) and AARCH64. -- cgit 1.4.1 From 65c3db86256b3907404623fe1c52e01c9d12ff97 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 21:03:59 +0100 Subject: Fix punctuation in connection with "e.g." --- .github/ISSUE_TEMPLATE/bug_report.md | 5 +++-- CONTRIBUTING.md | 2 +- docs/FAQ.md | 4 ++-- docs/INSTALL.md | 3 ++- docs/afl-fuzz_approach.md | 2 +- docs/best_practices.md | 4 ++-- docs/custom_mutators.md | 7 ++++--- docs/env_variables.md | 22 +++++++++++----------- docs/fuzzing_binary-only_targets.md | 6 +++--- docs/fuzzing_in_depth.md | 32 ++++++++++++++++---------------- docs/ideas.md | 2 +- docs/important_changes.md | 2 +- instrumentation/README.llvm.md | 2 +- instrumentation/README.lto.md | 2 +- utils/README.md | 2 +- utils/afl_network_proxy/README.md | 6 ++++-- 16 files changed, 54 insertions(+), 49 deletions(-) (limited to 'utils') diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 31152cd2..0d80f4a3 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -8,8 +8,9 @@ assignees: '' --- **IMPORTANT** -1. You have verified that the issue to be present in the current `dev` branch -2. Please supply the command line options and relevant environment variables, e.g. a copy-paste of the contents of `out/default/fuzzer_setup` +1. You have verified that the issue to be present in the current `dev` branch. +2. Please supply the command line options and relevant environment variables, + e.g., a copy-paste of the contents of `out/default/fuzzer_setup`. Thank you for making AFL++ better! diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0268b2e5..0ab4f8ec 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,7 +17,7 @@ project, or added a file in a directory we already format, otherwise run: Regarding the coding style, please follow the AFL style. No camel case at all and use AFL's macros wherever possible -(e.g. WARNF, FATAL, MAP_SIZE, ...). +(e.g., WARNF, FATAL, MAP_SIZE, ...). Remember that AFL++ has to build and run on many platforms, so generalize your Makefiles/GNUmakefile (or your patches to our pre-existing diff --git a/docs/FAQ.md b/docs/FAQ.md index 49444999..27250415 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -21,7 +21,7 @@ If you find an interesting or important question missing, submit it via This already resulted in a much advanced AFL. Until the end of 2019, the AFL++ team had grown to four active developers which then implemented their own research and features, making it now by far the most flexible and feature rich guided fuzzer available as open source. - And in independent fuzzing benchmarks it is one of the best fuzzers available, e.g. [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html). + And in independent fuzzing benchmarks it is one of the best fuzzers available, e.g., [Fuzzbench Report](https://www.fuzzbench.com/reports/2020-08-03/index.html).

@@ -123,7 +123,7 @@ If you find an interesting or important question missing, submit it via Sending the same input again and again should take the exact same path through the target every time. If that is the case, the stability is 100%. - If, however, randomness happens, e.g. a thread reading other external data, + If, however, randomness happens, e.g., a thread reading other external data, reaction to timing, etc., then in some of the re-executions with the same data the edge coverage result will be different accross runs. Those edges that change are then flagged "unstable". diff --git a/docs/INSTALL.md b/docs/INSTALL.md index ab6e735b..c1e22e36 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -69,7 +69,8 @@ These build options exist: * NO_PYTHON - disable python support * NO_SPLICING - disables splicing mutation in afl-fuzz, not recommended for normal fuzzing * AFL_NO_X86 - if compiling on non-intel/amd platforms -* LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config (e.g. Debian) +* LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config + (e.g., Debian) e.g.: `make ASAN_BUILD=1` diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index 242104f7..68f45891 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -504,7 +504,7 @@ directory. This includes: - `peak_rss_mb` - max rss usage reached during fuzzing in MB - `edges_found` - how many edges have been found - `var_byte_count` - how many edges are non-deterministic -- `afl_banner` - banner text (e.g. the target name) +- `afl_banner` - banner text (e.g., the target name) - `afl_version` - the version of AFL++ used - `target_mode` - default, persistent, qemu, unicorn, non-instrumented - `command_line` - full command line used for the fuzzing session diff --git a/docs/best_practices.md b/docs/best_practices.md index 15f8870c..6a406bde 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -48,7 +48,7 @@ this with persistent mode [instrumentation/README.persistent_mode.md](../instrum and you have a performance gain of x10 instead of a performance loss of over x10 - that is a x100 difference!). -If modifying the source is not an option (e.g. because you only have a binary +If modifying the source is not an option (e.g., because you only have a binary and perform binary fuzzing) you can also use a shared library with AFL_PRELOAD to emulate the network. This is also much faster than the real network would be. See [utils/socket_fuzzing/](../utils/socket_fuzzing/). @@ -123,7 +123,7 @@ Four steps are required to do this and it also requires quite some knowledge of Only exclude those functions from instrumentation that provide no value for coverage - that is if it does not process any fuzz data directly or - indirectly (e.g. hash maps, thread management etc.). If, however, a + indirectly (e.g., hash maps, thread management etc.). If, however, a function directly or indirectly handles fuzz data, then you should not put the function in a deny instrumentation list and rather live with the instability it comes with. diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index fc5ecbf9..6bee5413 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -124,7 +124,7 @@ def deinit(): # optional for Python additional test case. Note that this function is optional - but it makes sense to use it. You would only skip this if `post_process` is used to fix checksums etc. - so if you are using it e.g. as a post processing library. + so if you are using it, e.g., as a post processing library. Note that a length > 0 *must* be returned! - `describe` (optional): @@ -191,8 +191,9 @@ trimmed input. Here's a quick API description: This method is called at the start of each trimming operation and receives the initial buffer. It should return the amount of iteration steps possible - on this input (e.g. if your input has n elements and you want to remove them - one by one, return n, if you do a binary search, return log(n), and so on). + on this input (e.g., if your input has n elements and you want to remove + them one by one, return n, if you do a binary search, return log(n), and so + on). If your trimming algorithm doesn't allow to determine the amount of (remaining) steps easily (esp. while running), then you can alternatively diff --git a/docs/env_variables.md b/docs/env_variables.md index 715a60cb..771bf157 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -80,9 +80,9 @@ fairly broad use of environment variables instead: Setting `AFL_INST_RATIO` to 0 is a valid choice. This will instrument only the transitions between function entry points, but not individual branches. - Note that this is an outdated variable. A few instances (e.g. afl-gcc) still - support these, but state-of-the-art (e.g. LLVM LTO and LLVM PCGUARD) do not - need this. + Note that this is an outdated variable. A few instances (e.g., afl-gcc) + still support these, but state-of-the-art (e.g., LLVM LTO and LLVM PCGUARD) + do not need this. - `AFL_NO_BUILTIN` causes the compiler to generate code suitable for use with libtokencap.so (but perhaps running a bit slower than without the flag). @@ -319,7 +319,7 @@ checks or alter some of the more exotic semantics of the tool: afl-fuzz), setting `AFL_PYTHON_MODULE` to a Python module can also provide additional mutations. If `AFL_CUSTOM_MUTATOR_ONLY` is also set, all mutations will solely be performed with the custom mutator. This feature - allows to configure custom mutators which can be very helpful, e.g. fuzzing + allows to configure custom mutators which can be very helpful, e.g., fuzzing XML or other highly flexible structured input. For details, see [custom_mutators.md](custom_mutators.md). @@ -449,7 +449,7 @@ checks or alter some of the more exotic semantics of the tool: not crash the target again when the test case is given. To be able to still re-trigger these crashes, you can use the `AFL_PERSISTENT_RECORD` variable with a value of how many previous fuzz cases to keep prio a crash. If set to - e.g. 10, then the 9 previous inputs are written to out/default/crashes as + e.g., 10, then the 9 previous inputs are written to out/default/crashes as RECORD:000000,cnt:000000 to RECORD:000000,cnt:000008 and RECORD:000000,cnt:000009 being the crash case. NOTE: This option needs to be enabled in config.h first! @@ -493,7 +493,7 @@ checks or alter some of the more exotic semantics of the tool: This is especially useful when running multiple instances (`-M/-S` for example). Applied tags are `banner` and `afl_version`. `banner` corresponds to the name of the fuzzer provided through `-M/-S`. `afl_version` - corresponds to the currently running AFL++ version (e.g. `++3.0c`). Default + corresponds to the currently running AFL++ version (e.g., `++3.0c`). Default (empty/non present) will add no tags to the metrics. For more information, see [rpc_statsd.md](rpc_statsd.md). @@ -535,11 +535,11 @@ The QEMU wrapper used to instrument binary-only code supports several settings: - `AFL_DEBUG` will print the found entry point for the binary to stderr. Use this if you are unsure if the entry point might be wrong - but use it - directly, e.g. `afl-qemu-trace ./program`. + directly, e.g., `afl-qemu-trace ./program`. - `AFL_ENTRYPOINT` allows you to specify a specific entry point into the binary (this can be very good for the performance!). The entry point is - specified as hex address, e.g. `0x4004110`. Note that the address must be + specified as hex address, e.g., `0x4004110`. Note that the address must be the address of a basic block. - Setting `AFL_INST_LIBS` causes the translator to also instrument the code @@ -595,7 +595,7 @@ QEMU driver to provide a `main` loop for a user provided `stdin` rather than using in-memory test cases. * `AFL_FRIDA_EXCLUDE_RANGES` - See `AFL_QEMU_EXCLUDE_RANGES` * `AFL_FRIDA_INST_COVERAGE_FILE` - File to write DynamoRio format coverage -information (e.g. to be loaded within IDA lighthouse). +information (e.g., to be loaded within IDA lighthouse). * `AFL_FRIDA_INST_DEBUG_FILE` - File to write raw assembly of original blocks and their instrumented counterparts during block compilation. * `AFL_FRIDA_INST_JIT` - Enable the instrumentation of Just-In-Time compiled @@ -617,13 +617,13 @@ child on fork. * `AFL_FRIDA_INST_RANGES` - See `AFL_QEMU_INST_RANGES` * `AFL_FRIDA_INST_SEED` - Sets the initial seed for the hash function used to generate block (and hence edge) IDs. Setting this to a constant value may be -useful for debugging purposes, e.g. investigating unstable edges. +useful for debugging purposes, e.g., investigating unstable edges. * `AFL_FRIDA_INST_TRACE` - Log to stdout the address of executed blocks, implies `AFL_FRIDA_INST_NO_OPTIMIZE`. * `AFL_FRIDA_INST_TRACE_UNIQUE` - As per `AFL_FRIDA_INST_TRACE`, but each edge is logged only once, requires `AFL_FRIDA_INST_NO_OPTIMIZE`. * `AFL_FRIDA_INST_UNSTABLE_COVERAGE_FILE` - File to write DynamoRio format -coverage information for unstable edges (e.g. to be loaded within IDA +coverage information for unstable edges (e.g., to be loaded within IDA lighthouse). * `AFL_FRIDA_JS_SCRIPT` - Set the script to be loaded by the FRIDA scripting engine. See [here](Scripting.md) for details. diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index c3204212..a786fd8b 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -113,7 +113,7 @@ If you want to fuzz a binary-only library, then you can fuzz it with frida-gum via frida_mode/. You will have to write a harness to call the target function in the library, use afl-frida.c as a template. -You can also perform remote fuzzing with frida, e.g. if you want to fuzz on +You can also perform remote fuzzing with frida, e.g., if you want to fuzz on iPhone or Android devices, for this you can use [https://github.com/ttdennis/fpicker/](https://github.com/ttdennis/fpicker/) as an intermediate that uses AFL++ for fuzzing. @@ -198,8 +198,8 @@ It is at about 80-85% performance. Dyninst is a binary instrumentation framework similar to Pintool and DynamoRIO. However, whereas Pintool and DynamoRIO work at runtime, Dyninst instruments the target at load time and then let it run - or save the binary with the changes. -This is great for some things, e.g. fuzzing, and not so effective for others, -e.g. malware analysis. +This is great for some things, e.g., fuzzing, and not so effective for others, +e.g., malware analysis. So, what you can do with Dyninst is taking every basic block and putting AFL++'s instrumentation code in there - and then save the binary. Afterwards, just fuzz diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 96e709ab..4e1e001e 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -167,7 +167,7 @@ allows you to find bugs that would not necessarily result in a crash. Note that sanitizers have a huge impact on CPU (= less executions per second) and RAM usage. Also you should only run one afl-fuzz instance per sanitizer -type. This is enough because a use-after-free bug will be picked up, e.g. by +type. This is enough because a use-after-free bug will be picked up, e.g., by ASAN (address sanitizer) anyway when syncing to other fuzzing instances, so not all fuzzing instances need to be instrumented with ASAN. @@ -179,7 +179,7 @@ The following sanitizers have built-in support in AFL++: local variable that is defined and read before it is even set. Enabled with `export AFL_USE_MSAN=1` before compiling. * UBSAN = Undefined Behavior SANitizer, finds instances where - by the C and C++ - standards - undefined behavior happens, e.g. adding two signed integers + standards - undefined behavior happens, e.g., adding two signed integers together where the result is larger than a signed integer can hold. Enabled with `export AFL_USE_UBSAN=1` before compiling. * CFISAN = Control Flow Integrity SANitizer, finds instances where the control @@ -202,15 +202,15 @@ be looked up in the sanitizer documentation of llvm/clang. afl-fuzz, however, requires some specific parameters important for fuzzing to be set. If you want to set your own, it might bail and report what it is missing. -Note that some sanitizers cannot be used together, e.g. ASAN and MSAN, and -others often cannot work together because of target weirdness, e.g. ASAN and +Note that some sanitizers cannot be used together, e.g., ASAN and MSAN, and +others often cannot work together because of target weirdness, e.g., ASAN and CFISAN. You might need to experiment which sanitizers you can combine in a target (which means more instances can be run without a sanitized target, which is more effective). ### d) Modifying the target -If the target has features that make fuzzing more difficult, e.g. checksums, +If the target has features that make fuzzing more difficult, e.g., checksums, HMAC, etc. then modify the source code so that checks for these values are removed. This can even be done safely for source code used in operational products by eliminating these checks within these AFL++ specific blocks: @@ -250,7 +250,7 @@ Then build the target. (Usually with `make`) reporting via `export AFL_QUIET=1`. 2. sometimes configure and build systems error on warnings - these should be - disabled (e.g. `--disable-werror` for some configure scripts). + disabled (e.g., `--disable-werror` for some configure scripts). 3. in case the configure/build system complains about AFL++'s compiler and aborts then set `export AFL_NOOPT=1` which will then just behave like the @@ -354,7 +354,7 @@ You can find many good examples of starting files in the Use the AFL++ tool `afl-cmin` to remove inputs from the corpus that do not produce a new path in the target. -Put all files from step a) into one directory, e.g. INPUTS. +Put all files from step a) into one directory, e.g., INPUTS. If the target program is to be called by fuzzing as `bin/target -d INPUTFILE` the run afl-cmin like this: @@ -380,8 +380,8 @@ for i in *; do done ``` -This step can also be parallelized, e.g. with `parallel`. Note that this step is -rather optional though. +This step can also be parallelized, e.g., with `parallel`. Note that this step +is rather optional though. ### Done! @@ -503,7 +503,7 @@ can set the cache size (in MB) by setting the environment variable `AFL_TESTCACHE_SIZE`. There should be one main fuzzer (`-M main-$HOSTNAME` option) and as many -secondary fuzzers (e.g. `-S variant1`) as you have cores that you use. Every +secondary fuzzers (e.g., `-S variant1`) as you have cores that you use. Every -M/-S entry needs a unique name (that can be whatever), however, the same -o output directory location has to be used for all instances. @@ -522,7 +522,7 @@ All other secondaries should be used like this: * a quarter to a third with the MOpt mutator enabled: `-L 0` * run with a different power schedule, recommended are: `fast (default), explore, coe, lin, quad, exploit and rare` which you can set - with e.g. `-p explore` + with, e.g., `-p explore` * a few instances should use the old queue cycling with `-Z` Also, it is recommended to set `export AFL_IMPORT_FIRST=1` to load test cases @@ -547,7 +547,7 @@ A long list can be found at However, you can also sync AFL++ with honggfuzz, libfuzzer with `-entropic=1`, etc. Just show the main fuzzer (-M) with the `-F` option where the queue/work -directory of a different fuzzer is, e.g. `-F /src/target/honggfuzz`. Using +directory of a different fuzzer is, e.g., `-F /src/target/honggfuzz`. Using honggfuzz (with `-n 1` or `-n 2`) and libfuzzer in parallel is highly recommended! @@ -615,8 +615,8 @@ To restart an afl-fuzz run, just reuse the same command line but replace the `-i directory` with `-i -` or set `AFL_AUTORESUME=1`. If you want to add new seeds to a fuzzing campaign you can run a temporary -fuzzing instance, e.g. when your main fuzzer is using `-o out` and the new seeds -are in `newseeds/` directory: +fuzzing instance, e.g., when your main fuzzer is using `-o out` and the new +seeds are in `newseeds/` directory: ``` AFL_BENCH_JUST_ONE=1 AFL_FAST_CAL=1 afl-fuzz -i newseeds -o out -S newseeds -- ./target @@ -665,9 +665,9 @@ then you will not touch any of the other library APIs and features. ### h) How long to fuzz a target? This is a difficult question. Basically if no new path is found for a long time -(e.g. for a day or a week) then you can expect that your fuzzing won't be +(e.g., for a day or a week) then you can expect that your fuzzing won't be fruitful anymore. However, often this just means that you should switch out -secondaries for others, e.g. custom mutator modules, sync to very different +secondaries for others, e.g., custom mutator modules, sync to very different fuzzers, etc. Keep the queue/ directory (for future fuzzings of the same or similar targets) diff --git a/docs/ideas.md b/docs/ideas.md index 325e7031..8193983b 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -32,7 +32,7 @@ Mentor: any ## Support other programming languages Other programming languages also use llvm hence they could (easily?) supported -for fuzzing, e.g. mono, swift, go, kotlin native, fortran, ... +for fuzzing, e.g., mono, swift, go, kotlin native, fortran, ... GCC also supports: Objective-C, Fortran, Ada, Go, and D (according to [Gcc homepage](https://gcc.gnu.org/)) diff --git a/docs/important_changes.md b/docs/important_changes.md index 6cd00791..82de054f 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -44,7 +44,7 @@ behaviors and defaults: * `-i` input directory option now descends into subdirectories. It also does not fatal on crashes and too large files, instead it skips them and uses them for splicing mutations - * -m none is now default, set memory limits (in MB) with e.g. -m 250 + * -m none is now default, set memory limits (in MB) with, e.g., -m 250 * deterministic fuzzing is now disabled by default (unless using -M) and can be enabled with -D * a caching of test cases can now be performed and can be modified by diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index d16049fa..ac8f2f2a 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -40,7 +40,7 @@ The idea and much of the initial implementation came from Laszlo Szekeres. ## 2a) How to use this - short -Set the `LLVM_CONFIG` variable to the clang version you want to use, e.g. +Set the `LLVM_CONFIG` variable to the clang version you want to use, e.g.: ``` LLVM_CONFIG=llvm-config-9 make diff --git a/instrumentation/README.lto.md b/instrumentation/README.lto.md index b97e5799..a20175b1 100644 --- a/instrumentation/README.lto.md +++ b/instrumentation/README.lto.md @@ -71,7 +71,7 @@ use an outdated Linux distribution, read the next section. Installing the llvm snapshot builds is easy and mostly painless: In the following line, change `NAME` for your Debian or Ubuntu release name -(e.g. buster, focal, eon, etc.): +(e.g., buster, focal, eon, etc.): ``` echo deb http://apt.llvm.org/NAME/ llvm-toolchain-NAME NAME >> /etc/apt/sources.list diff --git a/utils/README.md b/utils/README.md index b8df0b47..b7eead8e 100644 --- a/utils/README.md +++ b/utils/README.md @@ -13,7 +13,7 @@ Here's a quick overview of the stuff you can find in this directory: - afl_proxy - skeleton file example to show how to fuzz something where you gather coverage data via - different means, e.g. hw debugger + different means, e.g., hw debugger - afl_untracer - fuzz binary-only libraries much faster but with less coverage than qemu_mode diff --git a/utils/afl_network_proxy/README.md b/utils/afl_network_proxy/README.md index d2c00be2..c478319a 100644 --- a/utils/afl_network_proxy/README.md +++ b/utils/afl_network_proxy/README.md @@ -6,7 +6,8 @@ Note that the impact on fuzzing speed will be huge, expect a loss of 90%. ## When to use this 1. when you have to fuzz a target that has to run on a system that cannot - contain the fuzzing output (e.g. /tmp too small and file system is read-only) + contain the fuzzing output (e.g., /tmp too small and file system is + read-only) 2. when the target instantly reboots on crashes 3. ... any other reason you would need this @@ -28,6 +29,7 @@ For most targets this hurts performance though so it is disabled by default. Run `afl-network-server` with your target with the -m and -t values you need. Important is the -i parameter which is the TCP port to listen on. e.g.: + ``` afl-network-server -i 1111 -m 25M -t 1000 -- /bin/target -f @@ ``` @@ -50,7 +52,7 @@ value itself should be 500-1000 higher than the one on afl-network-server. The TARGET can be an IPv4 or IPv6 address, or a host name that resolves to either. Note that also the outgoing interface can be specified with a '%' for -`afl-network-client`, e.g. `fe80::1234%eth0`. +`afl-network-client`, e.g., `fe80::1234%eth0`. Also make sure your default TCP window size is larger than your MAP_SIZE (130kb is a good value). -- cgit 1.4.1 From aa6586a761c47da1f202221c9a449b692cacbcae Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Thu, 2 Dec 2021 21:23:22 +0100 Subject: Fix typos - 1st run --- custom_mutators/gramatron/README.md | 6 +++--- docs/FAQ.md | 2 +- docs/fuzzing_in_depth.md | 2 +- instrumentation/README.llvm.md | 2 +- qemu_mode/libcompcov/README.md | 2 +- utils/optimin/README.md | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) (limited to 'utils') diff --git a/custom_mutators/gramatron/README.md b/custom_mutators/gramatron/README.md index 0167f620..2ed014cd 100644 --- a/custom_mutators/gramatron/README.md +++ b/custom_mutators/gramatron/README.md @@ -1,6 +1,6 @@ # GramaTron -Gramatron is a coverage-guided fuzzer that uses grammar automatons to perform +GramaTron is a coverage-guided fuzzer that uses grammar automatons to perform grammar-aware fuzzing. Technical details about our framework are available in the [ISSTA'21 paper](https://nebelwelt.net/files/21ISSTA.pdf). The artifact to reproduce the experiments presented in the paper are present @@ -9,11 +9,11 @@ grammars is presented below: # Compiling -Execute `./build_gramatron_mutator.sh` +Execute `./build_gramatron_mutator.sh`. # Running -You have to set the grammar file to use with `GRAMMATRON_AUTOMATION`: +You have to set the grammar file to use with `GRAMATRON_AUTOMATION`: ``` export AFL_DISABLE_TRIM=1 diff --git a/docs/FAQ.md b/docs/FAQ.md index 27250415..671957ef 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -125,7 +125,7 @@ If you find an interesting or important question missing, submit it via If, however, randomness happens, e.g., a thread reading other external data, reaction to timing, etc., then in some of the re-executions with the same data - the edge coverage result will be different accross runs. Those edges that + the edge coverage result will be different across runs. Those edges that change are then flagged "unstable". The more "unstable" edges, the more difficult for AFL++ to identify valid new paths. diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 7c464d81..011ba783 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -131,7 +131,7 @@ The following options are available when you instrument with LTO mode have to compile the target twice, once specifically with/for this mode by setting `AFL_LLVM_CMPLOG=1`, and pass this binary to afl-fuzz via the `-c` parameter. Note that you can compile also just a cmplog binary and use that - for both, however, there will be a performance penality. You can read more + for both, however, there will be a performance penalty. You can read more about this in [instrumentation/README.cmplog.md](../instrumentation/README.cmplog.md). diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index ac8f2f2a..c93cd312 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -125,7 +125,7 @@ For splitting memcmp, strncmp, etc., see Then there are different ways of instrumenting the target: -1. An better instrumentation strategy uses LTO and link time instrumentation. +1. A better instrumentation strategy uses LTO and link time instrumentation. Note that not all targets can compile in this mode, however, if it works it is the best option you can use. To go with this option, use afl-clang-lto/afl-clang-lto++. See [README.lto.md](README.lto.md). diff --git a/qemu_mode/libcompcov/README.md b/qemu_mode/libcompcov/README.md index 50f0d802..bb010d8f 100644 --- a/qemu_mode/libcompcov/README.md +++ b/qemu_mode/libcompcov/README.md @@ -31,7 +31,7 @@ The AFL_COMPCOV_LEVEL tells to QEMU and libcompcov how to log comaprisons. Level 1 logs just comparison with immediates / read-only memory and level 2 logs all the comparisons. -The library make use of https://github.com/ouadev/proc_maps_parser and so it is +The library makes use of https://github.com/ouadev/proc_maps_parser and so it is Linux specific. However, this is not a strict dependency, other UNIX operating systems can be supported by replacing the code related to the /proc/self/maps parsing. \ No newline at end of file diff --git a/utils/optimin/README.md b/utils/optimin/README.md index 507cb305..340022b8 100644 --- a/utils/optimin/README.md +++ b/utils/optimin/README.md @@ -77,7 +77,7 @@ For more details, see the paper [Seed Selection for Successful Fuzzing](https://dl.acm.org/doi/10.1145/3460319.3464795). If you use OptiMin in your research, please cite this paper. -Bibtex: +BibTeX: ```bibtex @inproceedings{Herrera:2021:FuzzSeedSelection, -- cgit 1.4.1 From 6eab6a55af74754cd0229012f1c9543c0c3cb2a7 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Sat, 4 Dec 2021 21:14:50 +0100 Subject: Fix spelling of "FRIDA mode" and "QEMU mode" --- TODO.md | 2 +- docs/INSTALL.md | 4 ++-- docs/features.md | 8 ++++---- docs/fuzzing_binary-only_targets.md | 35 ++++++++++++++++++----------------- docs/important_changes.md | 8 ++++---- instrumentation/README.llvm.md | 2 +- qemu_mode/libqasan/README.md | 2 +- utils/README.md | 2 +- utils/afl_untracer/README.md | 2 +- utils/aflpp_driver/README.md | 4 ++-- utils/qbdi_mode/README.md | 2 +- 11 files changed, 36 insertions(+), 35 deletions(-) (limited to 'utils') diff --git a/TODO.md b/TODO.md index 04f3abab..e6b095fc 100644 --- a/TODO.md +++ b/TODO.md @@ -16,7 +16,7 @@ ## Further down the road -qemu_mode/frida_mode: +QEMU mode/FRIDA mode: - non colliding instrumentation - rename qemu specific envs to AFL_QEMU (AFL_ENTRYPOINT, AFL_CODE_START/END, AFL_COMPCOV_LEVEL?) diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 08d3283e..9d1309fe 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -23,7 +23,7 @@ sudo apt-get install -y build-essential python3-dev automake git flex bison libg # try to install llvm 11 and install the distro default if that fails sudo apt-get install -y lld-11 llvm-11 llvm-11-dev clang-11 || sudo apt-get install -y lld llvm llvm-dev clang sudo apt-get install -y gcc-$(gcc --version|head -n1|sed 's/.* //'|sed 's/\..*//')-plugin-dev libstdc++-$(gcc --version|head -n1|sed 's/.* //'|sed 's/\..*//')-dev -sudo apt-get install -y ninja-build # for qemu_mode +sudo apt-get install -y ninja-build # for QEMU mode git clone https://github.com/AFLplusplus/AFLplusplus cd AFLplusplus make distrib @@ -33,7 +33,7 @@ sudo make install It is recommended to install the newest available gcc, clang and llvm-dev possible in your distribution! -Note that "make distrib" also builds instrumentation, qemu_mode, unicorn_mode +Note that "make distrib" also builds instrumentation, QEMU mode, unicorn_mode and more. If you just want plain AFL++, then do "make all". However, compiling and using at least instrumentation is highly recommended for much better results - hence in this case choose: diff --git a/docs/features.md b/docs/features.md index fb688a64..06b1bcbe 100644 --- a/docs/features.md +++ b/docs/features.md @@ -4,7 +4,7 @@ AFL++ supports llvm from 3.8 up to version 12, very fast binary fuzzing with QEMU 5.1 with laf-intel and redqueen, frida mode, unicorn mode, gcc plugin, full *BSD, Mac OS, Solaris and Android support and much, much, much more. -| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | frida_mode(9) | qemu_mode(10) |unicorn_mode(10) |coresight_mode(11)| +| Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) |unicorn_mode(10) |coresight_mode(11)| | -------------------------|:-------:|:---------:|:----------:|:----------------:|:----------------:|:----------------:|:----------------:| | Threadsafe counters | | x(3) | | | | | | | NeverZero | x86[_64]| x(1) | x | x | x | x | | @@ -37,9 +37,9 @@ QEMU 5.1 with laf-intel and redqueen, frida mode, unicorn mode, gcc plugin, full Among others, the following features and patches have been integrated: -* NeverZero patch for afl-gcc, instrumentation, qemu_mode and unicorn_mode which +* NeverZero patch for afl-gcc, instrumentation, QEMU mode and unicorn_mode which prevents a wrapping map value to zero, increases coverage -* Persistent mode, deferred forkserver and in-memory fuzzing for qemu_mode +* Persistent mode, deferred forkserver and in-memory fuzzing for QEMU mode * Unicorn mode which allows fuzzing of binaries from completely different platforms (integration provided by domenukk) * The new CmpLog instrumentation for LLVM and QEMU inspired by @@ -51,7 +51,7 @@ Among others, the following features and patches have been integrated: [https://github.com/puppet-meteor/MOpt-AFL](https://github.com/puppet-meteor/MOpt-AFL) * LLVM mode Ngram coverage by Adrian Herrera [https://github.com/adrianherrera/afl-ngram-pass](https://github.com/adrianherrera/afl-ngram-pass) -* LAF-Intel/CompCov support for instrumentation, qemu_mode and unicorn_mode +* LAF-Intel/CompCov support for instrumentation, QEMU mode and unicorn_mode (with enhanced capabilities) * Radamsa and honggfuzz mutators (as custom mutators). * QBDI mode to fuzz android native libraries via Quarkslab's diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index b3d9ca02..15155111 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -12,18 +12,18 @@ fuzzed with AFL++. ## TL;DR: -Qemu_mode in persistent mode is the fastest - if the stability is high enough. +QEMU mode in persistent mode is the fastest - if the stability is high enough. Otherwise, try RetroWrite, Dyninst, and if these fail, too, then try standard -qemu_mode with AFL_ENTRYPOINT to where you need it. +QEMU mode with `AFL_ENTRYPOINT` to where you need it. -If your target is a library, then use frida_mode. +If your target is a library, then use FRIDA mode. If your target is non-linux, then use unicorn_mode. ## Fuzzing binary-only targets with AFL++ -### Qemu_mode +### QEMU mode -Qemu_mode is the "native" solution to the program. It is available in the +QEMU mode is the "native" solution to the program. It is available in the ./qemu_mode/ directory and, once compiled, it can be accessed by the afl-fuzz -Q command line option. It is the easiest to use alternative and even works for cross-platform binaries. @@ -37,11 +37,12 @@ cd qemu_mode ./build_qemu_support.sh ``` -The following setup to use qemu_mode is recommended: +The following setup to use QEMU mode is recommended: + * run 1 afl-fuzz -Q instance with CMPLOG (`-c 0` + `AFL_COMPCOV_LEVEL=2`) * run 1 afl-fuzz -Q instance with QASAN (`AFL_USE_QASAN=1`) * run 1 afl-fuzz -Q instance with LAF (`AFL_PRELOAD=libcmpcov.so` + - `AFL_COMPCOV_LEVEL=2`), alternatively you can use frida_mode, just switch `-Q` + `AFL_COMPCOV_LEVEL=2`), alternatively you can use FRIDA mode, just switch `-Q` with `-O` and remove the LAF instance Then run as many instances as you have cores left with either -Q mode or - even @@ -49,16 +50,16 @@ better - use a binary rewriter like Dyninst, RetroWrite, ZAFL, etc. If [afl-dyninst](https://github.com/vanhauser-thc/afl-dyninst) works for your binary, then you can use afl-fuzz normally and it will have twice the speed -compared to qemu_mode (but slower than qemu persistent mode). Note that several +compared to QEMU mode (but slower than QEMU persistent mode). Note that several other binary rewriters exist, all with their advantages and caveats. -The speed decrease of qemu_mode is at about 50%. However, various options exist +The speed decrease of QEMU mode is at about 50%. However, various options exist to increase the speed: - using AFL_ENTRYPOINT to move the forkserver entry to a later basic block in the binary (+5-10% speed) - using persistent mode [qemu_mode/README.persistent.md](../qemu_mode/README.persistent.md) this will - result in a 150-300% overall speed increase - so 3-8x the original qemu_mode + result in a 150-300% overall speed increase - so 3-8x the original QEMU mode speed! - using AFL_CODE_START/AFL_CODE_END to only instrument specific parts @@ -71,7 +72,7 @@ conducive to parallelization. Note that there is also honggfuzz: [https://github.com/google/honggfuzz](https://github.com/google/honggfuzz) which -now has a qemu_mode, but its performance is just 1.5% ... +now has a QEMU mode, but its performance is just 1.5% ... If you like to code a customized fuzzer without much work, we highly recommend to check out our sister project libafl which supports QEMU, too: @@ -87,14 +88,14 @@ It is included in AFL++. For more information, see [qemu_mode/README.wine.md](../qemu_mode/README.wine.md). -### Frida_mode +### FRIDA mode -In frida_mode, you can fuzz binary-only targets as easily as with QEMU. -Frida_mode is sometimes faster and sometimes slower than Qemu_mode. It is also +In FRIDA mode, you can fuzz binary-only targets as easily as with QEMU mode. +FRIDA mode is sometimes faster and sometimes slower than QEMU mode. It is also newer, lacks COMPCOV, and has the advantage that it works on MacOS (both intel and M1). -To build frida_mode: +To build FRIDA mode: ```shell cd frida_mode @@ -149,11 +150,11 @@ For further information, check out If the goal is to fuzz a dynamic library, then there are two options available. For both, you need to write a small harness that loads and calls the library. -Then you fuzz this with either frida_mode or qemu_mode and either use +Then you fuzz this with either FRIDA mode or QEMU mode and either use `AFL_INST_LIBS=1` or `AFL_QEMU/FRIDA_INST_RANGES`. Another, less precise and slower option is to fuzz it with utils/afl_untracer/ -and use afl-untracer.c as a template. It is slower than frida_mode. +and use afl-untracer.c as a template. It is slower than FRIDA mode. For more information, see [utils/afl_untracer/README.md](../utils/afl_untracer/README.md). diff --git a/docs/important_changes.md b/docs/important_changes.md index 203c5a66..d5e67f75 100644 --- a/docs/important_changes.md +++ b/docs/important_changes.md @@ -5,8 +5,8 @@ changes. ## From version 3.00 onwards -With AFL++ 3.13-3.20, we introduce frida_mode (-O) to have an alternative for -binary-only fuzzing. It is slower than Qemu mode but works on MacOS, Android, +With AFL++ 3.13-3.20, we introduce FRIDA mode (`-O`) to have an alternative for +binary-only fuzzing. It is slower than QEMU mode but works on MacOS, Android, iOS etc. With AFL++ 3.15, we introduced the following changes from previous behaviors: @@ -31,8 +31,8 @@ behaviors and defaults: All instrumentation source code is now in the `instrumentation/` folder. * The gcc_plugin was replaced with a new version submitted by AdaCore that supports more features. Thank you! - * qemu_mode got upgraded to QEMU 5.1, but to be able to build this a current - ninja build tool version and python3 setuptools are required. qemu_mode also + * QEMU mode got upgraded to QEMU 5.1, but to be able to build this a current + ninja build tool version and python3 setuptools are required. QEMU mode also got new options like snapshotting, instrumenting specific shared libraries, etc. Additionally QEMU 5.1 supports more CPU targets so this is really worth it. diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index c93cd312..fa025643 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -218,7 +218,7 @@ by Jinghan Wang, et. al. Note that the original implementation (available [here](https://github.com/bitsecurerlab/afl-sensitive)) is built on top of AFL's -qemu_mode. This is essentially a port that uses LLVM vectorized instructions +QEMU mode. This is essentially a port that uses LLVM vectorized instructions (available from llvm versions 4.0.1 and higher) to achieve the same results when compiling source code. diff --git a/qemu_mode/libqasan/README.md b/qemu_mode/libqasan/README.md index 6a65c12b..41195933 100644 --- a/qemu_mode/libqasan/README.md +++ b/qemu_mode/libqasan/README.md @@ -9,7 +9,7 @@ and this runtime is injected via LD_PRELOAD (so works just for dynamically linked binaries). The usage is super simple, just set the env var `AFL_USE_QASAN=1` when fuzzing -in qemu mode (-Q). afl-fuzz will automatically set AFL_PRELOAD to load this +in QEMU mode (-Q). afl-fuzz will automatically set AFL_PRELOAD to load this library and enable the QASan instrumentation in afl-qemu-trace. For debugging purposes, we still suggest to run the original QASan as the diff --git a/utils/README.md b/utils/README.md index b7eead8e..5f5745b9 100644 --- a/utils/README.md +++ b/utils/README.md @@ -16,7 +16,7 @@ Here's a quick overview of the stuff you can find in this directory: different means, e.g., hw debugger - afl_untracer - fuzz binary-only libraries much faster but with - less coverage than qemu_mode + less coverage than QEMU mode - analysis_scripts - random -o out analysis scripts diff --git a/utils/afl_untracer/README.md b/utils/afl_untracer/README.md index 9f41618f..3fff5f83 100644 --- a/utils/afl_untracer/README.md +++ b/utils/afl_untracer/README.md @@ -5,7 +5,7 @@ afl-untracer is an example skeleton file which can easily be used to fuzz a closed source library. -It requires less memory and is x3-5 faster than qemu_mode, however, it is way +It requires less memory and is x3-5 faster than QEMU mode, however, it is way more course grained and does not provide interesting features like compcov or cmplog. diff --git a/utils/aflpp_driver/README.md b/utils/aflpp_driver/README.md index d534cd7f..8b9fe15f 100644 --- a/utils/aflpp_driver/README.md +++ b/utils/aflpp_driver/README.md @@ -22,10 +22,10 @@ or `@@` as command line parameters. ## aflpp_qemu_driver -Note that you can use the driver too for frida_mode (`-O`). +Note that you can use the driver too for FRIDA mode (`-O`). aflpp_qemu_driver is used for libfuzzer `LLVMFuzzerTestOneInput()` targets that -are to be fuzzed in qemu_mode. So compile them with clang/clang++, without +are to be fuzzed in QEMU mode. So compile them with clang/clang++, without -fsantize=fuzzer or afl-clang-fast, and link in libAFLQemuDriver.a: `clang++ -o fuzz fuzzer_harness.cc libAFLQemuDriver.a [plus required linking]`. diff --git a/utils/qbdi_mode/README.md b/utils/qbdi_mode/README.md index c8d46fca..02dd3c74 100755 --- a/utils/qbdi_mode/README.md +++ b/utils/qbdi_mode/README.md @@ -2,7 +2,7 @@ NOTE: this code is outdated and first would need to be adapted to the current AFL++ versions. -Try frida_mode or fpicker [https://github.com/ttdennis/fpicker/](https://github.com/ttdennis/fpicker/) first, maybe they suite your need. +Try FRIDA mode or fpicker [https://github.com/ttdennis/fpicker/](https://github.com/ttdennis/fpicker/) first, maybe they suite your need. ## 1) Introduction -- cgit 1.4.1 From 3506077fd6f250f3c080b58ea3bae117c3b122da Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Sat, 4 Dec 2021 22:28:05 +0100 Subject: Add missing blank lines and remove double blank lines --- custom_mutators/README.md | 1 + custom_mutators/gramatron/README.md | 1 + custom_mutators/libfuzzer/README.md | 4 +++- docs/custom_mutators.md | 8 +++++++- docs/fuzzing_binary-only_targets.md | 3 ++- docs/fuzzing_in_depth.md | 1 - frida_mode/README.md | 1 + unicorn_mode/README.md | 1 - unicorn_mode/samples/c/COMPILE.md | 1 + unicorn_mode/samples/speedtest/README.md | 13 ++++++++++--- utils/afl_untracer/README.md | 8 +++++++- utils/autodict_ql/readme.md | 17 ++++++++++------- utils/qbdi_mode/README.md | 8 ++++---- utils/qemu_persistent_hook/README.md | 2 +- 14 files changed, 48 insertions(+), 21 deletions(-) (limited to 'utils') diff --git a/custom_mutators/README.md b/custom_mutators/README.md index fa877f34..0289e150 100644 --- a/custom_mutators/README.md +++ b/custom_mutators/README.md @@ -15,6 +15,7 @@ In `./rust`, you will find rust bindings, including a simple example in `./rust/ If you use git to clone AFL++, then the following will incorporate our excellent grammar custom mutator: + ```sh git submodule update --init ``` diff --git a/custom_mutators/gramatron/README.md b/custom_mutators/gramatron/README.md index 2ed014cd..5e10f97b 100644 --- a/custom_mutators/gramatron/README.md +++ b/custom_mutators/gramatron/README.md @@ -34,6 +34,7 @@ afl-fuzz -i in -o out -- ./target E.g., ./preprocess/prep_automaton.sh ~/grammars/ruby/source.json PROGRAM ``` + - If the grammar has no self-embedding rules then you do not need to pass the stack limit parameter. However, if it does have self-embedding rules then you need to pass the stack limit parameter. We recommend starting with `5` and diff --git a/custom_mutators/libfuzzer/README.md b/custom_mutators/libfuzzer/README.md index fb3025f2..cb4773b7 100644 --- a/custom_mutators/libfuzzer/README.md +++ b/custom_mutators/libfuzzer/README.md @@ -11,9 +11,11 @@ Note that this is currently a simple implementation and it is missing two featur * Dictionary support To update the source, all that is needed is that FuzzerDriver.cpp has to receive + ``` #include "libfuzzer.inc" ``` + before the closing namespace bracket. It is also libfuzzer.inc where the configuration of the libfuzzer mutations @@ -21,4 +23,4 @@ are done. > Original repository: https://github.com/llvm/llvm-project > Path: compiler-rt/lib/fuzzer/*.{h|cpp} -> Source commit: df3e903655e2499968fc7af64fb5fa52b2ee79bb +> Source commit: df3e903655e2499968fc7af64fb5fa52b2ee79bb \ No newline at end of file diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 0e94ab10..2f632e1f 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -23,6 +23,7 @@ The custom mutator is passed to `afl-fuzz` via the `AFL_CUSTOM_MUTATOR_LIBRARY` or `AFL_PYTHON_MODULE` environment variable, and must export a fuzz function. Now AFL++ also supports multiple custom mutators which can be specified in the same `AFL_CUSTOM_MUTATOR_LIBRARY` environment variable like this. + ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="full/path/to/mutator_first.so;full/path/to/mutator_second.so" ``` @@ -38,6 +39,7 @@ performed with the custom mutator. ## 2) APIs C/C++: + ```c void *afl_custom_init(afl_state_t *afl, unsigned int seed); unsigned int afl_custom_fuzz_count(void *data, const unsigned char *buf, size_t buf_size); @@ -56,6 +58,7 @@ void afl_custom_deinit(void *data); ``` Python: + ```python def init(seed): pass @@ -233,7 +236,6 @@ Optionally, the following environment variables are supported: combined with a custom trimming routine (see below) because trimming can cause the same test breakage like havoc and splice. - - `AFL_PYTHON_ONLY` Deprecated and removed, use `AFL_CUSTOM_MUTATOR_ONLY` instead. @@ -268,9 +270,11 @@ In case your setup is different, set the necessary variables like this: ### Custom Mutator Preparation For C/C++ mutators, the source code must be compiled as a shared object: + ```bash gcc -shared -Wall -O3 example.c -o example.so ``` + Note that if you specify multiple custom mutators, the corresponding functions will be called in the order in which they are specified. e.g. first `post_process` function of `example_first.so` will be called and then that of @@ -279,12 +283,14 @@ will be called in the order in which they are specified. e.g. first ### Run C/C++ + ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="/full/path/to/example_first.so;/full/path/to/example_second.so" afl-fuzz /path/to/program ``` Python + ```bash export PYTHONPATH=`dirname /full/path/to/example.py` export AFL_PYTHON_MODULE=example diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index 15155111..eaed3a91 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -21,6 +21,7 @@ If your target is a library, then use FRIDA mode. If your target is non-linux, then use unicorn_mode. ## Fuzzing binary-only targets with AFL++ + ### QEMU mode QEMU mode is the "native" solution to the program. It is available in the @@ -175,6 +176,7 @@ An alternative solution are binary rewriters. They are faster then the solutions native to AFL++ but don't always work. ### ZAFL + ZAFL is a static rewriting platform supporting x86-64 C/C++, stripped/unstripped, and PIE/non-PIE binaries. Beyond conventional instrumentation, ZAFL's API enables transformation passes (e.g., laf-Intel, @@ -277,7 +279,6 @@ There are many binary-only fuzzing frameworks. Some are great for CTFs but don't work with large binaries, others are very slow but have good path discovery, some are very hard to set-up... - * Jackalope: [https://github.com/googleprojectzero/Jackalope](https://github.com/googleprojectzero/Jackalope) * Manticore: diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 2e030e7b..4a1ddf45 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -808,7 +808,6 @@ then color-codes the input based on which sections appear to be critical, and which are not; while not bulletproof, it can often offer quick insights into complex file formats. - ## 5. CI fuzzing Some notes on CI fuzzing - this fuzzing is different to normal fuzzing campaigns diff --git a/frida_mode/README.md b/frida_mode/README.md index e88bda8d..08f6b891 100644 --- a/frida_mode/README.md +++ b/frida_mode/README.md @@ -141,6 +141,7 @@ instances run CMPLOG mode and instrumentation of the binary is less frequent (only on CMP, SUB and CALL instructions) performance is not quite so critical. ## Advanced configuration options + * `AFL_FRIDA_DRIVER_NO_HOOK` - See `AFL_QEMU_DRIVER_NO_HOOK`. When using the QEMU driver to provide a `main` loop for a user provided `LLVMFuzzerTestOneInput`, this option configures the driver to read input from diff --git a/unicorn_mode/README.md b/unicorn_mode/README.md index 4c95e8f3..ee4a7b22 100644 --- a/unicorn_mode/README.md +++ b/unicorn_mode/README.md @@ -83,7 +83,6 @@ uses slightly older concepts, but describes the general ideas, at: [https://medium.com/@njvoss299/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf](https://medium.com/@njvoss299/afl-unicorn-fuzzing-arbitrary-binary-code-563ca28936bf) - The ['helper_scripts'](./helper_scripts) directory also contains several helper scripts that allow you to dump context from a running process, load it, and hook heap allocations. For details on how to use this check out the follow-up blog post to the one linked above. diff --git a/unicorn_mode/samples/c/COMPILE.md b/unicorn_mode/samples/c/COMPILE.md index 4e3cf568..e5265071 100644 --- a/unicorn_mode/samples/c/COMPILE.md +++ b/unicorn_mode/samples/c/COMPILE.md @@ -6,6 +6,7 @@ This shows a simple harness for unicornafl in C The target can be built using the `make` command. Just make sure you have built unicorn support first: + ```bash cd /path/to/afl/unicorn_mode ./build_unicorn_support.sh diff --git a/unicorn_mode/samples/speedtest/README.md b/unicorn_mode/samples/speedtest/README.md index 496d75cd..bd5ba8d0 100644 --- a/unicorn_mode/samples/speedtest/README.md +++ b/unicorn_mode/samples/speedtest/README.md @@ -35,7 +35,6 @@ cd python TODO: add results here. - ## Compiling speedtest_target.c You shouldn't need to compile simple_target.c since a X86_64 binary version is @@ -46,20 +45,28 @@ The pre-built binary (simple_target_x86_64.bin) was built using -g -O0 in gcc. Then load the binary and execute the main function directly. -## Addresses for the harness: +## Addresses for the harness + To find the address (in hex) of main, run: + ```bash objdump -M intel -D target | grep '
:' | cut -d" " -f1 ``` + To find all call sites to magicfn, run: + ```bash objdump -M intel -D target | grep '$' | cut -d":" -f1 ``` + For malloc callsites: + ```bash objdump -M intel -D target | grep '$' | cut -d":" -f1 ``` + And free callsites: + ```bash objdump -M intel -D target | grep '$' | cut -d":" -f1 -``` +``` \ No newline at end of file diff --git a/utils/afl_untracer/README.md b/utils/afl_untracer/README.md index 3fff5f83..cd87998b 100644 --- a/utils/afl_untracer/README.md +++ b/utils/afl_untracer/README.md @@ -28,28 +28,34 @@ To generate the `patches.txt` file for your target library use the The patches.txt file has to be pointed to by `AFL_UNTRACER_FILE`. To easily run the scripts without needing to run the GUI with Ghidra: + ``` /opt/ghidra/support/analyzeHeadless /tmp/ tmp$$ -import libtestinstr.so -postscript ./ghidra_get_patchpoints.java rm -rf /tmp/tmp$$ ``` + The file is created at `~/Desktop/patches.txt` ### Fuzzing Example (after modifying afl-untracer.c to your needs, compiling and creating patches.txt): + ``` LD_LIBRARY_PATH=/path/to/target/library AFL_UNTRACER_FILE=./patches.txt afl-fuzz -i in -o out -- ./afl-untracer ``` + (or even remote via afl-network-proxy). ### Testing and debugging For testing/debugging you can try: + ``` make DEBUG=1 AFL_UNTRACER_FILE=./patches.txt AFL_DEBUG=1 gdb ./afl-untracer ``` + and then you can easily set breakpoints to "breakpoint" and "fuzz". # Background @@ -57,4 +63,4 @@ and then you can easily set breakpoints to "breakpoint" and "fuzz". This idea is based on [UnTracer](https://github.com/FoRTE-Research/UnTracer-AFL) and modified by [Trapfuzz](https://github.com/googleprojectzero/p0tools/tree/master/TrapFuzz). This implementation is slower because the traps are not patched out with each -run, but on the other hand gives much better coverage information. +run, but on the other hand gives much better coverage information. \ No newline at end of file diff --git a/utils/autodict_ql/readme.md b/utils/autodict_ql/readme.md index 491ec85b..789cd152 100644 --- a/utils/autodict_ql/readme.md +++ b/utils/autodict_ql/readme.md @@ -6,12 +6,12 @@ Tokens are useful when you perform fuzzing on different parsers. The AFL++ `-x` switch enables the usage of dictionaries through your fuzzing campaign. If you are not familiar with Dictionaries in fuzzing, take a look [here](https://github.com/AFLplusplus/AFLplusplus/tree/stable/dictionaries) . - ## Why CodeQL ? -We basically developed this plugin on top of the CodeQL engine because it gives the user scripting features, it's easier and it's independent of the LLVM system. This means that a user can write his CodeQL scripts or modify the current scripts to improve or change the token generation algorithms based on different program analysis concepts. +We basically developed this plugin on top of the CodeQL engine because it gives the user scripting features, it's easier and it's independent of the LLVM system. This means that a user can write his CodeQL scripts or modify the current scripts to improve or change the token generation algorithms based on different program analysis concepts. ## CodeQL scripts + Currently, we pushed some scripts as defaults for Token generation. In addition, we provide every CodeQL script as an standalone script because it's easier to modify or test. Currently we provided the following CodeQL scripts : @@ -28,23 +28,26 @@ Currently we provided the following CodeQL scripts : You can write other CodeQL scripts to extract possible effective tokens if you think they can be useful. - ## Usage -Before you proceed to installation make sure that you have the following packages by installing them : +Before you proceed to installation make sure that you have the following packages by installing them: + ```shell sudo apt install build-essential libtool-bin python3-dev python3 automake git vim wget -y ``` + The usage of Autodict-QL is pretty easy. But let's describe it as: 1. First of all, you need to have CodeQL installed on the system. We make this possible with `build-codeql.sh` bash script. This script will install CodeQL completety and will set the required environment variables for your system. -Do the following : +Do the following: + ```shell # chmod +x codeql-build.sh # ./codeql-build.sh # source ~/.bashrc # codeql ``` + Then you should get: ```shell @@ -86,8 +89,8 @@ Commands: - This will create the final `tokens` dir for you and you are done, then pass the tokens path to AFL++'s `-x` flag. 5. Done! - ## More on dictionaries and tokens + Core developer of the AFL++ project Marc Heuse also developed a similar tool named `dict2file` which is a LLVM pass which can automatically extract useful tokens, in addition with LTO instrumentation mode, this dict2file is automatically generates token extraction. `Autodict-QL` plugin gives you scripting capability and you can do whatever you want to extract from the Codebase and it's up to you. In addition it's independent from LLVM system. On the other hand, you can also use Google dictionaries which have been made public in May 2020, but the problem of using Google dictionaries is that they are limited to specific file formats and specifications. For example, for testing binutils and ELF file format or AVI in FFMPEG, there are no pre-built dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` features to automatically generate dictionaries based on the target. @@ -97,4 +100,4 @@ In overall, fuzzing with dictionaries and well-generated tokens will give better There are 2 important points to remember : - If you combine `Autodict-QL` with AFL++ cmplog, you will get much better code coverage and hence better chances to discover new bugs. -- Do not forget to set `AFL_MAX_DET_EXTRAS` at least to the number of generated dictionaries. If you forget to set this environment variable, then AFL++ uses just 200 tokens and use the rest of them only probabilistically. So this will guarantee that your tokens will be used by AFL++. +- Do not forget to set `AFL_MAX_DET_EXTRAS` at least to the number of generated dictionaries. If you forget to set this environment variable, then AFL++ uses just 200 tokens and use the rest of them only probabilistically. So this will guarantee that your tokens will be used by AFL++. \ No newline at end of file diff --git a/utils/qbdi_mode/README.md b/utils/qbdi_mode/README.md index 02dd3c74..08558017 100755 --- a/utils/qbdi_mode/README.md +++ b/utils/qbdi_mode/README.md @@ -9,7 +9,6 @@ Try FRIDA mode or fpicker [https://github.com/ttdennis/fpicker/](https://github. The code in ./qbdi_mode allows you to build a standalone feature that using the QBDI framework to fuzz android native library. - ## 2) Build First download the Android NDK @@ -53,6 +52,7 @@ https://qbdi.quarkslab.com/ ``` For Android x86_64 + ``` https://github.com/QBDI/QBDI/releases/download/v0.7.0/QBDI-0.7.0-android-X86_64.tar.gz ``` @@ -86,7 +86,6 @@ Then run the build.sh this could build the afl-fuzz and also the qbdi template for android x86_64 - ### Example The demo-so.c is an vulnerable library, it has a function for test @@ -160,6 +159,7 @@ QBDI_NOINLINE int fuzz_func() { ``` Just compile it + ``` ./build.sh x86_64 ``` @@ -176,6 +176,7 @@ adb push ../../android-standalone-toolchain-x86_64/sysroot/usr/lib/x86_64-linux- ``` In android adb shell, run the loader to test if it runs + ``` cd /data/local/tmp export LD_LIBRARY_PATH=/data/local/tmp @@ -202,5 +203,4 @@ Now run `afl-fuzz` to fuzz the demo library ./afl-fuzz -i in -o out -- ./loader /data/local/tmp/libdemo.so @@ ``` -![screen1](assets/screen1.png) - +![screen1](assets/screen1.png) \ No newline at end of file diff --git a/utils/qemu_persistent_hook/README.md b/utils/qemu_persistent_hook/README.md index 3f908c22..3bbaef6b 100644 --- a/utils/qemu_persistent_hook/README.md +++ b/utils/qemu_persistent_hook/README.md @@ -16,4 +16,4 @@ mkdir in echo 0000 > in/in ../../afl-fuzz -Q -i in -o out -- ./test -``` +``` \ No newline at end of file -- cgit 1.4.1 From 89df436290c67b1c03122bfe5c68cf4f92e581c0 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Sun, 5 Dec 2021 19:03:48 +0100 Subject: Fix broken links - 1st run --- docs/INSTALL.md | 9 ++++---- docs/afl-fuzz_approach.md | 14 ++++++------ docs/env_variables.md | 11 +++++----- docs/fuzzing_binary-only_targets.md | 8 +++---- docs/fuzzing_in_depth.md | 9 ++++---- frida_mode/Scripting.md | 4 ++-- instrumentation/README.llvm.md | 43 ++++++++++++++++++++++++++++++++++++- utils/README.md | 2 +- 8 files changed, 72 insertions(+), 28 deletions(-) (limited to 'utils') diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 9d1309fe..906d3f8e 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -60,10 +60,9 @@ These build targets exist: * unit: perform unit tests (based on cmocka) * help: shows these build options -[Unless you are on Mac OS -X](https://developer.apple.com/library/archive/qa/qa1118/_index.html), you can -also build statically linked versions of the AFL++ binaries by passing the -`STATIC=1` argument to make: +[Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html), +you can also build statically linked versions of the AFL++ binaries by passing +the `STATIC=1` argument to make: ```shell make STATIC=1 @@ -169,5 +168,5 @@ sysctl kern.sysv.shmall=98304 ``` See -[https://www.spy-hill.com/help/apple/SharedMemory.html](https://www.spy-hill.com/help/apple/SharedMemory.html) +[http://www.spy-hill.com/help/apple/SharedMemory.html](http://www.spy-hill.com/help/apple/SharedMemory.html) for documentation for these settings and how to make them permanent. \ No newline at end of file diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index a72087c2..01888935 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -243,9 +243,10 @@ now. It tells you about the current stage, which can be any of: together two random inputs from the queue at some arbitrarily selected midpoint. - sync - a stage used only when `-M` or `-S` is set (see - [parallel_fuzzing.md](parallel_fuzzing.md)). No real fuzzing is involved, but - the tool scans the output from other fuzzers and imports test cases as - necessary. The first time this is done, it may take several minutes or so. + [fuzzing_in_depth.md:3c) Using multiple cores](fuzzing_in_depth.md#c-using-multiple-cores)). + No real fuzzing is involved, but the tool scans the output from other fuzzers + and imports test cases as necessary. The first time this is done, it may take + several minutes or so. The remaining fields should be fairly self-evident: there's the exec count progress indicator for the current stage, a global exec counter, and a benchmark @@ -254,8 +255,8 @@ to another, but the benchmark should be ideally over 500 execs/sec most of the time - and if it stays below 100, the job will probably take very long. The fuzzer will explicitly warn you about slow targets, too. If this happens, -see the [perf_tips.md](perf_tips.md) file included with the fuzzer for ideas on -how to speed things up. +see the [best_practices.md#improving-speed](best_practices.md#improving-speed) +for ideas on how to speed things up. ### Findings in depth @@ -396,7 +397,8 @@ comparing it to the number of logical cores on the system. If the value is shown in green, you are using fewer CPU cores than available on your system and can probably parallelize to improve performance; for tips on how -to do that, see [parallel_fuzzing.md](parallel_fuzzing.md). +to do that, see +[fuzzing_in_depth.md:3c) Using multiple cores](fuzzing_in_depth.md#c-using-multiple-cores). If the value is shown in red, your CPU is *possibly* oversubscribed, and running additional fuzzers may not give you any benefits. diff --git a/docs/env_variables.md b/docs/env_variables.md index 86ebf25c..0952b960 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -583,10 +583,11 @@ The QEMU wrapper used to instrument binary-only code supports several settings: The FRIDA wrapper used to instrument binary-only code supports many of the same options as `afl-qemu-trace`, but also has a number of additional advanced -options. These are listed in brief below (see [here](../frida_mode/README.md) -for more details). These settings are provided for compatibiltiy with QEMU mode, -the preferred way to configure FRIDA mode is through its -[scripting](../frida_mode/Scripting.md) support. +options. These are listed in brief below (see +[frida_mode/README.md](../frida_mode/README.md) for more details). These +settings are provided for compatibiltiy with QEMU mode, the preferred way to +configure FRIDA mode is through its [scripting](../frida_mode/Scripting.md) +support. * `AFL_FRIDA_DEBUG_MAPS` - See `AFL_QEMU_DEBUG_MAPS` * `AFL_FRIDA_DRIVER_NO_HOOK` - See `AFL_QEMU_DRIVER_NO_HOOK`. When using the @@ -627,7 +628,7 @@ the preferred way to configure FRIDA mode is through its coverage information for unstable edges (e.g., to be loaded within IDA lighthouse). * `AFL_FRIDA_JS_SCRIPT` - Set the script to be loaded by the FRIDA scripting - engine. See [here](Scripting.md) for details. + engine. See [frida_mode/Scripting.md](../frida_mode/Scripting.md) for details. * `AFL_FRIDA_OUTPUT_STDOUT` - Redirect the standard output of the target application to the named file (supersedes the setting of `AFL_DEBUG_CHILD`) * `AFL_FRIDA_OUTPUT_STDERR` - Redirect the standard error of the target diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index eaed3a91..fd18b5c1 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -107,10 +107,10 @@ For additional instructions and caveats, see [frida_mode/README.md](../frida_mode/README.md). If possible, you should use the persistent mode, see -[qemu_frida/README.md](../qemu_frida/README.md). The mode is approximately 2-5x -slower than compile-time instrumentation, and is less conducive to -parallelization. But for binary-only fuzzing, it gives a huge speed improvement -if it is possible to use. +[instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md). +The mode is approximately 2-5x slower than compile-time instrumentation, and is +less conducive to parallelization. But for binary-only fuzzing, it gives a huge +speed improvement if it is possible to use. If you want to fuzz a binary-only library, then you can fuzz it with frida-gum via frida_mode/. You will have to write a harness to call the target function in diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 4a1ddf45..29e8f817 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -153,12 +153,12 @@ only instrument parts of the target that you are interested in: There are many more options and modes available, however, these are most of the time less effective. See: -* [instrumentation/README.ctx.md](../instrumentation/README.ctx.md) -* [instrumentation/README.ngram.md](../instrumentation/README.ngram.md) +* [instrumentation/README.llvm.md#6) AFL++ Context Sensitive Branch Coverage](../instrumentation/README.llvm.md#6-afl-context-sensitive-branch-coverage) +* [instrumentation/README.llvm.md#7) AFL++ N-Gram Branch Coverage](../instrumentation/README.llvm.md#7-afl-n-gram-branch-coverage) AFL++ performs "never zero" counting in its bitmap. You can read more about this here: -* [instrumentation/README.neverzero.md](../instrumentation/README.neverzero.md) +* [instrumentation/README.llvm.md#8-neverzero-counters](../instrumentation/README.llvm.md#8-neverzero-counters) ### c) Selecting sanitizers @@ -474,7 +474,8 @@ is: ![resources/screenshot.png](resources/screenshot.png) -All labels are explained in [status_screen.md](status_screen.md). +All labels are explained in +[afl-fuzz_approach.md#understanding-the-status-screen](afl-fuzz_approach.md#understanding-the-status-screen). ### b) Keeping memory use and timeouts in check diff --git a/frida_mode/Scripting.md b/frida_mode/Scripting.md index 63ab1718..ad86fdd3 100644 --- a/frida_mode/Scripting.md +++ b/frida_mode/Scripting.md @@ -109,8 +109,8 @@ Afl.setPersistentAddress(address); A persistent hook can be implemented using a conventional shared object, sample source code for a hook suitable for the prototype of `LLVMFuzzerTestOneInput` -can be found in [hook/hook.c](hook/hook.c). This can be configured using code -similar to the following. +can be found in [hook/](hook/). This can be configured using code similar to the +following. ```js const path = Afl.module.path; diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index fa025643..ca9ce933 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -234,4 +234,45 @@ are 2-16. It is highly recommended to increase the MAP_SIZE_POW2 definition in config.h to at least 18 and maybe up to 20 for this as otherwise too many map collisions -occur. \ No newline at end of file +occur. + +## 8) NeverZero counters + +In larger, complex, or reiterative programs, the byte sized counters that +collect the edge coverage can easily fill up and wrap around. This is not that +much of an issue - unless, by chance, it wraps just to a value of zero when the +program execution ends. In this case, afl-fuzz is not able to see that the edge +has been accessed and will ignore it. + +NeverZero prevents this behavior. If a counter wraps, it jumps over the value 0 +directly to a 1. This improves path discovery (by a very small amount) at a very +low cost (one instruction per edge). + +(The alternative of saturated counters has been tested also and proved to be +inferior in terms of path discovery.) + +This is implemented in afl-gcc and afl-gcc-fast, however, for llvm_mode this is +optional if multithread safe counters are selected or the llvm version is below +9 - as there are severe performance costs in these cases. + +If you want to enable this for llvm versions below 9 or thread safe counters, +then set + +``` +export AFL_LLVM_NOT_ZERO=1 +``` + +In case you are on llvm 9 or greater and you do not want this behavior, then you +can set: + +``` +AFL_LLVM_SKIP_NEVERZERO=1 +``` + +If the target does not have extensive loops or functions that are called a lot +then this can give a small performance boost. + +Please note that the default counter implementations are not thread safe! + +Support for thread safe counters in mode LLVM CLASSIC can be activated with +setting `AFL_LLVM_THREADSAFE_INST=1`. \ No newline at end of file diff --git a/utils/README.md b/utils/README.md index 5f5745b9..debc86e8 100644 --- a/utils/README.md +++ b/utils/README.md @@ -48,7 +48,7 @@ Here's a quick overview of the stuff you can find in this directory: - defork - intercept fork() in targets - distributed_fuzzing - a sample script for synchronizing fuzzer instances - across multiple machines (see parallel_fuzzing.md). + across multiple machines. - libdislocator - like ASAN but lightweight. -- cgit 1.4.1 From 139db40cfc243d2de4eb1584e41867e5a44a06f9 Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Sun, 5 Dec 2021 19:31:43 +0100 Subject: Fix usage and punctuation in connection with "then" and "than" --- docs/fuzzing_binary-only_targets.md | 2 +- docs/fuzzing_in_depth.md | 32 ++++++++++++++++---------------- instrumentation/README.llvm.md | 2 +- utils/afl_network_proxy/README.md | 2 +- utils/afl_untracer/README.md | 2 +- utils/aflpp_driver/README.md | 26 +++++++++++++------------- 6 files changed, 33 insertions(+), 33 deletions(-) (limited to 'utils') diff --git a/docs/fuzzing_binary-only_targets.md b/docs/fuzzing_binary-only_targets.md index fd18b5c1..5434a22c 100644 --- a/docs/fuzzing_binary-only_targets.md +++ b/docs/fuzzing_binary-only_targets.md @@ -172,7 +172,7 @@ Fore more information, see ## Binary rewriters -An alternative solution are binary rewriters. They are faster then the solutions +An alternative solution are binary rewriters. They are faster than the solutions native to AFL++ but don't always work. ### ZAFL diff --git a/docs/fuzzing_in_depth.md b/docs/fuzzing_in_depth.md index 29e8f817..aaceb600 100644 --- a/docs/fuzzing_in_depth.md +++ b/docs/fuzzing_in_depth.md @@ -211,7 +211,7 @@ is more effective). ### d) Modifying the target If the target has features that make fuzzing more difficult, e.g., checksums, -HMAC, etc. then modify the source code so that checks for these values are +HMAC, etc., then modify the source code so that checks for these values are removed. This can even be done safely for source code used in operational products by eliminating these checks within these AFL++ specific blocks: @@ -244,16 +244,16 @@ Then build the target. (Usually with `make`) **NOTES** -1. sometimes configure and build systems are fickle and do not like stderr +1. Sometimes configure and build systems are fickle and do not like stderr output (and think this means a test failure) - which is something AFL++ likes to do to show statistics. It is recommended to disable AFL++ instrumentation reporting via `export AFL_QUIET=1`. -2. sometimes configure and build systems error on warnings - these should be +2. Sometimes configure and build systems error on warnings - these should be disabled (e.g., `--disable-werror` for some configure scripts). -3. in case the configure/build system complains about AFL++'s compiler and - aborts then set `export AFL_NOOPT=1` which will then just behave like the +3. In case the configure/build system complains about AFL++'s compiler and + aborts, then set `export AFL_NOOPT=1` which will then just behave like the real compiler. This option has to be unset again before building the target! #### configure @@ -304,8 +304,8 @@ that you want to fuzz, plus a few specific AFL++ functions around it. See [instrumentation/README.persistent_mode.md](../instrumentation/README.persistent_mode.md) for details. -Basically if you do not fuzz a target in persistent mode then you are just doing -it for a hobby and not professionally :-). +Basically if you do not fuzz a target in persistent mode, then you are just +doing it for a hobby and not professionally :-). ### g) libfuzzer fuzzer harnesses with LLVMFuzzerTestOneInput() @@ -418,9 +418,9 @@ Note that both scripts improve your fuzzing performance but also decrease your system protection against attacks! So set strong firewall rules and only expose SSH as a network service if you use these (which is highly recommended). -If you have an input corpus from step 2 then specify this directory with the -`-i` option. Otherwise create a new directory and create a file with any content -as test data in there. +If you have an input corpus from step 2, then specify this directory with the +`-i` option. Otherwise, create a new directory and create a file with any +content as test data in there. If you do not want anything special, the defaults are already usually best, hence all you need is to specify the seed input directory with the result of @@ -498,8 +498,8 @@ and not waste CPU time. ### c) Using multiple cores -If you want to seriously fuzz then use as many cores/threads as possible to fuzz -your target. +If you want to seriously fuzz, then use as many cores/threads as possible to +fuzz your target. On the same machine - due to the design of how AFL++ works - there is a maximum number of CPU cores/threads that are useful, use more and the overall @@ -658,7 +658,7 @@ An "easy" helper script for this is [https://github.com/vanhauser-thc/afl-cov](https://github.com/vanhauser-thc/afl-cov), just follow the README of that separate project. -If you see that an important area or a feature has not been covered so far then +If you see that an important area or a feature has not been covered so far, then try to find an input that is able to reach that and start a new secondary in that fuzzing campaign with that seed as input, let it run for a few minutes, then terminate it. The main node will pick it up and make it available to the @@ -668,13 +668,13 @@ AFL_TRY_AFFINITY=1` if you have no free core. Note that in nearly all cases you can never reach full coverage. A lot of functionality is usually dependent on exclusive options that would need individual fuzzing campaigns each with one of these options set. E.g., if you -fuzz a library to convert image formats and your target is the png to tiff API +fuzz a library to convert image formats and your target is the png to tiff API, then you will not touch any of the other library APIs and features. ### h) How long to fuzz a target? -This is a difficult question. Basically if no new path is found for a long time -(e.g., for a day or a week) then you can expect that your fuzzing won't be +This is a difficult question. Basically, if no new path is found for a long time +(e.g., for a day or a week), then you can expect that your fuzzing won't be fruitful anymore. However, often this just means that you should switch out secondaries for others, e.g., custom mutator modules, sync to very different fuzzers, etc. diff --git a/instrumentation/README.llvm.md b/instrumentation/README.llvm.md index ca9ce933..d220e52c 100644 --- a/instrumentation/README.llvm.md +++ b/instrumentation/README.llvm.md @@ -269,7 +269,7 @@ can set: AFL_LLVM_SKIP_NEVERZERO=1 ``` -If the target does not have extensive loops or functions that are called a lot +If the target does not have extensive loops or functions that are called a lot, then this can give a small performance boost. Please note that the default counter implementations are not thread safe! diff --git a/utils/afl_network_proxy/README.md b/utils/afl_network_proxy/README.md index c478319a..c34463e2 100644 --- a/utils/afl_network_proxy/README.md +++ b/utils/afl_network_proxy/README.md @@ -1,6 +1,6 @@ # afl-network-proxy -If you want to run afl-fuzz over the network than this is what you need :) +If you want to run afl-fuzz over the network, then this is what you need. :) Note that the impact on fuzzing speed will be huge, expect a loss of 90%. ## When to use this diff --git a/utils/afl_untracer/README.md b/utils/afl_untracer/README.md index cd87998b..da0e0c77 100644 --- a/utils/afl_untracer/README.md +++ b/utils/afl_untracer/README.md @@ -15,7 +15,7 @@ Supported is so far Intel (i386/x86_64) and AARCH64. ### Modify afl-untracer.c -Read and modify afl-untracer.c then `make`. +Read and modify afl-untracer.c, then `make`. To adapt afl-untracer.c to your needs, read the header of the file and then search and edit the `STEP 1`, `STEP 2` and `STEP 3` locations. diff --git a/utils/aflpp_driver/README.md b/utils/aflpp_driver/README.md index 8b9fe15f..c547aaea 100644 --- a/utils/aflpp_driver/README.md +++ b/utils/aflpp_driver/README.md @@ -5,20 +5,20 @@ aflpp_driver is used to compile directly libfuzzer `LLVMFuzzerTestOneInput()` targets. -Just do `afl-clang-fast++ -o fuzz fuzzer_harness.cc libAFLDriver.a [plus required linking]`. +Just do `afl-clang-fast++ -o fuzz fuzzer_harness.cc libAFLDriver.a [plus +required linking]`. -You can also sneakily do this little trick: -If this is the clang compile command to build for libfuzzer: - `clang++ -o fuzz -fsanitize=fuzzer fuzzer_harness.cc -lfoo` -then just switch `clang++` with `afl-clang-fast++` and our compiler will +You can also sneakily do this little trick: If this is the clang compile command +to build for libfuzzer: `clang++ -o fuzz -fsanitize=fuzzer fuzzer_harness.cc +-lfoo`, then just switch `clang++` with `afl-clang-fast++` and our compiler will magically insert libAFLDriver.a :) -To use shared-memory test cases, you need nothing to do. -To use stdin test cases, give `-` as the only command line parameter. -To use file input test cases, give `@@` as the only command line parameter. +To use shared-memory test cases, you need nothing to do. To use stdin test +cases, give `-` as the only command line parameter. To use file input test +cases, give `@@` as the only command line parameter. -IMPORTANT: if you use `afl-cmin` or `afl-cmin.bash` then either pass `-` -or `@@` as command line parameters. +IMPORTANT: if you use `afl-cmin` or `afl-cmin.bash`, then either pass `-` or +`@@` as command line parameters. ## aflpp_qemu_driver @@ -38,6 +38,6 @@ AFL_QEMU_PERSISTENT_HOOK=/path/to/aflpp_qemu_driver_hook.so afl-fuzz -Q ... -- . ``` if you use afl-cmin or `afl-showmap -C` with the aflpp_qemu_driver you need to -set the set same AFL_QEMU_... (or AFL_FRIDA_...) environment variables. -If you want to use afl-showmap (without -C) or afl-cmin.bash then you may not -set these environment variables and rather set `AFL_QEMU_DRIVER_NO_HOOK=1`. \ No newline at end of file +set the set same AFL_QEMU_... (or AFL_FRIDA_...) environment variables. If you +want to use afl-showmap (without -C) or afl-cmin.bash, then you may not set +these environment variables and rather set `AFL_QEMU_DRIVER_NO_HOOK=1`. \ No newline at end of file -- cgit 1.4.1 From bb506de0b809f97a4221ee1b6e040dcb5f9ca56a Mon Sep 17 00:00:00 2001 From: llzmb <46303940+llzmb@users.noreply.github.com> Date: Sun, 5 Dec 2021 19:04:45 +0100 Subject: Fix various missed issues - 1st run --- custom_mutators/gramatron/README.md | 43 ++++----- dictionaries/README.md | 20 ++-- docs/afl-fuzz_approach.md | 11 ++- docs/custom_mutators.md | 2 +- docs/env_variables.md | 2 +- docs/features.md | 4 +- utils/autodict_ql/readme.md | 180 ++++++++++++++++++++++-------------- utils/libdislocator/README.md | 29 +++--- 8 files changed, 168 insertions(+), 123 deletions(-) (limited to 'utils') diff --git a/custom_mutators/gramatron/README.md b/custom_mutators/gramatron/README.md index 5e10f97b..8aa0cc44 100644 --- a/custom_mutators/gramatron/README.md +++ b/custom_mutators/gramatron/README.md @@ -1,17 +1,17 @@ # GramaTron GramaTron is a coverage-guided fuzzer that uses grammar automatons to perform -grammar-aware fuzzing. Technical details about our framework are available -in the [ISSTA'21 paper](https://nebelwelt.net/files/21ISSTA.pdf). -The artifact to reproduce the experiments presented in the paper are present -in `artifact/`. Instructions to run a sample campaign and incorporate new -grammars is presented below: +grammar-aware fuzzing. Technical details about our framework are available in +the [ISSTA'21 paper](https://nebelwelt.net/files/21ISSTA.pdf). The artifact to +reproduce the experiments presented in the paper are present in `artifact/`. +Instructions to run a sample campaign and incorporate new grammars is presented +below: -# Compiling +## Compiling Execute `./build_gramatron_mutator.sh`. -# Running +## Running You have to set the grammar file to use with `GRAMATRON_AUTOMATION`: @@ -23,26 +23,27 @@ export GRAMATRON_AUTOMATION=grammars/ruby/source_automata.json afl-fuzz -i in -o out -- ./target ``` -# Adding and testing a new grammar +## Adding and testing a new grammar -- Specify in a JSON format for CFG. Examples are correspond `source.json` files +- Specify in a JSON format for CFG. Examples are correspond `source.json` files. - Run the automaton generation script (in `src/gramfuzz-mutator/preprocess`) which will place the generated automaton in the same folder. -``` -./preprocess/prep_automaton.sh [stack_limit] + ``` + ./preprocess/prep_automaton.sh [stack_limit] -E.g., ./preprocess/prep_automaton.sh ~/grammars/ruby/source.json PROGRAM -``` + E.g., ./preprocess/prep_automaton.sh ~/grammars/ruby/source.json PROGRAM + ``` -- If the grammar has no self-embedding rules then you do not need to pass the - stack limit parameter. However, if it does have self-embedding rules then you +- If the grammar has no self-embedding rules, then you do not need to pass the + stack limit parameter. However, if it does have self-embedding rules, then you need to pass the stack limit parameter. We recommend starting with `5` and - then increasing it if you need more complexity -- To sanity-check that the automaton is generating inputs as expected you can use the `test` binary housed in `src/gramfuzz-mutator` + then increasing it if you need more complexity. +- To sanity-check that the automaton is generating inputs as expected, you can + use the `test` binary housed in `src/gramfuzz-mutator`. -``` -./test SanityCheck + ``` + ./test SanityCheck -E.g., ./test SanityCheck ~/grammars/ruby/source_automata.json -``` \ No newline at end of file + E.g., ./test SanityCheck ~/grammars/ruby/source_automata.json + ``` \ No newline at end of file diff --git a/dictionaries/README.md b/dictionaries/README.md index f3b8a9e5..0b3b4d90 100644 --- a/dictionaries/README.md +++ b/dictionaries/README.md @@ -2,17 +2,17 @@ (See [../README.md](../README.md) for the general instruction manual.) -This subdirectory contains a set of dictionaries that can be used in -conjunction with the -x option to allow the fuzzer to effortlessly explore the -grammar of some of the more verbose data formats or languages. +This subdirectory contains a set of dictionaries that can be used in conjunction +with the -x option to allow the fuzzer to effortlessly explore the grammar of +some of the more verbose data formats or languages. -These sets were done by Michal Zalewski, various contributors, and imported -from oss-fuzz, go-fuzz and libfuzzer. +These sets were done by Michal Zalewski, various contributors, and imported from +oss-fuzz, go-fuzz and libfuzzer. Custom dictionaries can be added at will. They should consist of a reasonably-sized set of rudimentary syntax units that the fuzzer will then try -to clobber together in various ways. Snippets between 2 and 16 bytes are -usually the sweet spot. +to clobber together in various ways. Snippets between 2 and 16 bytes are usually +the sweet spot. Custom dictionaries can be created in two ways: @@ -34,9 +34,9 @@ In the file mode, every name field can be optionally followed by @, e.g.: `keyword_foo@1 = "foo"` Such entries will be loaded only if the requested dictionary level is equal or -higher than this number. The default level is zero; a higher value can be set -by appending @ to the dictionary file name, like so: +higher than this number. The default level is zero; a higher value can be set by +appending @ to the dictionary file name, like so: `-x path/to/dictionary.dct@2` -Good examples of dictionaries can be found in xml.dict and png.dict. +Good examples of dictionaries can be found in xml.dict and png.dict. \ No newline at end of file diff --git a/docs/afl-fuzz_approach.md b/docs/afl-fuzz_approach.md index 01888935..2da61cc4 100644 --- a/docs/afl-fuzz_approach.md +++ b/docs/afl-fuzz_approach.md @@ -468,7 +468,8 @@ cd ../../ sudo make install ``` -To learn more about remote monitoring and metrics visualization with StatsD, see [rpc_statsd.md](rpc_statsd.md). +To learn more about remote monitoring and metrics visualization with StatsD, see +[rpc_statsd.md](rpc_statsd.md). ### Addendum: status and plot files @@ -524,9 +525,9 @@ into each of them or deploy scripts to read the fuzzer statistics. Using `AFL_STATSD` (and the other related environment variables `AFL_STATSD_HOST`, `AFL_STATSD_PORT`, `AFL_STATSD_TAGS_FLAVOR`) you can automatically send metrics to your favorite StatsD server. Depending on your StatsD server, you will be -able to monitor, trigger alerts, or perform actions based on these metrics (e.g: -alert on slow exec/s for a new build, threshold of crashes, time since last -crash > X, etc.). +able to monitor, trigger alerts, or perform actions based on these metrics +(e.g.: alert on slow exec/s for a new build, threshold of crashes, time since +last crash > X, etc.). The selected metrics are a subset of all the metrics found in the status and in the plot file. The list is the following: `cycle_done`, `cycles_wo_finds`, @@ -537,6 +538,6 @@ the plot file. The list is the following: `cycle_done`, `cycles_wo_finds`, definitions can be found in the addendum above. When using multiple fuzzer instances with StatsD, it is *strongly* recommended -to setup the flavor (AFL_STATSD_TAGS_FLAVOR) to match your StatsD server. This +to setup the flavor (`AFL_STATSD_TAGS_FLAVOR`) to match your StatsD server. This will allow you to see individual fuzzer performance, detect bad ones, see the progress of each strategy... \ No newline at end of file diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index 2f632e1f..7b4e0516 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -276,7 +276,7 @@ gcc -shared -Wall -O3 example.c -o example.so ``` Note that if you specify multiple custom mutators, the corresponding functions -will be called in the order in which they are specified. e.g. first +will be called in the order in which they are specified. E.g., the first `post_process` function of `example_first.so` will be called and then that of `example_second.so`. diff --git a/docs/env_variables.md b/docs/env_variables.md index 0952b960..c45f4ab9 100644 --- a/docs/env_variables.md +++ b/docs/env_variables.md @@ -585,7 +585,7 @@ The FRIDA wrapper used to instrument binary-only code supports many of the same options as `afl-qemu-trace`, but also has a number of additional advanced options. These are listed in brief below (see [frida_mode/README.md](../frida_mode/README.md) for more details). These -settings are provided for compatibiltiy with QEMU mode, the preferred way to +settings are provided for compatibility with QEMU mode, the preferred way to configure FRIDA mode is through its [scripting](../frida_mode/Scripting.md) support. diff --git a/docs/features.md b/docs/features.md index 06b1bcbe..431d9eb1 100644 --- a/docs/features.md +++ b/docs/features.md @@ -1,7 +1,7 @@ # Important features of AFL++ AFL++ supports llvm from 3.8 up to version 12, very fast binary fuzzing with -QEMU 5.1 with laf-intel and redqueen, frida mode, unicorn mode, gcc plugin, full +QEMU 5.1 with laf-intel and redqueen, FRIDA mode, unicorn mode, gcc plugin, full *BSD, Mac OS, Solaris and Android support and much, much, much more. | Feature/Instrumentation | afl-gcc | llvm | gcc_plugin | FRIDA mode(9) | QEMU mode(10) |unicorn_mode(10) |coresight_mode(11)| @@ -30,7 +30,7 @@ QEMU 5.1 with laf-intel and redqueen, frida mode, unicorn mode, gcc plugin, full versions that write to a file to use with afl-fuzz' `-x` 8. the snapshot LKM is currently unmaintained due to too many kernel changes coming too fast :-( -9. frida mode is supported on Linux and MacOS for Intel and ARM +9. FRIDA mode is supported on Linux and MacOS for Intel and ARM 10. QEMU/Unicorn is only supported on Linux 11. Coresight mode is only available on AARCH64 Linux with a CPU with Coresight extension diff --git a/utils/autodict_ql/readme.md b/utils/autodict_ql/readme.md index 789cd152..f61026b7 100644 --- a/utils/autodict_ql/readme.md +++ b/utils/autodict_ql/readme.md @@ -2,21 +2,35 @@ ## What is this? -`Autodict-QL` is a plugin system that enables fast generation of Tokens/Dictionaries in a handy way that can be manipulated by the user (unlike The LLVM Passes that are hard to modify). This means that autodict-ql is a scriptable feature which basically uses CodeQL (a powerful semantic code analysis engine) to fetch information from a code base. +`Autodict-QL` is a plugin system that enables fast generation of +Tokens/Dictionaries in a handy way that can be manipulated by the user (unlike +The LLVM Passes that are hard to modify). This means that autodict-ql is a +scriptable feature which basically uses CodeQL (a powerful semantic code +analysis engine) to fetch information from a code base. -Tokens are useful when you perform fuzzing on different parsers. The AFL++ `-x` switch enables the usage of dictionaries through your fuzzing campaign. If you are not familiar with Dictionaries in fuzzing, take a look [here](https://github.com/AFLplusplus/AFLplusplus/tree/stable/dictionaries) . +Tokens are useful when you perform fuzzing on different parsers. The AFL++ `-x` +switch enables the usage of dictionaries through your fuzzing campaign. If you +are not familiar with Dictionaries in fuzzing, take a look +[here](https://github.com/AFLplusplus/AFLplusplus/tree/stable/dictionaries). -## Why CodeQL ? +## Why CodeQL? -We basically developed this plugin on top of the CodeQL engine because it gives the user scripting features, it's easier and it's independent of the LLVM system. This means that a user can write his CodeQL scripts or modify the current scripts to improve or change the token generation algorithms based on different program analysis concepts. +We basically developed this plugin on top of the CodeQL engine because it gives +the user scripting features, it's easier and it's independent of the LLVM +system. This means that a user can write his CodeQL scripts or modify the +current scripts to improve or change the token generation algorithms based on +different program analysis concepts. ## CodeQL scripts -Currently, we pushed some scripts as defaults for Token generation. In addition, we provide every CodeQL script as an standalone script because it's easier to modify or test. +Currently, we pushed some scripts as defaults for Token generation. In addition, +we provide every CodeQL script as an standalone script because it's easier to +modify or test. -Currently we provided the following CodeQL scripts : +Currently we provided the following CodeQL scripts: -`strcmp-str.ql` is used to extract strings that are related to the `strcmp` function. +`strcmp-str.ql` is used to extract strings that are related to the `strcmp` +function. `strncmp-str.ql` is used to extract the strings from the `strncmp` function. @@ -24,13 +38,18 @@ Currently we provided the following CodeQL scripts : `litool.ql` extracts Magic numbers as Hexadecimal format. -`strtool.ql` extracts strings with uses of a regex and dataflow concept to capture the string comparison functions. If `strcmp` is rewritten in a project as Mystrcmp or something like strmycmp, then this script can catch the arguments and these are valuable tokens. +`strtool.ql` extracts strings with uses of a regex and dataflow concept to +capture the string comparison functions. If `strcmp` is rewritten in a project +as Mystrcmp or something like strmycmp, then this script can catch the arguments +and these are valuable tokens. -You can write other CodeQL scripts to extract possible effective tokens if you think they can be useful. +You can write other CodeQL scripts to extract possible effective tokens if you +think they can be useful. ## Usage -Before you proceed to installation make sure that you have the following packages by installing them: +Before you proceed to installation make sure that you have the following +packages by installing them: ```shell sudo apt install build-essential libtool-bin python3-dev python3 automake git vim wget -y @@ -38,66 +57,91 @@ sudo apt install build-essential libtool-bin python3-dev python3 automake git vi The usage of Autodict-QL is pretty easy. But let's describe it as: -1. First of all, you need to have CodeQL installed on the system. We make this possible with `build-codeql.sh` bash script. This script will install CodeQL completety and will set the required environment variables for your system. -Do the following: - -```shell -# chmod +x codeql-build.sh -# ./codeql-build.sh -# source ~/.bashrc -# codeql -``` - -Then you should get: - -```shell -Usage: codeql ... -Create and query CodeQL databases, or work with the QL language. - -GitHub makes this program freely available for the analysis of open-source software and certain other uses, but it is -not itself free software. Type codeql --license to see the license terms. - - --license Show the license terms for the CodeQL toolchain. -Common options: - -h, --help Show this help text. - -v, --verbose Incrementally increase the number of progress messages printed. - -q, --quiet Incrementally decrease the number of progress messages printed. -Some advanced options have been hidden; try --help -v for a fuller view. -Commands: - query Compile and execute QL code. - bqrs Get information from .bqrs files. - database Create, analyze and process CodeQL databases. - dataset [Plumbing] Work with raw QL datasets. - test Execute QL unit tests. - resolve [Deep plumbing] Helper commands to resolve disk locations etc. - execute [Deep plumbing] Low-level commands that need special JVM options. - version Show the version of the CodeQL toolchain. - generate Generate formatted QL documentation. - github Commands useful for interacting with the GitHub API through CodeQL. -``` - -2. Compile your project with CodeQL: For using the Autodict-QL plugin, you need to compile the source of the target you want to fuzz with CodeQL. This is not something hard. - - First you need to create a CodeQL database of the project codebase, suppose we want to compile `libxml` with codeql. Go to libxml and issue the following commands: - - `./configure --disable-shared` - - `codeql create database libxml-db --language=cpp --command=make` - - Now you have the CodeQL database of the project :-) -3. The final step is to update the CodeQL database you created in step 2 (Suppose we are in `aflplusplus/utils/autodict_ql/` directory): - - `codeql database upgrade /home/user/libxml/libxml-db` +1. First of all, you need to have CodeQL installed on the system. We make this + possible with `build-codeql.sh` bash script. This script will install CodeQL + completety and will set the required environment variables for your system. + Do the following: + + ```shell + # chmod +x codeql-build.sh + # ./codeql-build.sh + # source ~/.bashrc + # codeql + ``` + + Then you should get: + + ```shell + Usage: codeql ... + Create and query CodeQL databases, or work with the QL language. + + GitHub makes this program freely available for the analysis of open-source software and certain other uses, but it is + not itself free software. Type codeql --license to see the license terms. + + --license Show the license terms for the CodeQL toolchain. + Common options: + -h, --help Show this help text. + -v, --verbose Incrementally increase the number of progress messages printed. + -q, --quiet Incrementally decrease the number of progress messages printed. + Some advanced options have been hidden; try --help -v for a fuller view. + Commands: + query Compile and execute QL code. + bqrs Get information from .bqrs files. + database Create, analyze and process CodeQL databases. + dataset [Plumbing] Work with raw QL datasets. + test Execute QL unit tests. + resolve [Deep plumbing] Helper commands to resolve disk locations etc. + execute [Deep plumbing] Low-level commands that need special JVM options. + version Show the version of the CodeQL toolchain. + generate Generate formatted QL documentation. + github Commands useful for interacting with the GitHub API through CodeQL. + ``` + +2. Compile your project with CodeQL: For using the Autodict-QL plugin, you need + to compile the source of the target you want to fuzz with CodeQL. This is not + something hard. + - First you need to create a CodeQL database of the project codebase, suppose + we want to compile `libxml` with codeql. Go to libxml and issue the + following commands: + - `./configure --disable-shared` + - `codeql create database libxml-db --language=cpp --command=make` + - Now you have the CodeQL database of the project :-) +3. The final step is to update the CodeQL database you created in step 2 + (Suppose we are in `aflplusplus/utils/autodict_ql/` directory): + - `codeql database upgrade /home/user/libxml/libxml-db` 4. Everything is set! Now you should issue the following to get the tokens: - - `python3 autodict-ql.py [CURRECT_DIR] [CODEQL_DATABASE_PATH] [TOKEN_PATH]` - - example : `python3 /home/user/AFLplusplus/utils/autodict_ql/autodict-ql.py $PWD /home/user/libxml/libxml-db tokens` - - This will create the final `tokens` dir for you and you are done, then pass the tokens path to AFL++'s `-x` flag. + - `python3 autodict-ql.py [CURRECT_DIR] [CODEQL_DATABASE_PATH] [TOKEN_PATH]` + - example: `python3 /home/user/AFLplusplus/utils/autodict_ql/autodict-ql.py + $PWD /home/user/libxml/libxml-db tokens` + - This will create the final `tokens` dir for you and you are done, then + pass the tokens path to AFL++'s `-x` flag. 5. Done! ## More on dictionaries and tokens -Core developer of the AFL++ project Marc Heuse also developed a similar tool named `dict2file` which is a LLVM pass which can automatically extract useful tokens, in addition with LTO instrumentation mode, this dict2file is automatically generates token extraction. `Autodict-QL` plugin gives you scripting capability and you can do whatever you want to extract from the Codebase and it's up to you. In addition it's independent from LLVM system. -On the other hand, you can also use Google dictionaries which have been made public in May 2020, but the problem of using Google dictionaries is that they are limited to specific file formats and specifications. For example, for testing binutils and ELF file format or AVI in FFMPEG, there are no pre-built dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` features to automatically generate dictionaries based on the target. - -I've personally prefered to use `Autodict-QL` or `dict2file` rather than Google dictionaries or any other manually generated dictionaries as `Autodict-QL` and `dict2file` are working based on the target. -In overall, fuzzing with dictionaries and well-generated tokens will give better results. - -There are 2 important points to remember : - -- If you combine `Autodict-QL` with AFL++ cmplog, you will get much better code coverage and hence better chances to discover new bugs. -- Do not forget to set `AFL_MAX_DET_EXTRAS` at least to the number of generated dictionaries. If you forget to set this environment variable, then AFL++ uses just 200 tokens and use the rest of them only probabilistically. So this will guarantee that your tokens will be used by AFL++. \ No newline at end of file +Core developer of the AFL++ project Marc Heuse also developed a similar tool +named `dict2file` which is a LLVM pass which can automatically extract useful +tokens, in addition with LTO instrumentation mode, this dict2file is +automatically generates token extraction. `Autodict-QL` plugin gives you +scripting capability and you can do whatever you want to extract from the +Codebase and it's up to you. In addition it's independent from LLVM system. On +the other hand, you can also use Google dictionaries which have been made public +in May 2020, but the problem of using Google dictionaries is that they are +limited to specific file formats and specifications. For example, for testing +binutils and ELF file format or AVI in FFMPEG, there are no pre-built +dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` +features to automatically generate dictionaries based on the target. + +I've personally preferred to use `Autodict-QL` or `dict2file` rather than Google +dictionaries or any other manually generated dictionaries as `Autodict-QL` and +`dict2file` are working based on the target. In overall, fuzzing with +dictionaries and well-generated tokens will give better results. + +There are 2 important points to remember: + +- If you combine `Autodict-QL` with AFL++ cmplog, you will get much better code + coverage and hence better chances to discover new bugs. +- Do not forget to set `AFL_MAX_DET_EXTRAS` at least to the number of generated + dictionaries. If you forget to set this environment variable, then AFL++ uses + just 200 tokens and use the rest of them only probabilistically. So this will + guarantee that your tokens will be used by AFL++. \ No newline at end of file diff --git a/utils/libdislocator/README.md b/utils/libdislocator/README.md index 64a5f14c..7150c205 100644 --- a/utils/libdislocator/README.md +++ b/utils/libdislocator/README.md @@ -10,8 +10,8 @@ heap-related security bugs in several ways: subsequent PROT_NONE page, causing most off-by-one reads and writes to immediately segfault, - - It adds a canary immediately below the allocated buffer, to catch writes - to negative offsets (won't catch reads, though), + - It adds a canary immediately below the allocated buffer, to catch writes to + negative offsets (won't catch reads, though), - It sets the memory returned by malloc() to garbage values, improving the odds of crashing when the target accesses uninitialized data, @@ -19,35 +19,34 @@ heap-related security bugs in several ways: - It sets freed memory to PROT_NONE and does not actually reuse it, causing most use-after-free bugs to segfault right away, - - It forces all realloc() calls to return a new address - and sets - PROT_NONE on the original block. This catches use-after-realloc bugs, + - It forces all realloc() calls to return a new address - and sets PROT_NONE + on the original block. This catches use-after-realloc bugs, - - It checks for calloc() overflows and can cause soft or hard failures - of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, + - It checks for calloc() overflows and can cause soft or hard failures of + alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, AFL_LD_HARD_FAIL). - Optionally, in platforms supporting it, huge pages can be used by passing USEHUGEPAGE=1 to make. - - Size alignment to `max_align_t` can be enforced with AFL_ALIGNED_ALLOC=1. - In this case, a tail canary is inserted in the padding bytes at the end - of the allocated zone. This reduce the ability of libdislocator to detect + - Size alignment to `max_align_t` can be enforced with AFL_ALIGNED_ALLOC=1. In + this case, a tail canary is inserted in the padding bytes at the end of the + allocated zone. This reduce the ability of libdislocator to detect off-by-one bugs but also it make slibdislocator compliant to the C standard. Basically, it is inspired by some of the non-default options available for the OpenBSD allocator - see malloc.conf(5) on that platform for reference. It is -also somewhat similar to several other debugging libraries, such as gmalloc -and DUMA - but is simple, plug-and-play, and designed specifically for fuzzing -jobs. +also somewhat similar to several other debugging libraries, such as gmalloc and +DUMA - but is simple, plug-and-play, and designed specifically for fuzzing jobs. Note that it does nothing for stack-based memory handling errors. The -fstack-protector-all setting for GCC / clang, enabled when using AFL_HARDEN, can catch some subset of that. The allocator is slow and memory-intensive (even the tiniest allocation uses up -4 kB of physical memory and 8 kB of virtual mem), making it completely unsuitable -for "production" uses; but it can be faster and more hassle-free than ASAN / MSAN -when fuzzing small, self-contained binaries. +4 kB of physical memory and 8 kB of virtual mem), making it completely +unsuitable for "production" uses; but it can be faster and more hassle-free than +ASAN / MSAN when fuzzing small, self-contained binaries. To use this library, run AFL++ like so: -- cgit 1.4.1