diff options
author | van Hauser <vh@thc.org> | 2021-12-07 15:18:32 +0100 |
---|---|---|
committer | GitHub <noreply@github.com> | 2021-12-07 15:18:32 +0100 |
commit | 1f6c72ea1baea69b2dc5b3a68bfacbc00652bc66 (patch) | |
tree | a5a7ed81710c1dec50f0aa661b53c0cd884a4da2 /docs/custom_mutators.md | |
parent | 5469112db90741cb06c0979313938d83e63f793f (diff) | |
parent | bb506de0b809f97a4221ee1b6e040dcb5f9ca56a (diff) | |
download | afl++-1f6c72ea1baea69b2dc5b3a68bfacbc00652bc66.tar.gz |
Merge pull request #1191 from llzmb/docs_quality_assurance
Docs content - quality assurance
Diffstat (limited to 'docs/custom_mutators.md')
-rw-r--r-- | docs/custom_mutators.md | 121 |
1 files changed, 66 insertions, 55 deletions
diff --git a/docs/custom_mutators.md b/docs/custom_mutators.md index b1dfd309..7b4e0516 100644 --- a/docs/custom_mutators.md +++ b/docs/custom_mutators.md @@ -1,16 +1,16 @@ # Custom Mutators in AFL++ This file describes how you can implement custom mutations to be used in AFL. -For now, we support C/C++ library and Python module, collectivelly named as the +For now, we support C/C++ library and Python module, collectively named as the custom mutator. -There is also experimental support for Rust in `custom_mutators/rust`. -Please refer to that directory for documentation. -Run ```cargo doc -p custom_mutator --open``` in that directory to view the -documentation in your web browser. +There is also experimental support for Rust in `custom_mutators/rust`. For +documentation, refer to that directory. Run `cargo doc -p custom_mutator --open` +in that directory to view the documentation in your web browser. Implemented by -- C/C++ library (`*.so`): Khaled Yakdan from Code Intelligence (<yakdan@code-intelligence.de>) +- C/C++ library (`*.so`): Khaled Yakdan from Code Intelligence + (<yakdan@code-intelligence.de>) - Python module: Christian Holler from Mozilla (<choller@mozilla.com>) ## 1) Introduction @@ -21,13 +21,17 @@ fuzzing by using libraries that perform mutations according to a given grammar. The custom mutator is passed to `afl-fuzz` via the `AFL_CUSTOM_MUTATOR_LIBRARY` or `AFL_PYTHON_MODULE` environment variable, and must export a fuzz function. -Now AFL also supports multiple custom mutators which can be specified in the same `AFL_CUSTOM_MUTATOR_LIBRARY` environment variable like this. +Now AFL++ also supports multiple custom mutators which can be specified in the +same `AFL_CUSTOM_MUTATOR_LIBRARY` environment variable like this. + ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="full/path/to/mutator_first.so;full/path/to/mutator_second.so" ``` -Please see [APIs](#2-apis) and [Usage](#3-usage) for detail. -The custom mutation stage is set to be the first non-deterministic stage (right before the havoc stage). +For details, see [APIs](#2-apis) and [Usage](#3-usage). + +The custom mutation stage is set to be the first non-deterministic stage (right +before the havoc stage). Note: If `AFL_CUSTOM_MUTATOR_ONLY` is set, all mutations will solely be performed with the custom mutator. @@ -35,6 +39,7 @@ performed with the custom mutator. ## 2) APIs C/C++: + ```c void *afl_custom_init(afl_state_t *afl, unsigned int seed); unsigned int afl_custom_fuzz_count(void *data, const unsigned char *buf, size_t buf_size); @@ -53,6 +58,7 @@ void afl_custom_deinit(void *data); ``` Python: + ```python def init(seed): pass @@ -101,7 +107,8 @@ def deinit(): # optional for Python - `init`: - This method is called when AFL++ starts up and is used to seed RNG and set up buffers and state. + This method is called when AFL++ starts up and is used to seed RNG and set + up buffers and state. - `queue_get` (optional): @@ -110,27 +117,26 @@ def deinit(): # optional for Python - `fuzz_count` (optional): - When a queue entry is selected to be fuzzed, afl-fuzz selects the number - of fuzzing attempts with this input based on a few factors. - If however the custom mutator wants to set this number instead on how often - it is called for a specific queue entry, use this function. - This function is most useful if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. + When a queue entry is selected to be fuzzed, afl-fuzz selects the number of + fuzzing attempts with this input based on a few factors. If, however, the + custom mutator wants to set this number instead on how often it is called + for a specific queue entry, use this function. This function is most useful + if `AFL_CUSTOM_MUTATOR_ONLY` is **not** used. - `fuzz` (optional): This method performs custom mutations on a given input. It also accepts an - additional test case. - Note that this function is optional - but it makes sense to use it. - You would only skip this if `post_process` is used to fix checksums etc. - so if you are using it e.g. as a post processing library. + additional test case. Note that this function is optional - but it makes + sense to use it. You would only skip this if `post_process` is used to fix + checksums etc. so if you are using it, e.g., as a post processing library. Note that a length > 0 *must* be returned! - `describe` (optional): When this function is called, it shall describe the current test case, - generated by the last mutation. This will be called, for example, - to name the written test case file after a crash occurred. - Using it can help to reproduce crashing mutations. + generated by the last mutation. This will be called, for example, to name + the written test case file after a crash occurred. Using it can help to + reproduce crashing mutations. - `havoc_mutation` and `havoc_mutation_probability` (optional): @@ -142,21 +148,21 @@ def deinit(): # optional for Python - `post_process` (optional): For some cases, the format of the mutated data returned from the custom - mutator is not suitable to directly execute the target with this input. - For example, when using libprotobuf-mutator, the data returned is in a - protobuf format which corresponds to a given grammar. In order to execute - the target, the protobuf data must be converted to the plain-text format - expected by the target. In such scenarios, the user can define the - `post_process` function. This function is then transforming the data into the - format expected by the API before executing the target. + mutator is not suitable to directly execute the target with this input. For + example, when using libprotobuf-mutator, the data returned is in a protobuf + format which corresponds to a given grammar. In order to execute the target, + the protobuf data must be converted to the plain-text format expected by the + target. In such scenarios, the user can define the `post_process` function. + This function is then transforming the data into the format expected by the + API before executing the target. This can return any python object that implements the buffer protocol and supports PyBUF_SIMPLE. These include bytes, bytearray, etc. - `queue_new_entry` (optional): - This methods is called after adding a new test case to the queue. - If the contents of the file was changed return True, False otherwise. + This methods is called after adding a new test case to the queue. If the + contents of the file was changed, return True, False otherwise. - `introspection` (optional): @@ -168,8 +174,8 @@ def deinit(): # optional for Python The last method to be called, deinitializing the state. -Note that there are also three functions for trimming as described in the -next section. +Note that there are also three functions for trimming as described in the next +section. ### Trimming Support @@ -177,8 +183,8 @@ The generic trimming routines implemented in AFL++ can easily destroy the structure of complex formats, possibly leading to a point where you have a lot of test cases in the queue that your Python module cannot process anymore but your target application still accepts. This is especially the case when your -target can process a part of the input (causing coverage) and then errors out -on the remaining input. +target can process a part of the input (causing coverage) and then errors out on +the remaining input. In such cases, it makes sense to implement a custom trimming routine. The API consists of multiple methods because after each trimming step, we have to go @@ -189,8 +195,9 @@ trimmed input. Here's a quick API description: This method is called at the start of each trimming operation and receives the initial buffer. It should return the amount of iteration steps possible - on this input (e.g. if your input has n elements and you want to remove them - one by one, return n, if you do a binary search, return log(n), and so on). + on this input (e.g., if your input has n elements and you want to remove + them one by one, return n, if you do a binary search, return log(n), and so + on). If your trimming algorithm doesn't allow to determine the amount of (remaining) steps easily (esp. while running), then you can alternatively @@ -202,21 +209,21 @@ trimmed input. Here's a quick API description: - `trim` (optional) This method is called for each trimming operation. It doesn't have any - arguments because we already have the initial buffer from `init_trim` and we - can memorize the current state in the data variables. This can also save + arguments because there is already the initial buffer from `init_trim` and + we can memorize the current state in the data variables. This can also save reparsing steps for each iteration. It should return the trimmed input buffer. - `post_trim` (optional) This method is called after each trim operation to inform you if your - trimming step was successful or not (in terms of coverage). If you receive - a failure here, you should reset your input to the last known good state. - In any case, this method must return the next trim iteration index (from 0 - to the maximum amount of steps you returned in `init_trim`). + trimming step was successful or not (in terms of coverage). If you receive a + failure here, you should reset your input to the last known good state. In + any case, this method must return the next trim iteration index (from 0 to + the maximum amount of steps you returned in `init_trim`). Omitting any of three trimming methods will cause the trimming to be disabled -and trigger a fallback to the builtin default trimming routine. +and trigger a fallback to the built-in default trimming routine. ### Environment Variables @@ -224,11 +231,10 @@ Optionally, the following environment variables are supported: - `AFL_CUSTOM_MUTATOR_ONLY` - Disable all other mutation stages. This can prevent broken test cases - (those that your Python module can't work with anymore) to fill up your - queue. Best combined with a custom trimming routine (see below) because - trimming can cause the same test breakage like havoc and splice. - + Disable all other mutation stages. This can prevent broken test cases (those + that your Python module can't work with anymore) to fill up your queue. Best + combined with a custom trimming routine (see below) because trimming can + cause the same test breakage like havoc and splice. - `AFL_PYTHON_ONLY` @@ -264,22 +270,27 @@ In case your setup is different, set the necessary variables like this: ### Custom Mutator Preparation For C/C++ mutators, the source code must be compiled as a shared object: + ```bash gcc -shared -Wall -O3 example.c -o example.so ``` -Note that if you specify multiple custom mutators, the corresponding functions will -be called in the order in which they are specified. e.g first `post_process` function of -`example_first.so` will be called and then that of `example_second.so`. + +Note that if you specify multiple custom mutators, the corresponding functions +will be called in the order in which they are specified. E.g., the first +`post_process` function of `example_first.so` will be called and then that of +`example_second.so`. ### Run C/C++ + ```bash export AFL_CUSTOM_MUTATOR_LIBRARY="/full/path/to/example_first.so;/full/path/to/example_second.so" afl-fuzz /path/to/program ``` Python + ```bash export PYTHONPATH=`dirname /full/path/to/example.py` export AFL_PYTHON_MODULE=example @@ -288,8 +299,8 @@ afl-fuzz /path/to/program ## 4) Example -Please see [example.c](../custom_mutators/examples/example.c) and -[example.py](../custom_mutators/examples/example.py) +See [example.c](../custom_mutators/examples/example.c) and +[example.py](../custom_mutators/examples/example.py). ## 5) Other Resources @@ -297,4 +308,4 @@ Please see [example.c](../custom_mutators/examples/example.c) and - [bruce30262/libprotobuf-mutator_fuzzing_learning](https://github.com/bruce30262/libprotobuf-mutator_fuzzing_learning/tree/master/4_libprotobuf_aflpp_custom_mutator) - [thebabush/afl-libprotobuf-mutator](https://github.com/thebabush/afl-libprotobuf-mutator) - [XML Fuzzing@NullCon 2017](https://www.agarri.fr/docs/XML_Fuzzing-NullCon2017-PUBLIC.pdf) - - [A bug detected by AFL + XML-aware mutators](https://bugs.chromium.org/p/chromium/issues/detail?id=930663) + - [A bug detected by AFL + XML-aware mutators](https://bugs.chromium.org/p/chromium/issues/detail?id=930663) \ No newline at end of file |