From 079f177cdaf43f017bf320912cd97f86dea586be Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sat, 8 Feb 2020 15:41:17 +0100 Subject: persistent mode doc --- qemu_mode/README.persistent.md | 99 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 qemu_mode/README.persistent.md (limited to 'qemu_mode/README.persistent.md') diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md new file mode 100644 index 00000000..6dba5a00 --- /dev/null +++ b/qemu_mode/README.persistent.md @@ -0,0 +1,99 @@ +# How to use the persistent mode in AFL++'s QEMU mode + +## 1) Introduction + +Persistent mode let you fuzz your target persistently between to +addresses - without forking for every fuzzing attempt. +This increases the speed by a factor between x2 and x5, hence it is +very, very valuable. + +The persistent mode is currently only available for x86/x86_64 targets. + + +## 2) How use the persistent mode + +### 2.1) The START address + +The start of the persistent mode has to be set with AFL_QEMU_PERSISTENT_ADDR. + +This address must be at the start of a function or the starting address of +basic block. This (as well as the RET address, see below) has to be defined +in hexadecimal with the 0x prefix. + +If the target is compiled with position independant code (PIE/PIC), you must +add 0x4000000000 to that address, because qemu loads to this base address. + +If this address is not valid, afl-fuzz will error during startup with the +message that the forkserver was not found. + + +### 2.2) the RET address + +The RET address is optional, and only needed if the the return should not be +at the end of the function to which the START address points into, but earlier. + +It is defined by setting AFL_QEMU_PERSISTENT_RET, and too 0x4000000000 has to +be set if the target is position independant. + + +### 2.3) the OFFSET + +If the START address is *not* the beginning of a function, and *no* RET has +been set (so the end of the loop will be at the end of the function), the +ESP pointer very likely has to be reset correctly. + +The value by which the ESP pointer has to be corrected has to set in the +variable AFL_QEMU_PERSISTENT_RETADDR_OFFSET + +Now to get this value right here some help: +1. use gdb on the target +2. set a breakpoint to your START address +3. set a breakpoint to the end of the same function +4. "run" the target with a valid commandline +5. at the first breakpoint print the ESP value with +``` +print $esp +``` +6. "continue" the target until the second breakpoint +7. again print the ESP value +8. calculate the difference between the two values - and this is the offset + + +### 2.4) resetting the register state + +It is very, very likely you need to reste the register state when starting +a new loop. Because of this you 99% of the time should set + +AFL_QEMU_PERSISTENT_GPR=1 + + +## 3) optional parameters + +### 3.1) loop counter value + +The more stable your loop in the target, the longer you can run it, the more +unstable it is the lower the loop count should be. A low value would be 100, +the maximum value should be 10000. The default is 1000. +This value can be set with AFL_QEMU_PERSISTENT_CNT + +This is the same concept as in the llvm_mode persistent mode with __AFL_LOOP(). + + +### 3.2) a hook for in-memory fuzzing + +You can increase the speed of the persistent mode even more by bypassing all +the reading of the fuzzing input via a file by reading directly into the +memory address space of the target process. + +All this needs is that the START address has a register pointing to the +memory buffer, and another register holding the value of the read length +(or pointing to the memory where that value is held). + +If the target reads from an input file you have to supply an input file +that is of least of the size that your fuzzing input will be (and do not +supply @@). + +An example that you can use with little modification for your target can +be found here: [examples/qemu_persistent_hook](../examples/qemu_persistent_hook) +This shared library is specified via AFL_QEMU_PERSISTENT_HOOK + -- cgit 1.4.1 From 49acc388dd9d318cfe7aa7766be7eea0daf2cbf1 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sun, 9 Feb 2020 09:29:56 +0100 Subject: update documentation --- README.md | 3 ++- docs/Changelog.md | 2 ++ qemu_mode/README.md | 31 +++++++++++++++++++++++-------- qemu_mode/README.persistent.md | 3 ++- 4 files changed, 29 insertions(+), 10 deletions(-) (limited to 'qemu_mode/README.persistent.md') diff --git a/README.md b/README.md index e7b216e7..c3e8dc48 100644 --- a/README.md +++ b/README.md @@ -61,9 +61,10 @@ | Feature/Instrumentation | afl-gcc | llvm_mode | gcc_plugin | qemu_mode | unicorn_mode | | ----------------------- |:-------:|:---------:|:----------:|:---------:|:------------:| - | laf-intel / CompCov | | x | | x86/arm | x86/arm | | NeverZero | x | x(1) | (2) | x | x | | Persistent mode | | x | x | x86 | x | + | laf-intel / CompCov | | x | | x86/arm | x86/arm | + | CmpLog | | x | | x | | | Whitelist | | x | x | | | | InsTrim | | x | | | | diff --git a/docs/Changelog.md b/docs/Changelog.md index f2c39e65..8b56603f 100644 --- a/docs/Changelog.md +++ b/docs/Changelog.md @@ -26,7 +26,9 @@ sending a mail to . - LLVM 11 is supported - CmpLog instrumentation using SanCov (see llvm_mode/README.cmplog) - qemu_mode: + - persistent mode is now also available for arm and aarch64 - CmpLog instrumentation for QEMU (-c afl-fuzz command line option) + for x86, x86_64, arm and aarch64 - AFL_PERSISTENT_HOOK callback module for persistent QEMU (see examples/qemu_persistent_hook) - added qemu_mode/README.persistent.md documentation diff --git a/qemu_mode/README.md b/qemu_mode/README.md index 95b75e9c..4198af14 100644 --- a/qemu_mode/README.md +++ b/qemu_mode/README.md @@ -71,7 +71,8 @@ must be an address of a basic block. ## 4) Bonus feature #2: persistent mode -AFL++'s QEMU mode now supports also persistent mode for x86 and x86_64 targets. +AFL++'s QEMU mode now supports also persistent mode for x86, x86_64, arm +and aarch64 targets. This increases the speed by several factors, however it is a bit of work to set up - but worth the effort. @@ -85,6 +86,7 @@ The option that enables QEMU CompareCoverage is AFL_COMPCOV_LEVEL. There is also ./libcompcov/ which implements CompareCoverage for *cmp functions (splitting memcmp, strncmp, etc. to make these conditions easier solvable by afl-fuzz). + AFL_COMPCOV_LEVEL=1 is to instrument comparisons with only immediate values / read-only memory. AFL_COMPCOV_LEVEL=2 instruments all comparison instructions and memory comparison functions when libcompcov @@ -93,11 +95,23 @@ AFL_COMPCOV_LEVEL=3 has the same effects of AFL_COMPCOV_LEVEL=2 but enables also the instrumentation of the floating-point comparisons on x86 and x86_64 (experimental). Integer comparison instructions are currently instrumented only -on the x86, x86_64 and ARM targets. +on the x86, x86_64, arm and aarch64 targets. Highly recommended. -## 6) Bonus feature #4: Wine mode +## 6) CMPLOG mode + +Another new feature is CMPLOG, which is based on the redqueen project. +Here all immidiates in CMP instructions are learned and put into a dynamic +dictionary and applied to all locations in the input that reached that +CMP, trying to solve and pass it. +This is a very effective feature and it is available for x86, x86_64, arm +and aarch64. + +To enable it you must pass on the command line of afl-fuzz: + -c /path/to/your/target + +## 7) Bonus feature #4: Wine mode AFL++ QEMU can use Wine to fuzz WIn32 PE binaries. Use the -W flag of afl-fuzz. @@ -105,7 +119,7 @@ Note that some binaries require user interaction with the GUI and must be patche For examples look [here](https://github.com/andreafioraldi/WineAFLplusplusDEMO). -## 7) Notes on linking +## 8) Notes on linking The feature is supported only on Linux. Supporting BSD may amount to porting the changes made to linux-user/elfload.c and applying them to @@ -126,7 +140,7 @@ practice, this means two things: Setting AFL_INST_LIBS=1 can be used to circumvent the .text detection logic and instrument every basic block encountered. -## 8) Benchmarking +## 9) Benchmarking If you want to compare the performance of the QEMU instrumentation with that of afl-gcc compiled code against the same target, you need to build the @@ -141,7 +155,7 @@ Comparative measurements of execution speed or instrumentation coverage will be fairly meaningless if the optimization levels or instrumentation scopes don't match. -## 9) Gotchas, feedback, bugs +## 10) Gotchas, feedback, bugs If you need to fix up checksums or do other cleanup on mutated test cases, see examples/post_library/ for a viable solution. @@ -162,7 +176,7 @@ with -march=core2, can help. Beyond that, this is an early-stage mechanism, so fields reports are welcome. You can send them to . -## 10) Alternatives: static rewriting +## 11) Alternatives: static rewriting Statically rewriting binaries just once, instead of attempting to translate them at run time, can be a faster alternative. That said, static rewriting is @@ -176,4 +190,5 @@ The best implementation is this one: The issue however is Dyninst which is not rewriting the binaries so that they run stable. A lot of crashes happen, especially in C++ programs that use throw/catch. Try it first, and if it works for you be happy as it is -2-3x as fast as qemu_mode. +2-3x as fast as qemu_mode, however usually not as fast as QEMU persistent mode. + diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md index 6dba5a00..e2e372d8 100644 --- a/qemu_mode/README.persistent.md +++ b/qemu_mode/README.persistent.md @@ -7,7 +7,8 @@ addresses - without forking for every fuzzing attempt. This increases the speed by a factor between x2 and x5, hence it is very, very valuable. -The persistent mode is currently only available for x86/x86_64 targets. +The persistent mode is currently only available for x86/x86_64, arm +and aarch64 targets. ## 2) How use the persistent mode -- cgit 1.4.1 From d84cd978d452fc8ab723aadd30e3db9e33bd7709 Mon Sep 17 00:00:00 2001 From: Andrea Fioraldi Date: Sun, 9 Feb 2020 11:27:49 +0100 Subject: persistent readme --- qemu_mode/README.persistent.md | 87 +++++++++++++++++++++++++++--------------- 1 file changed, 56 insertions(+), 31 deletions(-) (limited to 'qemu_mode/README.persistent.md') diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md index e2e372d8..e4ac5cee 100644 --- a/qemu_mode/README.persistent.md +++ b/qemu_mode/README.persistent.md @@ -10,67 +10,86 @@ very, very valuable. The persistent mode is currently only available for x86/x86_64, arm and aarch64 targets. - ## 2) How use the persistent mode ### 2.1) The START address -The start of the persistent mode has to be set with AFL_QEMU_PERSISTENT_ADDR. +The start of the persistent loop has to be set with AFL_QEMU_PERSISTENT_ADDR. -This address must be at the start of a function or the starting address of -basic block. This (as well as the RET address, see below) has to be defined -in hexadecimal with the 0x prefix. +This address can be the address of whatever instruction but the way in which +you setup persistent mode change if it is the starting instruction of a +function (suggested). This (as well as the RET address, see below) has to be +defined in hexadecimal with the 0x prefix or as a decimal value. If the target is compiled with position independant code (PIE/PIC), you must add 0x4000000000 to that address, because qemu loads to this base address. +On strange setups the base address set by QEMU for PIE executable may change, +you can check it printing the process map using AFL_QEMU_DEBUG_MAPS=1. If this address is not valid, afl-fuzz will error during startup with the message that the forkserver was not found. - ### 2.2) the RET address -The RET address is optional, and only needed if the the return should not be +The RET address is the last instruction of the persistent loop. +The emulator will emit a jump to START when translating the instruction at RET. +It is optional, and only needed if the the return should not be at the end of the function to which the START address points into, but earlier. +It is not set, QEMU will assume that START points to a function and will patch +the return address (on stack or in the link register) to return to START +(like WinAFL). + It is defined by setting AFL_QEMU_PERSISTENT_RET, and too 0x4000000000 has to be set if the target is position independant. - ### 2.3) the OFFSET +This option is for x86 only, arm doesn't save the return address on stack. + If the START address is *not* the beginning of a function, and *no* RET has -been set (so the end of the loop will be at the end of the function), the -ESP pointer very likely has to be reset correctly. +been set (so the end of the loop will be at the end of the function but START +will not be at the beginning), we need an offset from the ESP pointer to locate +the return address to patch. The value by which the ESP pointer has to be corrected has to set in the variable AFL_QEMU_PERSISTENT_RETADDR_OFFSET Now to get this value right here some help: 1. use gdb on the target -2. set a breakpoint to your START address -3. set a breakpoint to the end of the same function +2. set a breakpoint to the function in which START is contained +3. set a breakpoint to your START address 4. "run" the target with a valid commandline -5. at the first breakpoint print the ESP value with -``` -print $esp -``` +5. at the first breakpoint print the ESP value with `p $esp` and take not of it 6. "continue" the target until the second breakpoint 7. again print the ESP value 8. calculate the difference between the two values - and this is the offset - ### 2.4) resetting the register state -It is very, very likely you need to reste the register state when starting -a new loop. Because of this you 99% of the time should set +It is very, very likely you need to restore the general purpose registers state +when starting a new loop. Because of this you 99% of the time should set AFL_QEMU_PERSISTENT_GPR=1 +An example, is when you want to use main() as persistent START: -## 3) optional parameters +```c +int main(int argc, char **argv) { -### 3.1) loop counter value + if (argc < 2) return 1; + + // do stuffs + +} +``` + +If you don't save and restore the registers in x86_64, the paramteter argc +will be lost at the second execution of the loop. + +## 3) Optional parameters + +### 3.1) Loop counter value The more stable your loop in the target, the longer you can run it, the more unstable it is the lower the loop count should be. A low value would be 100, @@ -79,22 +98,28 @@ This value can be set with AFL_QEMU_PERSISTENT_CNT This is the same concept as in the llvm_mode persistent mode with __AFL_LOOP(). - -### 3.2) a hook for in-memory fuzzing +### 3.2) A hook for in-memory fuzzing You can increase the speed of the persistent mode even more by bypassing all the reading of the fuzzing input via a file by reading directly into the memory address space of the target process. -All this needs is that the START address has a register pointing to the -memory buffer, and another register holding the value of the read length -(or pointing to the memory where that value is held). +All this needs is that the START address has a register that can reach the +memory buffer or that the memory buffer is at a know location. You probably need +the value of the size of the buffer (maybe it is in a register when START is +hitted). + +The persistent hook will execute a function on every persistent iteration +(at the start START) defined in a shared object specified with +AFL_QEMU_PERSISTENT_HOOK=/path/to/hook.so. -If the target reads from an input file you have to supply an input file -that is of least of the size that your fuzzing input will be (and do not -supply @@). +The signature is: + +```c +void afl_persistent_hook(uint64_t* regs, uint64_t guest_base); +``` + +In this hook, you can inspect and change the saved GPR state at START. An example that you can use with little modification for your target can be found here: [examples/qemu_persistent_hook](../examples/qemu_persistent_hook) -This shared library is specified via AFL_QEMU_PERSISTENT_HOOK - -- cgit 1.4.1 From a86f740995ffe7c2a456390403d1c78df42d9dcd Mon Sep 17 00:00:00 2001 From: Andrea Fioraldi Date: Sun, 9 Feb 2020 11:31:34 +0100 Subject: typo --- qemu_mode/README.persistent.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'qemu_mode/README.persistent.md') diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md index e4ac5cee..6948c316 100644 --- a/qemu_mode/README.persistent.md +++ b/qemu_mode/README.persistent.md @@ -60,7 +60,7 @@ Now to get this value right here some help: 2. set a breakpoint to the function in which START is contained 3. set a breakpoint to your START address 4. "run" the target with a valid commandline -5. at the first breakpoint print the ESP value with `p $esp` and take not of it +5. at the first breakpoint print the ESP value with `p $esp` and take note of it 6. "continue" the target until the second breakpoint 7. again print the ESP value 8. calculate the difference between the two values - and this is the offset -- cgit 1.4.1 From 34a9419b8990fe36da3148c006f6278b20205e94 Mon Sep 17 00:00:00 2001 From: van Hauser Date: Sun, 9 Feb 2020 12:22:39 +0100 Subject: readme fixes --- PATCHES | 1 - qemu_mode/README.persistent.md | 40 +++++++++++++++++++++++----------------- 2 files changed, 23 insertions(+), 18 deletions(-) delete mode 120000 PATCHES (limited to 'qemu_mode/README.persistent.md') diff --git a/PATCHES b/PATCHES deleted file mode 120000 index b34f8c1d..00000000 --- a/PATCHES +++ /dev/null @@ -1 +0,0 @@ -docs/PATCHES \ No newline at end of file diff --git a/qemu_mode/README.persistent.md b/qemu_mode/README.persistent.md index 6948c316..c96a451b 100644 --- a/qemu_mode/README.persistent.md +++ b/qemu_mode/README.persistent.md @@ -2,7 +2,7 @@ ## 1) Introduction -Persistent mode let you fuzz your target persistently between to +Persistent mode let you fuzz your target persistently between two addresses - without forking for every fuzzing attempt. This increases the speed by a factor between x2 and x5, hence it is very, very valuable. @@ -16,10 +16,12 @@ and aarch64 targets. The start of the persistent loop has to be set with AFL_QEMU_PERSISTENT_ADDR. -This address can be the address of whatever instruction but the way in which -you setup persistent mode change if it is the starting instruction of a -function (suggested). This (as well as the RET address, see below) has to be -defined in hexadecimal with the 0x prefix or as a decimal value. +This address can be the address of whatever instruction. +Setting this address to the start of a function makes the usage simple. +If the address is however within a function, either RET or OFFSET (see below +in 2.2 and 2.3) have to be set. +This address (as well as the RET address, see below) has to be defined in +hexadecimal with the 0x prefix or as a decimal value. If the target is compiled with position independant code (PIE/PIC), you must add 0x4000000000 to that address, because qemu loads to this base address. @@ -36,8 +38,8 @@ The emulator will emit a jump to START when translating the instruction at RET. It is optional, and only needed if the the return should not be at the end of the function to which the START address points into, but earlier. -It is not set, QEMU will assume that START points to a function and will patch -the return address (on stack or in the link register) to return to START +If it is not set, QEMU will assume that START points to a function and will +patch the return address (on stack or in the link register) to return to START (like WinAFL). It is defined by setting AFL_QEMU_PERSISTENT_RET, and too 0x4000000000 has to @@ -45,25 +47,29 @@ be set if the target is position independant. ### 2.3) the OFFSET -This option is for x86 only, arm doesn't save the return address on stack. +This option is valid only for x86/x86_64 only, arm/aarch64 do not save the +return address on stack. If the START address is *not* the beginning of a function, and *no* RET has been set (so the end of the loop will be at the end of the function but START -will not be at the beginning), we need an offset from the ESP pointer to locate -the return address to patch. +will not be at the beginning of it), we need an offset from the ESP pointer +to locate the return address to patch. The value by which the ESP pointer has to be corrected has to set in the variable AFL_QEMU_PERSISTENT_RETADDR_OFFSET Now to get this value right here some help: 1. use gdb on the target -2. set a breakpoint to the function in which START is contained -3. set a breakpoint to your START address -4. "run" the target with a valid commandline -5. at the first breakpoint print the ESP value with `p $esp` and take note of it -6. "continue" the target until the second breakpoint -7. again print the ESP value -8. calculate the difference between the two values - and this is the offset +2. set a breakpoint to "main" (this is required for PIE/PIC binaries so the + addresses are set up) +3. "run" the target with a valid commandline +4. set a breakpoint to the function in which START is contained +5. set a breakpoint to your START address +6. "continue" to the function start breakpoint +6. print the ESP value with `print $esp` and take note of it +7. "continue" the target until the second breakpoint +8. again print the ESP value +9. calculate the difference between the two values - and this is the offset ### 2.4) resetting the register state -- cgit 1.4.1