1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
|
# Debugging
If you are using FRIDA mode and have hit some problems, then this guide may help
you to diagnose any problems you are encountering. This assumes you have
followed the [osx-lib](test/osx-lib) example to start fuzzing your target.
It should be noted that attempting to debug code using gdb which has been
instrumented in FRIDA is unlikely to be successful since the debugger will be
inserting breakpoints by patching the code in memory. FRIDA works by reading
this code and generating an instrumented copy to execute. In any case, unless
you are very familiar with the implementation of Stalker, the instrumented code
generated by FRIDA is likely to be very difficult to follow. For this reason,
the following debugging strategies are outlined below.
By convention, all files below should be provided with their path (they are
omitted for readability) and all items in `<braces>` are placeholders and should
be replaced accordingly.
## Select your version
Test with both the `dev` and `stable` branches of AFL++. The `dev` branch should
have the very latest version containing any fixes for identified issues. The
`stable` branch is updated less frequently, but equally might avoid a problem if
a regression has been introduced into the `dev` branch.
## Enable diagnostic information
Run your target specifying the `AFL_DEBUG_CHILD=1` environment variable. This
will print a lot more diagnostic information to the screen when the target
starts up. If you have a simple configuration issue, then you will likely see a
warning or error message in the output.
## Check your test harness
If any of the following steps fail, then there is a problem with your test
harness or your target library. Since this is running without FRIDA mode or
`afl-fuzz` that greatly reduces the search area for your defect. This is why it
is *VERY* important to carry out these basic steps first before taking on the
additional complexity of debugging with FRIDA mode or `afl-fuzz`.
- Run your harness outside of the fuzzer, passing it a representative seed as
it's input `./harness <input>`.
- Pass your harness multiple seeds to check that it is stable when running
multiple tests as it will when running in fork server mode `./harness <input1>
<intput2>`.
- Build your test harness with `CFLAGS=-fsanitize=address` and
`LDFLAGS=-fsanitize=address`. Then run it again with multiple inputs to check
for errors (note that when fuzzing, your harness should not be built with any
sanitizer options).
## Check the samples
FRIDA mode contains a number of different sample targets in the `test` folder.
Have a look through these and find one which is similar to your real target.
Check whether you have any issues running the sample target and make sure you
compare the command line used to launch the sample with the one you are using to
launch your real target very carefully to check for any differences. If
possible, start with one of these samples and gradually make changes one at a
time re-testing as you go until you have migrated it to run your own target.
## FRIDA mode
### Basic
First, just try running your target with `LD_PRELOAD=afl-frida-trace.so
./harness <input>`. An error here means that your defect occurs when running
with just FRIDA mode and isn't related to `afl-fuzz`.
Now you can try commenting out the implementation of `LLVMFuzzerTestOneInput` so
that the harness doesn't actually run your target library. This may also aid in
narrowing down the problem.
```c
int LLVMFuzzerTestOneInput(const unsigned char* data, size_t size){
// fpn_crashme(data, size);
return 0;
}
```
### Persistent mode
If your target is ok running in basic mode, you can try running it in persistent
mode (if that is the configuration you are having issues with) as follows (again
outside of `afl-fuzz`). This time, you will want to run it inside a debugger so
that you can use the debugger to send the `SIGCONT` signals (by continuing)
usually sent by `afl-fuzz` on each iteration.
```bash
gdb \
--ex 'set environment __AFL_PERSISTENT=1' \
--ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \
--ex 'set environment LD_PRELOAD=afl-frida-trace.so' \
--ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=<entry_address>' \
--args ./harness <input>
```
Note:
- You have to manually set the `__AFL_PERSISTENT` environment variable which is
usually passed by `afl-fuzz`.
- Setting breakpoints etc. is likely to interfere with FRIDA and cause spurious
errors.
If this is successful, you can try additionally loading the hook library:
```bash
gdb \
--ex 'set environment __AFL_PERSISTENT=1' \
--ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \
--ex 'set environment LD_PRELOAD=afl-frida-trace.so' \
--ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=<entry_address>' \
--ex 'set environment AFL_FRIDA_PERSISTENT_HOOK=frida_hook.so'
--args ./harness <input>
```
Note that the format of the hook used for FRIDA mode is subtly different to that
used when running in QEMU mode as shown below. Thus the DSO used for the hook is
not interchangeable.
```c
void afl_persistent_hook(GumCpuContext *regs, uint8_t *input_buf,
uint32_t input_buf_len);
void afl_persistent_hook(struct x86_64_regs *regs, uint64_t guest_base,
uint8_t *input_buf, uint32_t input_buf_len);
```
### ASAN
It is also possible to enable ASAN (if that is the configuration you are having
issues with) without having to use `afl-fuzz`. This can be done as follows:
Note:
- The name of the asan DSO may need to be changed depending on your platform.
- The asan DSO must appear first in the `LD_PRELOAD` environment variable.
```bash
LD_PRELOAD=libclang_rt.asan-x86_64.so:afl-frida-trace.so \
ASAN_OPTIONS=detect_leaks=false,halt_on_error=0 \
AFL_USE_FASAN=1 \
./harness <input>
```
Note that care should be taken to ensure that if you set `AFL_INST_LIBS=1`, you
use `AFL_FRIDA_INST_RANGES` or `AFL_FRIDA_EXCLUDE_RANGES` to exclude the ASAN
DSO from coverage. Failure to do so will result in ASAN attempting to sanitize
itself and as a result detecting failures when it attempts to update the shadow
maps.
## Printf
If you have an idea of where things are going wrong for you, then don't be
scared to add `printf` statements to either AFL++ or FRIDA mode itself to show
more diagnostic information. Just be sure to set `AFL_DEBUG=1` and
`AFL_DEBUG_CHILD=1` when you are testing it.
## Core dumps
Lastly, if your defect only occurs when using `afl-fuzz` (e.g., when using
`CMPLOG` which cannot be tested outside of `afl-fuzz` due to its need for a
shared memory mapping being created for it to record its data), it is possible
to enable the creation of a core dump for post-mortem analysis.
Firstly, check if your `/proc/sys/kernel/core_pattern` configuration is set to a
filename (AFL++ encourages you to set it to the value `core` in any case since
it doesn't want any handler applications getting in the way).
Next, set `ulimit -c unlimited` to remove any size limitations for core files.
Lastly, when you `afl-fuzz`, set the environment variable `AFL_DEBUG=1` to
enable the creation of the `core` file. The file should be created in the
working directory of the target application. If there is an existing `core` file
already there, then it may not be overwritten.
## Reach out
Get in touch on discord and ask for help. The groups are pretty active so
someone may well be able to offer some advice. Better still, if you are able to
create a minimal reproducer for your problem, it will make it easier to diagnose
the issue.
|
