v3.13c release (#951)

* persistent replay env setup

* implementation without testing

* complete implemenation, still no test

* fix

* fixes

* fixes

* documentation for AFL_PERSISTENT_RECORD

* afl-cmin: Allow @@ to be part of an arg

The previous implementation of "@@ handling"
in afl-cmin differed greatly from how it was
handled in afl-fuzz and how the documentation presented it.

It used to require that the @@ be its own argument separated
by whitespace and could not be used in situations like "--file=@@".

This change standardizes it to just look for @@ to be *in* an argument
in the same manner that afl-cmin.bash does, so that it will have
the expected and documented behavior.

* triage_crashes.sh: Allow @@ to be part of an arg

* triage_crashes.sh: Fix error reporting

* afl-showmap: Allow @@ to be part of an arg

The previous implementation of "@@ handling"
in afl-showmap differed greatly from how it was
handled in afl-fuzz and how the documentation presented it.

It used to require that the @@ be its own argument separated
by whitespace and could not be used in situations like "--file=@@".

This change standardizes it to use detect_file_args() like
everybody else does, so that it will have the expected and
documented behavior.

* afl-showmap: Unwind a change to keep it pre-C99 compatible

* v3.13a init

* ifdef for record

* changelog info

* AFL_PERSISTENT_RECORD not a default

* Add support for FRIDA mode

* support libraries for find_afl_binary

* remove warnings

* update dynamic list

* update changelog

* try to trigger github actions

* try to trigger github actions

* android: support host and target 32bit build

* remove InsTrim

* Fix support for afl-cmin and updated README

* integrate frida_mode, code-format

* update README

* Update custom_mutators.md

* fix compilation for llvm 3.8.0

* pass lib -ldl only on Linux platforms

* typos

* simpler argument processing

* -m32 support for docker container

* restructure havoc

* add introspection

* ensure one fuzzer sync per cycle, cycle introspection

* remove unneeded var

* add parallel builds

* add parallel builds

* Add network_proxy build targets to gitignore (#852)

All other build targets in utils/ are ignored except for these due to
the lack of file extension.

* Fixes: 6d2ac3e3140 ("fix grammar download script")

The git submodle entry point is "grammar_mutator" not "grammar-mutator"

The build script fails without this

* fix #if A == B always evalutes to true

* try to avoid CI build failure by updating apt packages

* fix k-ctx

* Initialalize the autodict-ql

Initialalize the autodict-ql
add codeql scripts

* update the codes, readme

- add readme
- add required qlpack.yml

* update readme

update readme

* Update readme

Update readme

* Update readme

Update readme

* rename python file

rename python file

* update

update

* Add shell command

Add shell command

* update readme

update readme

* Add support for standalone leak-sanitizer, introducting the environment
variable AFL_USE_LSAN.

AFL_USE_LSAN introduces the macro __AFL_CHECK_LEAK() which will check
for a memory leak when the macro is run. This is especially helpful
when using __AFL_LOOP().

If __AFL_LEAK_CHECK() is not used when AFL_USE_LSAN=1 is set,
the leak checker will run when the program exits.

* Replace __AFL_CHECK_LEAK with __AFL_LEAK_CHECK to be more proper.

Fix spelling mistakes.

Correctly call LSAN_ERROR not MSAN_ERROR.

* Some updates on readme

Some updates on readme

* Update readme

Update readme

* Updates

update

* finalize 1

commit final things

* space

space

* remove things

remove things

* Add python scripts

Add python scripts

* Update python scripts

Update python scripts

* new commit - change strings

new commit - change strings

* update qlpack name

update qlpack name

* remove unessential things

remove unessential things from scripts

* remove dirs

remove dirs

* Update readme

Update readme

* Add note

Add note

* Add `

Add `

* change cur

change current dir

* Fix typos,
Use symbolize=0 for LSAN,
Remove syntactic sugar.

* Remove check for exit_code on LSAN and replace it with check for
symbolize=0.

* Move definition of __AFL_LEAK_CHECK inside ifguards,
use LSAN_OPTIONS=print_suppressions=0

* revert Heiko's commit

* Fix Haiku references, no <sys/syscall.h> and missing defines for USEMMAP

* cleanup

* Add -lnetwork to dependencies for Haiku

* fix conflict

* Fix undeclared SYS_write on Haiku

* Declare private api __kern_write for Haiku

* better MacOs msg

* Haiku: create directory for debug_server, if not present

* add missing env

* better understandable directory creation logic

* android: disable sigaction inside debuggerd

check https://github.com/google/AFL/blob/master/docs/INSTALL#L173

* fix forkserver timeout error msg

* removed -lc++ linking for lto

* fix afl_custom_queue_new_entry when syncing

* update grammar-mutator, show better fuzzing strategy yields

* Update ideas.md

Hey, I noticed there was a spelling error in above documentation for GSOC '21. I have corrected it, you can have a look at it if you want.

* display dictionary usage in havoc only mode

* ui custom mutator only display

* add AFL_EXIT_ON_SEED_ISSUES

* afl-whatsup -d

* fix alive count in afl-whatsup

* update havoc

* ui update

* fix aflpp qemu hook

* qemu driver new api

* add readme

* update readme

* allow aflpp_qemu_driver_hook.o to fail

* fix writing stat file on exit

* remove duplicate plot file write

* fix warnings

* afl-whatsup -d fix

* fix ui

* update readme

* qemuafl

* fix compcovtest

* fix compcovtest

* fix compcovtest

* cmplog -l3: disable trimming, forcing input2stage for all

* autoformat with black

* fix nits

* Changes following code review

* fix nits

* update docs

* review

* Add newline

Add newline

* Update readme

fix typo in readme

* Add new line

Add new line

* fixes

* fix compcovtest

* fix compcovtest

* code format for frida mode

* reworked formatting in order to avoid gcc 8.3.0 warnings

* add idea of thread-safe target feedback

* fix-typo: "WIn32" -> "Win32"

* fix custom trim for increasing data

* drop support for llvm < 6.0

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

* Push to stable (#895)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>

* Push to stable (#927)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* push to stable (#931)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* Final push to stable (#936)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

* push to stable (#931) (#932)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* improve error msg

* Added documentation for wine LoadLibrary workaround (#933)

* Fix cmake target compilation command example (#934)

- Fix typo DCMAKE_C_COMPILERC -> DCMAKE_C_COMPILER.
- Add `cd build` after `mkdir build`.

* showmap passes queue items in alphabetical order

* added tmp files to gitignore

* lenient dict parsing, no map size enum for binary fuzzing

* added info about showmap queue directions

* update binary-only doc

* turn off map size detection if skip_bin_check is set

* Typo

* update docs

* update afl-system-config

* Set kill signal before using it in afl-showmap (#935)

* fix afl-cc help output

* add libafl to binary-only doc

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>

* final push to stable (really?) (#939)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

* push to stable (#931) (#932)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* improve error msg

* Added documentation for wine LoadLibrary workaround (#933)

* Fix cmake target compilation command example (#934)

- Fix typo DCMAKE_C_COMPILERC -> DCMAKE_C_COMPILER.
- Add `cd build` after `mkdir build`.

* showmap passes queue items in alphabetical order

* added tmp files to gitignore

* lenient dict parsing, no map size enum for binary fuzzing

* added info about showmap queue directions

* update binary-only doc

* turn off map size detection if skip_bin_check is set

* Typo

* update docs

* update afl-system-config

* Set kill signal before using it in afl-showmap (#935)

* fix afl-cc help output

* add libafl to binary-only doc

* update docs

* less executions on variable paths

* AFL_SKIP_CRASHES is obsolete since 3.0

* add AFL_TRY_AFFINITY

* Typo

* Typo

* Typo/wording

* tweaks

* typos

* fix afl-whatsup help output

* fix afl-plot output

* fix for MacOS

* fix cmpcov doc for qemu

* fix tmpfile removal

* update dockerfile

* Frida (#940)

* Added re2 test

* Added libpcap test

* Fix validation of setting of ADDR_NO_RANDOMIZE

* Added support for printing original and instrumented code

Co-authored-by: Your Name <you@example.com>

* Support for AFL_FRIDA_PERSISTENT_RET (#941)

Co-authored-by: Your Name <you@example.com>

* Changes to add missing exclusion of ranges (#943)

Co-authored-by: Your Name <you@example.com>

* add --afl-noopt to afl-cc

* docs: fix link to README in QuickStartGuide (#946)

* Support writing Stalker stats (#945)

* Support writing Stalker stats

* Fixed string handling in print functions

Co-authored-by: Your Name <you@example.com>

* afl-cmin help fix, aflpp_driver - + @@ support

* fix for afl-showmap

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>
Co-authored-by: Dag Heyman Kajevic <dag.heyman@gmail.com>

* Dev (#949)

* use atomic read-modify-write increment for LLVM CLASSIC

* Change other LLVM modes to atomic increments

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* work in progress: not working correctly yet

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* still not working

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* first working NeverZero implementation

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* add some comments

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

* push to stable (#931) (#932)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* improve error msg

* Added documentation for wine LoadLibrary workaround (#933)

* Fix cmake target compilation command example (#934)

- Fix typo DCMAKE_C_COMPILERC -> DCMAKE_C_COMPILER.
- Add `cd build` after `mkdir build`.

* showmap passes queue items in alphabetical order

* added tmp files to gitignore

* lenient dict parsing, no map size enum for binary fuzzing

* added info about showmap queue directions

* update binary-only doc

* turn off map size detection if skip_bin_check is set

* Typo

* update docs

* update afl-system-config

* Set kill signal before using it in afl-showmap (#935)

* fix afl-cc help output

* add libafl to binary-only doc

* update docs

* less executions on variable paths

* AFL_SKIP_CRASHES is obsolete since 3.0

* add AFL_TRY_AFFINITY

* Typo

* Typo

* Typo/wording

* tweaks

* typos

* fix afl-whatsup help output

* fix afl-plot output

* fix for MacOS

* fix cmpcov doc for qemu

* fix tmpfile removal

* update dockerfile

* Frida (#940)

* Added re2 test

* Added libpcap test

* Fix validation of setting of ADDR_NO_RANDOMIZE

* Added support for printing original and instrumented code

Co-authored-by: Your Name <you@example.com>

* Support for AFL_FRIDA_PERSISTENT_RET (#941)

Co-authored-by: Your Name <you@example.com>

* Changes to add missing exclusion of ranges (#943)

Co-authored-by: Your Name <you@example.com>

* add --afl-noopt to afl-cc

* docs: fix link to README in QuickStartGuide (#946)

* Support writing Stalker stats (#945)

* Support writing Stalker stats

* Fixed string handling in print functions

Co-authored-by: Your Name <you@example.com>

* afl-cmin help fix, aflpp_driver - + @@ support

* fix for afl-showmap

* support new env var AFL_LLVM_THREADSAFE_INST to enable atomic counters.
add new test case for that.

* add documentation for AFL_LLVM_THREADSAFE_INST

* add support for AFL_LLVM_THREADSAFE_INST to other LLVM passes

* add missing include for _exit()

* threadsafe doc fixes, code format

* Wording: "never zero" -> NeverZero

* fix afl_custom_post_process with multiple custom mutators

* fix docs

* debug ck_write

* fixed potential diff by 0

* fixes

* fix classic threadsafe counters

Co-authored-by: van Hauser <vh@thc.org>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>
Co-authored-by: Dag Heyman Kajevic <dag.heyman@gmail.com>

* v3.13c release (#950)

* use atomic read-modify-write increment for LLVM CLASSIC

* Change other LLVM modes to atomic increments

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* work in progress: not working correctly yet

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* still not working

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* first working NeverZero implementation

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* add some comments

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

* push to stable (#931) (#932)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382cab75832721d859c3e731da071435d.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168dda766d2f25f15c15c4d3ecf21d0667.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382cab75832721d859c3e731da071435d commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* improve error msg

* Added documentation for wine LoadLibrary workaround (#933)

* Fix cmake target compilation command example (#934)

- Fix typo DCMAKE_C_COMPILERC -> DCMAKE_C_COMPILER.
- Add `cd build` after `mkdir build`.

* showmap passes queue items in alphabetical order

* added tmp files to gitignore

* lenient dict parsing, no map size enum for binary fuzzing

* added info about showmap queue directions

* update binary-only doc

* turn off map size detection if skip_bin_check is set

* Typo

* update docs

* update afl-system-config

* Set kill signal before using it in afl-showmap (#935)

* fix afl-cc help output

* add libafl to binary-only doc

* update docs

* less executions on variable paths

* AFL_SKIP_CRASHES is obsolete since 3.0

* add AFL_TRY_AFFINITY

* Typo

* Typo

* Typo/wording

* tweaks

* typos

* fix afl-whatsup help output

* fix afl-plot output

* fix for MacOS

* fix cmpcov doc for qemu

* fix tmpfile removal

* update dockerfile

* Frida (#940)

* Added re2 test

* Added libpcap test

* Fix validation of setting of ADDR_NO_RANDOMIZE

* Added support for printing original and instrumented code

Co-authored-by: Your Name <you@example.com>

* Support for AFL_FRIDA_PERSISTENT_RET (#941)

Co-authored-by: Your Name <you@example.com>

* Changes to add missing exclusion of ranges (#943)

Co-authored-by: Your Name <you@example.com>

* add --afl-noopt to afl-cc

* docs: fix link to README in QuickStartGuide (#946)

* Support writing Stalker stats (#945)

* Support writing Stalker stats

* Fixed string handling in print functions

Co-authored-by: Your Name <you@example.com>

* afl-cmin help fix, aflpp_driver - + @@ support

* fix for afl-showmap

* support new env var AFL_LLVM_THREADSAFE_INST to enable atomic counters.
add new test case for that.

* add documentation for AFL_LLVM_THREADSAFE_INST

* add support for AFL_LLVM_THREADSAFE_INST to other LLVM passes

* add missing include for _exit()

* threadsafe doc fixes, code format

* Wording: "never zero" -> NeverZero

* fix afl_custom_post_process with multiple custom mutators

* fix docs

* debug ck_write

* fixed potential diff by 0

* fixes

* fix classic threadsafe counters

* v3.13c release

Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>
Co-authored-by: Dag Heyman Kajevic <dag.heyman@gmail.com>

Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Jiangen Jiao <joeyjiaojg@qq.com>
Co-authored-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: R. Elliott Childre <elliottchildre329@gmail.com>
Co-authored-by: microsvuln <55649192+Microsvuln@users.noreply.github.com>
Co-authored-by: Joshua Rogers <jrogers@opera.com>
Co-authored-by: begasus <begasus@gmail.com>
Co-authored-by: Ujjwal Kirti <64329707+ujjwalkirti@users.noreply.github.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: veritas501 <hxzene@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>
Co-authored-by: Dag Heyman Kajevic <dag.heyman@gmail.com>

1 files changed, 100 insertions, 0 deletions

diff --git a/utils/autodict_ql/readme.md b/utils/autodict_ql/readme.md
new file mode 100644
index 00000000..9170f552
--- /dev/null
+++ b/utils/autodict_ql/readme.md
@@ -0,0 +1,100 @@
+# Autodict-QL - Optimal Token Generation for Fuzzing
+
+## What is this?
+
+`Autodict-QL` is a plugin system that enables fast generation of Tokens/Dictionaries in a handy way that can be manipulated by the user (unlike The LLVM Passes that are hard to modify). This means that autodict-ql is a scriptable feature which basically uses CodeQL (a powerful semantic code analysis engine) to fetch information from a code base.
+
+Tokens are useful when you perform fuzzing on different parsers. The AFL++ `-x` switch enables the usage of dictionaries through your fuzzing campaign. If you are not familiar with Dictionaries in fuzzing, take a look [here](https://github.com/AFLplusplus/AFLplusplus/tree/stable/dictionaries) .
+
+
+## Why CodeQL ?
+We basically developed this plugin on top of the CodeQL engine because it gives the user scripting features, it's easier and it's independent of the LLVM system. This means that a user can write his CodeQL scripts or modify the current scripts to improve or change the token generation algorithms based on different program analysis concepts.
+
+
+## CodeQL scripts
+Currently, we pushed some scripts as defaults for Token generation. In addition, we provide every CodeQL script as an standalone script because it's easier to modify or test.
+
+Currently we provided the following CodeQL scripts :
+
+`strcmp-str.ql` is used to extract strings that are related to the `strcmp` function.
+
+`strncmp-str.ql` is used to extract the strings from the `strncmp` function.
+
+`memcmp-str.ql` is used to extract the strings from the `memcmp` function.
+
+`litool.ql` extracts Magic numbers as Hexadecimal format.
+
+`strtool.ql` extracts strings with uses of a regex and dataflow concept to capture the string comparison functions. If `strcmp` is rewritten in a project as Mystrcmp or something like strmycmp, then this script can catch the arguments and these are valuable tokens.
+
+You can write other CodeQL scripts to extract possible effective tokens if you think they can be useful.
+
+
+## Usage
+
+Before you proceed to installation make sure that you have the following packages by installing them :
+```shell
+sudo apt install build-essential libtool-bin python3-dev python3 automake git vim wget -y
+```
+The usage of Autodict-QL is pretty easy. But let's describe it as:
+
+1. First of all, you need to have CodeQL installed on the system. we make this possible with `build-codeql.sh` bash script. This script will install CodeQL completety and will set the required environment variables for your system.
+Do the following :
+```shell
+# chmod +x codeql-build.sh
+# ./codeql-build.sh
+# source ~/.bashrc
+# codeql 
+```
+Then you should get:
+
+```shell
+Usage: codeql <command> <argument>...
+Create and query CodeQL databases, or work with the QL language.
+
+GitHub makes this program freely available for the analysis of open-source software and certain other uses, but it is
+not itself free software. Type codeql --license to see the license terms.
+
+      --license              Show the license terms for the CodeQL toolchain.
+Common options:
+  -h, --help                 Show this help text.
+  -v, --verbose              Incrementally increase the number of progress messages printed.
+  -q, --quiet                Incrementally decrease the number of progress messages printed.
+Some advanced options have been hidden; try --help -v for a fuller view.
+Commands:
+  query     Compile and execute QL code.
+  bqrs      Get information from .bqrs files.
+  database  Create, analyze and process CodeQL databases.
+  dataset   [Plumbing] Work with raw QL datasets.
+  test      Execute QL unit tests.
+  resolve   [Deep plumbing] Helper commands to resolve disk locations etc.
+  execute   [Deep plumbing] Low-level commands that need special JVM options.
+  version   Show the version of the CodeQL toolchain.
+  generate  Generate formatted QL documentation.
+  github    Commands useful for interacting with the GitHub API through CodeQL.
+```
+
+2. Compile your project with CodeQL: For using the Autodict-QL plugin, you need to compile the source of the target you want to fuzz with CodeQL. This is not something hard.
+	- First you need to create a CodeQL database of the project codebase, suppose we want to compile `libxml` with codeql. Go to libxml and issue the following commands:
+		- `./configure --disable-shared`
+		- `codeql create database libxml-db --language=cpp --command=make`
+			- Now you have the CodeQL database of the project :-)
+3. The final step is to update the CodeQL database you created in step 2 (Suppose we are in `aflplusplus/utils/autodict_ql/` directory):
+	- `codeql database upgrade /home/user/libxml/libxml-db`
+4. Everything is set! Now you should issue the following to get the tokens:
+	- `python3 autodict-ql.py [CURRECT_DIR] [CODEQL_DATABASE_PATH] [TOKEN_PATH]`
+		- example : `python3 /home/user/AFLplusplus/utils/autodict_ql/autodict-ql.py $PWD /home/user/libxml/libxml-db tokens`
+			- This will create the final `tokens` dir for you and you are done, then pass the tokens path to AFL++'s `-x` flag.
+5. Done! 
+
+
+## More on dictionaries and tokens
+Core developer of the AFL++ project Marc Heuse also developed a similar tool named `dict2file` which is a LLVM pass which can automatically extract useful tokens, in addition with LTO instrumentation mode, this dict2file is automatically generates token extraction. `Autodict-QL` plugin gives you scripting capability and you can do whatever you want to extract from the Codebase and it's up to you. In addition it's independent from LLVM system.
+On the other hand, you can also use Google dictionaries which have been made public in May 2020, but the problem of using Google dictionaries is that they are limited to specific file formats and specifications. For example, for testing binutils and ELF file format or AVI in FFMPEG, there are no prebuilt dictionaries, so it is highly recommended to use `Autodict-QL` or `Dict2File` features to automatically generate dictionaries based on the target.
+
+I've personally prefered to use `Autodict-QL` or `dict2file` rather than Google dictionaries or any other manually generated dictionaries as `Autodict-QL` and `dict2file` are working based on the target.
+In overall, fuzzing with dictionaries and well-generated tokens will give better results.
+
+There are 2 important points to remember :
+
+- If you combine `Autodict-QL` with AFL++ cmplog, you will get much better code coverage and hence better chances to discover new bugs.
+- Do not forget to set `AFL_MAX_DET_EXTRAS` at least to the number of generated dictionaries. If you forget to set this environment variable, then AFL++ uses just 200 tokens and use the rest of them only probabilistically. So this will guarantee that your tokens will be used by AFL++.