push to stable (#1983)

* Output afl-clang-fast stuffs only if necessary (#1912)

* afl-cc header

* afl-cc common declarations

 - Add afl-cc-state.c
 - Strip includes, find_object, debug/be_quiet/have_*/callname setting from afl-cc.c
 - Use debugf_args in main
 - Modify execvp stuffs to fit new aflcc struct

* afl-cc show usage

* afl-cc mode selecting

1. compiler_mode by callname in argv[0]
2. compiler_mode by env "AFL_CC_COMPILER"
3. compiler_mode/instrument_mode by command line options "--afl-..."
4. instrument_mode/compiler_mode by various env vars including "AFL_LLVM_INSTRUMENT"
5. final checking steps
6. print "... - mode: %s-%s\n"
7. determine real argv[0] according to compiler_mode

* afl-cc macro defs

* afl-cc linking behaviors

* afl-cc fsanitize behaviors

* afl-cc misc

* afl-cc body update

* afl-cc all-in-one

formated with custom-format.py

* nits

---------

Co-authored-by: vanhauser-thc <vh@thc.org>

* changelog

* update grammar mutator

* lto llvm 12+

* docs(custom_mutators): fix missing ':' (#1953)

* Fix broken LTO mode and response file support (#1948)

* Strip `-Wl,-no-undefined` during compilation (#1952)

Make the compiler wrapper stripping `-Wl,-no-undefined` in addition to `-Wl,--no-undefined`.
Both versions of the flag are accepted by clang and, therefore, used by building systems in the wild (e.g., samba will not build without this fix).

* Remove dead code in write_to_testcase (#1955)

The custom_mutators_count check in if case is duplicate with if condition.
The else case is custom_mutators_count == 0, neither custom_mutator_list iteration nor sent check needed.

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>

* update qemuafl

* WIP: Add ability to generate drcov trace using QEMU backend (#1956)

* Document new drcov QEMU plugin

* Add link to lightkeeper for QEMU drcov file loading

---------

Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>

* code format

* changelog

* sleep on uid != 0 afl-system-config

* fix segv about skip_next, warn on unsupported cases of linking options (#1958)

* todos

* ensure afl-cc only allows available compiler modes

* update grammar mutator

* disable aslr on apple

* fix for arm64

* help selective instrumentation

* typos

* macos

* add compiler test script

* apple fixes

* bump nyx submodules (#1963)

* fix docs

* update changelog

* update grammar mutator

* improve compiler test script

* gcc asan workaround (#1966)

* fix github merge fuckup

* fix

* Fix afl-cc (#1968)

- Check if too many cmdline params here, each time before insert a new param.
 - Check if it is "-fsanitize=..." before we do sth.
 - Remove improper param_st transfer.

* Avoid adding llvmnative instrumentation when linking rust sanitizer runtime (#1969)

* Dynamic instrumentation filtering for LLVM native (#1971)

* Add two dynamic instrumentation filter methods to runtime

* Always use pc-table with native pcguard

* Add make_symbol_list.py and README

* changelog

* todos

* new forkserver check

* fix

* nyx test for CI

* improve nyx docs

* Fixes to afl-cc and documentation (#1974)

* Always compile with -ldl when building for CODE_COVERAGE

When building with CODE_COVERAGE, the afl runtime contains code that
calls `dladdr` which requires -ldl. Under most circumstances, clang
already adds this (e.g. when building with pc-table), but there are some
circumstances where it isn't added automatically.

* Add visibility declaration to __afl_connected

When building with hidden visibility, the use of __AFL_LOOP inside such
code can cause linker errors due to __afl_connected being declared
"hidden".

* Update docs to clarify that CODE_COVERAGE=1 is required for dynamic_covfilter

* nits

* nyx build script updates

* test error output

* debug ci

* debug ci

* Improve afl-cc (#1975)

* update response file support

 - full support of rsp file
 - fix some segv issues

* Improve afl-cc

 - remove dead code about allow/denylist options of sancov
 - missing `if (!aflcc->have_msan)`
 - add docs for each function
 - typo

* enable nyx

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* fix ci

* clean test script

* NO_NYX

* NO_NYX

* fix ci

* debug ci

* fix ci

* finalize ci fix

* Enhancement on Deterministic stage (#1972)

* fuzzer: init commit based on aflpp 60dc37a8cf09f8e9048e4b6a2204d6c90b27655a

* fuzzers: adding the skip variables and initialize

* log: profile the det/havoc finding

* log: add profile log output

* fuzzers: sperate log/skipdet module

* fuzzers: add quick eff_map calc

* fuzzers: add skip_eff_map in fuzz_one

* fuzzers: mark whole input space in eff_map

* fuzzers: add undet bit threshold to skip some seeds

* fuzzers: fix one byte overflow

* fuzzers: fix overflow

* fix code format

* add havoc only again

* code format

* remove log to INTROSPECTION, rename skipdet module

* rename skipdet module

* remove log to stats

* clean redundant code

* code format

* remove redundant code format check

* remove redundant doc

* remove redundant objects

* clean files

* change -d to default skipdet

* disable deterministic when using CUSTOM_MUTATOR

* revert fix

* final touches for skipdet

* remove unused var

* remove redundant eff struct (#1977)

* update QEMU-Nyx submodule (#1978)

* update QEMU-Nyx submodule (#1980)

* Fix type in AFL_NOOPT env variable in afl-cc help message (#1982)

* nits

* 2024 v4.10c release

* fixes

---------

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com>
Co-authored-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Nils Bars <nils.bars@rub.de>
Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com>
Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>
Co-authored-by: Sergej Schumilo <sergej@schumilo.de>
Co-authored-by: Christian Holler (:decoder) <choller@mozilla.com>
Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com>
Co-authored-by: Khaled Yakdan <yakdan@code-intelligence.com>

1 files changed, 59 insertions, 1 deletions

diff --git a/include/afl-fuzz.h b/include/afl-fuzz.h
index f1813df6..c24f39e2 100644
--- a/include/afl-fuzz.h
+++ b/include/afl-fuzz.h
@@ -10,7 +10,7 @@
                      Dominik Maier <mail@dmnk.co>
 
    Copyright 2016, 2017 Google Inc. All rights reserved.
-   Copyright 2019-2023 AFLplusplus Project. All rights reserved.
+   Copyright 2019-2024 AFLplusplus Project. All rights reserved.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
@@ -149,6 +149,48 @@ struct tainted {
 
 };
 
+struct inf_profile {
+
+  u32 inf_skipped_bytes;               /* Inference Stage Profiling         */
+  u64 inf_execs_cost, inf_time_cost;
+
+};
+
+/* ToDo: add cmplog profile as well */
+struct havoc_profile {
+
+  u32 queued_det_stage,                 /* Det/Havoc Stage Profiling        */
+      queued_havoc_stage, total_queued_det, edge_det_stage, edge_havoc_stage,
+      total_det_edge;
+
+  u64 det_stage_time, havoc_stage_time, total_det_time;
+
+};
+
+struct skipdet_entry {
+
+  u8  continue_inf, done_eff;
+  u32 undet_bits, quick_eff_bytes;
+
+  u8 *skip_eff_map,                     /* we'v finish the eff_map          */
+      *done_inf_map;                    /* some bytes are not done yet      */
+
+};
+
+struct skipdet_global {
+
+  u8 use_skip_havoc;
+
+  u32 undet_bits_threshold;
+
+  u64 last_cov_undet;
+
+  u8 *virgin_det_bits;                  /* global fuzzed bits               */
+
+  struct inf_profile *inf_prof;
+
+};
+
 struct queue_entry {
 
   u8 *fname;                            /* File name for the test case      */
@@ -203,6 +245,8 @@ struct queue_entry {
 
   struct queue_entry *mother;           /* queue entry this based on        */
 
+  struct skipdet_entry *skipdet_e;
+
 };
 
 struct extra_data {
@@ -247,6 +291,8 @@ enum {
   /* 19 */ STAGE_CUSTOM_MUTATOR,
   /* 20 */ STAGE_COLORIZATION,
   /* 21 */ STAGE_ITS,
+  /* 22 */ STAGE_INF,
+  /* 23 */ STAGE_QUICK,
 
   STAGE_NUM_MAX
 
@@ -782,6 +828,11 @@ typedef struct afl_state {
    * is too large) */
   struct queue_entry **q_testcase_cache;
 
+  /* Global Profile Data for deterministic/havoc-splice stage */
+  struct havoc_profile *havoc_prof;
+
+  struct skipdet_global *skipdet_g;
+
 #ifdef INTROSPECTION
   char  mutation[8072];
   char  m_tmp[4096];
@@ -1232,6 +1283,13 @@ AFL_RAND_RETURN rand_next(afl_state_t *afl);
 /* probability between 0.0 and 1.0 */
 double rand_next_percent(afl_state_t *afl);
 
+/* SkipDet Functions */
+
+u8 skip_deterministic_stage(afl_state_t *, u8 *, u8 *, u32, u64);
+u8 is_det_timeout(u64, u8);
+
+void plot_profile_data(afl_state_t *, struct queue_entry *);
+
 /**** Inline routines ****/
 
 /* Generate a random number (from 0 to limit - 1). This may