From 730713193a236dd63592bb70dbd3ef7cf062c268 Mon Sep 17 00:00:00 2001 From: Resery <50428593+Resery@users.noreply.github.com> Date: Mon, 19 Feb 2024 15:44:05 -0600 Subject: replaced unicornafl with unicorn (#2003) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * push to stable (#1983) * Output afl-clang-fast stuffs only if necessary (#1912) * afl-cc header * afl-cc common declarations - Add afl-cc-state.c - Strip includes, find_object, debug/be_quiet/have_*/callname setting from afl-cc.c - Use debugf_args in main - Modify execvp stuffs to fit new aflcc struct * afl-cc show usage * afl-cc mode selecting 1. compiler_mode by callname in argv[0] 2. compiler_mode by env "AFL_CC_COMPILER" 3. compiler_mode/instrument_mode by command line options "--afl-..." 4. instrument_mode/compiler_mode by various env vars including "AFL_LLVM_INSTRUMENT" 5. final checking steps 6. print "... - mode: %s-%s\n" 7. determine real argv[0] according to compiler_mode * afl-cc macro defs * afl-cc linking behaviors * afl-cc fsanitize behaviors * afl-cc misc * afl-cc body update * afl-cc all-in-one formated with custom-format.py * nits --------- Co-authored-by: vanhauser-thc * changelog * update grammar mutator * lto llvm 12+ * docs(custom_mutators): fix missing ':' (#1953) * Fix broken LTO mode and response file support (#1948) * Strip `-Wl,-no-undefined` during compilation (#1952) Make the compiler wrapper stripping `-Wl,-no-undefined` in addition to `-Wl,--no-undefined`. Both versions of the flag are accepted by clang and, therefore, used by building systems in the wild (e.g., samba will not build without this fix). * Remove dead code in write_to_testcase (#1955) The custom_mutators_count check in if case is duplicate with if condition. The else case is custom_mutators_count == 0, neither custom_mutator_list iteration nor sent check needed. Signed-off-by: Xeonacid * update qemuafl * WIP: Add ability to generate drcov trace using QEMU backend (#1956) * Document new drcov QEMU plugin * Add link to lightkeeper for QEMU drcov file loading --------- Co-authored-by: Jean-Romain Garnier * code format * changelog * sleep on uid != 0 afl-system-config * fix segv about skip_next, warn on unsupported cases of linking options (#1958) * todos * ensure afl-cc only allows available compiler modes * update grammar mutator * disable aslr on apple * fix for arm64 * help selective instrumentation * typos * macos * add compiler test script * apple fixes * bump nyx submodules (#1963) * fix docs * update changelog * update grammar mutator * improve compiler test script * gcc asan workaround (#1966) * fix github merge fuckup * fix * Fix afl-cc (#1968) - Check if too many cmdline params here, each time before insert a new param. - Check if it is "-fsanitize=..." before we do sth. - Remove improper param_st transfer. * Avoid adding llvmnative instrumentation when linking rust sanitizer runtime (#1969) * Dynamic instrumentation filtering for LLVM native (#1971) * Add two dynamic instrumentation filter methods to runtime * Always use pc-table with native pcguard * Add make_symbol_list.py and README * changelog * todos * new forkserver check * fix * nyx test for CI * improve nyx docs * Fixes to afl-cc and documentation (#1974) * Always compile with -ldl when building for CODE_COVERAGE When building with CODE_COVERAGE, the afl runtime contains code that calls `dladdr` which requires -ldl. Under most circumstances, clang already adds this (e.g. when building with pc-table), but there are some circumstances where it isn't added automatically. * Add visibility declaration to __afl_connected When building with hidden visibility, the use of __AFL_LOOP inside such code can cause linker errors due to __afl_connected being declared "hidden". * Update docs to clarify that CODE_COVERAGE=1 is required for dynamic_covfilter * nits * nyx build script updates * test error output * debug ci * debug ci * Improve afl-cc (#1975) * update response file support - full support of rsp file - fix some segv issues * Improve afl-cc - remove dead code about allow/denylist options of sancov - missing `if (!aflcc->have_msan)` - add docs for each function - typo * enable nyx * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * fix ci * clean test script * NO_NYX * NO_NYX * fix ci * debug ci * fix ci * finalize ci fix * Enhancement on Deterministic stage (#1972) * fuzzer: init commit based on aflpp 60dc37a8cf09f8e9048e4b6a2204d6c90b27655a * fuzzers: adding the skip variables and initialize * log: profile the det/havoc finding * log: add profile log output * fuzzers: sperate log/skipdet module * fuzzers: add quick eff_map calc * fuzzers: add skip_eff_map in fuzz_one * fuzzers: mark whole input space in eff_map * fuzzers: add undet bit threshold to skip some seeds * fuzzers: fix one byte overflow * fuzzers: fix overflow * fix code format * add havoc only again * code format * remove log to INTROSPECTION, rename skipdet module * rename skipdet module * remove log to stats * clean redundant code * code format * remove redundant code format check * remove redundant doc * remove redundant objects * clean files * change -d to default skipdet * disable deterministic when using CUSTOM_MUTATOR * revert fix * final touches for skipdet * remove unused var * remove redundant eff struct (#1977) * update QEMU-Nyx submodule (#1978) * update QEMU-Nyx submodule (#1980) * Fix type in AFL_NOOPT env variable in afl-cc help message (#1982) * nits * 2024 v4.10c release * fixes --------- Signed-off-by: Xeonacid Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com> Co-authored-by: Xeonacid Co-authored-by: Nils Bars Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com> Co-authored-by: Jean-Romain Garnier Co-authored-by: Sergej Schumilo Co-authored-by: Christian Holler (:decoder) Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com> Co-authored-by: Khaled Yakdan * replaced unicornafl with unicorn The submodule of unicorn cannot be imported through unicornafl.*_const here. If we want to use the *_const module, we should reference `from unicorn.*_const import *` like this instead of importing the entire contents of the *_const module via unicornafl。 --------- Signed-off-by: Xeonacid Co-authored-by: van Hauser Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com> Co-authored-by: Xeonacid Co-authored-by: Nils Bars Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com> Co-authored-by: Jean-Romain Garnier Co-authored-by: Sergej Schumilo Co-authored-by: Christian Holler (:decoder) Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com> Co-authored-by: Khaled Yakdan --- unicorn_mode/helper_scripts/unicorn_loader.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_loader.py b/unicorn_mode/helper_scripts/unicorn_loader.py index c48a7572..740af1c8 100644 --- a/unicorn_mode/helper_scripts/unicorn_loader.py +++ b/unicorn_mode/helper_scripts/unicorn_loader.py @@ -21,10 +21,10 @@ import zlib # Unicorn imports from unicornafl import * -from unicornafl.arm_const import * -from unicornafl.arm64_const import * -from unicornafl.x86_const import * -from unicornafl.mips_const import * +from unicorn.arm_const import * +from unicorn.arm64_const import * +from unicorn.x86_const import * +from unicorn.mips_const import * # If Capstone libraries are availible (only check once) try: -- cgit 1.4.1 From 80158de3e801fa7dc1d4e36ec88cb767997f478e Mon Sep 17 00:00:00 2001 From: Resery <50428593+Resery@users.noreply.github.com> Date: Tue, 20 Feb 2024 07:01:37 -0600 Subject: Catch invalid frees (#2008) 1. There isn't a need to check all chunks when address == 0 2. If the address is not in chunks, the program may want to free an object that doesn't exist. There may be a "double-free" or "invalid-free" vulnerability. (This patch is from the repo named "Battelle/afl-unicorn") --- unicorn_mode/helper_scripts/unicorn_loader.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_loader.py b/unicorn_mode/helper_scripts/unicorn_loader.py index 740af1c8..cef39f7e 100644 --- a/unicorn_mode/helper_scripts/unicorn_loader.py +++ b/unicorn_mode/helper_scripts/unicorn_loader.py @@ -148,6 +148,9 @@ class UnicornSimpleHeap(object): return new_chunk_addr def free(self, addr): + if addr == 0: + return False + for chunk in self._chunks: if chunk.is_buffer_in_chunk(addr, 1): if self._debug_print: @@ -159,7 +162,8 @@ class UnicornSimpleHeap(object): self._uc.mem_unmap(chunk.actual_addr, chunk.total_size) self._chunks.remove(chunk) return True - return False + # Freed an object that doesn't exist. Maybe 'dobule-free' or 'invalid free' vulnerability here. + self._uc.force_crash(UcError(UC_ERR_FETCH_UNMAPPED)) # Implements basic guard-page functionality def __check_mem_access(self, uc, access, address, size, value, user_data): -- cgit 1.4.1 From 340d6aa97cd8fa18e8c7650ac9067e1b2688e8bb Mon Sep 17 00:00:00 2001 From: Resery <50428593+Resery@users.noreply.github.com> Date: Wed, 21 Feb 2024 05:42:55 -0600 Subject: unicornafl: fix malloc of size 0 (#2010) * bugfix: free a chunk with a size of 0, it will cause 1 byte oob. Malloc does not check the size. Generally, malloc(0) should return 0 but there will return two pages. Free will use is_buffer_in_chunk to check whether the address is in the chunk. At that time, the chunk.data_addr == total_size . Free pass address and "1" to is_buffer_in_chunk. So cause 1 byte out-of-bound. * typo --- unicorn_mode/helper_scripts/unicorn_loader.py | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_loader.py b/unicorn_mode/helper_scripts/unicorn_loader.py index cef39f7e..d0995f83 100644 --- a/unicorn_mode/helper_scripts/unicorn_loader.py +++ b/unicorn_mode/helper_scripts/unicorn_loader.py @@ -101,6 +101,10 @@ class UnicornSimpleHeap(object): # - Allocate at least 1 4k page of memory to make Unicorn happy # - Add guard pages at the start and end of the region total_chunk_size = UNICORN_PAGE_SIZE + ALIGN_PAGE_UP(size) + UNICORN_PAGE_SIZE + + if size == 0: + return 0 + # Gross but efficient way to find space for the chunk: chunk = None for addr in range(self.HEAP_MIN_ADDR, self.HEAP_MAX_ADDR, UNICORN_PAGE_SIZE): -- cgit 1.4.1 From 1e01ccc8fd717b067b697d5b7353a5d587f6a484 Mon Sep 17 00:00:00 2001 From: Resery <50428593+Resery@users.noreply.github.com> Date: Tue, 27 Feb 2024 02:43:50 -0600 Subject: unicornafl: Add UAF chcker to loader (#2009) * impl uaf chcker By adding a list of freed chunks, add the chunk to the list during free, check whether the allocated block is in the freed chunk list during malloc, and if so, remove the chunk from the freed chunk list, in __check_mem_access check whether the address is in the freed chunk list. This enables the detection of uaf. * make uaf_check be configruable --- unicorn_mode/helper_scripts/unicorn_loader.py | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_loader.py b/unicorn_mode/helper_scripts/unicorn_loader.py index d0995f83..a83e7000 100644 --- a/unicorn_mode/helper_scripts/unicorn_loader.py +++ b/unicorn_mode/helper_scripts/unicorn_loader.py @@ -87,9 +87,10 @@ class UnicornSimpleHeap(object): _uc = None # Unicorn engine instance to interact with _chunks = [] # List of all known chunks + _chunks_freed = [] # List of all freed chunks _debug_print = False # True to print debug information - def __init__(self, uc, debug_print=False): + def __init__(self, uc, debug_print=False， uaf_check=False): self._uc = uc self._debug_print = debug_print @@ -111,6 +112,13 @@ class UnicornSimpleHeap(object): try: self._uc.mem_map(addr, total_chunk_size, UC_PROT_READ | UC_PROT_WRITE) chunk = self.HeapChunk(addr, total_chunk_size, size) + + if self.uaf_check: + for chunk_freed in self._chunks_freed: + if chunk_freed.is_buffer_in_chunk(chunk.data_addr, 1): + self._chunks_freed.remove(chunk_freed) + break + if self._debug_print: print( "Allocating 0x{0:x}-byte chunk @ 0x{1:016x}".format( @@ -164,6 +172,10 @@ class UnicornSimpleHeap(object): ) ) self._uc.mem_unmap(chunk.actual_addr, chunk.total_size) + + if self.uaf_check: + self._chunks_freed.append(chunk) + self._chunks.remove(chunk) return True # Freed an object that doesn't exist. Maybe 'dobule-free' or 'invalid free' vulnerability here. @@ -187,6 +199,15 @@ class UnicornSimpleHeap(object): # Force a memory-based crash uc.force_crash(UcError(UC_ERR_READ_PROT)) + if self.uaf_check: + for chunk in self._chunks_freed: + if address >= chunk.actual_addr and ( + (address + size) <= (chunk.actual_addr + chunk.total_size) + ): + if chunk.is_buffer_in_chunk(address, size): + print("Use-after-free @ 0x{0:016x}".format(address)) + uc.force_crash(UcError(UC_ERR_FETCH_UNMAPPED)) + # --------------------------- # ---- Loading function -- cgit 1.4.1 From 603136efa032d62eec14720a7435ce0a6d143bb6 Mon Sep 17 00:00:00 2001 From: Joachim Hyrathon Date: Tue, 27 Feb 2024 16:46:07 +0800 Subject: unicornafl: Fix dump_regs() type errors in pwndbg dumper (#2005) * push to stable (#1983) * Output afl-clang-fast stuffs only if necessary (#1912) * afl-cc header * afl-cc common declarations - Add afl-cc-state.c - Strip includes, find_object, debug/be_quiet/have_*/callname setting from afl-cc.c - Use debugf_args in main - Modify execvp stuffs to fit new aflcc struct * afl-cc show usage * afl-cc mode selecting 1. compiler_mode by callname in argv[0] 2. compiler_mode by env "AFL_CC_COMPILER" 3. compiler_mode/instrument_mode by command line options "--afl-..." 4. instrument_mode/compiler_mode by various env vars including "AFL_LLVM_INSTRUMENT" 5. final checking steps 6. print "... - mode: %s-%s\n" 7. determine real argv[0] according to compiler_mode * afl-cc macro defs * afl-cc linking behaviors * afl-cc fsanitize behaviors * afl-cc misc * afl-cc body update * afl-cc all-in-one formated with custom-format.py * nits --------- Co-authored-by: vanhauser-thc * changelog * update grammar mutator * lto llvm 12+ * docs(custom_mutators): fix missing ':' (#1953) * Fix broken LTO mode and response file support (#1948) * Strip `-Wl,-no-undefined` during compilation (#1952) Make the compiler wrapper stripping `-Wl,-no-undefined` in addition to `-Wl,--no-undefined`. Both versions of the flag are accepted by clang and, therefore, used by building systems in the wild (e.g., samba will not build without this fix). * Remove dead code in write_to_testcase (#1955) The custom_mutators_count check in if case is duplicate with if condition. The else case is custom_mutators_count == 0, neither custom_mutator_list iteration nor sent check needed. Signed-off-by: Xeonacid * update qemuafl * WIP: Add ability to generate drcov trace using QEMU backend (#1956) * Document new drcov QEMU plugin * Add link to lightkeeper for QEMU drcov file loading --------- Co-authored-by: Jean-Romain Garnier * code format * changelog * sleep on uid != 0 afl-system-config * fix segv about skip_next, warn on unsupported cases of linking options (#1958) * todos * ensure afl-cc only allows available compiler modes * update grammar mutator * disable aslr on apple * fix for arm64 * help selective instrumentation * typos * macos * add compiler test script * apple fixes * bump nyx submodules (#1963) * fix docs * update changelog * update grammar mutator * improve compiler test script * gcc asan workaround (#1966) * fix github merge fuckup * fix * Fix afl-cc (#1968) - Check if too many cmdline params here, each time before insert a new param. - Check if it is "-fsanitize=..." before we do sth. - Remove improper param_st transfer. * Avoid adding llvmnative instrumentation when linking rust sanitizer runtime (#1969) * Dynamic instrumentation filtering for LLVM native (#1971) * Add two dynamic instrumentation filter methods to runtime * Always use pc-table with native pcguard * Add make_symbol_list.py and README * changelog * todos * new forkserver check * fix * nyx test for CI * improve nyx docs * Fixes to afl-cc and documentation (#1974) * Always compile with -ldl when building for CODE_COVERAGE When building with CODE_COVERAGE, the afl runtime contains code that calls `dladdr` which requires -ldl. Under most circumstances, clang already adds this (e.g. when building with pc-table), but there are some circumstances where it isn't added automatically. * Add visibility declaration to __afl_connected When building with hidden visibility, the use of __AFL_LOOP inside such code can cause linker errors due to __afl_connected being declared "hidden". * Update docs to clarify that CODE_COVERAGE=1 is required for dynamic_covfilter * nits * nyx build script updates * test error output * debug ci * debug ci * Improve afl-cc (#1975) * update response file support - full support of rsp file - fix some segv issues * Improve afl-cc - remove dead code about allow/denylist options of sancov - missing `if (!aflcc->have_msan)` - add docs for each function - typo * enable nyx * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * debug ci * fix ci * clean test script * NO_NYX * NO_NYX * fix ci * debug ci * fix ci * finalize ci fix * Enhancement on Deterministic stage (#1972) * fuzzer: init commit based on aflpp 60dc37a8cf09f8e9048e4b6a2204d6c90b27655a * fuzzers: adding the skip variables and initialize * log: profile the det/havoc finding * log: add profile log output * fuzzers: sperate log/skipdet module * fuzzers: add quick eff_map calc * fuzzers: add skip_eff_map in fuzz_one * fuzzers: mark whole input space in eff_map * fuzzers: add undet bit threshold to skip some seeds * fuzzers: fix one byte overflow * fuzzers: fix overflow * fix code format * add havoc only again * code format * remove log to INTROSPECTION, rename skipdet module * rename skipdet module * remove log to stats * clean redundant code * code format * remove redundant code format check * remove redundant doc * remove redundant objects * clean files * change -d to default skipdet * disable deterministic when using CUSTOM_MUTATOR * revert fix * final touches for skipdet * remove unused var * remove redundant eff struct (#1977) * update QEMU-Nyx submodule (#1978) * update QEMU-Nyx submodule (#1980) * Fix type in AFL_NOOPT env variable in afl-cc help message (#1982) * nits * 2024 v4.10c release * fixes --------- Signed-off-by: Xeonacid Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com> Co-authored-by: Xeonacid Co-authored-by: Nils Bars Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com> Co-authored-by: Jean-Romain Garnier Co-authored-by: Sergej Schumilo Co-authored-by: Christian Holler (:decoder) Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com> Co-authored-by: Khaled Yakdan * Fix dump_regs() type errors in Python TypeError will occur as gdb api return some strange type of values that json can't serialize, this would fix this issue * Update reg_val is None condition --------- Signed-off-by: Xeonacid Co-authored-by: van Hauser Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com> Co-authored-by: Xeonacid Co-authored-by: Nils Bars Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com> Co-authored-by: Jean-Romain Garnier Co-authored-by: Sergej Schumilo Co-authored-by: Christian Holler (:decoder) Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com> Co-authored-by: Khaled Yakdan --- unicorn_mode/helper_scripts/unicorn_dumper_pwndbg.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_dumper_pwndbg.py b/unicorn_mode/helper_scripts/unicorn_dumper_pwndbg.py index 7e97f6a7..76eaf54f 100644 --- a/unicorn_mode/helper_scripts/unicorn_dumper_pwndbg.py +++ b/unicorn_mode/helper_scripts/unicorn_dumper_pwndbg.py @@ -111,12 +111,14 @@ def dump_regs(): reg_state = {} for reg in pwndbg.gdblib.regs.all: reg_val = pwndbg.gdblib.regs[reg] + if reg_val is None: + continue # current dumper script looks for register values to be hex strings # reg_str = "0x{:08x}".format(reg_val) # if "64" in get_arch(): # reg_str = "0x{:016x}".format(reg_val) # reg_state[reg.strip().strip('$')] = reg_str - reg_state[reg.strip().strip("$")] = reg_val + reg_state[reg.strip().strip("$")] = int(reg_val) return reg_state -- cgit 1.4.1 From 2bf92848ff7f48155b8fa031543ab3410dc556d6 Mon Sep 17 00:00:00 2001 From: Alex Schmith Date: Wed, 3 Apr 2024 05:57:09 -0400 Subject: Fixed unicorn_dumper_gdb.py for updated version of gef (#2045) Updated unicorn_dumper_gdb.py to support new gef api and replaced deprecated functions . The functions that are not in the new gef api are read_memory(), and current_arch(). Also replaced some deprecated functions with the updated versions of them. replaced read_memory() with GefMemoryManager.read() as read_memory(). read_memory() is in legacy-gef-api replaced current_arch with gef.arch.registers replaced get_process_maps() with gef.memory.maps (just depreacated) replaced get_register() with gef.arch.register() --- unicorn_mode/helper_scripts/unicorn_dumper_gdb.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) (limited to 'unicorn_mode/helper_scripts') diff --git a/unicorn_mode/helper_scripts/unicorn_dumper_gdb.py b/unicorn_mode/helper_scripts/unicorn_dumper_gdb.py index 1ac4c9f3..a202ac0e 100644 --- a/unicorn_mode/helper_scripts/unicorn_dumper_gdb.py +++ b/unicorn_mode/helper_scripts/unicorn_dumper_gdb.py @@ -89,8 +89,8 @@ def dump_arch_info(): def dump_regs(): reg_state = {} - for reg in current_arch.all_registers: - reg_val = get_register(reg) + for reg in gef.arch.registers: + reg_val = gef.arch.register(reg) reg_state[reg.strip().strip("$")] = reg_val return reg_state @@ -101,7 +101,9 @@ def dump_process_memory(output_dir): final_segment_list = [] # GEF: - vmmap = get_process_maps() + vmmap = gef.memory.maps + memory = GefMemoryManager() + if not vmmap: print("No address mapping information found") return final_segment_list @@ -126,7 +128,7 @@ def dump_process_memory(output_dir): if entry.is_readable() and not "(deleted)" in entry.path: try: # Compress and dump the content to a file - seg_content = read_memory(entry.page_start, entry.size) + seg_content = memory.read(entry.page_start, entry.size) if seg_content == None: print( "Segment empty: @0x{0:016x} (size:UNKNOWN) {1}".format( -- cgit 1.4.1