From 038fef962c3d85fe7e37fcd8717270654f927881 Mon Sep 17 00:00:00 2001 From: vanhauser-thc Date: Thu, 8 Feb 2024 12:46:00 +0100 Subject: performance --- docs/INSTALL.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'docs/INSTALL.md') diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 84bbe3ea..9f53afed 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -69,14 +69,15 @@ These build targets exist: [Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html), you can also build statically linked versions of the AFL++ binaries by passing -the `STATIC=1` argument to make: +the `PERFORMANCE=1` argument to make: ```shell -make STATIC=1 +make PERFORMANCE=1 ``` These build options exist: +* PERFORMANCE - compile with performance options that make the binary not transferable to other systems. Recommended! * STATIC - compile AFL++ static * CODE_COVERAGE - compile the target for code coverage (see docs/instrumentation/README.llvm.md) * ASAN_BUILD - compiles AFL++ with memory sanitizer for debug purposes -- cgit 1.4.1 From 037a14f62175226850f7a1b27f376843097b0e30 Mon Sep 17 00:00:00 2001 From: Sean McBride Date: Thu, 8 Feb 2024 21:15:59 -0500 Subject: Fixed issue #1981: document PATH correctly based on homebrew version - removed reference to M1 as M2 and M3 exist now too. - Also use current name and spelling of "macOS". --- docs/INSTALL.md | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) (limited to 'docs/INSTALL.md') diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 84bbe3ea..1ac303ce 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -96,9 +96,9 @@ These build options exist: e.g.: `make LLVM_CONFIG=llvm-config-14` -## MacOS X on x86 and arm64 (M1) +## macOS on x86_64 and arm64 -MacOS has some gotchas due to the idiosyncrasies of the platform. +macOS has some gotchas due to the idiosyncrasies of the platform. To build AFL, install llvm (and perhaps gcc) from brew and follow the general instructions for Linux. If possible, avoid Xcode at all cost. @@ -107,17 +107,20 @@ instructions for Linux. If possible, avoid Xcode at all cost. brew install wget git make cmake llvm gdb coreutils ``` -Be sure to setup `PATH` to point to the correct clang binaries and use the -freshly installed clang, clang++, llvm-config, gmake and coreutils, e.g.: +Depending on your macOS system + brew version, brew may be installed in different places. +You can check with `brew info llvm` to know where, then create a variable for it: ```shell -# Depending on your MacOS system + brew version it is either -export PATH="/opt/homebrew/opt/llvm/bin:$PATH" +export HOMEBREW_BASE="/opt/homebrew/opt" # or -export PATH="/usr/local/opt/llvm/bin:/usr/local/opt/coreutils/libexec/gnubin:$PATH" -# you can check with "brew info llvm" +export HOMEBREW_BASE="/usr/local/opt" +``` -export PATH="/usr/local/bin:$PATH" +Be sure to setup `PATH` to point to the correct clang binaries and use the +freshly installed clang, clang++, llvm-config, gmake and coreutils, e.g.: + +```shell +export PATH="$HOMEBREW_BASE/coreutils/libexec/gnubin:/usr/local/bin:$HOMEBREW_BASE/llvm/bin:$PATH" export CC=clang export CXX=clang++ gmake @@ -130,9 +133,9 @@ sudo gmake install `afl-gcc` will fail unless you have GCC installed, but that is using outdated instrumentation anyway. `afl-clang` might fail too depending on your PATH setup. But you don't want neither, you want `afl-clang-fast` anyway :) Note that -`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on MacOS. +`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on macOS. -The crash reporting daemon that comes by default with MacOS X will cause +The crash reporting daemon that comes by default with macOS will cause problems with fuzzing. You need to turn it off: ``` @@ -145,16 +148,16 @@ and definitely don't look POSIX-compliant. This means two things: - Fuzzing will be probably slower than on Linux. In fact, some folks report considerable performance gains by running the jobs inside a Linux VM on - MacOS X. + macOS. - Some non-portable, platform-specific code may be incompatible with the AFL++ forkserver. If you run into any problems, set `AFL_NO_FORKSRV=1` in the environment before starting afl-fuzz. -User emulation mode of QEMU does not appear to be supported on MacOS X, so +User emulation mode of QEMU does not appear to be supported on macOS, so black-box instrumentation mode (`-Q`) will not work. However, FRIDA mode (`-O`) -works on both x86 and arm64 MacOS boxes. +works on both x86 and arm64 macOS boxes. -MacOS X supports SYSV shared memory used by AFL's instrumentation, but the +macOS supports SYSV shared memory used by AFL's instrumentation, but the default settings aren't usable with AFL++. The default settings on 10.14 seem to be: -- cgit 1.4.1 From 808022d3e010e2385c7094a4e9933f80b9953e2b Mon Sep 17 00:00:00 2001 From: Sean McBride Date: Thu, 15 Feb 2024 15:34:17 -0500 Subject: Fixed #1865: many updates to INSTALL.md for macOS - moved shared mem instructions to before building so that tests during build don't fail - corrected path to README.llvm.md - updated some macOS spelling - added missing sudo - misc other changes from reading the document carefully --- docs/INSTALL.md | 100 +++++++++++++++++++++----------------------------------- 1 file changed, 37 insertions(+), 63 deletions(-) (limited to 'docs/INSTALL.md') diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 5260a65c..3089aab2 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -21,7 +21,7 @@ If you want to build AFL++ yourself, you have many options. The easiest choice is to build and install everything: NOTE: depending on your Debian/Ubuntu/Kali/... release, replace `-14` with -whatever llvm version is available. We recommend llvm 13, 14, 15 or 16. +whatever llvm version is available. We recommend llvm 13 or newer. ```shell sudo apt-get update @@ -67,7 +67,7 @@ These build targets exist: * unit: perform unit tests (based on cmocka) * help: shows these build options -[Unless you are on Mac OS X](https://developer.apple.com/library/archive/qa/qa1118/_index.html), +[Unless you are on macOS](https://developer.apple.com/library/archive/qa/qa1118/_index.html), you can also build statically linked versions of the AFL++ binaries by passing the `PERFORMANCE=1` argument to make: @@ -77,10 +77,10 @@ make PERFORMANCE=1 These build options exist: -* PERFORMANCE - compile with performance options that make the binary not transferable to other systems. Recommended! -* STATIC - compile AFL++ static -* CODE_COVERAGE - compile the target for code coverage (see docs/instrumentation/README.llvm.md) -* ASAN_BUILD - compiles AFL++ with memory sanitizer for debug purposes +* PERFORMANCE - compile with performance options that make the binary not transferable to other systems. Recommended (except on macOS)! +* STATIC - compile AFL++ static (does not work on macOS) +* CODE_COVERAGE - compile the target for code coverage (see [README.llvm.md](../instrumentation/README.llvm.md)) +* ASAN_BUILD - compiles AFL++ with address sanitizer for debug purposes * UBSAN_BUILD - compiles AFL++ tools with undefined behaviour sanitizer for debug purposes * DEBUG - no optimization, -ggdb3, all warnings and -Werror * LLVM_DEBUG - shows llvm deprecation warnings @@ -92,7 +92,7 @@ These build options exist: * NO_NYX - disable building nyx mode dependencies * NO_CORESIGHT - disable building coresight (arm64 only) * NO_UNICORN_ARM64 - disable building unicorn on arm64 -* AFL_NO_X86 - if compiling on non-intel/amd platforms +* AFL_NO_X86 - if compiling on non-Intel/AMD platforms * LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config (e.g., Debian) e.g.: `make LLVM_CONFIG=llvm-config-14` @@ -101,8 +101,19 @@ e.g.: `make LLVM_CONFIG=llvm-config-14` macOS has some gotchas due to the idiosyncrasies of the platform. -To build AFL, install llvm (and perhaps gcc) from brew and follow the general -instructions for Linux. If possible, avoid Xcode at all cost. +macOS supports SYSV shared memory used by AFL++'s instrumentation, but the +default settings aren't sufficient. Before even building, increase +them by running the provided script: + +```shell +sudo afl-system-config +``` + +See +[https://www.spy-hill.com/help/apple/SharedMemory.html](https://www.spy-hill.com/help/apple/SharedMemory.html) +for documentation for the shared memory settings and how to make them permanent. + +Next, to build AFL++, install the following packages from brew: ```shell brew install wget git make cmake llvm gdb coreutils @@ -113,38 +124,37 @@ You can check with `brew info llvm` to know where, then create a variable for it ```shell export HOMEBREW_BASE="/opt/homebrew/opt" -# or +``` + +or + +```shell export HOMEBREW_BASE="/usr/local/opt" ``` -Be sure to setup `PATH` to point to the correct clang binaries and use the -freshly installed clang, clang++, llvm-config, gmake and coreutils, e.g.: +Set `PATH` to point to the brew clang, clang++, llvm-config, gmake and coreutils. +Also use the brew clang compiler; the Xcode clang compiler must not be used. ```shell export PATH="$HOMEBREW_BASE/coreutils/libexec/gnubin:/usr/local/bin:$HOMEBREW_BASE/llvm/bin:$PATH" export CC=clang export CXX=clang++ -gmake -cd frida_mode -gmake -cd .. -sudo gmake install ``` -`afl-gcc` will fail unless you have GCC installed, but that is using outdated -instrumentation anyway. `afl-clang` might fail too depending on your PATH setup. -But you don't want neither, you want `afl-clang-fast` anyway :) Note that -`afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on macOS. +Then build following the general Linux instructions. -The crash reporting daemon that comes by default with macOS will cause -problems with fuzzing. You need to turn it off: +If everything worked, you should then have `afl-clang-fast` installed, which you can check with: -``` -launchctl unload -w /System/Library/LaunchAgents/com.apple.ReportCrash.plist -sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.ReportCrash.Root.plist +```shell +which afl-clang-fast ``` -The `fork()` semantics on OS X are a bit unusual compared to other unix systems +Note that `afl-clang-lto`, `afl-gcc-fast` and `qemu_mode` are not working on macOS. + +The crash reporting daemon that comes by default with macOS will cause +problems with fuzzing. You need to turn it off, which you can do with `afl-system-config`. + +The `fork()` semantics on macOS are a bit unusual compared to other unix systems and definitely don't look POSIX-compliant. This means two things: - Fuzzing will be probably slower than on Linux. In fact, some folks report @@ -157,39 +167,3 @@ and definitely don't look POSIX-compliant. This means two things: User emulation mode of QEMU does not appear to be supported on macOS, so black-box instrumentation mode (`-Q`) will not work. However, FRIDA mode (`-O`) works on both x86 and arm64 macOS boxes. - -macOS supports SYSV shared memory used by AFL's instrumentation, but the -default settings aren't usable with AFL++. The default settings on 10.14 seem to -be: - -```bash -$ ipcs -M -IPC status from as of XXX -shminfo: - shmmax: 4194304 (max shared memory segment size) - shmmin: 1 (min shared memory segment size) - shmmni: 32 (max number of shared memory identifiers) - shmseg: 8 (max shared memory segments per process) - shmall: 1024 (max amount of shared memory in pages) -``` - -To temporarily change your settings to something minimally usable with AFL++, -run these commands as root: - -```bash -sysctl kern.sysv.shmmax=8388608 -sysctl kern.sysv.shmall=4096 -``` - -If you're running more than one instance of AFL, you likely want to make -`shmall` bigger and increase `shmseg` as well: - -```bash -sysctl kern.sysv.shmmax=8388608 -sysctl kern.sysv.shmseg=48 -sysctl kern.sysv.shmall=98304 -``` - -See -[http://www.spy-hill.com/help/apple/SharedMemory.html](http://www.spy-hill.com/help/apple/SharedMemory.html) -for documentation for these settings and how to make them permanent. -- cgit 1.4.1