diff options
| author | Chris Ball <chris@printf.net> | 2023-10-02 03:23:09 -0700 |
|---|---|---|
| committer | Chris Ball <chris@printf.net> | 2023-10-02 04:11:43 -0700 |
| commit | b9db6b1254c9bf3a47c171bb96468628e9bd00f2 (patch) | |
| tree | bf68f202352170edb26933fc4f64418399177db1 | |
| parent | 49a1d81191aea5c7d068ad0051f39fc579ebfa63 (diff) | |
| download | afl++-b9db6b1254c9bf3a47c171bb96468628e9bd00f2.tar.gz | |
benchmark: add a README, lower default runs from 5 to 3
| -rw-r--r-- | benchmark/README.md | 46 | ||||
| -rw-r--r-- | benchmark/benchmark.ipynb | 3502 | ||||
| -rw-r--r-- | benchmark/benchmark.py | 2 |
3 files changed, 110 insertions, 3440 deletions
diff --git a/benchmark/README.md b/benchmark/README.md new file mode 100644 index 00000000..66f7f59e --- /dev/null +++ b/benchmark/README.md @@ -0,0 +1,46 @@ +# American Fuzzy Lop plus plus (AFL++) + +## benchmarking + +This directory contains benchmarking tools that allow you to compare one machine +with another in terms of raw ability to execute a fuzzing target repeatedly. + +To achieve this, we use a sample program ("test-instr.c") where each path is +equally likely, supply it a single seed, and tell AFL to exit after one run of +deterministic mutations against that seed. + +Usage: + +``` +cd aflplusplus/benchmark +python3 benchmark.py + [*] Using 16 fuzzers for multicore fuzzing (use --fuzzers to override) + [*] Ready, starting benchmark... + [*] Compiling the test-instr-persist-shmem fuzzing harness for the benchmark to use. + [*] multicore test-instr-persist-shmem run 1 of 3, execs/s: 846065.81 + [*] multicore test-instr-persist-shmem run 2 of 3, execs/s: 849694.03 + [*] multicore test-instr-persist-shmem run 3 of 3, execs/s: 850757.52 + [*] Average AFL execs/sec for this test across all runs was: 848839.12 + [*] Average total execs/sec for this test across all runs was: 833138.28 + [*] Results have been written to benchmark-results.jsonl +``` + +By default, the script will use a number of parallel fuzzers equal to your +available CPUs/threads (change with `--fuzzers`), and will perform each test +three times and average the result (change with `--runs`). + +The script will use multicore fuzzing instead of singlecore by default (change +with `--mode singlecore`) and use a persistent-mode shared memory harness for +optimal speed (change with `--target test-instr`). + +Each run writes results to [benchmark-results.jsonl](benchmark-results.jsonl) +in [JSON Lines](https://jsonlines.org/) format, ready to be pulled in to other +tools such as [jq -cs](https://jqlang.github.io/jq/) or +[pandas](https://pandas.pydata.org/) for analysis. + +## Data analysis + +There is sample data in [benchmark-results.jsonl](benchmark-results.jsonl), and +a Jupyter notebook for exploring the results and suggesting their meaning at +[benchmark.ipynb](benchmark.ipynb). + diff --git a/benchmark/benchmark.ipynb b/benchmark/benchmark.ipynb index b1ef6b0f..7ff90fcd 100644 --- a/benchmark/benchmark.ipynb +++ b/benchmark/benchmark.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 502, + "execution_count": 12, "metadata": {}, "outputs": [], "source": [ @@ -20,14 +20,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Translate the JSON Lines entries into a single pandas DataFrame.\n", + "### Translate the JSON Lines entries into a single pandas DataFrame\n", "\n", "We have JSON Lines in [benchmark-results.jsonl](benchmark-results.jsonl) that look like this:" ] }, { "cell_type": "code", - "execution_count": 503, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -89,7 +89,7 @@ }, { "cell_type": "code", - "execution_count": 504, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -371,7 +371,7 @@ "[5 rows x 37 columns]" ] }, - "execution_count": 504, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -404,7 +404,7 @@ }, { "cell_type": "code", - "execution_count": 505, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -418,1647 +418,65 @@ }, { "cell_type": "code", - "execution_count": 506, + "execution_count": 16, "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "<div>\n", - "<style scoped>\n", - " .dataframe tbody tr th:only-of-type {\n", - " vertical-align: middle;\n", - " }\n", - "\n", - " .dataframe tbody tr th {\n", - " vertical-align: top;\n", - " }\n", - "\n", - " .dataframe thead th {\n", - " text-align: right;\n", - " }\n", - "</style>\n", - "<table border=\"1\" class=\"dataframe\">\n", - " <thead>\n", - " <tr style=\"text-align: right;\">\n", - " <th></th>\n", - " <th>execs_per_sec</th>\n", - " <th>parallel_fuzzers</th>\n", - " <th>afl_persistent_config</th>\n", - " <th>afl_system_config</th>\n", - " <th>label</th>\n", - " </tr>\n", - " </thead>\n", - " <tbody>\n", - " <tr>\n", - " <th>108</th>\n", - " <td>135613.26</td>\n", - " <td>1.0</td>\n", - " <td>True</td>\n", - " <td>True</td>\n", - " <td>Singlecore: Persistent mode/shared memory + ke...</td>\n", - " </tr>\n", - " <tr>\n", - " <th>117</th>\n", - " <td>135613.26</td>\n", - " <td>10.0</td>\n", - " <td>True</td>\n", - " <td>True</td>\n", - " <td>Singlecore: Persistent mode/shared memory + ke...</td>\n", - " </tr>\n", - " </tbody>\n", - "</table>\n", - "</div>" - ], - "text/plain": [ - " execs_per_sec parallel_fuzzers afl_persistent_config \\\n", - "108 135613.26 1.0 True \n", - "117 135613.26 10.0 True \n", - "\n", - " afl_system_config label \n", - "108 True Singlecore: Persistent mode/shared memory + ke... \n", - "117 True Singlecore: Persistent mode/shared memory + ke... " - ] - }, - "execution_count": 506, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "def build_graphdf_from_query(query: pd.DataFrame):\n", " \"\"\"Build a table suitable for graphing from a subset of the dataframe.\"\"\"\n", " graphdata = []\n", " max_fuzzers = int(query[[\"targets.test-instr-persist-shmem.multicore.fuzzers_used\", \"targets.test-instr.multicore.fuzzers_used\"]].max(axis=1).max(axis=0))\n", " for _, row in query.iterrows():\n", - " if row[\"targets.test-instr-persist-shmem.multicore.total_execs_per_sec\"] > 0:\n", - " execs_per_sec = row[\"targets.test-instr-persist-shmem.multicore.total_execs_per_sec\"]\n", - " afl_execs_per_sec = row[\"targets.test-instr-persist-shmem.multicore.afl_execs_per_sec\"]\n", - " parallel_fuzzers = row[\"targets.test-instr-persist-shmem.multicore.fuzzers_used\"]\n", - " afl_persistent_config = row[\"config.afl_persistent_config\"]\n", - " afl_system_config = row[\"config.afl_system_config\"]\n", - " label = \"shmem-multicore\"\n", - " if afl_persistent_config:\n", - " label += \"+persist-conf\"\n", - " if afl_system_config:\n", - " label += \"+system-conf\"\n", - " if label == \"shmem-multicore+persist-conf+system-conf\":\n", - " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Persistent mode/shared memory + kernel config\"})\n", - " graphdata.append({\"execs_per_sec\": afl_execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: afl_execs: Persistent mode/shared memory + kernel config\"})\n", - " if label == \"shmem-multicore\":\n", - " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Persistent mode/shared memory without kernel config\"})\n", - " if row[\"targets.test-instr.multicore.total_execs_per_sec\"] > 0:\n", - " execs_per_sec = row[\"targets.test-instr.multicore.total_execs_per_sec\"]\n", - " parallel_fuzzers = row[\"targets.test-instr.multicore.fuzzers_used\"]\n", - " afl_persistent_config = row[\"config.afl_persistent_config\"]\n", - " afl_system_config = row[\"config.afl_system_config\"]\n", - " label = \"base-multicore\"\n", - " if afl_persistent_config:\n", - " label += \"+persist-conf\"\n", - " if afl_system_config:\n", - " label += \"+system-conf\"\n", + " for target in [\"test-instr-persist-shmem\", \"test-instr\"]:\n", + " for mode in [\"multicore\", \"singlecore\"]:\n", + " label = \"\"\n", + " if not row[f\"targets.{target}.{mode}.total_execs_per_sec\"] > 0:\n", + " continue\n", + " execs_per_sec = row[f\"targets.{target}.{mode}.total_execs_per_sec\"]\n", + " afl_execs_per_sec = row[f\"targets.{target}.{mode}.afl_execs_per_sec\"]\n", + " parallel_fuzzers = row[f\"targets.{target}.{mode}.fuzzers_used\"]\n", + " afl_persistent_config = row[\"config.afl_persistent_config\"]\n", + " afl_system_config = row[\"config.afl_system_config\"]\n", + " if target == \"test-instr-persist-shmem\":\n", + " label += \"shmem\"\n", + " else:\n", + " label += \"base\"\n", + " if mode == \"multicore\":\n", + " label += \"-multicore\"\n", + " else:\n", + " label += \"-singlecore\"\n", + " if afl_persistent_config:\n", + " label += \"+persist-conf\"\n", + " if afl_system_config:\n", + " label += \"+system-conf\"\n", + " \n", + " if label == \"shmem-multicore+persist-conf+system-conf\":\n", + " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Persistent mode/shared memory + kernel config\"})\n", + " graphdata.append({\"execs_per_sec\": afl_execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: afl_execs: Persistent mode/shared memory + kernel config\"})\n", + " if label == \"shmem-multicore\":\n", + " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Persistent mode/shared memory without kernel config\"})\n", " if label == \"base-multicore+persist-conf+system-conf\":\n", - " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Non-persistent mode + kernel config\"}) \n", - " if row[\"targets.test-instr-persist-shmem.singlecore.total_execs_per_sec\"] > 0:\n", - " execs_per_sec = row[\"targets.test-instr-persist-shmem.singlecore.total_execs_per_sec\"]\n", - " parallel_fuzzers = row[\"targets.test-instr-persist-shmem.singlecore.fuzzers_used\"]\n", - " afl_persistent_config = row[\"config.afl_persistent_config\"]\n", - " afl_system_config = row[\"config.afl_system_config\"]\n", - " label = \"shmem-singlecore\"\n", - " if afl_persistent_config:\n", - " label += \"+persist-conf\"\n", - " if afl_system_config:\n", - " label += \"+system-conf\"\n", - " if label == \"shmem-singlecore+persist-conf+system-conf\":\n", - " for i in range(1, max_fuzzers + 1):\n", - " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": float(i), \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Singlecore: Persistent mode/shared memory + kernel config\"})\n", - " if row[\"targets.test-instr.singlecore.total_execs_per_sec\"] > 0:\n", - " execs_per_sec = row[\"targets.test-instr.singlecore.total_execs_per_sec\"]\n", - " parallel_fuzzers = row[\"targets.test-instr.singlecore.fuzzers_used\"]\n", - " afl_persistent_config = row[\"config.afl_persistent_config\"]\n", - " afl_system_config = row[\"config.afl_system_config\"]\n", - " label = \"base-singlecore\"\n", - " if afl_persistent_config:\n", - " label += \"+persist-conf\"\n", - " if afl_system_config:\n", - " label += \"+system-conf\"\n", - " if label == \"base-singlecore+persist-conf+system-conf\":\n", - " for i in range(1, max_fuzzers + 1):\n", - " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": float(i), \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Singlecore: Non-persistent mode + kernel config\"})\n", - "\n", + " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": parallel_fuzzers, \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Multicore: Non-persistent mode + kernel config\"})\n", + " if label == \"shmem-singlecore+persist-conf+system-conf\":\n", + " for i in range(1, max_fuzzers + 1):\n", + " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": float(i), \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Singlecore: Persistent mode/shared memory + kernel config\"})\n", + " if label == \"base-singlecore+persist-conf+system-conf\":\n", + " for i in range(1, max_fuzzers + 1):\n", + " graphdata.append({\"execs_per_sec\": execs_per_sec, \"parallel_fuzzers\": float(i), \"afl_persistent_config\": afl_persistent_config, \"afl_system_config\": afl_system_config, \"label\": \"Singlecore: Non-persistent mode + kernel config\"})\n", " return pd.DataFrame.from_records(graphdata).sort_values(\"label\", ascending=False)\n", "\n", - "graphdf = build_graphdf_from_query(i7)\n", - "graphdf.head(2)" + "graphdf = build_graphdf_from_query(i7)" ] }, { "cell_type": "code", - "execution_count": 507, + "execution_count": 17, "metadata": {}, "outputs": [ { "data": { - "application/vnd.plotly.v1+json": { - "config": { - "plotlyServerURL": "https://plot.ly" - }, - "data": [ - { - "hovertemplate": "Configuration=Multicore: Non-persistent mode + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: Non-persistent mode + kernel config", - "line": { - "color": "#636efa", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: Non-persistent mode + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 11019.3, - 21111.92, - 30568.82, - 38963.07, - 47693.65, - 55718.73, - 64156.79, - 72176.39, - 61257.76, - 67507.14, - 73183.59, - 79167.7, - 85202.55, - 91594.86, - 97682.33, - 103765.38, - 84547.71, - 86611.67, - 88657, - 90134.42, - 91033.28, - 91637.86, - 92747.81, - 93207.38, - 92970.87, - 94162.8, - 94237.96, - 91716.1, - 94072.6, - 95644.79, - 94880.56, - 93460.57, - 94194.83, - 93853.08, - 95147.78, - 92697.06 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Multicore: Persistent mode/shared memory + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: Persistent mode/shared memory + kernel config", - "line": { - "color": "#EF553B", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: Persistent mode/shared memory + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 133591.26, - 255995.07, - 380246.34, - 490254.72, - 598698.16, - 716786.21, - 824873.02, - 942712.02, - 709716.24, - 779115.44, - 851918.03, - 914375.37, - 990573.31, - 1060551.02, - 1134650.66, - 1203287.99, - 1022498.84, - 1058151.58, - 1076742.64, - 1091743.7, - 1062616.36, - 1081621.57, - 1132929.86, - 1133825.45, - 1114215.27, - 1122210.96, - 1130627.8, - 1135890.71, - 1137390.94, - 1142969.21, - 1149056.35, - 1142131.87, - 1128973.67, - 1116863.53, - 1117913.34, - 1124962.18 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Multicore: Persistent mode/shared memory without kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: Persistent mode/shared memory without kernel config", - "line": { - "color": "#00cc96", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: Persistent mode/shared memory without kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 90851.4, - 176159.32, - 260268.78, - 336355.99, - 413750, - 492578.2, - 565737.17, - 640579.35, - 491284.66, - 540759.63, - 591144.78, - 638938.52, - 687954.18, - 734886.87, - 784210.26, - 834811.24, - 714178.66, - 734804.4, - 740714.93, - 750425.99, - 752625.52, - 760661.34, - 758401.65, - 767985.22, - 785474.61, - 789679.78, - 790483.94, - 798176.69, - 792763.39, - 788972.72, - 789307.69, - 777440.02, - 767985.22, - 775967.55, - 764863.41, - 768616.31 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Multicore: afl_execs: Persistent mode/shared memory + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: afl_execs: Persistent mode/shared memory + kernel config", - "line": { - "color": "#ab63fa", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: afl_execs: Persistent mode/shared memory + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 134423.5, - 258490.04, - 383777.45, - 496249.48, - 613089.31, - 730366.19, - 844187.32, - 962846.18, - 997414.74, - 1034757.73, - 1070703.42, - 1104249.08, - 1131176.88, - 1164076.48, - 1198824.47, - 1227578.7, - 1272311.96, - 1295688.8, - 1314398.6, - 1328581.94, - 1342660.66, - 1363930.3, - 1377043.72, - 1375818.24, - 1361687.56, - 1369637.56, - 1375444.16, - 1349599.77, - 1321658.08, - 1301868.24, - 1276904.9, - 1243444.8, - 1243981.21, - 1234425.98, - 1244349.38, - 1250454.58 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Singlecore: Non-persistent mode + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Singlecore: Non-persistent mode + kernel config", - "line": { - "color": "#FFA15A", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Singlecore: Non-persistent mode + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96, - 11038.96 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Singlecore: Persistent mode/shared memory + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Singlecore: Persistent mode/shared memory + kernel config", - "line": { - "color": "#19d3f3", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Singlecore: Persistent mode/shared memory + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "xaxis": "x", - "y": [ - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26, - 135613.26 - ], - "yaxis": "y" - } - ], - "layout": { - "height": 400, - "legend": { - "title": { - "text": "Configuration" - }, - "tracegroupgap": 0 - }, - "template": { - "data": { - "bar": [ - { - "error_x": { - "color": "#2a3f5f" - }, - "error_y": { - "color": "#2a3f5f" - }, - "marker": { - "line": { - "color": "#E5ECF6", - "width": 0.5 - }, - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "bar" - } - ], - "barpolar": [ - { - "marker": { - "line": { - "color": "#E5ECF6", - "width": 0.5 - }, - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "barpolar" - } - ], - "carpet": [ - { - "aaxis": { - "endlinecolor": "#2a3f5f", - "gridcolor": "white", - "linecolor": "white", - "minorgridcolor": "white", - "startlinecolor": "#2a3f5f" - }, - "baxis": { - "endlinecolor": "#2a3f5f", - "gridcolor": "white", - "linecolor": "white", - "minorgridcolor": "white", - "startlinecolor": "#2a3f5f" - }, - "type": "carpet" - } - ], - "choropleth": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "choropleth" - } - ], - "contour": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "contour" - } - ], - "contourcarpet": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "contourcarpet" - } - ], - "heatmap": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "heatmap" - } - ], - "heatmapgl": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "heatmapgl" - } - ], - "histogram": [ - { - "marker": { - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "histogram" - } - ], - "histogram2d": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "histogram2d" - } - ], - "histogram2dcontour": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "histogram2dcontour" - } - ], - "mesh3d": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "mesh3d" - } - ], - "parcoords": [ - { - "line": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "parcoords" - } - ], - "pie": [ - { - "automargin": true, - "type": "pie" - } - ], - "scatter": [ - { - "fillpattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - }, - "type": "scatter" - } - ], - "scatter3d": [ - { - "line": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatter3d" - } - ], - "scattercarpet": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattercarpet" - } - ], - "scattergeo": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattergeo" - } - ], - "scattergl": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattergl" - } - ], - "scattermapbox": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattermapbox" - } - ], - "scatterpolar": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterpolar" - } - ], - "scatterpolargl": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterpolargl" - } - ], - "scatterternary": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterternary" - } - ], - "surface": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "surface" - } - ], - "table": [ - { - "cells": { - "fill": { - "color": "#EBF0F8" - }, - "line": { - "color": "white" - } - }, - "header": { - "fill": { - "color": "#C8D4E3" - }, - "line": { - "color": "white" - } - }, - "type": "table" - } - ] - }, - "layout": { - "annotationdefaults": { - "arrowcolor": "#2a3f5f", - "arrowhead": 0, - "arrowwidth": 1 - }, - "autotypenumbers": "strict", - "coloraxis": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "colorscale": { - "diverging": [ - [ - 0, - "#8e0152" - ], - [ - 0.1, - "#c51b7d" - ], - [ - 0.2, - "#de77ae" - ], - [ - 0.3, - "#f1b6da" - ], - [ - 0.4, - "#fde0ef" - ], - [ - 0.5, - "#f7f7f7" - ], - [ - 0.6, - "#e6f5d0" - ], - [ - 0.7, - "#b8e186" - ], - [ - 0.8, - "#7fbc41" - ], - [ - 0.9, - "#4d9221" - ], - [ - 1, - "#276419" - ] - ], - "sequential": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "sequentialminus": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ] - }, - "colorway": [ - "#636efa", - "#EF553B", - "#00cc96", - "#ab63fa", - "#FFA15A", - "#19d3f3", - "#FF6692", - "#B6E880", - "#FF97FF", - "#FECB52" - ], - "font": { - "color": "#2a3f5f" - }, - "geo": { - "bgcolor": "white", - "lakecolor": "white", - "landcolor": "#E5ECF6", - "showlakes": true, - "showland": true, - "subunitcolor": "white" - }, - "hoverlabel": { - "align": "left" - }, - "hovermode": "closest", - "mapbox": { - "style": "light" - }, - "paper_bgcolor": "white", - "plot_bgcolor": "#E5ECF6", - "polar": { - "angularaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "bgcolor": "#E5ECF6", - "radialaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - } - }, - "scene": { - "xaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - }, - "yaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - }, - "zaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - } - }, - "shapedefaults": { - "line": { - "color": "#2a3f5f" - } - }, - "ternary": { - "aaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "baxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "bgcolor": "#E5ECF6", - "caxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - } - }, - "title": { - "x": 0.05 - }, - "xaxis": { - "automargin": true, - "gridcolor": "white", - "linecolor": "white", - "ticks": "", - "title": { - "standoff": 15 - }, - "zerolinecolor": "white", - "zerolinewidth": 2 - }, - "yaxis": { - "automargin": true, - "gridcolor": "white", - "linecolor": "white", - "ticks": "", - "title": { - "standoff": 15 - }, - "zerolinecolor": "white", - "zerolinewidth": 2 - } - } - }, - "title": { - "text": "Fuzzer performance" - }, - "width": 1200, - "xaxis": { - "anchor": "y", - "domain": [ - 0, - 1 - ], - "tickvals": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36 - ], - "title": { - "text": "Number of parallel fuzzers" - } - }, - "yaxis": { - "anchor": "x", - "domain": [ - 0, - 1 - ], - "ticktext": [ - "1x", - "26x", - "51x", - "75x", - "100x", - "125x" - ], - "tickvals": [ - 11019.3, - 284224.18399999995, - 557429.068, - 830633.9519999999, - 1103838.836, - 1377043.72 - ], - "title": { - "text": "Fuzz target executions per second" - } - } - } - }, - "text/html": [ - "<div> <div id=\"80d54fb8-8ae8-4543-ba3c-c1846c8bbc5b\" class=\"plotly-graph-div\" style=\"height:400px; width:1200px;\"></div> <script type=\"text/javascript\"> require([\"plotly\"], function(Plotly) { window.PLOTLYENV=window.PLOTLYENV || {}; if (document.getElementById(\"80d54fb8-8ae8-4543-ba3c-c1846c8bbc5b\")) { Plotly.newPlot( \"80d54fb8-8ae8-4543-ba3c-c1846c8bbc5b\", [{\"hovertemplate\":\"Configuration=Multicore: Non-persistent mode + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: Non-persistent mode + kernel config\",\"line\":{\"color\":\"#636efa\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: Non-persistent mode + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[11019.3,21111.92,30568.82,38963.07,47693.65,55718.73,64156.79,72176.39,61257.76,67507.14,73183.59,79167.7,85202.55,91594.86,97682.33,103765.38,84547.71,86611.67,88657.0,90134.42,91033.28,91637.86,92747.81,93207.38,92970.87,94162.8,94237.96,91716.1,94072.6,95644.79,94880.56,93460.57,94194.83,93853.08,95147.78,92697.06],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Multicore: Persistent mode\\u002fshared memory + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: Persistent mode\\u002fshared memory + kernel config\",\"line\":{\"color\":\"#EF553B\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: Persistent mode\\u002fshared memory + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[133591.26,255995.07,380246.34,490254.72,598698.16,716786.21,824873.02,942712.02,709716.24,779115.44,851918.03,914375.37,990573.31,1060551.02,1134650.66,1203287.99,1022498.84,1058151.58,1076742.64,1091743.7,1062616.36,1081621.57,1132929.86,1133825.45,1114215.27,1122210.96,1130627.8,1135890.71,1137390.94,1142969.21,1149056.35,1142131.87,1128973.67,1116863.53,1117913.34,1124962.18],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Multicore: Persistent mode\\u002fshared memory without kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: Persistent mode\\u002fshared memory without kernel config\",\"line\":{\"color\":\"#00cc96\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: Persistent mode\\u002fshared memory without kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[90851.4,176159.32,260268.78,336355.99,413750.0,492578.2,565737.17,640579.35,491284.66,540759.63,591144.78,638938.52,687954.18,734886.87,784210.26,834811.24,714178.66,734804.4,740714.93,750425.99,752625.52,760661.34,758401.65,767985.22,785474.61,789679.78,790483.94,798176.69,792763.39,788972.72,789307.69,777440.02,767985.22,775967.55,764863.41,768616.31],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\",\"line\":{\"color\":\"#ab63fa\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[134423.5,258490.04,383777.45,496249.48,613089.31,730366.19,844187.32,962846.18,997414.74,1034757.73,1070703.42,1104249.08,1131176.88,1164076.48,1198824.47,1227578.7,1272311.96,1295688.8,1314398.6,1328581.94,1342660.66,1363930.3,1377043.72,1375818.24,1361687.56,1369637.56,1375444.16,1349599.77,1321658.08,1301868.24,1276904.9,1243444.8,1243981.21,1234425.98,1244349.38,1250454.58],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Singlecore: Non-persistent mode + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Singlecore: Non-persistent mode + kernel config\",\"line\":{\"color\":\"#FFA15A\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Singlecore: Non-persistent mode + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96,11038.96],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Singlecore: Persistent mode\\u002fshared memory + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Singlecore: Persistent mode\\u002fshared memory + kernel config\",\"line\":{\"color\":\"#19d3f3\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Singlecore: Persistent mode\\u002fshared memory + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0],\"xaxis\":\"x\",\"y\":[135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26,135613.26],\"yaxis\":\"y\",\"type\":\"scatter\"}], {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Number of parallel fuzzers\"},\"tickvals\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Fuzz target executions per second\"},\"tickvals\":[11019.3,284224.18399999995,557429.068,830633.9519999999,1103838.836,1377043.72],\"ticktext\":[\"1x\",\"26x\",\"51x\",\"75x\",\"100x\",\"125x\"]},\"legend\":{\"title\":{\"text\":\"Configuration\"},\"tracegroupgap\":0},\"title\":{\"text\":\"Fuzzer performance\"},\"width\":1200,\"height\":400}, {\"responsive\": true} ).then(function(){\n", - " \n", - "var gd = document.getElementById('80d54fb8-8ae8-4543-ba3c-c1846c8bbc5b');\n", - "var x = new MutationObserver(function (mutations, observer) {{\n", - " var display = window.getComputedStyle(gd).display;\n", - " if (!display || display === 'none') {{\n", - " console.log([gd, 'removed!']);\n", - " Plotly.purge(gd);\n", - " observer.disconnect();\n", - " }}\n", - "}});\n", - "\n", - "// Listen for the removal of the full notebook cells\n", - "var notebookContainer = gd.closest('#notebook-container');\n", - "if (notebookContainer) {{\n", - " x.observe(notebookContainer, {childList: true});\n", - "}}\n", - "\n", - "// Listen for the clearing of the current output cell\n", - "var outputEl = gd.closest('.output');\n", - "if (outputEl) {{\n", - " x.observe(outputEl, {childList: true});\n", - "}}\n", - "\n", - " }) }; }); </script> </div>" + "image/svg+xml": [ + "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"1200\" height=\"400\" style=\"\" viewBox=\"0 0 1200 400\"><rect x=\"0\" y=\"0\" width=\"1200\" height=\"400\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-4c17ba\"><g class=\"clips\"><clipPath id=\"clip4c17baxyplot\" class=\"plotclip\"><rect width=\"632\" height=\"220\"/></clipPath><clipPath class=\"axesclip\" id=\"clip4c17bax\"><rect x=\"80\" y=\"0\" width=\"632\" height=\"400\"/></clipPath><clipPath class=\"axesclip\" id=\"clip4c17bay\"><rect x=\"0\" y=\"100\" width=\"1200\" height=\"220\"/></clipPath><clipPath class=\"axesclip\" id=\"clip4c17baxy\"><rect x=\"80\" y=\"100\" width=\"632\" height=\"220\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"632\" height=\"220\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(98.06,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(116.11,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(134.17000000000002,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(152.23000000000002,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(170.29000000000002,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(188.34,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(206.4,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(224.46,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(242.51,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(260.57,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(278.63,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(296.69,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(314.74,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(332.8,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(350.86,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(368.91,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(386.97,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(405.03,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(423.09,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(441.14,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(459.2,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(477.26,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(495.31,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(513.37,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(531.4300000000001,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(549.49,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(567.54,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(585.6,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(603.66,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(621.71,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(639.77,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(657.83,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(675.89,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(693.94,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,309)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,269.4)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,229.8)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,190.2)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,150.6)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,111)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,310.6)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip4c17baxyplot)\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace57c310\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,209L72.23,203.68L90.29,202.52L126.4,200.14L144.46,201.72L270.86,195.56L288.91,198.34L397.26,197.15L415.31,197.09L632,197.16\" style=\"vector-effect: none; fill: none; stroke: rgb(99, 110, 250); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace634c70\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,191.23L36.11,155.48L54.17,139.54L72.23,123.82L90.29,106.7L108.34,91.04L126.4,73.95L144.46,107.73L162.51,97.67L180.57,87.11L198.63,78.06L216.69,67.02L234.74,56.87L270.86,36.19L288.91,62.39L306.97,57.22L325.03,54.53L343.09,52.35L361.14,56.58L379.2,53.82L397.26,46.38L415.31,46.25L433.37,49.1L487.54,45.95L505.6,45.74L541.71,44.05L559.77,45.05L595.89,48.71L613.94,48.56L632,47.54\" style=\"vector-effect: none; fill: none; stroke: rgb(239, 85, 59); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace9a1458\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,197.43L36.11,172.87L54.17,161.84L108.34,128.6L126.4,117.75L144.46,139.39L162.51,132.22L216.69,110.88L234.74,104.08L270.86,89.59L288.91,107.08L306.97,104.09L325.03,103.23L343.09,101.83L361.14,101.51L379.2,100.34L397.26,100.67L415.31,99.28L433.37,96.75L487.54,94.9L505.6,95.69L541.71,96.19L559.77,97.91L577.83,99.28L595.89,98.12L613.94,99.73L632,99.19\" style=\"vector-effect: none; fill: none; stroke: rgb(0, 204, 150); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter tracea2f422\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,191.11L36.11,154.97L54.17,138.67L126.4,71.04L144.46,66.03L198.63,50.54L216.69,46.64L270.86,32.66L288.91,26.18L325.03,20.08L343.09,18.02L361.14,15.98L379.2,12.9L397.26,11L415.31,11.18L433.37,13.23L451.43,12.07L469.49,11.23L487.54,14.98L505.6,19.03L523.66,21.9L541.71,25.51L559.77,30.36L577.83,30.29L595.89,31.67L632,29.35\" style=\"vector-effect: none; fill: none; stroke: rgb(171, 99, 250); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace461284\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,209L632,209\" style=\"vector-effect: none; fill: none; stroke: rgb(255, 161, 90); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace7157c1\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,190.94L632,190.94\" style=\"vector-effect: none; fill: none; stroke: rgb(25, 211, 243); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" transform=\"translate(80,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(98.06,0)\">2</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(116.11,0)\">3</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(134.17000000000002,0)\">4</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(152.23000000000002,0)\">5</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(170.29000000000002,0)\">6</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(188.34,0)\">7</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(206.4,0)\">8</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(224.46,0)\">9</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(242.51,0)\">10</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(260.57,0)\">11</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(278.63,0)\">12</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(296.69,0)\">13</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(314.74,0)\">14</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(332.8,0)\">15</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(350.86,0)\">16</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(368.91,0)\">17</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(386.97,0)\">18</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(405.03,0)\">19</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(423.09,0)\">20</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(441.14,0)\">21</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(459.2,0)\">22</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(477.26,0)\">23</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(495.31,0)\">24</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(513.37,0)\">25</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(531.4300000000001,0)\">26</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(549.49,0)\">27</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(567.54,0)\">28</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(585.6,0)\">29</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(603.66,0)\">30</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(621.71,0)\">31</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(639.77,0)\">32</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(657.83,0)\">33</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(675.89,0)\">34</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(693.94,0)\">35</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(712,0)\">36</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,309)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,269.4)\">26x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,229.8)\">51x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,190.2)\">75x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,150.6)\">100x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,111)\">125x</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-4c17ba\"><g class=\"clips\"/><clipPath id=\"legend4c17ba\"><rect width=\"463\" height=\"143\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(724.64,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"463\" height=\"143\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend4c17ba)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Configuration</text><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: Non-persistent mode + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(99, 110, 250); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: Persistent mode/shared memory + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(239, 85, 59); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,70.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: Persistent mode/shared memory without kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(0, 204, 150); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,89.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: afl_execs: Persistent mode/shared memory + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(171, 99, 250); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,108.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Singlecore: Non-persistent mode + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(255, 161, 90); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,127.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Singlecore: Persistent mode/shared memory + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(25, 211, 243); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"60\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Fuzzer performance</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"396\" y=\"360.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Number of parallel fuzzers</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,24.684375000000003,210)\" x=\"24.684375000000003\" y=\"210\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Fuzz target executions per second</text></g></g></svg>" ] }, "metadata": {}, @@ -2066,7 +484,6 @@ } ], "source": [ - "\n", "import numpy as np\n", "pd.options.plotting.backend = \"plotly\"\n", "\n", @@ -2088,7 +505,8 @@ "# Update the primary Y-axis with custom tick labels\n", "fig.update_yaxes(tickvals=tickvals, ticktext=ticktext)\n", "fig.update_xaxes(tickvals=list(range(1,36+1)))\n", - "fig.update_layout(width=1200, height=400)\n" + "fig.update_layout(width=1200, height=400)\n", + "fig.show(\"svg\")\n" ] }, { @@ -2100,7 +518,7 @@ }, { "cell_type": "code", - "execution_count": 508, + "execution_count": 18, "metadata": {}, "outputs": [ { @@ -2241,7 +659,7 @@ "5.0 135613.26 " ] }, - "execution_count": 508, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -2259,7 +677,7 @@ }, { "cell_type": "code", - "execution_count": 509, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -2318,7 +736,7 @@ "<IPython.core.display.Markdown object>" ] }, - "execution_count": 509, + "execution_count": 19, "metadata": {}, "output_type": "execute_result" } @@ -2402,7 +820,7 @@ }, { "cell_type": "code", - "execution_count": 521, + "execution_count": 20, "metadata": {}, "outputs": [ { @@ -2537,7 +955,7 @@ "149 6.15 " ] }, - "execution_count": 521, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -2549,7 +967,7 @@ }, { "cell_type": "code", - "execution_count": 516, + "execution_count": 21, "metadata": {}, "outputs": [ { @@ -2611,7 +1029,7 @@ "153 True Multicore: afl_execs: Persistent mode/shared m... " ] }, - "execution_count": 516, + "execution_count": 21, "metadata": {}, "output_type": "execute_result" } @@ -2623,1796 +1041,13 @@ }, { "cell_type": "code", - "execution_count": 514, + "execution_count": 22, "metadata": {}, "outputs": [ { "data": { - "application/vnd.plotly.v1+json": { - "config": { - "plotlyServerURL": "https://plot.ly" - }, - "data": [ - { - "hovertemplate": "Configuration=Multicore: Persistent mode/shared memory + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: Persistent mode/shared memory + kernel config", - "line": { - "color": "#636efa", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: Persistent mode/shared memory + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36, - 37, - 38, - 39, - 40, - 41, - 42, - 43, - 44, - 45, - 46, - 47, - 48, - 49, - 50, - 51, - 52, - 53, - 54, - 55, - 56, - 57, - 58, - 59, - 60, - 61, - 62, - 63, - 64, - 65, - 66, - 67, - 68, - 69, - 70, - 71, - 72, - 73, - 74, - 75, - 76, - 77, - 78, - 79, - 80, - 81, - 82, - 83, - 84, - 85, - 86, - 87, - 88, - 89, - 90, - 91, - 92, - 93, - 94, - 95, - 96, - 97, - 98, - 99, - 100, - 101, - 102, - 103, - 104, - 105, - 106, - 107, - 108, - 109, - 110, - 111, - 112, - 113, - 114, - 115, - 116, - 117, - 118, - 119, - 120, - 121, - 122, - 123, - 124, - 125, - 126, - 127, - 128, - 129, - 130, - 131, - 132, - 133, - 134, - 135, - 136, - 137, - 138, - 139, - 140, - 141, - 142, - 143, - 144, - 145, - 146, - 147, - 148, - 149, - 150, - 151, - 152, - 153, - 154, - 155, - 156, - 157, - 158, - 159, - 160, - 161, - 162, - 163, - 164, - 165, - 166, - 167, - 168, - 169, - 170, - 171, - 172, - 173, - 174, - 175, - 176, - 177, - 178, - 179, - 180, - 181, - 182, - 183, - 184, - 185, - 186, - 187, - 188, - 189, - 190, - 191, - 192, - 193, - 194, - 195, - 196, - 197, - 198, - 199, - 200 - ], - "xaxis": "x", - "y": [ - 105839.1, - 210819.47, - 316229.21, - 413256.46, - 518632.73, - 619884.69, - 702256.76, - 804131.53, - 847288.04, - 943139.75, - 1024439.07, - 1113578.57, - 1162772.81, - 1265283.48, - 1346295.34, - 1421319.66, - 1389055.03, - 1336294.29, - 1354421.12, - 1341083.87, - 1267487.8, - 1228006.44, - 1164952.24, - 1117569.89, - 1027015.81, - 1011333.83, - 993703.26, - 983824.21, - 969783.14, - 953522.94, - 937704.89, - 914207.81, - 898800.31, - 896437.34, - 895982.76, - 896412.07, - 888119.63, - 874168.22, - 858049.53, - 839870.71, - 830337.88, - 833695.19, - 832246.18, - 831472, - 820819.59, - 809235.61, - 796104.63, - 779748.67, - 651915.77, - 658310.11, - 664906.42, - 670874.88, - 679223.43, - 680460.23, - 683449.31, - 683776.32, - 642820.96, - 656234.7, - 664079.06, - 668815.96, - 677202.95, - 682183.78, - 687798.53, - 695065.41, - 653862.76, - 662255.65, - 667137.25, - 674252.38, - 674769.27, - 676779.76, - 678994.74, - 677707.66, - 643636.07, - 645011.41, - 647860.09, - 650443.61, - 655762.21, - 655894.58, - 659395.36, - 660632.48, - 636619.43, - 636679.37, - 641091.3, - 642134.44, - 644188.04, - 645646.45, - 648190.91, - 645188.95, - 623576.69, - 623521.92, - 627188.81, - 631735.31, - 632169.99, - 632104.98, - 636441.28, - 636413.09, - 627747.11, - 618580.91, - 610289.87, - 602517.1, - 587297.53, - 574782.55, - 559428.41, - 543938.93, - 538120.81, - 531555.69, - 526211.57, - 521796.77, - 510905.75, - 501877.11, - 490630.76, - 479788.36, - 475372.8, - 470776.14, - 466163.29, - 463421.1, - 454522.49, - 446589.03, - 438649.5, - 428977.64, - 424292.46, - 418867.74, - 414980.74, - 412384.2, - 404224.17, - 396310.94, - 389392.67, - 381715.99, - 377825.09, - 374215.78, - 370708.15, - 368402.32, - 361940.65, - 355502.6, - 349589.01, - 342501.52, - 339897.15, - 337989.11, - 335723.22, - 334039.83, - 329127.04, - 323073.5, - 318652.21, - 313198.69, - 311591.71, - 310530.42, - 309390.36, - 307977.56, - 306559.97, - 305066.46, - 302157.29, - 301097.52, - 300138.11, - 299824.88, - 298130.63, - 297838.27, - 296189.21, - 294695.11, - 292518.21, - 291030.08, - 290207.87, - 289828.9, - 289247.99, - 288295.58, - 287447.58, - 286405.31, - 284755.57, - 284046.79, - 283176.27, - 283053.87, - 282178.36, - 281522.15, - 280377.23, - 279885.55, - 278793.08, - 277923.89, - 277069.78, - 277124.24, - 276576.49, - 276020.64, - 275328, - 275029.74, - 274030.79, - 273612.38, - 273285.13, - 273039.33, - 272819.57, - 272960.42, - 272388.01, - 272311.26, - 272115.97, - 272056.42, - 271835.4, - 271397.63, - 271867.2, - 271065.21, - 270797.96, - 270150.29, - 269442.01, - 268674.91 - ], - "yaxis": "y" - }, - { - "hovertemplate": "Configuration=Multicore: afl_execs: Persistent mode/shared memory + kernel config<br>Number of parallel fuzzers=%{x}<br>Fuzz target executions per second=%{y}<extra></extra>", - "legendgroup": "Multicore: afl_execs: Persistent mode/shared memory + kernel config", - "line": { - "color": "#EF553B", - "dash": "solid" - }, - "marker": { - "symbol": "circle" - }, - "mode": "lines", - "name": "Multicore: afl_execs: Persistent mode/shared memory + kernel config", - "orientation": "v", - "showlegend": true, - "type": "scatter", - "x": [ - 1, - 2, - 3, - 4, - 5, - 6, - 7, - 8, - 9, - 10, - 11, - 12, - 13, - 14, - 15, - 16, - 17, - 18, - 19, - 20, - 21, - 22, - 23, - 24, - 25, - 26, - 27, - 28, - 29, - 30, - 31, - 32, - 33, - 34, - 35, - 36, - 37, - 38, - 39, - 40, - 41, - 42, - 43, - 44, - 45, - 46, - 47, - 48, - 49, - 50, - 51, - 52, - 53, - 54, - 55, - 56, - 57, - 58, - 59, - 60, - 61, - 62, - 63, - 64, - 65, - 66, - 67, - 68, - 69, - 70, - 71, - 72, - 73, - 74, - 75, - 76, - 77, - 78, - 79, - 80, - 81, - 82, - 83, - 84, - 85, - 86, - 87, - 88, - 89, - 90, - 91, - 92, - 93, - 94, - 95, - 96, - 97, - 98, - 99, - 100, - 101, - 102, - 103, - 104, - 105, - 106, - 107, - 108, - 109, - 110, - 111, - 112, - 113, - 114, - 115, - 116, - 117, - 118, - 119, - 120, - 121, - 122, - 123, - 124, - 125, - 126, - 127, - 128, - 129, - 130, - 131, - 132, - 133, - 134, - 135, - 136, - 137, - 138, - 139, - 140, - 141, - 142, - 143, - 144, - 145, - 146, - 147, - 148, - 149, - 150, - 151, - 152, - 153, - 154, - 155, - 156, - 157, - 158, - 159, - 160, - 161, - 162, - 163, - 164, - 165, - 166, - 167, - 168, - 169, - 170, - 171, - 172, - 173, - 174, - 175, - 176, - 177, - 178, - 179, - 180, - 181, - 182, - 183, - 184, - 185, - 186, - 187, - 188, - 189, - 190, - 191, - 192, - 193, - 194, - 195, - 196, - 197, - 198, - 199, - 200 - ], - "xaxis": "x", - "y": [ - 107126.58, - 214213.66, - 322468.69, - 427406.37, - 535728.44, - 643227.5, - 746997.96, - 852324.44, - 898199.42, - 994921.42, - 1086491.72, - 1188114.32, - 1275638.92, - 1373504.32, - 1456611.99, - 1547050.37, - 1556304.25, - 1498337.65, - 1480610.39, - 1442181.26, - 1380390.08, - 1315149.42, - 1250840.52, - 1198962.91, - 1113901.96, - 1112866.02, - 1109572.68, - 1112386.81, - 1104839.85, - 1088259.95, - 1072951.12, - 1038335.3, - 1039005.59, - 1055683.11, - 1074708.24, - 1088882.64, - 1084369.02, - 1061476.09, - 1037330.81, - 1001283.07, - 1011982.42, - 1039061.07, - 1060191.68, - 1069379.92, - 1051676.06, - 1025702.93, - 1000795.88, - 959941, - 928333.9, - 936392.44, - 947163.68, - 958614.58, - 973982.54, - 976113.12, - 983432.87, - 985159.38, - 949664.42, - 960540.52, - 971717.37, - 978223.94, - 995090.76, - 1000123.55, - 1006856.18, - 1013280.29, - 977531.19, - 988260.54, - 996765.65, - 1006933, - 1016151.03, - 1020419.88, - 1024544.66, - 1027862.2, - 989415.52, - 999208.44, - 1009747.84, - 1016122.1, - 1026766.44, - 1032416.84, - 1037369.06, - 1037677.89, - 1001527.34, - 1008569.78, - 1024112.93, - 1033177.84, - 1035389.26, - 1040484.52, - 1047416.67, - 1043614.54, - 1014160.19, - 1019409.94, - 1033667.5, - 1040422.32, - 1045409.98, - 1048162.62, - 1050384.15, - 1050304.88, - 1037251.1, - 1023279.61, - 1009889.86, - 996157.16, - 973425.48, - 960922.5, - 941705.52, - 927206.03, - 919716.12, - 907116.8, - 898444.05, - 889678.68, - 871535.65, - 858369.28, - 839357.6, - 828077.49, - 817619.99, - 806563.34, - 795820.84, - 789602.32, - 769744.98, - 754704.16, - 739965.24, - 721357.74, - 705584.89, - 689179.3, - 674153.86, - 668264.05, - 648129.94, - 630733.08, - 614518.38, - 598284.67, - 580642.38, - 562735.32, - 547668.6, - 540727.65, - 519637, - 499189.04, - 482457.86, - 458655.34, - 453087.56, - 445650.76, - 438779.54, - 434421.26, - 422130, - 403403.62, - 391528.74, - 374715.06, - 372678.44, - 371466.33, - 369815.4, - 367734.06, - 366332.54, - 365256.89, - 362078.84, - 361083.46, - 359994.43, - 359950.89, - 357498.64, - 357285.14, - 355405.08, - 354127.44, - 351793.59, - 350348, - 349438.44, - 349188.38, - 348377.38, - 347124.06, - 346480.82, - 345660.61, - 344352.86, - 343903.25, - 342402.74, - 342935.7, - 342089.26, - 341369.47, - 340166.19, - 339692.96, - 339204.8, - 338925.12, - 337700.46, - 338203.76, - 337556.9, - 336873.92, - 336399.84, - 336455.79, - 335823.56, - 335587.52, - 335620.09, - 334996.68, - 334980.98, - 335404.84, - 335051.8, - 334887.42, - 335150.96, - 334773.71, - 335035.28, - 334596.91, - 336065.8, - 335034.33, - 334931.36, - 334191.98, - 332929.11, - 331957.22 - ], - "yaxis": "y" - } - ], - "layout": { - "height": 400, - "legend": { - "title": { - "text": "Configuration" - }, - "tracegroupgap": 0 - }, - "template": { - "data": { - "bar": [ - { - "error_x": { - "color": "#2a3f5f" - }, - "error_y": { - "color": "#2a3f5f" - }, - "marker": { - "line": { - "color": "#E5ECF6", - "width": 0.5 - }, - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "bar" - } - ], - "barpolar": [ - { - "marker": { - "line": { - "color": "#E5ECF6", - "width": 0.5 - }, - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "barpolar" - } - ], - "carpet": [ - { - "aaxis": { - "endlinecolor": "#2a3f5f", - "gridcolor": "white", - "linecolor": "white", - "minorgridcolor": "white", - "startlinecolor": "#2a3f5f" - }, - "baxis": { - "endlinecolor": "#2a3f5f", - "gridcolor": "white", - "linecolor": "white", - "minorgridcolor": "white", - "startlinecolor": "#2a3f5f" - }, - "type": "carpet" - } - ], - "choropleth": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "choropleth" - } - ], - "contour": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "contour" - } - ], - "contourcarpet": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "contourcarpet" - } - ], - "heatmap": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "heatmap" - } - ], - "heatmapgl": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "heatmapgl" - } - ], - "histogram": [ - { - "marker": { - "pattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - } - }, - "type": "histogram" - } - ], - "histogram2d": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "histogram2d" - } - ], - "histogram2dcontour": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "histogram2dcontour" - } - ], - "mesh3d": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "type": "mesh3d" - } - ], - "parcoords": [ - { - "line": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "parcoords" - } - ], - "pie": [ - { - "automargin": true, - "type": "pie" - } - ], - "scatter": [ - { - "fillpattern": { - "fillmode": "overlay", - "size": 10, - "solidity": 0.2 - }, - "type": "scatter" - } - ], - "scatter3d": [ - { - "line": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatter3d" - } - ], - "scattercarpet": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattercarpet" - } - ], - "scattergeo": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattergeo" - } - ], - "scattergl": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattergl" - } - ], - "scattermapbox": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scattermapbox" - } - ], - "scatterpolar": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterpolar" - } - ], - "scatterpolargl": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterpolargl" - } - ], - "scatterternary": [ - { - "marker": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "type": "scatterternary" - } - ], - "surface": [ - { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - }, - "colorscale": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "type": "surface" - } - ], - "table": [ - { - "cells": { - "fill": { - "color": "#EBF0F8" - }, - "line": { - "color": "white" - } - }, - "header": { - "fill": { - "color": "#C8D4E3" - }, - "line": { - "color": "white" - } - }, - "type": "table" - } - ] - }, - "layout": { - "annotationdefaults": { - "arrowcolor": "#2a3f5f", - "arrowhead": 0, - "arrowwidth": 1 - }, - "autotypenumbers": "strict", - "coloraxis": { - "colorbar": { - "outlinewidth": 0, - "ticks": "" - } - }, - "colorscale": { - "diverging": [ - [ - 0, - "#8e0152" - ], - [ - 0.1, - "#c51b7d" - ], - [ - 0.2, - "#de77ae" - ], - [ - 0.3, - "#f1b6da" - ], - [ - 0.4, - "#fde0ef" - ], - [ - 0.5, - "#f7f7f7" - ], - [ - 0.6, - "#e6f5d0" - ], - [ - 0.7, - "#b8e186" - ], - [ - 0.8, - "#7fbc41" - ], - [ - 0.9, - "#4d9221" - ], - [ - 1, - "#276419" - ] - ], - "sequential": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ], - "sequentialminus": [ - [ - 0, - "#0d0887" - ], - [ - 0.1111111111111111, - "#46039f" - ], - [ - 0.2222222222222222, - "#7201a8" - ], - [ - 0.3333333333333333, - "#9c179e" - ], - [ - 0.4444444444444444, - "#bd3786" - ], - [ - 0.5555555555555556, - "#d8576b" - ], - [ - 0.6666666666666666, - "#ed7953" - ], - [ - 0.7777777777777778, - "#fb9f3a" - ], - [ - 0.8888888888888888, - "#fdca26" - ], - [ - 1, - "#f0f921" - ] - ] - }, - "colorway": [ - "#636efa", - "#EF553B", - "#00cc96", - "#ab63fa", - "#FFA15A", - "#19d3f3", - "#FF6692", - "#B6E880", - "#FF97FF", - "#FECB52" - ], - "font": { - "color": "#2a3f5f" - }, - "geo": { - "bgcolor": "white", - "lakecolor": "white", - "landcolor": "#E5ECF6", - "showlakes": true, - "showland": true, - "subunitcolor": "white" - }, - "hoverlabel": { - "align": "left" - }, - "hovermode": "closest", - "mapbox": { - "style": "light" - }, - "paper_bgcolor": "white", - "plot_bgcolor": "#E5ECF6", - "polar": { - "angularaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "bgcolor": "#E5ECF6", - "radialaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - } - }, - "scene": { - "xaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - }, - "yaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - }, - "zaxis": { - "backgroundcolor": "#E5ECF6", - "gridcolor": "white", - "gridwidth": 2, - "linecolor": "white", - "showbackground": true, - "ticks": "", - "zerolinecolor": "white" - } - }, - "shapedefaults": { - "line": { - "color": "#2a3f5f" - } - }, - "ternary": { - "aaxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "baxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - }, - "bgcolor": "#E5ECF6", - "caxis": { - "gridcolor": "white", - "linecolor": "white", - "ticks": "" - } - }, - "title": { - "x": 0.05 - }, - "xaxis": { - "automargin": true, - "gridcolor": "white", - "linecolor": "white", - "ticks": "", - "title": { - "standoff": 15 - }, - "zerolinecolor": "white", - "zerolinewidth": 2 - }, - "yaxis": { - "automargin": true, - "gridcolor": "white", - "linecolor": "white", - "ticks": "", - "title": { - "standoff": 15 - }, - "zerolinecolor": "white", - "zerolinewidth": 2 - } - } - }, - "title": { - "text": "Fuzzer performance" - }, - "width": 1200, - "xaxis": { - "anchor": "y", - "domain": [ - 0, - 1 - ], - "tickvals": [ - 0, - 5, - 10, - 15, - 20, - 25, - 30, - 35, - 40, - 45, - 50, - 55, - 60, - 65, - 70, - 75, - 80, - 85, - 90, - 95, - 100, - 105, - 110, - 115, - 120, - 125, - 130, - 135, - 140, - 145, - 150, - 155, - 160, - 165, - 170, - 175, - 180, - 185, - 190, - 195, - 200 - ], - "title": { - "text": "Number of parallel fuzzers" - } - }, - "yaxis": { - "anchor": "x", - "domain": [ - 0, - 1 - ], - "ticktext": [ - "10x", - "36x", - "62x", - "89x", - "115x", - "141x" - ], - "tickvals": [ - 105839.1, - 395932.13, - 686025.1599999999, - 976118.1899999998, - 1266211.22, - 1556304.25 - ], - "title": { - "text": "Fuzz target executions per second" - } - } - } - }, - "text/html": [ - "<div> <div id=\"12f69392-4283-47e6-a491-90bdd9508929\" class=\"plotly-graph-div\" style=\"height:400px; width:1200px;\"></div> <script type=\"text/javascript\"> require([\"plotly\"], function(Plotly) { window.PLOTLYENV=window.PLOTLYENV || {}; if (document.getElementById(\"12f69392-4283-47e6-a491-90bdd9508929\")) { Plotly.newPlot( \"12f69392-4283-47e6-a491-90bdd9508929\", [{\"hovertemplate\":\"Configuration=Multicore: Persistent mode\\u002fshared memory + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: Persistent mode\\u002fshared memory + kernel config\",\"line\":{\"color\":\"#636efa\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: Persistent mode\\u002fshared memory + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0,37.0,38.0,39.0,40.0,41.0,42.0,43.0,44.0,45.0,46.0,47.0,48.0,49.0,50.0,51.0,52.0,53.0,54.0,55.0,56.0,57.0,58.0,59.0,60.0,61.0,62.0,63.0,64.0,65.0,66.0,67.0,68.0,69.0,70.0,71.0,72.0,73.0,74.0,75.0,76.0,77.0,78.0,79.0,80.0,81.0,82.0,83.0,84.0,85.0,86.0,87.0,88.0,89.0,90.0,91.0,92.0,93.0,94.0,95.0,96.0,97.0,98.0,99.0,100.0,101.0,102.0,103.0,104.0,105.0,106.0,107.0,108.0,109.0,110.0,111.0,112.0,113.0,114.0,115.0,116.0,117.0,118.0,119.0,120.0,121.0,122.0,123.0,124.0,125.0,126.0,127.0,128.0,129.0,130.0,131.0,132.0,133.0,134.0,135.0,136.0,137.0,138.0,139.0,140.0,141.0,142.0,143.0,144.0,145.0,146.0,147.0,148.0,149.0,150.0,151.0,152.0,153.0,154.0,155.0,156.0,157.0,158.0,159.0,160.0,161.0,162.0,163.0,164.0,165.0,166.0,167.0,168.0,169.0,170.0,171.0,172.0,173.0,174.0,175.0,176.0,177.0,178.0,179.0,180.0,181.0,182.0,183.0,184.0,185.0,186.0,187.0,188.0,189.0,190.0,191.0,192.0,193.0,194.0,195.0,196.0,197.0,198.0,199.0,200.0],\"xaxis\":\"x\",\"y\":[105839.1,210819.47,316229.21,413256.46,518632.73,619884.69,702256.76,804131.53,847288.04,943139.75,1024439.07,1113578.57,1162772.81,1265283.48,1346295.34,1421319.66,1389055.03,1336294.29,1354421.12,1341083.87,1267487.8,1228006.44,1164952.24,1117569.89,1027015.81,1011333.83,993703.26,983824.21,969783.14,953522.94,937704.89,914207.81,898800.31,896437.34,895982.76,896412.07,888119.63,874168.22,858049.53,839870.71,830337.88,833695.19,832246.18,831472.0,820819.59,809235.61,796104.63,779748.67,651915.77,658310.11,664906.42,670874.88,679223.43,680460.23,683449.31,683776.32,642820.96,656234.7,664079.06,668815.96,677202.95,682183.78,687798.53,695065.41,653862.76,662255.65,667137.25,674252.38,674769.27,676779.76,678994.74,677707.66,643636.07,645011.41,647860.09,650443.61,655762.21,655894.58,659395.36,660632.48,636619.43,636679.37,641091.3,642134.44,644188.04,645646.45,648190.91,645188.95,623576.69,623521.92,627188.81,631735.31,632169.99,632104.98,636441.28,636413.09,627747.11,618580.91,610289.87,602517.1,587297.53,574782.55,559428.41,543938.93,538120.81,531555.69,526211.57,521796.77,510905.75,501877.11,490630.76,479788.36,475372.8,470776.14,466163.29,463421.1,454522.49,446589.03,438649.5,428977.64,424292.46,418867.74,414980.74,412384.2,404224.17,396310.94,389392.67,381715.99,377825.09,374215.78,370708.15,368402.32,361940.65,355502.6,349589.01,342501.52,339897.15,337989.11,335723.22,334039.83,329127.04,323073.5,318652.21,313198.69,311591.71,310530.42,309390.36,307977.56,306559.97,305066.46,302157.29,301097.52,300138.11,299824.88,298130.63,297838.27,296189.21,294695.11,292518.21,291030.08,290207.87,289828.9,289247.99,288295.58,287447.58,286405.31,284755.57,284046.79,283176.27,283053.87,282178.36,281522.15,280377.23,279885.55,278793.08,277923.89,277069.78,277124.24,276576.49,276020.64,275328.0,275029.74,274030.79,273612.38,273285.13,273039.33,272819.57,272960.42,272388.01,272311.26,272115.97,272056.42,271835.4,271397.63,271867.2,271065.21,270797.96,270150.29,269442.01,268674.91],\"yaxis\":\"y\",\"type\":\"scatter\"},{\"hovertemplate\":\"Configuration=Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\\u003cbr\\u003eNumber of parallel fuzzers=%{x}\\u003cbr\\u003eFuzz target executions per second=%{y}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\",\"legendgroup\":\"Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\",\"line\":{\"color\":\"#EF553B\",\"dash\":\"solid\"},\"marker\":{\"symbol\":\"circle\"},\"mode\":\"lines\",\"name\":\"Multicore: afl_execs: Persistent mode\\u002fshared memory + kernel config\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0,37.0,38.0,39.0,40.0,41.0,42.0,43.0,44.0,45.0,46.0,47.0,48.0,49.0,50.0,51.0,52.0,53.0,54.0,55.0,56.0,57.0,58.0,59.0,60.0,61.0,62.0,63.0,64.0,65.0,66.0,67.0,68.0,69.0,70.0,71.0,72.0,73.0,74.0,75.0,76.0,77.0,78.0,79.0,80.0,81.0,82.0,83.0,84.0,85.0,86.0,87.0,88.0,89.0,90.0,91.0,92.0,93.0,94.0,95.0,96.0,97.0,98.0,99.0,100.0,101.0,102.0,103.0,104.0,105.0,106.0,107.0,108.0,109.0,110.0,111.0,112.0,113.0,114.0,115.0,116.0,117.0,118.0,119.0,120.0,121.0,122.0,123.0,124.0,125.0,126.0,127.0,128.0,129.0,130.0,131.0,132.0,133.0,134.0,135.0,136.0,137.0,138.0,139.0,140.0,141.0,142.0,143.0,144.0,145.0,146.0,147.0,148.0,149.0,150.0,151.0,152.0,153.0,154.0,155.0,156.0,157.0,158.0,159.0,160.0,161.0,162.0,163.0,164.0,165.0,166.0,167.0,168.0,169.0,170.0,171.0,172.0,173.0,174.0,175.0,176.0,177.0,178.0,179.0,180.0,181.0,182.0,183.0,184.0,185.0,186.0,187.0,188.0,189.0,190.0,191.0,192.0,193.0,194.0,195.0,196.0,197.0,198.0,199.0,200.0],\"xaxis\":\"x\",\"y\":[107126.58,214213.66,322468.69,427406.37,535728.44,643227.5,746997.96,852324.44,898199.42,994921.42,1086491.72,1188114.32,1275638.92,1373504.32,1456611.99,1547050.37,1556304.25,1498337.65,1480610.39,1442181.26,1380390.08,1315149.42,1250840.52,1198962.91,1113901.96,1112866.02,1109572.68,1112386.81,1104839.85,1088259.95,1072951.12,1038335.3,1039005.59,1055683.11,1074708.24,1088882.64,1084369.02,1061476.09,1037330.81,1001283.07,1011982.42,1039061.07,1060191.68,1069379.92,1051676.06,1025702.93,1000795.88,959941.0,928333.9,936392.44,947163.68,958614.58,973982.54,976113.12,983432.87,985159.38,949664.42,960540.52,971717.37,978223.94,995090.76,1000123.55,1006856.18,1013280.29,977531.19,988260.54,996765.65,1006933.0,1016151.03,1020419.88,1024544.66,1027862.2,989415.52,999208.44,1009747.84,1016122.1,1026766.44,1032416.84,1037369.06,1037677.89,1001527.34,1008569.78,1024112.93,1033177.84,1035389.26,1040484.52,1047416.67,1043614.54,1014160.19,1019409.94,1033667.5,1040422.32,1045409.98,1048162.62,1050384.15,1050304.88,1037251.1,1023279.61,1009889.86,996157.16,973425.48,960922.5,941705.52,927206.03,919716.12,907116.8,898444.05,889678.68,871535.65,858369.28,839357.6,828077.49,817619.99,806563.34,795820.84,789602.32,769744.98,754704.16,739965.24,721357.74,705584.89,689179.3,674153.86,668264.05,648129.94,630733.08,614518.38,598284.67,580642.38,562735.32,547668.6,540727.65,519637.0,499189.04,482457.86,458655.34,453087.56,445650.76,438779.54,434421.26,422130.0,403403.62,391528.74,374715.06,372678.44,371466.33,369815.4,367734.06,366332.54,365256.89,362078.84,361083.46,359994.43,359950.89,357498.64,357285.14,355405.08,354127.44,351793.59,350348.0,349438.44,349188.38,348377.38,347124.06,346480.82,345660.61,344352.86,343903.25,342402.74,342935.7,342089.26,341369.47,340166.19,339692.96,339204.8,338925.12,337700.46,338203.76,337556.9,336873.92,336399.84,336455.79,335823.56,335587.52,335620.09,334996.68,334980.98,335404.84,335051.8,334887.42,335150.96,334773.71,335035.28,334596.91,336065.8,335034.33,334931.36,334191.98,332929.11,331957.22],\"yaxis\":\"y\",\"type\":\"scatter\"}], {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Number of parallel fuzzers\"},\"tickvals\":[0,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195,200]},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Fuzz target executions per second\"},\"tickvals\":[105839.1,395932.13,686025.1599999999,976118.1899999998,1266211.22,1556304.25],\"ticktext\":[\"10x\",\"36x\",\"62x\",\"89x\",\"115x\",\"141x\"]},\"legend\":{\"title\":{\"text\":\"Configuration\"},\"tracegroupgap\":0},\"title\":{\"text\":\"Fuzzer performance\"},\"width\":1200,\"height\":400}, {\"responsive\": true} ).then(function(){\n", - " \n", - "var gd = document.getElementById('12f69392-4283-47e6-a491-90bdd9508929');\n", - "var x = new MutationObserver(function (mutations, observer) {{\n", - " var display = window.getComputedStyle(gd).display;\n", - " if (!display || display === 'none') {{\n", - " console.log([gd, 'removed!']);\n", - " Plotly.purge(gd);\n", - " observer.disconnect();\n", - " }}\n", - "}});\n", - "\n", - "// Listen for the removal of the full notebook cells\n", - "var notebookContainer = gd.closest('#notebook-container');\n", - "if (notebookContainer) {{\n", - " x.observe(notebookContainer, {childList: true});\n", - "}}\n", - "\n", - "// Listen for the clearing of the current output cell\n", - "var outputEl = gd.closest('.output');\n", - "if (outputEl) {{\n", - " x.observe(outputEl, {childList: true});\n", - "}}\n", - "\n", - " }) }; }); </script> </div>" + "image/svg+xml": [ + "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"1200\" height=\"400\" style=\"\" viewBox=\"0 0 1200 400\"><rect x=\"0\" y=\"0\" width=\"1200\" height=\"400\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-860347\"><g class=\"clips\"><clipPath id=\"clip860347xyplot\" class=\"plotclip\"><rect width=\"632\" height=\"220\"/></clipPath><clipPath class=\"axesclip\" id=\"clip860347x\"><rect x=\"80\" y=\"0\" width=\"632\" height=\"400\"/></clipPath><clipPath class=\"axesclip\" id=\"clip860347y\"><rect x=\"0\" y=\"100\" width=\"1200\" height=\"220\"/></clipPath><clipPath class=\"axesclip\" id=\"clip860347xy\"><rect x=\"80\" y=\"100\" width=\"632\" height=\"220\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"632\" height=\"220\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(92.7,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(108.58,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(124.46000000000001,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(140.34,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(156.22,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(172.1,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(187.98000000000002,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(203.86,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(219.74,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(235.62,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(251.5,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(267.38,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(283.26,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(299.14,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(315.02,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(330.89,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(346.77,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(362.65,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(378.53,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(394.41,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(410.29,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(426.17,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(442.05,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(457.93,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(473.81,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(489.69,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(505.57,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(521.45,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(537.3299999999999,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(553.21,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(569.0899999999999,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(584.96,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(600.84,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(616.72,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(632.6,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(648.48,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(664.36,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(680.24,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(696.12,0)\" d=\"M0,100v220\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,309)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,269.4)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,229.8)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,190.2)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,150.6)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,111)\" d=\"M80,0h632\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"/><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip860347xyplot)\"><g class=\"scatterlayer mlayer\"><g class=\"trace scatter tracefbee6d\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,209L15.88,138.83L19.06,127.58L22.23,113.68L25.41,107.79L34.93,71.44L38.11,64.72L44.46,39.67L47.64,29.43L50.81,33.83L53.99,41.03L57.17,38.56L60.34,40.38L63.52,50.43L66.69,55.82L73.05,70.89L76.22,83.25L85.75,89.15L88.92,91.06L95.28,95.44L98.45,98.65L101.63,100.75L104.8,101.08L111.16,101.08L114.33,102.21L127.04,110.1L130.21,109.64L136.56,109.95L139.74,111.4L146.09,114.77L149.27,117.01L152.44,134.46L155.62,133.58L168.32,130.56L171.5,130.15L174.67,130.11L177.85,135.7L184.2,132.8L187.38,132.15L193.73,130.32L196.9,129.56L200.08,128.57L203.26,134.19L212.78,131.41L215.96,131.34L225.49,130.94L228.66,135.59L238.19,134.66L241.37,133.93L250.89,133.27L254.07,136.54L263.6,135.79L266.77,135.51L276.3,135.37L279.48,138.32L285.83,137.83L289.01,137.21L301.71,136.57L304.88,137.76L314.41,141.2L317.59,143.28L327.12,149.2L330.29,149.99L342.99,153.71L346.17,154.94L352.52,157.95L355.7,158.56L368.4,161.4L371.58,162.48L381.11,165.53L384.28,166.27L393.81,168.27L396.98,169.35L406.51,171.87L409.69,172.36L419.22,174.04L422.39,174.92L431.92,177.05L435.1,177.31L444.62,178.52L447.8,179.35L457.33,180.91L460.5,181.06L558.95,185.63L562.13,185.62L632,186.77\" style=\"vector-effect: none; fill: none; stroke: rgb(99, 110, 250); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g><g class=\"trace scatter trace7ed52e\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M0,208.82L22.23,107.1L25.41,100.84L44.46,24.61L47.64,12.26L50.81,11L53.99,18.91L57.17,21.33L60.34,26.58L73.05,59.78L76.22,71.39L85.75,71.6L88.92,72.63L95.28,76.98L98.45,81.71L101.63,81.62L104.8,79.34L111.16,74.81L114.33,75.42L120.68,81.84L123.86,86.76L127.04,85.3L130.21,81.61L133.39,78.72L136.56,77.47L139.74,79.89L142.91,83.43L146.09,86.83L149.27,92.41L152.44,96.72L155.62,95.62L165.15,90.49L168.32,90.2L174.67,88.97L177.85,93.81L193.73,86.92L196.9,86L200.08,85.13L203.26,90.01L215.96,84.74L219.14,84.15L225.49,83.14L228.66,88.38L241.37,83.29L244.54,82.51L250.89,81.8L254.07,86.73L257.25,85.77L260.42,83.65L263.6,82.41L266.77,82.11L273.13,80.47L276.3,80.99L279.48,85.01L282.65,84.29L285.83,82.34L289.01,81.42L298.53,80.06L301.71,80.07L314.41,87.46L317.59,90.57L320.76,92.27L323.94,94.9L327.12,96.88L330.29,97.9L336.64,100.8L339.82,102L349.35,108.87L352.52,110.41L365.23,115.66L368.4,118.37L387.46,131.42L390.63,132.22L400.16,139.56L403.34,141.78L412.86,148.69L416.04,149.63L428.74,160.84L431.92,161.6L444.62,165.82L447.8,168.38L454.15,172.3L457.33,172.57L539.9,176.75L543.08,176.85L568.48,177.46L571.66,177.53L628.82,178L632,178.13\" style=\"vector-effect: none; fill: none; stroke: rgb(239, 85, 59); stroke-opacity: 1; stroke-width: 2px; opacity: 1;\"/></g><g class=\"points\"/><g class=\"text\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" transform=\"translate(92.7,0) rotate(90,0,327)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">5</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(108.58,0) rotate(90,0,327)\">10</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(124.46000000000001,0) rotate(90,0,327)\">15</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(140.34,0) rotate(90,0,327)\">20</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(156.22,0) rotate(90,0,327)\">25</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(172.1,0) rotate(90,0,327)\">30</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(187.98000000000002,0) rotate(90,0,327)\">35</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(203.86,0) rotate(90,0,327)\">40</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(219.74,0) rotate(90,0,327)\">45</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(235.62,0) rotate(90,0,327)\">50</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(251.5,0) rotate(90,0,327)\">55</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(267.38,0) rotate(90,0,327)\">60</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(283.26,0) rotate(90,0,327)\">65</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(299.14,0) rotate(90,0,327)\">70</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(315.02,0) rotate(90,0,327)\">75</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(330.89,0) rotate(90,0,327)\">80</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(346.77,0) rotate(90,0,327)\">85</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(362.65,0) rotate(90,0,327)\">90</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(378.53,0) rotate(90,0,327)\">95</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(394.41,0) rotate(90,0,327)\">100</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(410.29,0) rotate(90,0,327)\">105</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(426.17,0) rotate(90,0,327)\">110</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(442.05,0) rotate(90,0,327)\">115</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(457.93,0) rotate(90,0,327)\">120</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(473.81,0) rotate(90,0,327)\">125</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(489.69,0) rotate(90,0,327)\">130</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(505.57,0) rotate(90,0,327)\">135</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(521.45,0) rotate(90,0,327)\">140</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(537.3299999999999,0) rotate(90,0,327)\">145</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(553.21,0) rotate(90,0,327)\">150</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(569.0899999999999,0) rotate(90,0,327)\">155</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(584.96,0) rotate(90,0,327)\">160</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(600.84,0) rotate(90,0,327)\">165</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(616.72,0) rotate(90,0,327)\">170</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(632.6,0) rotate(90,0,327)\">175</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(648.48,0) rotate(90,0,327)\">180</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(664.36,0) rotate(90,0,327)\">185</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(680.24,0) rotate(90,0,327)\">190</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(696.12,0) rotate(90,0,327)\">195</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"333\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(712,0) rotate(90,0,327)\">200</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,309)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">10x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,269.4)\">36x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,229.8)\">62x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,190.2)\">89x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,150.6)\">115x</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,111)\">141x</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-860347\"><g class=\"clips\"/><clipPath id=\"legend860347\"><rect width=\"463\" height=\"67\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(724.64,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"463\" height=\"67\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend860347)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Configuration</text><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: Persistent mode/shared memory + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(99, 110, 250); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g><g class=\"groups\" transform=\"\"><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Multicore: afl_execs: Persistent mode/shared memory + kernel config</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(239, 85, 59); stroke-opacity: 1; stroke-width: 2px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"457.34375\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"60\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Fuzzer performance</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"396\" y=\"374.20625\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Number of parallel fuzzers</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,24.684375000000003,210)\" x=\"24.684375000000003\" y=\"210\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Fuzz target executions per second</text></g></g></svg>" ] }, "metadata": {}, @@ -4436,7 +1071,8 @@ "# Update the primary Y-axis with custom tick labels\n", "r6a_fig.update_yaxes(tickvals=tickvals, ticktext=ticktext)\n", "r6a_fig.update_xaxes(tickvals=list(range(0,200+1, 5)))\n", - "r6a_fig.update_layout(width=1200, height=400)" + "r6a_fig.update_layout(width=1200, height=400)\n", + "r6a_fig.show(\"svg\")" ] }, { @@ -4449,19 +1085,7 @@ "\n", "Does this mean that AFL++ is a bad fuzzer, or that AWS tricked us and gave us a 16-thread machine instead of a 192-thread one?\n", "\n", - "No -- the most likely cause here (based on a tip from @eqv) is that we're actually saturating the Linux kernel's ability to service system calls. We could look to reduce these, but there's another option available to us, which is to try running more system-call-servicers (read: kernels) at once, on this machine. One way to do that is to use hardware virtualization with KVM,\n", - "and if it is true that this particular fuzzer setup bottlenecks around 16 fuzzers, then we might expect an optimal number of KVM\n", - "kernels running on this machine to be around 16/192 == 8, each with 16 fuzzers in parallel, and perhaps a shared (in-memory?)\n", - "filesystem for the fuzzing queue." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Measuring system call saturation and experimenting with extra KVM hosts\n", - "\n", - "Coming soon!" + "No, probably not -- the most likely causes here are a problem with our Python harness, or potentially that we're already saturating the Linux kernel's ability to service system calls, although we're definitely hitting such a limit way earlier than expected. A good way to test this theory would be to run more system-call-servicers (read: kernels!) at once on this machine; one way to do that is to use hardware virtualization with KVM. " ] } ], diff --git a/benchmark/benchmark.py b/benchmark/benchmark.py index da32167a..e0dea299 100644 --- a/benchmark/benchmark.py +++ b/benchmark/benchmark.py @@ -64,7 +64,7 @@ cpu_count = multiprocessing.cpu_count() parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter) parser.add_argument("-b", "--basedir", help="directory to use for temp files", type=str, default="/tmp/aflpp-benchmark") parser.add_argument("-d", "--debug", help="show verbose debugging output", action="store_true") -parser.add_argument("-r", "--runs", help="how many runs to average results over", type=int, default=5) +parser.add_argument("-r", "--runs", help="how many runs to average results over", type=int, default=3) parser.add_argument("-f", "--fuzzers", help="how many afl-fuzz workers to use", type=int, default=cpu_count) parser.add_argument("-m", "--mode", help="pick modes", action="append", default=["multicore"], choices=mode_names) parser.add_argument("-c", "--comment", help="add a comment about your setup", type=str, default="") |