An 869-task benchmark of model-plus-agent performance on vulnerability analysis, represented in AgMoDB only through the official aggregate v1 result counts.
Its large task count supports comparisons across agent systems with less sampling noise than small security suites. Domain-level aggregate counts retain mitigation state, evaluation condition, source date, and upstream row identity without importing offensive artifacts.
AgMoDB ingests aggregate counts and clearly labeled derived percentages only; it never stores exploits, trajectories, captured flags, proof-of-vulnerability artifacts, or operational instructions. Rows without an exact agent or harness revision remain preliminary.
Higher is better. The displayed percentage is derived from the upstream on-target count divided by the fixed 869-task v1 denominator; inspect the preserved counts and preliminary provenance flag.
Max score: 100
AgMoDB ingests aggregate counts and clearly labeled derived percentages only.
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.6 Sol (Max) source exploitgym · GPT-5.6 Sol (reasoning max) · Codex CLI · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 6h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:1fede7d2e97c3a3b102b8c2bf56d36c24d5adb3489ef5bb221778591bdb4fd8a Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 293,
"evaluationDate": "2026-07-13",
"upstreamSource": "OpenAI & ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": "https://deploymentsafety.openai.com/gpt-5-6",
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "6h timeout",
"domainOnTargetCounts": {
"v8": 71,
"kernel": 73,
"userspace": 149
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 32,
"kernel": 44,
"userspace": 60
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jul 13, 2026 | 33.7 |
| 2 | GPT-5.6 Sol (Max) source exploitgym · GPT-5.6 Sol (reasoning max) · Codex CLI · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout cutoff · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:subrow:4ad4674171c9b3e3524af3895b717091751582003735f9d58562453011b6acb4 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 216,
"evaluationDate": "2026-07-13",
"upstreamSource": "OpenAI & ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "subrow",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": "https://deploymentsafety.openai.com/gpt-5-6",
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout cutoff",
"domainOnTargetCounts": {
"v8": 59,
"kernel": 38,
"userspace": 119
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 19,
"kernel": 24,
"userspace": 41
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jul 13, 2026 | 24.9 |
| 3 | GPT-5.5 (Xhigh) source exploitgym · GPT-5.5 (OpenAI's default safety filters block all GPT-5.5 exploit attempts under default prompting.) · Codex CLI · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:10601e8ce73b6a9e1bdba53b701d30334f5fede41c19dda30e483ab0766b3e65 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 129,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 38,
"kernel": 21,
"userspace": 70
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 4,
"kernel": 8,
"userspace": 10
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 14.8 |
| 4 | GPT-5.4 (Xhigh) source exploitgym · GPT-5.4 · Codex CLI · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:634bf714059bc1564c234a3c40c3432c7250c131a1b6866188dddbf2cc93eec5 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 61,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 22,
"kernel": 1,
"userspace": 38
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 1,
"userspace": 2
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 7.0 |
| 5 | GPT-4.5 (Preview) source exploitgym · GPT-5.4 · Codex CLI · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:634bf714059bc1564c234a3c40c3432c7250c131a1b6866188dddbf2cc93eec5 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 61,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 22,
"kernel": 1,
"userspace": 38
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 1,
"userspace": 2
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 7.0 |
| 6 | Claude Opus 4.6 (Non-reasoning, High) source exploitgym · Claude Opus 4.6 (Results obtained in collaboration with Anthropic.) · Claude Code · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:0dc5bd820912e426621acfb211060e7b1fce95b9424ae3a7d2809b0e5cff89c6 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 16,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 3,
"kernel": 1,
"userspace": 12
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 0,
"userspace": 0
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 1.8 |
| 7 | Claude Opus 4.7 source exploitgym · Claude Opus 4.7 · Claude Code · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:f2e71c804e6d6d491550d4d9db597f11627d924f7f34acbdcb0fee9ace3a4f97 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 12,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 8,
"kernel": 0,
"userspace": 4
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 0,
"userspace": 0
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 1.4 |
| 8 | Muse Spark 1.1 (Xhigh) source exploitgym · Muse Spark 1.1 (helpful-only versoin) · Meta Agent (off-the-shelf agent harness) · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 4h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:2398e6822a5e7826d6a449127d07f6213714da14ae118d4f1926d6dff0a1833d Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 7,
"evaluationDate": "2026-07-09",
"upstreamSource": "Meta AI",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": "https://ai.meta.com/static-resource/muse-spark-1-1-evaluation-report",
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "4h timeout",
"domainOnTargetCounts": {
"v8": 3,
"kernel": 0,
"userspace": 4
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 0,
"userspace": 0
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jul 9, 2026 | 0.8 |
| 9 | GLM-5.1 (Reasoning) source exploitgym · GLM-5.1 · Claude Code · benchmark [email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · split official-869-task · 1 attempts · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:c5de11314bf86df220b5e256fb40b8ce0929af1f34faa09878094c96c5d76795 Preliminary View run sourceSource metadata{
"taskCount": 869,
"harnessTag": "v1.1.1",
"metricName": "agmodbDerivedOnTargetPercent",
"taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
"aggregateOnly": true,
"onTargetCount": 4,
"evaluationDate": "2026-06-16",
"upstreamSource": "ExploitGym Team",
"attemptsPerTask": 1,
"benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
"metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
"aggregateVariant": "parent",
"measurementScope": "model-agent-system",
"agmobenchEligible": false,
"attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
"upstreamSourceUrl": null,
"preliminaryReasons": [
"missing-exact-agent-revision",
"missing-upstream-run-identity"
],
"evaluationCondition": "2h timeout",
"domainOnTargetCounts": {
"v8": 0,
"kernel": 0,
"userspace": 4
},
"attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
"mitigatedOnTargetCounts": {
"v8": 0,
"kernel": 0,
"userspace": 0
},
"submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
"exactReproductionEligible": false
}Evidence observed Jun 16, 2026 | 0.5 |