End-to-end cybersecurity CTF benchmark covering 40 real-world capture-the-flag challenges across web exploitation, reverse engineering, cryptography, and forensics. Measures whether an autonomous agent can plan, execute tools, and capture flags without human guidance.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Adversarial, multi-step agentic tasks grounded in real CTF problems; each challenge requires tool use and long-horizon planning; Anthropic and xAI report scores in their model cards so cross-vendor comparability is good.
Small benchmark (~40 challenges) so single-task variance is high; some recent results come from model-card subsets of 37–39 problems rather than the full set — superscript footnotes on cybench.github.io disclose this.
Higher is better. End-to-end % of CTF challenges solved without subtask guidance.
Small benchmark (~40 challenges) so single-task variance is high.
| # | Model | Score |
|---|---|---|
| 1 | 100.0 | |
| 2 | 96.0 | |
| 3 | 93.0 | |
| 4 | 82.0 | |
| 5 | 82.0 | |
| 6 | 65.4 | |
| 7 | 60.0 | |
| 8 | 60.0 | |
| 9 | 43.0 | |
| 10 | 42.0 | |
| 11 | 42.0 | |
| 12 | 38.0 | |
| 13 | 35.0 | |
| 14 | 35.0 | |
| 15 | 30.0 | |
| 16 | 30.0 | |
| 17 | 22.5 | |
| 18 | 22.5 | |
| 19 | 20.0 | |
| 20 | 20.0 | |
| 21 | 17.5 | |
| 22 | 17.5 | |
| 23 | 12.5 | |
| 24 | 10.0 | |
| 25 | 10.0 | |
| 26 | 10.0 | |
| 27 | 7.5 | |
| 28 | 7.5 | |
| 29 | 7.5 | |
| 30 | 7.5 | |
| 31 | 5.0 |