Cybersecurity benchmark testing LLMs on capture-the-flag (CTF) challenges — web exploitation, cryptography, reverse engineering, and binary exploitation tasks.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests applied security knowledge and creative problem-solving. Practical cybersecurity evaluation.
Score scale appears to be 0–100 but with non-standard distribution. Niche domain.
Higher is better. Score range 9–42 (challenge completion metric).
Score scale appears to be 0–100 but with non-standard distribution.
| # | Model | Score |
|---|---|---|
| 1 | 9.0 | |
| 2 | 1.5 | |
| 3 | 1.1 | |
| 4 | 1.0 | |
| 5 | 1.0 | |
| 6 | 0.9 | |
| 7 | 0.9 | |
| 8 | 0.9 | |
| 9 | 0.9 | |
| 10 | 0.9 | |
| 11 | 0.9 | |
| 12 | 0.9 | |
| 13 | 0.8 | |
| 14 | 0.8 | |
| 15 | 0.8 | |
| 16 | 0.8 | |
| 17 | 0.8 | |
| 18 | 0.8 | |
| 19 | 0.8 | |
| 20 | 0.8 | |
| 21 | 0.8 | |
| 22 | 0.8 | |
| 23 | 0.8 | |
| 24 | 0.8 | |
| 25 | 0.8 | |
| 26 | 0.7 | |
| 27 | 0.7 | |
| 28 | 0.7 | |
| 29 | 0.7 | |
| 30 | 0.7 | |
| 31 | 0.7 | |
| 32 | 0.7 | |
| 33 | 0.7 | |
| 34 | 0.7 | |
| 35 | 0.7 | |
| 36 | 0.7 | |
| 37 | 0.7 | |
| 38 | 0.7 | |
| 39 | 0.6 | |
| 40 | 0.6 | |
| 41 | 0.6 | |
| 42 | 0.6 | |
| 43 | 0.6 | |
| 44 | 0.6 | |
| 45 | 0.6 | |
| 46 | 0.6 | |
| 47 | 0.6 | |
| 48 | 0.6 | |
| 49 | 0.6 | |
| 50 | 0.6 |