Evaluates AI models on cybersecurity vulnerability reproduction — given a CVE and target environment, the model must find and reproduce the vulnerability. Measures offensive security capability as a proxy for defensive potential.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests real-world security capability on actual CVEs. Directly relevant to defensive AI applications.
Dual-use benchmark — high scores indicate both defensive and offensive capability. Not publicly available.
Higher is better. Percentage of vulnerabilities successfully reproduced.
Max score: 100
Dual-use benchmark — high scores indicate both defensive and offensive capability.
| # | Model | Score |
|---|---|---|
| 1 | 83.1 | |
| 2 | 73.8 | |
| 3 | 73.1 | |
| 4 | 66.3 |