Frontier Risk Evaluation for National Security and Public Safety. 1,010 expert-crafted adversarial prompts testing model safeguards against dual-use risks. Measures both Average Risk Score (willingness to generate harmful content) and Over-Refusal Score.
Tests safety-utility tradeoff; expert-crafted adversarial prompts; measures over-refusal alongside risk.
500 public + 510 private prompts; adversarial focus may not reflect typical usage patterns.
Lower is better. Average Risk Score is mean harmful-content compliance; Over-Refusal Score is a separate lower-is-better utility metric and must not be combined.
500 public + 510 private prompts.
| # | Model | Score |
|---|---|---|
| 1 | 5.8 | |
| 2 | 6.3 | |
| 3 | 7.6 | |
| 4 | 8.2 | |
| 5 | 9.5 | |
| 6 | 9.6 | |
| 7 | 10.8 | |
| 8 | 12.4 | |
| 9 | 12.8 | |
| 10 | 13.0 | |
| 11 | 13.0 | |
| 12 | 13.0 | |
| 13 | 13.6 |