Proportion of incorrect answers out of all non-correct responses in the AA-Omniscience benchmark. Measures how often a model confabulates rather than abstaining.
Directly isolates hallucination tendency. Models that abstain when uncertain score better. Captures the critical distinction between 'I don't know' and confidently wrong.
Can be gamed by excessive abstention. Should be read alongside accuracy to assess overall reliability.
Lower is better. Percentage (0-100) of non-correct responses that are incorrect (vs. abstained).
Max score: 100
Can be gamed by excessive abstention.
| # | Model | Score |
|---|---|---|
| 1 | 11.4 | |
| 2 | 14.2 | |
| 3 | 15.1 | |
| 4 | 17.6 | |
| 5 | 18.4 | |
| 6 | 20.0 | |
| 7 | 20.9 | |
| 8 | 22.6 | |
| 9 | 24.7 | |
| 10 | 25.4 | |
| 11 | 25.6 | |
| 12 | 26.3 | |
| 13 | 27.1 | |
| 14 | 27.3 | |
| 15 | 27.6 | |
| 16 | 27.6 | |
| 17 | 28.0 | |
| 18 | 28.2 | |
| 19 | 28.4 | |
| 20 | 28.6 | |
| 21 | 29.1 | |
| 22 | 29.7 | |
| 23 | 30.5 | |
| 24 | 31.3 | |
| 25 | 31.5 | |
| 26 | 31.5 | |
| 27 | 32.9 | |
| 28 | 32.9 | |
| 29 | 33.0 | |
| 30 | 33.3 | |
| 31 | 34.2 | |
| 32 | 34.3 | |
| 33 | 34.4 | |
| 34 | 34.5 | |
| 35 | 36.3 | |
| 36 | 37.3 | |
| 37 | 37.6 | |
| 38 | 38.1 | |
| 39 | 38.9 | |
| 40 | 39.2 | |
| 41 | 39.3 | |
| 42 | 39.4 | |
| 43 | 40.3 | |
| 44 | 40.3 | |
| 45 | 41.0 | |
| 46 | 41.7 | |
| 47 | 41.8 | |
| 48 | 42.0 | |
| 49 | 43.3 | |
| 50 | 44.8 |