GPQA Diamond measures graduate-level scientific reasoning with expert-written physics, chemistry, and biology questions. It is useful for comparing how models perform on hard, domain-specific analysis.
Questions are authored and validated by domain experts to target genuine scientific reasoning; hard distractors and technical framing make shallow pattern matching less effective.
Subject coverage is narrow, so scores do not generalize to all reasoning tasks; short QA format does not capture longer scientific workflows with tools or citations.
Higher is better. Accuracy percentage on expert-level questions. Top models reach 60-75%.
Max score: 100
Based on ~198 samples, ~1,500 input tokens and ~200 output tokens per sample (1,700 total per sample).
Estimates from GPQA Diamond paper. Actual costs vary with reasoning effort and multi-turn prompting.
Subject coverage is narrow, so scores do not generalize to all reasoning tasks.
| # | Model | Score | Est. Eval Cost | Score/$ |
|---|---|---|---|---|
| 1 | 96.3 | $4.95 | 19.5 | |
| 2 | 96.1 | $4.95 | 19.4 | |
| 3 | 95.3 | $0.37 | 257 | |
| 4 | 94.9 | $0.83 | 114 | |
| 5 | 94.9 | $4.95 | 19.2 | |
| 6 | 94.6 | — | — | |
| 7 | 94.5 | $0.37 | 255 | |
| 8 | 94.2 | $2.48 | 38.1 | |
| 9 | 94.1 | $0.54 | 174 | |
| 10 | 94.1 | $1.98 | 47.5 | |
| 11 | 94.1 | $1.07 | 88.0 | |
| 12 | 93.9 | $4.95 | 19.0 | |
| 13 | 93.8 | $0.54 | 174 | |
| 14 | 93.7 | $4.95 | 18.9 | |
| 15 | 93.7 | $2.48 | 37.9 | |
| 16 | 93.7 | $2.48 | 37.9 | |
| 17 | 93.5 | $0.83 | 112 | |
| 18 | 93.5 | $0.83 | 112 | |
| 19 | 93.5 | $0.37 | 252 | |
| 20 | 93.5 | $0.54 | 173 | |
| 21 | 93.5 | $0.83 | 112 | |
| 22 | 93.5 | $2.67 | 35.0 | |
| 23 | 93.5 | $1.48 | 63.0 | |
| 24 | 93.4 | $4.95 | 18.9 | |
| 25 | 93.2 | $2.67 | 34.9 | |
| 26 | 93.2 | $2.48 | 37.7 | |
| 27 | 93.1 | $4.95 | 18.8 | |
| 28 | 93.1 | $0.83 | 112 | |
| 29 | 93.1 | $1.98 | 47.0 | |
| 30 | 92.9 | $0.14 | 680 | |
| 31 | 92.8 | $0.55 | 169 | |
| 32 | 92.8 | $0.83 | 112 | |
| 33 | 92.8 | $1.98 | 46.9 | |
| 34 | 92.8 | $0.37 | 250 | |
| 35 | 92.7 | $0.83 | 111 | |
| 36 | 92.6 | $2.67 | 34.6 | |
| 37 | 92.6 | $4.95 | 18.7 | |
| 38 | 92.6 | $1.98 | 46.8 | |
| 39 | 92.5 | $1.07 | 86.5 | |
| 40 | 92.4 | $0.02 | 4.4k | |
| 41 | 92.3 | $0.06 | 1.5k | |
| 42 | 92.3 | $1.04 | 88.8 | |
| 43 | 92.2 | $0.80 | 115 | |
| 44 | 92.2 | $0.83 | 111 | |
| 45 | 92.1 | $0.0000 | — | |
| 46 | 92.1 | $0.80 | 115 | |
| 47 | 92.0 | $0.37 | 248 | |
| 48 | 92.0 | $1.34 | 68.8 | |
| 49 | 92.0 | $2.48 | 37.2 | |
| 50 | 91.9 | $2.48 | 37.1 |