Tests scientific reasoning by combining provided science facts with broad common knowledge — modeled after open-book exams where students must apply, not just recall, knowledge.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests reasoning over given facts, not just knowledge recall. Science-focused.
Only 7 model scores. Score is 0–1 ratio. Older benchmark likely saturated for frontier models.
Higher is better. Accuracy ratio (0–1). Random baseline: ~0.25 (4-choice).
Only 7 model scores.
| # | Model | Score |
|---|---|---|
| 1 | 0.9 | |
| 2 | 0.9 | |
| 3 | 0.6 | |
| 4 | 0.5 | |
| 5 | 0.5 | |
| 6 | 0.5 | |
| 7 | 0.5 | |
| 8 | 0.5 | |
| 9 | 0.3 |