Boolean Questions — naturally occurring yes/no questions from Google search queries paired with Wikipedia passages. Tests reading comprehension and factual reasoning.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Based on real user questions. Simple binary format makes scoring unambiguous.
Largely saturated (top models >90%). In training data for most models. Score is 0–1 ratio.
Higher is better. Accuracy ratio (0–1). Random baseline: 0.5.
Largely saturated (top models >90%).
| # | Model | Score |
|---|---|---|
| 1 | 1.0 | |
| 2 | 1.0 | |
| 3 | 1.0 | |
| 4 | 0.9 | |
| 5 | 0.9 | |
| 6 | 0.9 | |
| 7 | 0.9 | |
| 8 | 0.9 | |
| 9 | 0.9 | |
| 10 | 0.9 | |
| 11 | 0.9 | |
| 12 | 0.9 | |
| 13 | 0.9 | |
| 14 | 0.9 | |
| 15 | 0.9 | |
| 16 | 0.9 | |
| 17 | 0.9 | |
| 18 | 0.9 | |
| 19 | 0.9 | |
| 20 | 0.9 | |
| 21 | 0.9 | |
| 22 | 0.9 | |
| 23 | 0.9 | |
| 24 | 0.9 | |
| 25 | 0.9 | |
| 26 | 0.9 | |
| 27 | 0.9 | |
| 28 | 0.9 | |
| 29 | 0.9 | |
| 30 | 0.9 | |
| 31 | 0.9 | |
| 32 | 0.9 | |
| 33 | 0.9 | |
| 34 | 0.9 | |
| 35 | 0.9 | |
| 36 | 0.9 | |
| 37 | 0.9 | |
| 38 | 0.9 | |
| 39 | 0.9 | |
| 40 | 0.9 | |
| 41 | 0.9 | |
| 42 | 0.9 | |
| 43 | 0.9 | |
| 44 | 0.9 | |
| 45 | 0.9 | |
| 46 | 0.9 | |
| 47 | 0.9 | |
| 48 | 0.9 | |
| 49 | 0.9 | |
| 50 | 0.9 |