Big-Bench Hard from the Open LLM Leaderboard — 23 tasks from BIG-Bench that are challenging for language models, testing diverse reasoning capabilities.
Covers 23 reasoning tasks spanning deduction, symbolic manipulation, and linguistic puzzles; standardized evaluation enables consistent tracking across open-model releases.
Retired by Hugging Face on March 13, 2025; retained scores are historical. Fixed and older task set raises contamination risk for recent training corpora; benchmark tasks are short-form and do not capture extended tool-augmented reasoning.
Higher is better. Percentage accuracy.
Max score: 100
Retired by Hugging Face on March 13, 2025.
| # | Model | Score |
|---|---|---|
| 1 | 77.8 | |
| 2 | 75.0 | |
| 3 | 73.2 | |
| 4 | 72.7 | |
| 5 | 72.7 | |
| 6 | 72.1 | |
| 7 | 71.5 | |
| 8 | 71.3 | |
| 9 | 71.2 | |
| 10 | 71.0 | |
| 11 | 70.9 | |
| 12 | 70.8 | |
| 13 | 70.2 | |
| 14 | 70.0 | |
| 15 | Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) source benchpress Evidence observed Apr 13, 2026 | 70.0 |
| 16 | 69.8 | |
| 17 | 69.8 | |
| 18 | 69.8 | |
| 19 | 69.6 | |
| 20 | 69.4 | |
| 21 | 69.2 | |
| 22 | 69.2 | |
| 23 | 69.2 | |
| 24 | 69.2 | |
| 25 | 69.1 | |
| 26 | 69.0 | |
| 27 | 69.0 | |
| 28 | 68.9 | |
| 29 | 68.8 | |
| 30 | 68.7 | |
| 31 | 68.3 | |
| 32 | 68.3 | |
| 33 | 68.1 | |
| 34 | 67.9 | |
| 35 | 67.9 | |
| 36 | 67.6 | |
| 37 | 67.5 | |
| 38 | 67.3 | |
| 39 | 67.3 | |
| 40 | 67.0 | |
| 41 | 66.9 | |
| 42 | 66.9 | |
| 43 | 66.9 | |
| 44 | 66.7 | |
| 45 | 66.6 | |
| 46 | 66.4 | |
| 47 | 66.3 | |
| 48 | 66.3 | |
| 49 | 66.3 | |
| 50 | 66.3 |