Hard subset of LongBench v2 — requires multi-hop reasoning, synthesis across distant parts of long documents, and inference beyond literal retrieval.
Tests genuine long-context comprehension, not just retrieval. Good discriminator for reasoning over long inputs.
Harder tasks may have more ambiguous correct answers. Fewer evaluated models than the easy subset.
Higher is better. Accuracy percentage (0–100%).
Harder tasks may have more ambiguous correct answers.
| # | Model | Score |
|---|---|---|
| 1 | 76.9 | |
| 2 | 72.5 | |
| 3 | 72.2 | |
| 4 | 71.6 | |
| 5 | 68.9 | |
| 6 | 64.8 | |
| 7 | 60.2 | |
| 8 | 59.7 | |
| 9 | 58.2 | |
| 10 | 57.9 | |
| 11 | 57.4 | |
| 12 | 57.4 | |
| 13 | 57.4 | |
| 14 | 56.9 | |
| 15 | 56.9 | |
| 16 | 55.7 | |
| 17 | 55.4 | |
| 18 | 55.1 | |
| 19 | 54.9 | |
| 20 | 54.9 | |
| 21 | 54.8 | |
| 22 | 54.5 | |
| 23 | 54.0 | |
| 24 | 53.6 | |
| 25 | 53.1 | |
| 26 | 51.8 | |
| 27 | 51.8 | |
| 28 | 51.4 | |
| 29 | 51.4 | |
| 30 | 51.3 | |
| 31 | 51.2 | |
| 32 | 51.2 | |
| 33 | 51.1 | |
| 34 | 51.0 | |
| 35 | 51.0 | |
| 36 | 51.0 | |
| 37 | 51.0 | |
| 38 | 50.9 | |
| 39 | 50.8 | |
| 40 | 50.8 | |
| 41 | 50.2 | |
| 42 | 50.1 | |
| 43 | 50.1 | |
| 44 | 50.1 | |
| 45 | 50.0 | |
| 46 | 50.0 | |
| 47 | 50.0 | |
| 48 | 50.0 | |
| 49 | 49.9 | |
| 50 | 49.8 |