Tests LLM knowledge of recently published fiction — a contamination-resistant benchmark since the books postdate model training cutoffs.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Novel contamination resistance via recency. Tests reading comprehension on real literary works.
Score is 0–1 ratio. Effectiveness depends on training cutoff timing. May disadvantage models with older cutoffs regardless of capability.
Higher is better. Accuracy ratio (0–1). Top models score up to 1.0.
Score is 0–1 ratio.
| # | Model | Score |
|---|---|---|
| 1 | 1.2 | |
| 2 | 1.1 | |
| 3 | 1.1 | |
| 4 | 1.1 | |
| 5 | 1.0 | |
| 6 | 1.0 | |
| 7 | 1.0 | |
| 8 | 1.0 | |
| 9 | 1.0 | |
| 10 | 1.0 | |
| 11 | 1.0 | |
| 12 | 1.0 | |
| 13 | 1.0 | |
| 14 | 1.0 | |
| 15 | 1.0 | |
| 16 | 1.0 | |
| 17 | 1.0 | |
| 18 | 0.9 | |
| 19 | 0.9 | |
| 20 | 0.9 | |
| 21 | 0.9 | |
| 22 | 0.9 | |
| 23 | 0.9 | |
| 24 | 0.9 | |
| 25 | 0.9 | |
| 26 | 0.9 | |
| 27 | 0.9 | |
| 28 | 0.9 | |
| 29 | 0.9 | |
| 30 | 0.9 | |
| 31 | 0.9 | |
| 32 | 0.9 | |
| 33 | 0.9 | |
| 34 | 0.9 | |
| 35 | 0.9 | |
| 36 | 0.9 | |
| 37 | 0.9 | |
| 38 | 0.9 | |
| 39 | 0.9 | |
| 40 | 0.9 | |
| 41 | 0.9 | |
| 42 | 0.9 | |
| 43 | 0.9 | |
| 44 | 0.9 | |
| 45 | 0.9 | |
| 46 | 0.9 | |
| 47 | 0.9 | |
| 48 | 0.9 | |
| 49 | 0.9 | |
| 50 | 0.9 |