Tests word prediction in narrative contexts — models must predict the final word of passages where understanding the broader narrative context is essential.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests genuine narrative comprehension beyond surface-level pattern matching.
Only 3 model scores. Older benchmark (2016). Score is 0–1 ratio. Likely saturated.
Higher is better. Accuracy ratio (0–1).
Only 3 model scores.
| # | Model | Score |
|---|---|---|
| 1 | 0.8 | |
| 2 | 0.7 | |
| 3 | 0.7 |