Video understanding benchmark — 900 human-annotated videos (short/medium/long) testing temporal reasoning, scene understanding, and multimodal comprehension with and without subtitles.
Tests temporal understanding across video lengths, not just single-frame analysis. Subtitle ablation reveals how much models rely on text vs. visual information.
Video understanding is still nascent — even top models struggle with long videos. Subtitle-assisted scores can mask weak visual understanding.
Higher is better. Accuracy percentage.
Max score: 100
Video understanding is still nascent — even top models struggle with long videos.
| # | Model | Score |
|---|---|---|
| 1 | 81.3 | |
| 2 | 81.3 | |
| 3 | 77.2 | |
| 4 | 77.2 | |
| 5 | 75.0 | |
| 6 | 75.0 | |
| 7 | 68.9 | |
| 8 | 63.3 | |
| 9 | 62.9 | |
| 10 | 51.2 |