AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Benchmarks

Video-MME

Video understanding benchmark — 900 human-annotated videos (short/medium/long) testing temporal reasoning, scene understanding, and multimodal comprehension with and without subtitles.

multimodalvideomme

Strengths

Tests temporal understanding across video lengths, not just single-frame analysis. Subtitle ablation reveals how much models rely on text vs. visual information.

Caveats

Video understanding is still nascent — even top models struggle with long videos. Subtitle-assisted scores can mask weak visual understanding.

How to interpret scores

Contamination risk: moderateFreshness: periodic

Higher is better. Accuracy percentage.

Max score: 100

Relevant use cases

Video analysisContent moderationMedia understandingVideo Content AnalysisTemporal Reasoning

Caveat to keep in mind

Video understanding is still nascent — even top models struggle with long videos.

Leaderboard

Last synced: Nov 22, 2026, 12:18 PM
#ModelProviderScorePrice/M
1
Gemini 1.5 Pro (May '24)
source videomme
Evidence observed Mar 14, 2026
Google81.3$0.00
2
Gemini 1.5 Pro (Sep '24)
source videomme
Evidence observed Nov 22, 2026
Google81.3$0.00
3
GPT-4o (Nov '24)
source videomme
Evidence observed Nov 22, 2026
OpenAI77.2$4.38
4
GPT-4o (Aug '24)
source videomme
Evidence observed Mar 26, 2026
OpenAI77.2$4.38
5
Gemini 1.5 Flash (Sep '24)
source videomme
Evidence observed Nov 22, 2026
Google75.0$0.00
6
Gemini 1.5 Flash (May '24)
source videomme
Evidence observed Mar 14, 2026
Google75.0$0.00
7
GPT-4o mini
source videomme
Evidence observed Nov 22, 2026
OpenAI68.9$0.26
8
GPT-4
source videomme
Evidence observed Nov 22, 2026
OpenAI63.3$37.50
9
Claude 3.5 Sonnet (Oct '24)
source videomme
Evidence observed Nov 22, 2026
Anthropic62.9$6.00
10
Qwen: Qwen VL Max
source videomme
Evidence observed Nov 22, 2026
Alibaba51.2$1.40