Evaluates models on scientific chart comprehension and reasoning from arXiv papers. Models must interpret complex visualizations and answer multi-step reasoning questions. No-tools variant tests pure vision+reasoning.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests genuine multimodal understanding of scientific figures. Questions require multi-hop reasoning, not just OCR.
No-tools variant disadvantages models that rely on code execution for chart data extraction.
Higher is better. Accuracy on chart reasoning questions without tool access.
Max score: 100
No-tools variant disadvantages models that rely on code execution for chart data extraction.
| # | Model | Score |
|---|---|---|
| 1 | 86.1 | |
| 2 | 82.1 | |
| 3 | 69.1 |