OCR quality sub-score from the IDP Leaderboard, based on the OlmOCR benchmark testing math equations, tables, reading order, headers/footers, and scanned documents across 1,403 pages and 7,010 unit tests.
Unit-test-driven OCR evaluation; tests practical extraction tasks including math, tables, and reading order; diverse document sources (arXiv, Internet Archive, Library of Congress).
Biased toward English academic documents; OCR alone doesn't capture semantic understanding.
Higher is better. Pass rate across 7,010 unit tests, 0-100.
Max score: 100
Biased toward English academic documents.
| # | Model | Score |
|---|---|---|
| 1 | 87.4 | |
| 2 | 82.0 | |
| 3 | 81.0 | |
| 4 | 81.0 | |
| 5 | 79.1 | |
| 6 | 77.7 | |
| 7 | 77.2 | |
| 8 | 76.8 | |
| 9 | 75.4 | |
| 10 | 75.3 | |
| 11 | 75.3 | |
| 12 | 74.1 | |
| 13 | 73.9 | |
| 14 | 71.9 | |
| 15 | 69.9 | |
| 16 | 69.8 | |
| 17 | 69.6 | |
| 18 | 68.4 | |
| 19 | 64.8 | |
| 20 | 61.2 | |
| 21 | 61.2 | |
| 22 | 58.7 | |
| 23 | 49.4 | |
| 24 | 47.0 | |
| 25 | 38.3 | |
| 26 | 38.2 | |
| 27 | 35.2 |