BIg Bench for LaRge-scale Database Grounded Text-to-SQL Evaluation. 95 databases across 37 professional domains; measures whether a model can translate natural-language questions into correct SQL over realistic schemas with joins, aggregations, and business rules. Leaderboard hosted on GitHub Pages.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Realistic business-domain schemas (not synthetic); large held-out test set with blind evaluation; well-maintained public leaderboard.
English-only questions; dev-set accuracy is widely reported but can overstate generalization — prefer Test (%) when available; primarily SQLite dialect.
Higher is better. Execution accuracy (% of generated queries that return the correct result set). Test-set score preferred; dev-set used as fallback.
English-only questions.
| # | Model | Score |
|---|---|---|
| 1 | 75.3 | |
| 2 | 70.2 | |
| 3 | 68.1 | |
| 4 | 66.8 | |
| 5 | 66.8 | |
| 6 | 64.0 | |
| 7 | 62.9 | |
| 8 | 60.9 | |
| 9 | 59.9 | |
| 10 | 56.7 | |
| 11 | 55.8 | |
| 12 | 54.9 | |
| 13 | 49.0 | |
| 14 | 39.3 | |
| 15 | 36.5 | |
| 16 | 33.0 |