Chatbot Arena ELO rating for coding-specific prompts — how humans rate model responses to programming questions in blind tests.
Human judges score coding responses holistically across correctness, readability, and explanation quality; blind pairwise setup reduces brand and provider bias.
Voters may reward verbose explanations even when concise answers are more correct; crowd judges often cannot execute code, so subtle runtime bugs may be missed. Data: Arena (lmarena.ai) leaderboard dataset, https://huggingface.co/datasets/lmarena-ai/leaderboard-dataset, licensed CC-BY-4.0; rows are converted to the ELO fields shown here.
Higher is better. ELO-like rating for coding tasks.
Voters may reward verbose explanations even when concise answers are more correct.
| # | Model | Score |
|---|---|---|
| 1 | 1819.0 | |
| 2 | 1773.0 | |
| 3 | 1772.0 | |
| 4 | 1769.0 | |
| 5 | 1764.0 | |
| 6 | 1743.0 | |
| 7 | 1742.0 | |
| 8 | 1723.0 | |
| 9 | 1716.0 | |
| 10 | 1692.0 | |
| 11 | 1685.0 | |
| 12 | 1683.0 | |
| 13 | 1682.0 | |
| 14 | 1681.0 | |
| 15 | 1674.0 | |
| 16 | 1674.0 | |
| 17 | 1670.0 | |
| 18 | 1668.0 | |
| 19 | 1659.0 | |
| 20 | 1658.0 | |
| 21 | 1657.0 | |
| 22 | 1656.0 | |
| 23 | 1653.0 | |
| 24 | 1650.0 | |
| 25 | 1647.0 | |
| 26 | 1643.0 | |
| 27 | 1639.0 | |
| 28 | 1638.0 | |
| 29 | 1631.0 | |
| 30 | 1626.0 | |
| 31 | 1626.0 | |
| 32 | 1623.0 | |
| 33 | 1622.0 | |
| 34 | 1621.0 | |
| 35 | 1618.0 | |
| 36 | 1618.0 | |
| 37 | 1615.0 | |
| 38 | 1610.0 | |
| 39 | 1608.0 | |
| 40 | 1595.0 | |
| 41 | 1590.0 | |
| 42 | 1589.0 | |
| 43 | 1584.0 | |
| 44 | 1582.0 | |
| 45 | 1582.0 | |
| 46 | 1581.0 | |
| 47 | 1581.0 | |
| 48 | 1581.0 | |
| 49 | 1578.0 | |
| 50 | 1573.0 |