Tests LLMs on tactical chess puzzles — finding the best move or sequence of moves in positions that require combinatorial search and pattern recognition.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Tests structured reasoning and planning in a well-defined domain. Objectively verifiable correct answers.
Score is 0–1 ratio. Chess-specific capability may not generalize. Models may have memorized common puzzles.
Higher is better. Accuracy ratio (0–1). Top models: ~0.55.
Score is 0–1 ratio.
| # | Model | Score |
|---|---|---|
| 1 | 4.1 | |
| 2 | 2.7 | |
| 3 | 2.0 | |
| 4 | 0.8 | |
| 5 | 0.7 | |
| 6 | 0.7 | |
| 7 | 0.7 | |
| 8 | 0.7 | |
| 9 | 0.6 | |
| 10 | 0.6 | |
| 11 | 0.6 | |
| 12 | 0.6 | |
| 13 | 0.6 | |
| 14 | 0.6 | |
| 15 | 0.6 | |
| 16 | 0.6 | |
| 17 | 0.6 | |
| 18 | 0.6 | |
| 19 | 0.6 | |
| 20 | 0.6 | |
| 21 | 0.6 | |
| 22 | 0.6 | |
| 23 | 0.5 | |
| 24 | 0.5 | |
| 25 | 0.5 | |
| 26 | 0.5 | |
| 27 | 0.5 | |
| 28 | 0.5 | |
| 29 | 0.5 | |
| 30 | 0.5 | |
| 31 | 0.5 | |
| 32 | 0.5 | |
| 33 | 0.5 | |
| 34 | 0.5 | |
| 35 | 0.5 | |
| 36 | 0.5 | |
| 37 | 0.5 | |
| 38 | 0.5 | |
| 39 | 0.5 | |
| 40 | 0.5 | |
| 41 | 0.5 | |
| 42 | 0.5 | |
| 43 | 0.5 | |
| 44 | 0.5 | |
| 45 | 0.4 | |
| 46 | 0.4 | |
| 47 | 0.4 | |
| 48 | 0.4 | |
| 49 | 0.4 | |
| 50 | 0.4 |