Tests a model's ability to operate as an agentic terminal assistant — executing multi-step shell commands, debugging errors, and navigating file systems.
Covers realistic terminal workflows such as multi-step command chains, recovery from failures, and filesystem navigation; execution-style tasks reward correct actions instead of fluent explanations.
Results depend heavily on harness design and tool permissions, so model-only comparisons are imperfect; benchmark scenarios still under-sample large-repo architectural refactors.
Higher is better. Pass rate on complex agentic terminal tasks.
Max score: 100
Based on ~200 samples, ~500 input tokens and ~2,000 output tokens per sample (2,500 total per sample).
Estimates from TerminalBench paper. Actual costs vary with reasoning effort and multi-turn prompting.
Results depend heavily on harness design and tool permissions, so model-only comparisons are imperfect.
| # | Model | Score | Est. Eval Cost | Score/$ |
|---|---|---|---|---|
| 1 | 80.0 | $0.02 | 3.6k | |
| 2 | 80.0 | $0.35 | 231 | |
| 3 | 80.0 | $0.02 | 3.3k | |
| 4 | 75.8 | $0.08 | 913 | |
| 5 | 75.8 | $0.0000 | — | |
| 6 | 75.8 | $0.0000 | — | |
| 7 | 75.8 | $0.17 | 439 | |
| 8 | 75.8 | $0.0000 | — | |
| 9 | 75.8 | $0.0000 | — | |
| 10 | 75.8 | $0.0000 | — | |
| 11 | 75.8 | $0.0000 | — | |
| 12 | 75.8 | $4.25 | 17.8 | |
| 13 | 75.8 | $0.03 | 2.6k | |
| 14 | 75.8 | $0.0000 | — | |
| 15 | 75.8 | $1.17 | 65.0 | |
| 16 | 75.8 | $0.10 | 743 | |
| 17 | 75.8 | $0.53 | 144 | |
| 18 | 75.8 | $0.0000 | — | |
| 19 | 69.4 | $10.50 | 6.61 | |
| 20 | 65.9 | $8.40 | 7.85 | |
| 21 | 62.9 | $21.00 | 2.99 | |
| 22 | 62.9 | $8.40 | 7.49 | |
| 23 | 62.9 | $5.00 | 12.6 | |
| 24 | 62.1 | $8.40 | 7.40 | |
| 25 | 61.4 | $8.40 | 7.31 | |
| 26 | 60.6 | $12.50 | 4.85 | |
| 27 | 60.6 | $8.40 | 7.22 | |
| 28 | 59.8 | $12.50 | 4.79 | |
| 29 | 58.3 | $10.50 | 5.56 | |
| 30 | 57.6 | $6.25 | 9.21 | |
| 31 | 57.6 | $12.50 | 4.61 | |
| 32 | 57.6 | $5.00 | 11.5 | |
| 33 | 57.6 | $5.00 | 11.5 | |
| 34 | 54.5 | $10.50 | 5.19 | |
| 35 | 53.8 | $5.00 | 10.8 | |
| 36 | 53.0 | $5.77 | 9.18 | |
| 37 | 53.0 | $6.30 | 8.42 | |
| 38 | 53.0 | $5.77 | 9.18 | |
| 39 | 52.3 | $1.88 | 27.9 | |
| 40 | 52.3 | $1.88 | 27.9 | |
| 41 | 52.3 | $12.50 | 4.18 | |
| 42 | 51.5 | $10.50 | 4.91 | |
| 43 | 50.8 | $3.25 | 15.6 | |
| 44 | 50.8 | $1.90 | 26.7 | |
| 45 | 49.2 | $0.0000 | — | |
| 46 | 49.2 | $12.50 | 3.94 | |
| 47 | 49.2 | $0.51 | 96.5 | |
| 48 | 48.5 | $10.50 | 4.62 | |
| 49 | 47.0 | $5.77 | 8.13 | |
| 50 | 47.0 | $10.50 | 4.47 |