Scientific computing benchmark that tests a model's ability to write code for scientific simulations, data analysis, and numerical methods.
Bridges coding and domain expertise with tasks that require scientific context, not just syntactic correctness; includes simulation and numerical workflows that are underrepresented in generic coding benchmarks.
Coverage is limited to selected scientific domains, so transfer to other research areas is uncertain; benchmark tasks are narrower than full end-to-end scientific software projects.
Higher is better. Measures successful completion of scientific coding tasks.
Max score: 100
Based on ~338 samples, ~2,000 input tokens and ~3,000 output tokens per sample (5,000 total per sample).
Estimates from SciCode paper. Actual costs vary with reasoning effort and multi-turn prompting.
Coverage is limited to selected scientific domains, so transfer to other research areas is uncertain.
| # | Model | Score | Est. Eval Cost | Score/$ |
|---|---|---|---|---|
| 1 | 66.9 | $22.98 | 2.91 | |
| 2 | 65.0 | $22.98 | 2.83 | |
| 3 | 63.1 | $57.46 | 1.10 | |
| 4 | 62.6 | $0.0000 | — | |
| 5 | 61.8 | $22.98 | 2.69 | |
| 6 | 61.0 | $11.49 | 5.31 | |
| 7 | 61.0 | $57.46 | 1.06 | |
| 8 | 60.9 | $57.46 | 1.06 | |
| 9 | 60.9 | $1.18 | 51.8 | |
| 10 | 60.8 | $11.49 | 5.29 | |
| 11 | 60.4 | $22.98 | 2.63 | |
| 12 | 60.3 | $22.98 | 2.62 | |
| 13 | 59.8 | $0.0000 | — | |
| 14 | 59.7 | $5.15 | 11.6 | |
| 15 | 59.5 | $17.24 | 3.45 | |
| 16 | 59.3 | $22.98 | 2.58 | |
| 17 | 59.0 | $5.41 | 10.9 | |
| 18 | 58.9 | $3.41 | 17.3 | |
| 19 | 58.8 | $5.15 | 11.4 | |
| 20 | 58.8 | $5.15 | 11.4 | |
| 21 | 58.7 | $57.46 | 1.02 | |
| 22 | 58.7 | $13.52 | 4.34 | |
| 23 | 58.6 | $22.98 | 2.55 | |
| 24 | 58.4 | $5.15 | 11.3 | |
| 25 | 58.3 | $5.15 | 11.3 | |
| 26 | 57.8 | $7.44 | 7.77 | |
| 27 | 57.8 | $22.98 | 2.51 | |
| 28 | 57.6 | $11.49 | 5.01 | |
| 29 | 57.4 | $7.44 | 7.72 | |
| 30 | 57.4 | $5.15 | 11.1 | |
| 31 | 57.4 | $22.98 | 2.50 | |
| 32 | 57.3 | $11.49 | 4.99 | |
| 33 | 57.2 | $4.31 | 13.3 | |
| 34 | 57.1 | $22.98 | 2.48 | |
| 35 | 57.1 | $22.98 | 2.48 | |
| 36 | 56.8 | $1.12 | 50.9 | |
| 37 | 56.7 | $57.46 | 0.99 | |
| 38 | 56.6 | $4.31 | 13.1 | |
| 39 | 56.5 | $7.44 | 7.60 | |
| 40 | 56.5 | $57.46 | 0.98 | |
| 41 | 56.4 | $57.46 | 0.98 | |
| 42 | 56.4 | $28.73 | 1.96 | |
| 43 | 56.4 | $22.98 | 2.45 | |
| 44 | 56.1 | $33.80 | 1.66 | |
| 45 | 55.9 | $7.44 | 7.52 | |
| 46 | 55.9 | $1.42 | 39.4 | |
| 47 | 55.8 | $11.49 | 4.86 | |
| 48 | 55.8 | $33.80 | 1.65 | |
| 49 | 55.7 | $57.46 | 0.97 | |
| 50 | 55.7 | $0.0000 | — |