General Software Optimization — tests LLMs on code optimization tasks, measuring ability to improve runtime performance, memory usage, and algorithmic efficiency.
This definition has not completed editorial and provenance review. Verify its metric, version, source, and evaluation configuration upstream before using it for a model-selection decision.
Uniquely tests optimization ability rather than just correctness. Practical relevance to software performance.
Score is 0–1 ratio. Relatively new with few established baselines. Max score ~0.27.
Higher is better. Optimization success ratio (0–1).
Score is 0–1 ratio.
| # | Model | Score |
|---|---|---|
| 1 | 100.0 | |
| 2 | 99.9 | |
| 3 | 99.6 | |
| 4 | 60.0 | |
| 5 | 60.0 | |
| 6 | 60.0 | |
| 7 | 37.6 | |
| 8 | 37.5 | |
| 9 | 37.3 | |
| 10 | 37.3 | |
| 11 | 16.7 | |
| 12 | 15.7 | |
| 13 | 13.9 | |
| 14 | 7.3 | |
| 15 | 6.5 | |
| 16 | 6.1 | |
| 17 | 5.6 | |
| 18 | 5.3 | |
| 19 | 4.5 | |
| 20 | 4.0 | |
| 21 | 3.7 | |
| 22 | 2.9 | |
| 23 | 2.7 | |
| 24 | 2.3 | |
| 25 | 2.2 | |
| 26 | 2.0 | |
| 27 | 1.6 | |
| 28 | 1.6 | |
| 29 | 1.5 | |
| 30 | 1.4 | |
| 31 | 1.3 | |
| 32 | 1.1 | |
| 33 | 1.1 | |
| 34 | 1.0 | |
| 35 | 1.0 | |
| 36 | 1.0 | |
| 37 | 1.0 | |
| 38 | 0.9 | |
| 39 | 0.8 | |
| 40 | 0.8 | |
| 41 | 0.7 | |
| 42 | 0.7 | |
| 43 | 0.7 | |
| 44 | 0.7 | |
| 45 | 0.6 | |
| 46 | 0.6 | |
| 47 | 0.5 | |
| 48 | 0.5 | |
| 49 | 0.5 | |
| 50 | 0.5 |