SWE-Bench Pro Private evaluates coding agents on 276 long-horizon tasks from proprietary codebases that are distinct from the public and held-out open-source splits.
Proprietary repositories reduce direct training exposure; executable tests exercise industrial codebases unavailable in the public split.
OpenAI estimates that roughly 30% of SWE-Bench Pro tasks are broken, so held-out code does not guarantee a valid task or grader. Private, public, and held-out scores are different splits, and each result remains specific to its harness, effort, budget, and turn cap.
Higher is better. Percentage of tasks resolved correctly.
OpenAI estimates that roughly 30% of SWE-Bench Pro tasks are broken, so held-out code does not guarantee a valid task or grader.
| # | Model | Score |
|---|---|---|
| 1 | 51.5 | |
| 2 | 47.1 | |
| 3 | 44.7 | |
| 4 | 43.4 | |
| 5 | 43.4 | |
| 6 | 32.2 | |
| 7 | 27.7 | |
| 8 | 23.8 | |
| 9 | 23.4 | |
| 10 | 17.9 |