A version-pinned 74-task evaluation of model-plus-agent systems across seven difficult computer-use, research, coding, and reasoning domains, released as Frontier-Bench v0.1.0.
Historical structured Harbor leaderboard rows preserve the model, agent, organization, reasoning effort, accuracy, result date, standard error, token use, cost, and source record identity. Immutable dataset and leaderboard UUIDs keep these results distinct from later benchmarks.
Retired from collection on September 24, 2026 because the upstream v0.1 leaderboard is no longer available; retained scores are historical. Scores measure systems and harnesses. Frontier-Bench v0.1 is distinct from Terminal-Bench 3 and 4 and its rows must never be aliased or merged; compare only rows bound to the exact dataset-version and leaderboard UUIDs.
Higher is better. The upstream metric is accuracy, not pass@1; inspect effort, agent, trial count, standard error, tokens, and cost with the score.
Max score: 100
Retired from collection on September 24, 2026 because the upstream v0.1 leaderboard is no longer available.
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 5 (Fast) source frontier_bench · Opus 5 (max) · max · mini-SWE-agent · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record e9d504e6-a814-4cc4-99cf-033054408797 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://github.com/SWE-agent/mini-swe-agent",
"label": "mini-SWE-agent"
},
"modelDisplay": {
"url": "https://www.anthropic.com/news/claude-opus-5",
"label": "Opus 5"
},
"rowCreatedAt": "2026-07-24T20:02:59.210369+00:00",
"rowUpdatedAt": "2026-07-24T20:43:15.176386+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-24",
"attemptsPerTask": 1,
"leaderboardRank": 1,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-07-24",
"actualTotalTokens": 7280891794,
"agentOrganization": {
"url": "https://www.princeton.edu",
"label": "Princeton"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"actualTotalCostUsd": 5973.57,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.65,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 24, 2026 | 43.5 |
| 2 | GPT-5.6 Sol (Max) source frontier_bench · GPT-5.6 Sol (max) · max · Codex · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 4ce39786-946e-447b-bf36-9650ff7749ac View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://openai.com/codex/",
"label": "Codex"
},
"modelDisplay": {
"url": "https://developers.openai.com/api/docs/models/gpt-5.6-sol",
"label": "GPT-5.6 Sol"
},
"rowCreatedAt": "2026-07-23T00:48:29.293634+00:00",
"rowUpdatedAt": "2026-08-03T16:30:20.547554+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-13",
"attemptsPerTask": 1,
"leaderboardRank": 2,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-07-09",
"actualTotalTokens": 5779656478,
"agentOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"actualTotalCostUsd": 3963.86,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.58,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 34.6 |
| 3 | Anthropic: Claude Fable 5 (batch) source frontier_bench · Fable 5 (max) · max · Claude Code · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record fd93299a-0c43-4ec3-8aee-421fdbef4b8d View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://claude.com/claude-code",
"label": "Claude Code"
},
"modelDisplay": {
"url": "https://www.anthropic.com/news/claude-fable-5-mythos-5",
"label": "Fable 5"
},
"rowCreatedAt": "2026-07-23T00:48:22.980255+00:00",
"rowUpdatedAt": "2026-08-03T16:30:35.947107+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-13",
"attemptsPerTask": 1,
"leaderboardRank": 3,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-06-09",
"actualTotalTokens": 3583230945,
"agentOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"actualTotalCostUsd": 7062.39,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.71,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 34.0 |
| 4 | Anthropic: Claude Opus 4.8 (Fast) source frontier_bench · Opus 4.8 (max) · max · Claude Code · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 46eaaf38-415f-443a-a68b-f81c46e49be6 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://claude.com/claude-code",
"label": "Claude Code"
},
"modelDisplay": {
"url": "https://www.anthropic.com/news/claude-opus-4-8",
"label": "Opus 4.8"
},
"rowCreatedAt": "2026-07-23T00:48:35.616121+00:00",
"rowUpdatedAt": "2026-07-28T19:48:42.571115+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-12",
"attemptsPerTask": 1,
"leaderboardRank": 4,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-05-28",
"actualTotalTokens": 5208776503,
"agentOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"actualTotalCostUsd": 5701.74,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.55,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 21.1 |
| 5 | GPT-5.6 Terra (Max) source frontier_bench · GPT-5.6 Terra (max) · max · Codex · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 8519e479-95e6-44a4-a108-1834df711b28 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://openai.com/codex/",
"label": "Codex"
},
"modelDisplay": {
"url": "https://developers.openai.com/api/docs/models/gpt-5.6-terra",
"label": "GPT-5.6 Terra"
},
"rowCreatedAt": "2026-07-23T00:48:42.022884+00:00",
"rowUpdatedAt": "2026-07-29T16:40:03.684642+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-12",
"attemptsPerTask": 1,
"leaderboardRank": 5,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-07-09",
"actualTotalTokens": 7024607919,
"agentOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"actualTotalCostUsd": 2484.52,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.4,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 20.8 |
| 6 | Grok 4.5 (High) source frontier_bench · Grok 4.5 (xhigh) · xhigh · Cursor CLI · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 2ccf029d-62fe-4756-a624-b2d376895a32 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://cursor.com/docs/cli/overview",
"label": "Cursor CLI"
},
"modelDisplay": {
"url": "https://docs.x.ai/developers/models/grok-4.5",
"label": "Grok 4.5"
},
"rowCreatedAt": "2026-07-23T00:49:01.181248+00:00",
"rowUpdatedAt": "2026-07-28T19:49:06.124157+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-15",
"attemptsPerTask": 1,
"leaderboardRank": 6,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-07-08",
"actualTotalTokens": 1238993118,
"agentOrganization": {
"url": "https://cursor.com",
"label": "Cursor"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://x.ai",
"label": "xAI"
},
"actualTotalCostUsd": 960.04,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.47,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 15.7 |
| 7 | Anthropic: Claude Sonnet 5 (batch) source frontier_bench · Sonnet 5 (max) · max · Claude Code · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 95ef728a-41bc-4882-adfa-0d30ce50671c View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://claude.com/claude-code",
"label": "Claude Code"
},
"modelDisplay": {
"url": "https://www.anthropic.com/news/claude-sonnet-5",
"label": "Sonnet 5"
},
"rowCreatedAt": "2026-07-23T00:48:48.31335+00:00",
"rowUpdatedAt": "2026-07-28T19:47:56.288937+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-12",
"attemptsPerTask": 1,
"leaderboardRank": 7,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-06-30",
"actualTotalTokens": 17948790860,
"agentOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"actualTotalCostUsd": 7944.04,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.5,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 14.6 |
| 8 | GPT-5.6 Luna (Max) source frontier_bench · GPT-5.6 Luna (max) · max · Codex · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 5651c446-0517-409c-9632-aafd5ca76e93 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://openai.com/codex/",
"label": "Codex"
},
"modelDisplay": {
"url": "https://developers.openai.com/api/docs/models/gpt-5.6-luna",
"label": "GPT-5.6 Luna"
},
"rowCreatedAt": "2026-07-23T00:48:54.777793+00:00",
"rowUpdatedAt": "2026-07-29T16:40:18.252544+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-12",
"attemptsPerTask": 1,
"leaderboardRank": 8,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-07-09",
"actualTotalTokens": 11872804231,
"agentOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://openai.com",
"label": "OpenAI"
},
"actualTotalCostUsd": 1632.44,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 1.25,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 14.3 |
| 9 | GLM-5.2 (Max) source frontier_bench · GLM 5.2 (max) · max · Claude Code · benchmark v0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · split official-leaderboard · 1 attempts · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 95e83401-73ec-4317-8e4b-d0d579fcfb71 View run sourceSource metadata{
"taskCount": 74,
"metricName": "accuracy",
"releaseTag": "v0.1.0",
"agentDisplay": {
"url": "https://claude.com/claude-code",
"label": "Claude Code"
},
"modelDisplay": {
"url": "https://z.ai/blog/glm-5.2",
"label": "GLM 5.2"
},
"rowCreatedAt": "2026-07-23T00:49:14.299058+00:00",
"rowUpdatedAt": "2026-07-28T19:48:54.14958+00:00",
"leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
"releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
"datasetPackage": "frontier-bench/frontier-bench",
"evaluationDate": "2026-07-19",
"attemptsPerTask": 1,
"leaderboardRank": 9,
"metricSemantics": "native-harbor-accuracy-not-pass-at-k",
"datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
"measurementScope": "model-agent-system",
"modelReleaseDate": "2026-06-13",
"actualTotalTokens": 3276045131,
"agentOrganization": {
"url": "https://www.anthropic.com",
"label": "Anthropic"
},
"agmobenchEligible": false,
"leaderboardStatus": "display",
"modelOrganization": {
"url": "https://z.ai",
"label": "Z.ai"
},
"actualTotalCostUsd": 3766.5,
"attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
"upstreamTrialCount": 0,
"accuracyStandardError": 0.97,
"attemptsProvenanceUrls": [
"https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
"https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
]
}Evidence observed Jul 23, 2026 | 4.6 |