AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Models

GPT-5.6 Sol (Max)

July 9, 2026Last synced Nov 22, 2026, 8:17 AM1050K context
Effort
  • No reasoning
  • Low
  • Medium
  • High
  • Xhigh
  • Max

Blended Price

$8.00/M

Input Price

$4.00/M

Output Price

$20.00/M

Speed

—

TTFT

—

Benchmark Scores

Intelligence Index

Performance Radar

GPT-5.6 Sol (Max)
Field Average

Hidden, no data: AgMoBench Agentic, Output Speed

External Benchmarks

AA-Omniscience Accuracy

How GPT-5.6 Sol (Max) Compares

Axes

Filters

$0.00 – $67.5
≥ 1.4

Providers

ai2ai21-labsalibabaanthropicarceeawsazurebaidu
openai
zai
meta
nvidia
xai
upstage
deepseek
alibaba
mistral
nous-research
aws
google
anthropic
xiaomi
stepfun
kimi
reflection
minimax
azure
inclusionai
reka-ai
cohere
bytedance_seed
inception
tencent
thinking-machines
Bubble size = Context Window
8191.00
2000000.00
Compare with other models
—
Coding Index
—
Math Index
—
MMLU Pro
— / 100
GPQA Diamond
94.1 / 100
HLE
49.5 / 100
LiveCodeBench
— / 100
SciCode
57.1 / 100
MATH-500
— / 100
AIME
— / 30
AIME 2025
— / 30
IFBench
72.7 / 100
LCR
84.0 / 100
Terminal-Bench Hard
65.9 / 100
τ²-Bench
85.1 / 100
aa_omniscience
59.4 / 100
AA-Omniscience Hallucination Rateaa_omniscience
92.2 / 100
AA-Omniscience Indexaa_omniscience
22.0 / 100
ARC-AGI-1arc_agi
96.5 / 100
ARC-AGI-1 Cost per Taskarc_agi
0.5
ARC-AGI-2arc_agi
92.5 / 100
ARC-AGI-2 Cost per Taskarc_agi
1.4
BullshitBenchbullshitbench
48.0 / 100
Arena ELO: Codingchatbot_arena
1615.0
Ale Benchepoch_ai
2176.9
Apex Agentsepoch_ai
3.3
Arc Agi 2epoch_ai
0.9
BALROGepoch_ai
0.6
Btf3epoch_ai
0.1
Chess Puzzlesepoch_ai
0.6
Critptepoch_ai
0.3
Cursorbenchepoch_ai
0.4
DeepSWE (Epoch Import, Version Unverified)epoch_ai
0.7 / 100
Dtbenchepoch_ai
1.0
Ebr Benchepoch_ai
0.4
Eci Scoresepoch_ai
165.9
Epoch Capabilities Indexepoch_ai
162.0
Forecastbenchepoch_ai
65.7
Frontiermath Erdosepoch_ai
0.0
Frontiermath Tier 4 V2epoch_ai
0.8
Frontiermath Tiers 1 3 V2epoch_ai
0.9
Frontiersweepoch_ai
0.3
Furniture Assemblyepoch_ai
0.6
Gdp Pdfepoch_ai
0.3
Lmcaepoch_ai
58.4
Mystery Game Puzzlesepoch_ai
0.6
OSWorld 2.0 (Epoch Import)epoch_ai
0.3 / 100
OTIS Mock AIME 2024–2025epoch_ai
1.0
Posttrainbenchepoch_ai
0.3
Proofbenchepoch_ai
0.8
Scicodeepoch_ai
0.6
SimpleQA Verified (Epoch)epoch_ai
0.7
LegalBenchlegalbench
87.0 / 100
LiveBench Codinglivebench
89.7 / 100
LiveBench Data Analysislivebench
79.8 / 100
LiveBench Instruction Followinglivebench
71.8 / 100
LiveBench Languagelivebench
87.7 / 100
LiveBench Mathlivebench
96.2 / 100
LiveBench Reasoninglivebench
91.7 / 100
OpenRouter Token Volumeopenrouter_rankings
1640000000000.0
OpenRouter Weekly Trendopenrouter_rankings
4.0
PRBench Legalseal
50.5
Vectara Factual Consistencyvectara_hallucination
87.6 / 100
Vectara Hallucination Ratevectara_hallucination
12.4 / 100
WeirdMLweirdml
88.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.114815,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.114815,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"2ab08f641c16bcf3c7a59c38b35fa2e3\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"b4ed41aee6b00420557f098bd05f4165\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-19

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"8cf5cb76df0b13c1684b346a2012df83\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"c6e9a78d12dc070d7dab18a1f667a410\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-15

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"e7f673a6db195dac7f2956e3b1651145\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4d3b46c6ffe9404ca1c0ebcfb5938883\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-13

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"b57a26ca4df83acaa40eadb6632c7c61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ee35c9dbbb033dc7de85467eedb85d54\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-06

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-02

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-01

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"8bfe2e142f29acec39df406a6631f81a\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"92bfaa47028eae49163960f4c5a298a0\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-30

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"16793624264c3bdded076b4e26af5a90\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"e7fb37556a7353e7da435c4c7b78ca50\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-29

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"284ac3aff89a7d8a3aca179ee1e5dd61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6e0849a05a2c572b1050e5900a372e2b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-28

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"397d7634d348db2ae8df5180db7b475d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"882404aedf04dbdde2b6c0d731212a4f\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-24

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"58c953e3f78e2282d0259b52304c8ef8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef004f05f102510151398f73e966a509\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"2f3beadbd513a221b41d3816ff651f54\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f1d0d3b7e367e4a7909882624b1caca0\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"c985e4804ffc840374d9852c6b60005c\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0a820529c69911b0c1a63d8b2d3dbf0f\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-21

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-04

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-03

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"07d05a64baf8bd11a2e8505b7f9a1cc0\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4cf045387256dca83a7f8275d8d2157b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-01

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"3a9d42354f8128c11c15f5932e4a2fc7\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a865a80739604261a4cb1abb00b293d4\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-21

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"04f6ba019798e7ba90a11d7da82ed138\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"8f93bcbb2793025d982217c174632a0c\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-13

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"1c13f7ca05a41dddb3e8dbf221993d7e\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a339ccee4fdf8453a492fc58c884889b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-07

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"6bd4d2a79abbba43993aa72466c90f28\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"909105c66efc7bcdbd1ddf40343ec110\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-06

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"fe316c967e356fc8f0ae7405b6a374ea\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1102d97f1e4c2cf4cd67b86832b6396b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-05

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"cfc9b87c645d3ef2103e359fd0f9f521\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0d65f6e0efe0d5cae15e8e2fe5a0994b\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-30

7.8 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.6 Sol (Max) · effort max · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-6-sol-max

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-6-sol",
  "modelNotes": null,
  "modelsEtag": "W/\"bbdf5725ef7a703a7f0970e913262b6f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6529094085f9ca7ce5f3c2e1957d3b9c\"",
  "upstreamModelId": "openai-gpt-5-6-sol-max",
  "environmentCount": 25,
  "modelReleaseDate": "2026-07-09T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 25064.11,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-24

7.8 / 100
ExploitGym v1exploitgym

[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · official-869-task · GPT-5.6 Sol (reasoning max) · harness Codex CLI · 1 attempt · 2h timeout cutoff · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:subrow:4ad4674171c9b3e3524af3895b717091751582003735f9d58562453011b6acb4

Preliminary

View run source

Source metadata
{
  "taskCount": 869,
  "harnessTag": "v1.1.1",
  "metricName": "agmodbDerivedOnTargetPercent",
  "taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
  "aggregateOnly": true,
  "onTargetCount": 216,
  "evaluationDate": "2026-07-13",
  "upstreamSource": "OpenAI & ExploitGym Team",
  "attemptsPerTask": 1,
  "benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
  "metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
  "aggregateVariant": "subrow",
  "measurementScope": "model-agent-system",
  "agmobenchEligible": false,
  "attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
  "upstreamSourceUrl": "https://deploymentsafety.openai.com/gpt-5-6",
  "preliminaryReasons": [
    "missing-exact-agent-revision",
    "missing-upstream-run-identity"
  ],
  "evaluationCondition": "2h timeout cutoff",
  "domainOnTargetCounts": {
    "v8": 59,
    "kernel": 38,
    "userspace": 119
  },
  "attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
  "mitigatedOnTargetCounts": {
    "v8": 19,
    "kernel": 24,
    "userspace": 41
  },
  "submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
  "exactReproductionEligible": false
}

Evidence observed 2026-07-13

24.9 / 100
ExploitGym v1exploitgym

[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · official-869-task · GPT-5.6 Sol (reasoning max) · harness Codex CLI · 1 attempt · 6h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:1fede7d2e97c3a3b102b8c2bf56d36c24d5adb3489ef5bb221778591bdb4fd8a

Preliminary

View run source

Source metadata
{
  "taskCount": 869,
  "harnessTag": "v1.1.1",
  "metricName": "agmodbDerivedOnTargetPercent",
  "taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
  "aggregateOnly": true,
  "onTargetCount": 293,
  "evaluationDate": "2026-07-13",
  "upstreamSource": "OpenAI & ExploitGym Team",
  "attemptsPerTask": 1,
  "benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
  "metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
  "aggregateVariant": "parent",
  "measurementScope": "model-agent-system",
  "agmobenchEligible": false,
  "attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
  "upstreamSourceUrl": "https://deploymentsafety.openai.com/gpt-5-6",
  "preliminaryReasons": [
    "missing-exact-agent-revision",
    "missing-upstream-run-identity"
  ],
  "evaluationCondition": "6h timeout",
  "domainOnTargetCounts": {
    "v8": 71,
    "kernel": 73,
    "userspace": 149
  },
  "attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
  "mitigatedOnTargetCounts": {
    "v8": 32,
    "kernel": 44,
    "userspace": 60
  },
  "submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
  "exactReproductionEligible": false
}

Evidence observed 2026-07-13

33.7 / 100
Frontier-Bench v0.1frontier_bench

vv0.1.0@eb4af26c249b5770e788aa9e3acd2f6ef21f50c2 · official-leaderboard · GPT-5.6 Sol (max) · harness Codex · effort max · 1 attempt · harbor:frontier-bench/frontier-bench@2f325140-b410-452e-87b3-978ffcb4e1ea · record 4ce39786-946e-447b-bf36-9650ff7749ac

View run source

Source metadata
{
  "taskCount": 74,
  "metricName": "accuracy",
  "releaseTag": "v0.1.0",
  "agentDisplay": {
    "url": "https://openai.com/codex/",
    "label": "Codex"
  },
  "modelDisplay": {
    "url": "https://developers.openai.com/api/docs/models/gpt-5.6-sol",
    "label": "GPT-5.6 Sol"
  },
  "rowCreatedAt": "2026-07-23T00:48:29.293634+00:00",
  "rowUpdatedAt": "2026-08-03T16:30:20.547554+00:00",
  "leaderboardId": "076032b9-6e20-458a-b56d-5aea2488b3f9",
  "releaseCommit": "eb4af26c249b5770e788aa9e3acd2f6ef21f50c2",
  "datasetPackage": "frontier-bench/frontier-bench",
  "evaluationDate": "2026-07-13",
  "attemptsPerTask": 1,
  "leaderboardRank": 2,
  "metricSemantics": "native-harbor-accuracy-not-pass-at-k",
  "datasetVersionId": "2f325140-b410-452e-87b3-978ffcb4e1ea",
  "measurementScope": "model-agent-system",
  "modelReleaseDate": "2026-07-09",
  "actualTotalTokens": 5779656478,
  "agentOrganization": {
    "url": "https://openai.com",
    "label": "OpenAI"
  },
  "agmobenchEligible": false,
  "leaderboardStatus": "display",
  "modelOrganization": {
    "url": "https://openai.com",
    "label": "OpenAI"
  },
  "actualTotalCostUsd": 3963.86,
  "attemptsProvenance": "frontier-bench-v0.1-official-command-uses-harbor-default",
  "upstreamTrialCount": 0,
  "accuracyStandardError": 1.58,
  "attemptsProvenanceUrls": [
    "https://github.com/harbor-framework/frontier-bench/blob/eb4af26c249b5770e788aa9e3acd2f6ef21f50c2/README.md#running-the-benchmark",
    "https://github.com/harbor-framework/harbor/blob/v0.14.0/src/harbor/models/job/config.py#L318-L323"
  ]
}

Evidence observed 2026-07-23

34.6 / 100
Terminal-Bench 2.1terminal_bench

v2.1 · official-leaderboard · GPT-5.6 Sol (max) · harness [email protected] · effort max · 5 attempts · harbor:terminal-bench/terminal-bench-2-1 · provider gpt-5.6-sol · record leaderboard/submissions/2026-07-10-gpt-5-6-sol-max-codex.json · run fa34c325-6014-59c4-b45b-4ed08c8719ae

View run source

Source metadata
{
  "passAt2": 0.8348,
  "passAt3": 0.8663,
  "passAt4": 0.8809,
  "passAt5": 0.8876,
  "taskCount": 89,
  "actualTokens": {
    "output": 5935747,
    "cachedInput": 540552541,
    "uncachedInput": 25266704
  },
  "datasetPackage": "terminal-bench/terminal-bench-2-1",
  "leaderboardUrl": "https://www.tbench.ai/leaderboard/terminal-bench/2.1",
  "pullRequestUrl": "https://github.com/harbor-framework/terminal-bench-2-1/pull/102",
  "agentDisplayUrl": "https://openai.com/codex/",
  "modelDisplayUrl": "https://developers.openai.com/api/docs/models/gpt-5.6-sol",
  "actualTrialCount": 445,
  "agentOrganization": "OpenAI",
  "modelOrganization": "OpenAI",
  "actualTotalCostUsd": 574.68,
  "rewardHacksPercent": 7.19,
  "accuracyStandardError": 1.28,
  "submissionDownloadUrl": "https://raw.githubusercontent.com/harbor-framework/terminal-bench-2-1/main/leaderboard/submissions/2026-07-10-gpt-5-6-sol-max-codex.json",
  "disqualifiedTrialCount": 32,
  "expectedAttemptsPerTask": 5,
  "sourceFilterReasoningEffort": "max",
  "actualAverageTrialDurationSeconds": 431
}

Evidence observed 2026-07-10

76.2 / 100