AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Models

GPT-5.4 (Xhigh)

March 5, 2026Last synced Nov 22, 2026, 8:17 AM1050K context
Effort
  • No reasoning
  • Low
  • Xhigh

Blended Price

$5.63/M

Input Price

$2.50/M

Output Price

$15.00/M

Speed

—

TTFT

—

Benchmark Scores

Intelligence Index

Performance Radar

GPT-5.4 (Xhigh)
Field Average

Hidden, no data: AgMoBench Agentic, Output Speed

External Benchmarks

AA-Omniscience Accuracy

How GPT-5.4 (Xhigh) Compares

Axes

Filters

$0.00 – $67.5
≥ 1.4

Providers

ai2ai21-labsalibabaanthropicarceeawsazurebaidu
openai
zai
meta
nvidia
xai
upstage
deepseek
alibaba
mistral
nous-research
aws
google
anthropic
xiaomi
stepfun
kimi
reflection
minimax
azure
inclusionai
reka-ai
cohere
bytedance_seed
inception
tencent
thinking-machines
Bubble size = Context Window
8191.00
2000000.00
Compare with other models
—
Coding Index
—
Math Index
—
MMLU Pro
— / 100
GPQA Diamond
92.0 / 100
HLE
43.7 / 100
LiveCodeBench
— / 100
SciCode
— / 100
MATH-500
— / 100
AIME
— / 30
AIME 2025
— / 30
IFBench
73.9 / 100
LCR
82.0 / 100
Terminal-Bench Hard
57.6 / 100
τ²-Bench
87.1 / 100
aa_omniscience
50.0 / 100
AA-Omniscience Indexaa_omniscience
5.7 / 100
CyberGymanthropic_launch
66.3 / 100
ARC-AGI-1arc_agi
93.7 / 100
ARC-AGI-1 Cost per Taskarc_agi
0.6
ARC-AGI-2arc_agi
74.0 / 100
ARC-AGI-2 Cost per Taskarc_agi
1.5
OSWorld (Unversioned Matrix Import)benchmark_matrix
75.0 / 100
SWE-bench Probenchmark_matrix
57.7
Terminal-Bench 2.0benchmark_matrix
75.1
AA-Omniscience Hallucination RatePredicted
93.3 / 100
Aider PolyglotPredicted
92.2 / 100
AIMEPredicted
0.3 / 30
AIME 2025Predicted
1.0 / 30
AlpacaEval 2.0 LCPredicted
29.9 / 100
AlpacaEval 2.0 RawPredicted
21.0 / 100
BFCL (Berkeley Function Calling)Predicted
59.3
BigCodeBench CompletePredicted
39.8 / 100
BigCodeBench InstructPredicted
31.2 / 100
AA Intelligence Index (Matrix)Predicted
60.4
AA Long Context Reasoning (Matrix)Predicted
76.3
AIME 2024Predicted
99.6
AIME 2025 (Matrix)Predicted
100.0
Arena-Hard AutoPredicted
85.2
BrowseCompPredicted
80.3
BRUMO 2025Predicted
99.7
CMIMC 2025Predicted
90.9
CritPtPredicted
23.4
GPQA Diamond (Matrix)Predicted
93.3
GSM8KPredicted
96.9
HLE (Matrix)Predicted
49.8
HMMT Feb 2025Predicted
97.7
HMMT Nov 2025Predicted
90.3
HumanEvalPredicted
96.2
IFBench (Matrix)Predicted
71.0
IFEvalPredicted
93.4
IMO 2025Predicted
45.4
LiveCodeBench (Matrix)Predicted
89.0
MATH-500 (Matrix)Predicted
98.9
MathArena Apex 2025Predicted
11.1
MMLUPredicted
92.0
MMLU-Pro (Matrix)Predicted
88.7
MMMU-ProPredicted
80.9
MRCR v2Predicted
82.3
SimpleQAPredicted
69.4
SMT 2025Predicted
95.0
SWE-bench ProPredicted
52.8
Tau-Bench Telecom (Matrix)Predicted
98.8
Terminal-Bench 2.0Predicted
68.6
Terminal-Bench 1.0Predicted
53.8
USAMO 2025Predicted
39.4
Video-MMUPredicted
86.0
browsecompPredicted
80.3
Aider PolyglotPredicted
0.6
BALROGPredicted
0.0
BIG-Bench HardPredicted
3.0
BoolQPredicted
0.6
CAD-EvalPredicted
2.1
CyBenchPredicted
0.9
DeepResearchBenchPredicted
0.5
FictionLiveBenchPredicted
1.1
GdpvalPredicted
0.6
GeoBenchPredicted
0.0
GSM8K (Epoch)Predicted
2.4
GSOPredicted
0.4
HellaSwagPredicted
0.0
HlePredicted
0.5
Lech Mazur WritingPredicted
8.8
METR Time HorizonsPredicted
51.9
PIQAPredicted
0.8
The Agent CompanyPredicted
59.6
TriviaQAPredicted
25.6
VPCTPredicted
0.7
WinoGrandePredicted
0.8
FrontierMathPredicted
39.4 / 100
GAIA Level 1Predicted
86.1
GAIA Level 2Predicted
74.4
GAIA Level 3Predicted
93.7
GAIAPredicted
73.9 / 100
LiveCodeBenchPredicted
0.9 / 100
LongBench v2 EasyPredicted
81.8
LongBench v2 HardPredicted
28.1
LongBench v2Predicted
24.7 / 100
MATH-500Predicted
0.9 / 100
MathVistaPredicted
73.3 / 100
MLE-bench (Original)Predicted
68.1 / 100
MMLU ProPredicted
0.9 / 100
MMMUPredicted
86.3 / 100
MMTU Table UnderstandingPredicted
70.0 / 100
MT-BenchPredicted
7.8 / 10
NoLiMa (NIAH)Predicted
93.7 / 100
OCRBench v2Predicted
87.0 / 100
Open LLM AveragePredicted
10.6 / 100
Open LLM: BBHPredicted
35.8 / 100
Open LLM: GPQAPredicted
26.2 / 100
Open LLM: IFEvalPredicted
22.6 / 100
Open LLM: MATH Level 5Predicted
1.3 / 100
Open LLM: MMLU-PROPredicted
21.1 / 100
Open LLM: MUSRPredicted
37.8 / 100
RE-BenchPredicted
100.0
SimpleBenchPredicted
72.5 / 100
simpleqaPredicted
64.9
SWE-bench LitePredicted
80.1 / 100
SWE-bench VerifiedPredicted
86.4 / 100
tau-bench RetailPredicted
91.5 / 100
WebArenaPredicted
0.2 / 100
WildBenchPredicted
36.4
BullshitBenchbullshitbench
48.0 / 100
Arena ELO: Codingchatbot_arena
1462.0
Arena ELO: Creative Writingchatbot_arena
1431.0
arena_elo_documentchatbot_arena
1473.0
Chatbot Arena ELOchatbot_arena
1469.0
arena_elo_searchchatbot_arena
1198.0
arena_elo_visionchatbot_arena
1301.0
Apex Agentsepoch_ai
3.4
Arc Agi 2epoch_ai
0.7
Chess Puzzlesepoch_ai
0.5
Cl Benchepoch_ai
0.3
Cl Bench Lifeepoch_ai
0.2
Critptepoch_ai
0.2
DeepSWE (Epoch Import, Version Unverified)epoch_ai
0.5 / 100
Dtbenchepoch_ai
0.9
Ebr Benchepoch_ai
0.3
Enigma Evalepoch_ai
0.2
Epoch Capabilities Indexepoch_ai
156.1
Frontiermath Tier 4 V2epoch_ai
0.5
Frontiermath Tiers 1 3 V2epoch_ai
0.8
Furniture Assemblyepoch_ai
0.4
GSOepoch_ai
0.3
Hleepoch_ai
0.4
Lmcaepoch_ai
52.0
METR Time Horizonsepoch_ai
0.7
Mirrorcodeepoch_ai
0.2
Mystery Game Puzzlesepoch_ai
0.4
OTIS Mock AIME 2024–2025epoch_ai
1.0
Posttrainbenchepoch_ai
0.0
Proofbenchepoch_ai
0.6
Scicodeepoch_ai
0.6
SimpleQA Verified (Epoch)epoch_ai
0.5
EQ-Bench 3eqbench
1328.0
FrontierMathfrontiermath
47.6 / 100
IDP Coreidp-leaderboard
84.4 / 100
IDP OlmOCRidp-leaderboard
81.0 / 100
IDP OmniDocidp-leaderboard
85.3 / 100
IDP Overallidp-leaderboard
83.5 / 100
LegalBenchlegalbench
86.0 / 100
LiveBench Codinglivebench
77.5 / 100
LiveBench Data Analysislivebench
79.3 / 100
LiveBench Instruction Followinglivebench
70.2 / 100
LiveBench Languagelivebench
82.6 / 100
LiveBench Mathlivebench
94.1 / 100
LiveBench Overalllivebench
80.3 / 100
LiveBench Reasoninglivebench
88.1 / 100
AIME 2026matharena
99.2
MedQA (USMLE)medqa
96.1
MMMU-Prommmu
81.2
OSWorld (Unversioned Matrix Import)openai_launch
75.0 / 100
GDPvalopenai_launch
83.0 / 100
OpenRouter Token Volumeopenrouter_rankings
327000000000.0
OpenRouter Weekly Trendopenrouter_rankings
37.0
EnigmaEvalseal
16.0
MCP Atlasseal
70.6
SWE-Bench Pro (Private)seal
43.4
SWE-Bench Pro (Public)seal
59.1
Vectara Factual Consistencyvectara_hallucination
93.0 / 100
Vectara Hallucination Ratevectara_hallucination
7.0 / 100
WeirdMLweirdml
57.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"2ab08f641c16bcf3c7a59c38b35fa2e3\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"b4ed41aee6b00420557f098bd05f4165\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-19

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"8cf5cb76df0b13c1684b346a2012df83\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"c6e9a78d12dc070d7dab18a1f667a410\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-15

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"e7f673a6db195dac7f2956e3b1651145\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4d3b46c6ffe9404ca1c0ebcfb5938883\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-13

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"b57a26ca4df83acaa40eadb6632c7c61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ee35c9dbbb033dc7de85467eedb85d54\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-06

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-02

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-01

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"8bfe2e142f29acec39df406a6631f81a\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"92bfaa47028eae49163960f4c5a298a0\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-30

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"16793624264c3bdded076b4e26af5a90\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"e7fb37556a7353e7da435c4c7b78ca50\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-29

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"284ac3aff89a7d8a3aca179ee1e5dd61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6e0849a05a2c572b1050e5900a372e2b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-28

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"397d7634d348db2ae8df5180db7b475d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"882404aedf04dbdde2b6c0d731212a4f\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-24

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"58c953e3f78e2282d0259b52304c8ef8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef004f05f102510151398f73e966a509\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"2f3beadbd513a221b41d3816ff651f54\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f1d0d3b7e367e4a7909882624b1caca0\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"c985e4804ffc840374d9852c6b60005c\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0a820529c69911b0c1a63d8b2d3dbf0f\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-21

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-04

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-03

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"07d05a64baf8bd11a2e8505b7f9a1cc0\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4cf045387256dca83a7f8275d8d2157b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-01

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"3a9d42354f8128c11c15f5932e4a2fc7\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a865a80739604261a4cb1abb00b293d4\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-21

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"04f6ba019798e7ba90a11d7da82ed138\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"8f93bcbb2793025d982217c174632a0c\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-13

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"1c13f7ca05a41dddb3e8dbf221993d7e\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a339ccee4fdf8453a492fc58c884889b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-07

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"6bd4d2a79abbba43993aa72466c90f28\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"909105c66efc7bcdbd1ddf40343ec110\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-06

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"fe316c967e356fc8f0ae7405b6a374ea\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1102d97f1e4c2cf4cd67b86832b6396b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-05

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"cfc9b87c645d3ef2103e359fd0f9f521\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0d65f6e0efe0d5cae15e8e2fe5a0994b\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-30

0.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · GPT-5.4 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:openai-gpt-5-4-2026-03-05-high

View run source

Source metadata
{
  "provider": "OpenAI",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "openai-gpt-5-4-2026-03-05-thinking",
  "modelNotes": null,
  "modelsEtag": "W/\"bbdf5725ef7a703a7f0970e913262b6f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6529094085f9ca7ce5f3c2e1957d3b9c\"",
  "upstreamModelId": "openai-gpt-5-4-2026-03-05-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5187.41,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-24

0.2 / 100
ExploitGym v1exploitgym

[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00 · official-869-task · GPT-5.4 · harness Codex CLI · 1 attempt · 2h timeout · exploitgym:[email protected]@dcdbc88cbef13624328cc6d059a007a5d29b3c00:869-tasks · record exploitgym:v1:parent:634bf714059bc1564c234a3c40c3432c7250c131a1b6866188dddbf2cc93eec5

Preliminary

View run source

Source metadata
{
  "taskCount": 869,
  "harnessTag": "v1.1.1",
  "metricName": "agmodbDerivedOnTargetPercent",
  "taskListUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/dcdbc88cbef13624328cc6d059a007a5d29b3c00/data/task_ids/v1.txt",
  "aggregateOnly": true,
  "onTargetCount": 61,
  "evaluationDate": "2026-06-16",
  "upstreamSource": "ExploitGym Team",
  "attemptsPerTask": 1,
  "benchmarkCommit": "dcdbc88cbef13624328cc6d059a007a5d29b3c00",
  "metricSemantics": "AgMoDB-derived on_target count divided by the fixed v1 task count",
  "aggregateVariant": "parent",
  "measurementScope": "model-agent-system",
  "agmobenchEligible": false,
  "attemptsSemantics": "one submitted result trial per benchmark task, not 869 attempts per task",
  "upstreamSourceUrl": null,
  "preliminaryReasons": [
    "missing-exact-agent-revision",
    "missing-upstream-run-identity"
  ],
  "evaluationCondition": "2h timeout",
  "domainOnTargetCounts": {
    "v8": 22,
    "kernel": 1,
    "userspace": 38
  },
  "attemptsProvenanceUrl": "https://github.com/sunblaze-ucb/exploitgym/blob/603efb3461d17ed05718ca778d2715db772d9385/docs/submission.md#resultsjson",
  "mitigatedOnTargetCounts": {
    "v8": 0,
    "kernel": 1,
    "userspace": 2
  },
  "submissionContractCommit": "603efb3461d17ed05718ca778d2715db772d9385",
  "exactReproductionEligible": false
}

Evidence observed 2026-06-16

7.0 / 100
Senior SWE-Bench v2026.06senior_swe_bench

v2026.06 · official-100-task · GPT-5.4 (xhigh) · harness mini-swe-agent · effort xhigh · 1 attempt · pass@1 · harbor:snorkel-ai/senior-swe-bench-v2026.06 · provider openai/gpt-5.4 · record gpt-5-4-mini-swe · run c243aeb40c0c89716ec5a6e2056652f84b289975286619d6e25927d25cbb69c6

View run source

Source metadata
{
  "taskCount": 100,
  "agentGroup": "mini-swe-agent:gpt-5.4",
  "basicPassAt1": 41.05263157894737,
  "basicPassAt3": 54.736842105263165,
  "datasetPackage": "snorkel-ai/senior-swe-bench-v2026.06",
  "publicTaskCount": 50,
  "tastefulPassAt1": 16.842105263157894,
  "tastefulPassAt3": 27.368421052631582,
  "agentDescription": "OpenAI GPT-5.4 via mini-swe-agent harness (Portkey→OpenAI)",
  "privateTaskCount": 50,
  "upstreamProvider": "openai",
  "upstreamRunCount": 50,
  "averageAgentSteps": 86.26315789473684,
  "averageOutputTokens": 53854.221052631576,
  "averageTotalCostUsd": 3.071602650526316,
  "averageOutputCostUsd": 0.8078133157894736,
  "basicAllThreePassRate": 24.210526315789473,
  "tastefulAllThreePassRate": 6.315789473684211
}

Evidence observed 2026-07-22

16.8 / 100
Senior SWE-Bench v2026.06senior_swe_bench

v2026.06 · official-100-task · GPT-5.4 (xhigh) · harness mini-swe-agent · effort xhigh · 1 attempt · pass@1 · harbor:snorkel-ai/senior-swe-bench-v2026.06 · provider openai/gpt-5.4 · record gpt-5-4-mini-swe · run 90509a67fed6a6131978e82ddf4b66f2848116a98bd228326ce0482485a2feb3

View run source

Source metadata
{
  "taskCount": 100,
  "agentGroup": "mini-swe-agent:gpt-5.4",
  "basicPassAt1": 48.86363636363637,
  "datasetPackage": "snorkel-ai/senior-swe-bench-v2026.06",
  "publicTaskCount": 50,
  "tastefulPassAt1": 13.636363636363635,
  "agentDescription": "OpenAI GPT-5.4 via mini-swe-agent harness (Portkey→OpenAI)",
  "privateTaskCount": 50,
  "upstreamProvider": "openai",
  "upstreamRunCount": 50,
  "averageAgentSteps": 81.7840909090909,
  "averageOutputTokens": 50491.568181818184,
  "averageTotalCostUsd": 2.9705425000000005,
  "averageOutputCostUsd": 0.7573735227272728
}

Evidence observed 2026-06-30

13.6 / 100
SWE-rebenchswe_rebench

vcontinuous-leaderboard · tasks-created:2026-02-01:2026-05-15 · gpt-5.4-2026-03-05-medium · harness swe-rebench-react@tools · effort medium · 5 attempts · pass@1 · swe-rebench:standardized:128k-or-model-limit · provider gpt-5.4-2026-03-05-medium__tools · record gpt-5.4-2026-03-05-medium__tools:1769904000000:1778803200000

View run source

Source metadata
{
  "passAt5": 70.65868263473054,
  "developer": "openai",
  "instanceType": "model",
  "contextPolicy": "128k unless the model supports less",
  "attemptsPerTask": 5,
  "totalTokenUsage": 814025.7628742515,
  "modelReleaseDate": "2026-03-05",
  "scoreStandardError": 0.8122550877994328,
  "cachedTokenPercentage": 81.4876775099086,
  "modelReleaseTimestamp": 1772668800000,
  "standardContextTokens": 128000,
  "averageInstanceCostUsd": 0.6071112652694611,
  "potentialContamination": true,
  "aggregateTaskRangeTimestamp": {
    "to": 1778803200000,
    "from": 1769904000000
  },
  "publishedTaskRangeTimestamp": {
    "to": 1778803200000,
    "from": 1769904000000
  }
}

Evidence observed 2026-05-15

57.6 / 100