AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Models

Claude Sonnet 5.5

Anthropic's Sonnet-class model for well-scoped everyday work, a direct upgrade to Claude Sonnet 5, strong at building features and fixing bugs, with adaptive reasoning effort tiers.

September 28, 2026Last synced Nov 22, 2026, 8:17 AM1000K context128K output
Effort
  • Default
  • Low
  • Medium
  • High
  • Xhigh

Blended Price

$4.00/M

Input Price

$2.00/M

Output Price

$10.00/M

Speed

134 tok/s

TTFT

352.94s

Benchmark Scores

Intelligence Index
54.5
Coding Index
—
Math Index
—
MMLU Pro
— / 100
GPQA Diamond
— / 100
HLE
55.0 / 100
LiveCodeBench
— / 100
SciCode
61.0 / 100
MATH-500
— / 100
AIME
— / 30
AIME 2025
— / 30
IFBench
— / 100
LCR
82.7 / 100
Terminal-Bench Hard
— / 100
τ²-Bench
— / 100

Performance Radar

Claude Sonnet 5.5
Field Average

External Benchmarks

AA-Omniscience Accuracyaa_omniscience
51.5 / 100
AA-Omniscience Hallucination Rateaa_omniscience
43.3 / 100
AA-Omniscience Indexaa_omniscience
30.5 / 100
ARC-AGI-1arc_agi
67.0 / 100
ARC-AGI-1 Cost per Taskarc_agi
0.1
ARC-AGI-2arc_agi
24.0 / 100
ARC-AGI-2 Cost per Taskarc_agi
0.1
BullshitBenchbullshitbench
60.0 / 100
Arena ELO: Codingchatbot_arena
1772.0
Chatbot Arena ELOchatbot_arena
1470.0
arena_elo_visionchatbot_arena
1286.0
Apex Agentsepoch_ai
0.8
Blueprint Bench 2epoch_ai
0.4
Chess Puzzlesepoch_ai
0.6
Critptepoch_ai
0.3
Cursorbenchepoch_ai
0.5
Dtbenchepoch_ai
1.0
Frontiermath Erdosepoch_ai
0.0
Frontiermath Tier 4 V2epoch_ai
0.8
Frontiermath Tiers 1 3 V2epoch_ai
0.9
Frontiersweepoch_ai
0.6
Furniture Assemblyepoch_ai
0.8
Gdp Pdfepoch_ai
0.5
Lmcaepoch_ai
68.3
Mirrorcodeepoch_ai
1.0
Mystery Game Puzzlesepoch_ai
0.7
OTIS Mock AIME 2024–2025epoch_ai
1.0
Proofbenchepoch_ai
1.0
Scicodeepoch_ai
0.6
SimpleQA Verified (Epoch)epoch_ai
0.5
Spireclimbepoch_ai
0.0
Vending Bench 2epoch_ai
11165.3
LiveBench Codinglivebench
77.2 / 100
LiveBench Data Analysislivebench
59.5 / 100
LiveBench Instruction Followinglivebench
70.5 / 100
LiveBench Languagelivebench
83.4 / 100
LiveBench Mathlivebench
96.7 / 100
LiveBench Reasoninglivebench
91.6 / 100
OpenRouter Token Volumeopenrouter_rankings
2880000000000.0
OpenRouter Weekly Trendopenrouter_rankings
19.0
SimpleBenchsimplebench
75.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5487.227968,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

1.2 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 8169.902276,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

72.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4839.68249,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 10244.13243,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

35.5 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 7394.466286,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

24.3 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4796.883118,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

0.6 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
  "modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 5250.163516,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

4.9 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low

View run source

Source metadata
{
  "provider": "Anthropic",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "anthropic-claude-sonnet-5-5",
  "modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "anthropic-claude-sonnet-5-5-low",
  "environmentCount": 25,
  "modelReleaseDate": "2026-09-28T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 4756.680228,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

0.4 / 100
SWE-rebenchswe_rebench

vcontinuous-leaderboard · tasks-created:2026-05-15:2026-07-01 · Sonnet 5 [high] · harness swe-rebench-react@tools · effort high · 5 attempts · pass@1 · swe-rebench:standardized:128k-or-model-limit · provider Sonnet 5 [high]__tools · record Sonnet 5 [high]__tools:1778803200000:1782864000000

View run source

Source metadata
{
  "passAt5": 74.77477477477478,
  "developer": "anthropic",
  "instanceType": "model",
  "contextPolicy": "128k unless the model supports less",
  "attemptsPerTask": 5,
  "totalTokenUsage": 4645616.863063063,
  "modelReleaseDate": "2026-06-30",
  "scoreStandardError": 0.9448728361893257,
  "cachedTokenPercentage": 96.41145528118132,
  "modelReleaseTimestamp": 1782777600000,
  "standardContextTokens": 128000,
  "averageInstanceCostUsd": 1.4258611596396396,
  "potentialContamination": true,
  "aggregateTaskRangeTimestamp": {
    "to": 1782864000000,
    "from": 1778803200000
  },
  "publishedTaskRangeTimestamp": {
    "to": 1782864000000,
    "from": 1778803200000
  }
}

Evidence observed 2026-07-01

56.8 / 100

How Claude Sonnet 5.5 Compares

Axes

Filters

$0.00 – $67.5
≥ 1.4

Providers

ai2ai21-labsalibabaanthropicarceeawsazurebaidu
openai
zai
meta
nvidia
xai
upstage
deepseek
alibaba
mistral
nous-research
aws
google
anthropic
xiaomi
stepfun
kimi
reflection
minimax
azure
inclusionai
reka-ai
cohere
bytedance_seed
inception
tencent
thinking-machines
Bubble size = Context Window
8191.00
2000000.00
Compare with other models