Anthropic's Sonnet-class model for well-scoped everyday work, a direct upgrade to Claude Sonnet 5, strong at building features and fixing bugs, with adaptive reasoning effort tiers.
Blended Price
$4.00/M
Input Price
$2.00/M
Output Price
$10.00/M
Speed
134 tok/s
TTFT
352.94s
Hidden, no data: AgMoBench Overall, AgMoBench Reasoning, AgMoBench Coding, AgMoBench Math, AgMoBench Agentic, AgMoBench Robustness
* Shared benchmarks shown in their place: HLE, SciCode, LCR
Axes
Filters
Providers
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-20
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-18
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-17
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-16
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-10
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-09
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-06
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-04
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-11-02
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-10-28
v3 · public-demo · Claude Sonnet 5.5 (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the standard harness.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5487.227968,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 8169.902276,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the standard harness.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4839.68249,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (High) · effort high · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-high-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'high' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-high-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 10244.13243,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 (XHigh) · effort xhigh · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-xhigh
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'xhigh' reasoning effort using the standard harness.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-xhigh",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 7394.466286,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the provider adapter harness.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4796.883118,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 - Provider Adapter (Medium) · effort medium · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-medium-provider-adapter
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5-provider-adapter",
"modelNotes": "Evaluation conducted with 'medium' reasoning effort using the provider adapter harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-medium-provider-adapter",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 5250.163516,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
v3 · public-demo · Claude Sonnet 5.5 (Low) · effort low · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:anthropic-claude-sonnet-5-5-low
{
"provider": "Anthropic",
"modelType": "CoT",
"modelsUrl": "https://arcprize.org/media/data/models.json",
"modelGroup": "anthropic-claude-sonnet-5-5",
"modelNotes": "Evaluation conducted with 'low' reasoning effort using the standard harness. Partial dataset coverage: scores use full catalog denominators; costs include recorded usage only, excluding unrecorded attempts. Some model actions have unmeasured token usage; reported costs are recorded-usage lower bounds, not verified full-run costs.",
"modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
"scoreMetric": "relative-human-action-efficiency",
"evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
"evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
"upstreamModelId": "anthropic-claude-sonnet-5-5-low",
"environmentCount": 25,
"modelReleaseDate": "2026-09-28T00:00:00.000Z",
"upstreamDatasetId": "v3_Semi_Private",
"actionBudgetPolicy": "5x-human-baseline-median-per-level",
"technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
"catalogMatchAliases": [],
"agentHarnessPublished": false,
"verificationAuthority": "ARC Prize Foundation",
"upstreamRunIdPublished": false,
"actualEvaluationCostUsd": 4756.680228,
"providerModelIdPublished": false
}Evidence observed 2026-10-21
vcontinuous-leaderboard · tasks-created:2026-05-15:2026-07-01 · Sonnet 5 [high] · harness swe-rebench-react@tools · effort high · 5 attempts · pass@1 · swe-rebench:standardized:128k-or-model-limit · provider Sonnet 5 [high]__tools · record Sonnet 5 [high]__tools:1778803200000:1782864000000
{
"passAt5": 74.77477477477478,
"developer": "anthropic",
"instanceType": "model",
"contextPolicy": "128k unless the model supports less",
"attemptsPerTask": 5,
"totalTokenUsage": 4645616.863063063,
"modelReleaseDate": "2026-06-30",
"scoreStandardError": 0.9448728361893257,
"cachedTokenPercentage": 96.41145528118132,
"modelReleaseTimestamp": 1782777600000,
"standardContextTokens": 128000,
"averageInstanceCostUsd": 1.4258611596396396,
"potentialContamination": true,
"aggregateTaskRangeTimestamp": {
"to": 1782864000000,
"from": 1778803200000
},
"publishedTaskRangeTimestamp": {
"to": 1782864000000,
"from": 1778803200000
}
}Evidence observed 2026-07-01