AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Models

Gemini 3.1 Pro Preview

February 19, 2026Last synced Nov 22, 2026, 8:17 AM1049K context

Blended Price

$4.50/M

Input Price

$2.00/M

Output Price

$12.00/M

Speed

135 tok/s

TTFT

18.97s

Benchmark Scores

Intelligence Index

Performance Radar

Gemini 3.1 Pro Preview
Field Average

External Benchmarks

AA-Omniscience Accuracy

How Gemini 3.1 Pro Preview Compares

Axes

Filters

$0.00 – $67.5
≥ 1.4

Providers

ai2ai21-labsalibabaanthropicarceeawsazurebaidu
openai
zai
meta
nvidia
xai
upstage
deepseek
alibaba
mistral
nous-research
aws
google
anthropic
xiaomi
stepfun
kimi
reflection
minimax
azure
inclusionai
reka-ai
cohere
bytedance_seed
inception
tencent
thinking-machines
Bubble size = Context Window
8191.00
2000000.00
Compare with other models
—
Coding Index
—
Math Index
—
MMLU Pro
— / 100
GPQA Diamond
94.1 / 100
HLE
47.0 / 100
LiveCodeBench
— / 100
SciCode
58.7 / 100
MATH-500
— / 100
AIME
— / 30
AIME 2025
— / 30
IFBench
77.1 / 100
LCR
82.0 / 100
Terminal-Bench Hard
53.8 / 100
τ²-Bench
95.6 / 100
aa_omniscience
54.9 / 100
AA-Omniscience Hallucination Rateaa_omniscience
50.9 / 100
AA-Omniscience Indexaa_omniscience
31.9 / 100
Finance Agent v1.1anthropic_launch
59.7 / 100
HLE (with tools)anthropic_launch
51.4 / 100
ARC-AGI-1arc_agi
98.0 / 100
ARC-AGI-1 Cost per Taskarc_agi
0.5
ARC-AGI-2arc_agi
77.1 / 100
ARC-AGI-2 Cost per Taskarc_agi
1.0
ARC-AGI-1benchmark_matrix
90.0 / 100
ARC-AGI-2benchmark_matrix
77.1 / 100
AA Intelligence Index (Matrix)benchmark_matrix
57.0
AIME 2024benchmark_matrix
98.0
AIME 2025 (Matrix)benchmark_matrix
100.0
AIME 2026benchmark_matrix
97.0
BrowseCompbenchmark_matrix
85.9
Chatbot Arena ELO (Matrix)benchmark_matrix
1490.0
Codeforces Ratingbenchmark_matrix
2700.0
CritPtbenchmark_matrix
17.7
GPQA Diamond (Matrix)benchmark_matrix
94.3
HLE (Matrix)benchmark_matrix
44.4
HumanEvalbenchmark_matrix
95.0
IFEvalbenchmark_matrix
89.2
LiveCodeBench (Matrix)benchmark_matrix
82.0
MATH-500 (Matrix)benchmark_matrix
98.5
MathArena Apex 2025benchmark_matrix
33.5
MathVisionbenchmark_matrix
69.8
MMLUbenchmark_matrix
92.6
MMLU-Pro (Matrix)benchmark_matrix
89.5
MMMU-Probenchmark_matrix
80.5
MRCR v2benchmark_matrix
84.9
OSWorld (Unversioned Matrix Import)benchmark_matrix
72.0 / 100
SciCode (Matrix)benchmark_matrix
58.9
SimpleQAbenchmark_matrix
72.1
SWE-bench Probenchmark_matrix
54.2
Terminal-Bench 2.0benchmark_matrix
68.5
browsecompbenchmark_matrix
85.9
FrontierMathbenchmark_matrix
40.0 / 100
MMMUbenchmark_matrix
87.5 / 100
SimpleBenchbenchmark_matrix
79.6 / 100
simpleqabenchmark_matrix
72.1
SWE-bench Verifiedbenchmark_matrix
80.6 / 100
tau-bench Retailbenchmark_matrix
90.5 / 100
AA-Omniscience Hallucination RatePredicted
93.9 / 100
Aider PolyglotPredicted
85.6 / 100
AIMEPredicted
0.3 / 30
AIME 2025Predicted
0.9 / 30
AlpacaEval 2.0 LCPredicted
14.3 / 100
AlpacaEval 2.0 RawPredicted
9.5 / 100
BFCL (Berkeley Function Calling)Predicted
56.8
BigCodeBench CompletePredicted
37.4 / 100
BigCodeBench InstructPredicted
29.2 / 100
AA Long Context Reasoning (Matrix)Predicted
76.8
Arena-Hard AutoPredicted
74.5
BRUMO 2025Predicted
99.8
CMIMC 2025Predicted
38.4
GSM8KPredicted
84.3
HMMT Feb 2025Predicted
97.9
HMMT Nov 2025Predicted
93.3
IFBench (Matrix)Predicted
72.8
IMO 2025Predicted
46.9
SMT 2025Predicted
95.5
Tau-Bench Telecom (Matrix)Predicted
99.0
Terminal-Bench 1.0Predicted
55.9
USAMO 2025Predicted
69.2
Video-MMUPredicted
94.0
Aider PolyglotPredicted
0.3
BALROGPredicted
0.0
BIG-Bench HardPredicted
3.0
BoolQPredicted
0.6
CAD-EvalPredicted
2.3
CyBenchPredicted
1.1
DeepResearchBenchPredicted
0.5
FictionLiveBenchPredicted
0.7
GdpvalPredicted
0.6
GeoBenchPredicted
0.0
GSM8K (Epoch)Predicted
1.8
GSOPredicted
0.7
HellaSwagPredicted
0.0
HlePredicted
0.5
Lech Mazur WritingPredicted
8.8
METR Time HorizonsPredicted
57.3
PIQAPredicted
0.8
The Agent CompanyPredicted
56.4
TriviaQAPredicted
27.7
VPCTPredicted
0.8
WinoGrandePredicted
1.6
GAIA Level 1Predicted
84.3
GAIA Level 2Predicted
69.0
GAIA Level 3Predicted
89.4
GAIAPredicted
71.0 / 100
LiveCodeBenchPredicted
0.9 / 100
LongBench v2 EasyPredicted
56.7
LongBench v2 HardPredicted
28.7
LongBench v2Predicted
25.3 / 100
MATH-500Predicted
0.9 / 100
MathVistaPredicted
57.7 / 100
MLE-bench (Original)Predicted
67.1 / 100
MMLU ProPredicted
0.9 / 100
MMTU Table UnderstandingPredicted
73.9 / 100
MT-BenchPredicted
7.8 / 10
NoLiMa (NIAH)Predicted
93.2 / 100
OCRBench v2Predicted
86.0 / 100
Open LLM AveragePredicted
18.4 / 100
Open LLM: BBHPredicted
49.3 / 100
Open LLM: GPQAPredicted
27.4 / 100
Open LLM: IFEvalPredicted
24.7 / 100
Open LLM: MATH Level 5Predicted
53.9 / 100
Open LLM: MMLU-PROPredicted
33.3 / 100
Open LLM: MUSRPredicted
39.8 / 100
RE-BenchPredicted
100.0
SWE-bench LitePredicted
65.7 / 100
WebArenaPredicted
4.3 / 100
WildBenchPredicted
34.6
browsecompbrowsecomp
85.9
BullshitBenchbullshitbench
37.0 / 100
Arena ELO: Codingchatbot_arena
1446.0
Arena ELO: Creative Writingchatbot_arena
1490.0
arena_elo_documentchatbot_arena
1443.0
Chatbot Arena ELOchatbot_arena
1480.0
arena_elo_visionchatbot_arena
1296.0
Ale Benchepoch_ai
1054.8
Algotuneepoch_ai
2.0
Apex Agentsepoch_ai
0.4
Arc Agi 2epoch_ai
0.8
BALROGepoch_ai
0.6
Blueprint Bench 2epoch_ai
0.3
Chess Puzzlesepoch_ai
0.5
Cl Benchepoch_ai
0.2
Cl Bench Lifeepoch_ai
0.2
Critptepoch_ai
0.2
DeepResearchBenchepoch_ai
0.4
DeepSWE (Epoch Import, Version Unverified)epoch_ai
0.1 / 100
Dtbenchepoch_ai
0.9
Ebr Benchepoch_ai
0.1
Eci Scoresepoch_ai
157.6
Enigma Evalepoch_ai
0.4
Epoch Capabilities Indexepoch_ai
154.9
Exploitbenchepoch_ai
0.2
Forecastbenchepoch_ai
63.9
Frontiercodeepoch_ai
0.0
Frontiermath Tier 4 V2epoch_ai
0.3
Frontiermath Tiers 1 3 V2epoch_ai
0.6
Frontiersweepoch_ai
15.0
Furniture Assemblyepoch_ai
0.3
Gbaevalepoch_ai
0.0
Gdp Pdfepoch_ai
0.2
GeoBenchepoch_ai
0.8
GSOepoch_ai
0.2
Hleepoch_ai
0.5
Lmcaepoch_ai
51.2
METR Time Horizonsepoch_ai
0.8
Mirrorcodeepoch_ai
0.1
Mystery Game Puzzlesepoch_ai
0.3
OTIS Mock AIME 2024–2025epoch_ai
1.0
Posttrainbenchepoch_ai
0.2
Processed Data For Eciepoch_ai
0.2
Proofbenchepoch_ai
0.3
Scicodeepoch_ai
0.6
SimpleQA Verified (Epoch)epoch_ai
0.7
Vending Bench 2epoch_ai
911.2
WebDev Arenaepoch_ai
1446.6
EQ-Bench 3eqbench
1302.0
FrontierMathfrontiermath
36.9 / 100
IDP Coreidp-leaderboard
89.6 / 100
IDP OlmOCRidp-leaderboard
69.8 / 100
IDP OmniDocidp-leaderboard
85.3 / 100
IDP Overallidp-leaderboard
81.6 / 100
LegalBenchlegalbench
10.1 / 100
LiveBench Codinglivebench
66.7 / 100
LiveBench Data Analysislivebench
78.5 / 100
LiveBench Instruction Followinglivebench
79.1 / 100
LiveBench Languagelivebench
85.4 / 100
LiveBench Mathlivebench
91.0 / 100
LiveBench Overalllivebench
79.9 / 100
LiveBench Reasoninglivebench
84.0 / 100
AIME 2026matharena
98.3
MedQA (USMLE)medqa
96.4
MMMU-Prommmu
80.5
OpenRouter Token Volumeopenrouter_rankings
369000000000.0
OpenRouter Weekly Trendopenrouter_rankings
23.0
EnigmaEvalseal
36.8
MCP Atlasseal
78.2
MultiChallengeseal
71.4
SWE-Bench Pro (Private)seal
32.2
SWE-Bench Pro (Public)seal
46.1
SimpleBenchsimplebench
79.6 / 100
SWE-bench Verifiedswebench
80.6 / 100
Vectara Factual Consistencyvectara_hallucination
89.6 / 100
Vectara Hallucination Ratevectara_hallucination
10.4 / 100
WeirdMLweirdml
72.1 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"ec09fbed9f211f59ec54cca2227c3309\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5814b0938ba4a54ba31da46fe17d4380\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"d8dfdc9958a2e1502b54603b03fbc557\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"aaa10807f3dcaae3ce6c15be7539e733\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-18

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"f4d8307b32b2fefe010860bedd0c5dd8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef15ebd7644a875688eda094d716da9a\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-17

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"69aae253e532553a920bf4deb50ca495\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"5206202f381a21893de82dcc2db93ae9\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-16

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"513aaaa353972ca0baa9c95cb95e04dd\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"169653a17d103c8dbca682167ffccaa6\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-10

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"3d7dcca4fb88130b9e7ec186fd9ae23f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1e46e6ab8c83fb1a7b19f10be398adf2\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-09

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"8ea4d2292a0a8c759aa7e80a2041122d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"94fd3d68aacca744ff3c39f63a187685\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-06

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-04

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"c4baed683b2f33c40537f2338682475f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"48d1fd5bf96ae9583213f6699f12f2cc\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-11-02

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"8fc20a5217998d26c40ad2306d648194\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"095b2fa20aac96121092dab07e18cc8e\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-28

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"400b32b66e8c1b37fc50ff586b033048\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9314d4bea0d9d001c69a11614dc8d93c\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-21

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"2ab08f641c16bcf3c7a59c38b35fa2e3\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"b4ed41aee6b00420557f098bd05f4165\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-19

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"8cf5cb76df0b13c1684b346a2012df83\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"c6e9a78d12dc070d7dab18a1f667a410\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-15

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"57cc4a7689e563a0ffb7d275eb3a889b\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"19d71f8c9a2056c0be4bb4fd4a75a325\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-14

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"e7f673a6db195dac7f2956e3b1651145\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4d3b46c6ffe9404ca1c0ebcfb5938883\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-13

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"b57a26ca4df83acaa40eadb6632c7c61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ee35c9dbbb033dc7de85467eedb85d54\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-06

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-02

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"b52a6e82589fb2c719168d69aa0bb015\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f187ceaff4d7f6156e77c1f1254aafc8\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-10-01

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"8bfe2e142f29acec39df406a6631f81a\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"92bfaa47028eae49163960f4c5a298a0\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-30

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"16793624264c3bdded076b4e26af5a90\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"e7fb37556a7353e7da435c4c7b78ca50\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-29

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"284ac3aff89a7d8a3aca179ee1e5dd61\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6e0849a05a2c572b1050e5900a372e2b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-28

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"397d7634d348db2ae8df5180db7b475d\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"882404aedf04dbdde2b6c0d731212a4f\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-24

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"58c953e3f78e2282d0259b52304c8ef8\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"ef004f05f102510151398f73e966a509\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"2f3beadbd513a221b41d3816ff651f54\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"f1d0d3b7e367e4a7909882624b1caca0\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-22

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"c985e4804ffc840374d9852c6b60005c\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0a820529c69911b0c1a63d8b2d3dbf0f\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-21

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-04

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"a8dc9c600e07efa11ca8fb622181783f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"946807139d47f4bc368591b95885dfb4\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-03

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"07d05a64baf8bd11a2e8505b7f9a1cc0\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"4cf045387256dca83a7f8275d8d2157b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-09-01

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"3a9d42354f8128c11c15f5932e4a2fc7\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a865a80739604261a4cb1abb00b293d4\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-21

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"a2c874a4c5dac6490bc20617a93e7b62\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"66731344a65b8943f176406bd6afd4bb\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-20

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"04f6ba019798e7ba90a11d7da82ed138\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"8f93bcbb2793025d982217c174632a0c\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-13

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"1c13f7ca05a41dddb3e8dbf221993d7e\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"a339ccee4fdf8453a492fc58c884889b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-07

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"6bd4d2a79abbba43993aa72466c90f28\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"909105c66efc7bcdbd1ddf40343ec110\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-06

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"fe316c967e356fc8f0ae7405b6a374ea\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"1102d97f1e4c2cf4cd67b86832b6396b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-08-05

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"2fc19b6caba8b52765ca17eab04fc437\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"9485c7ef206d9b12cdfdec6a631df56b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-31

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"cfc9b87c645d3ef2103e359fd0f9f521\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"0d65f6e0efe0d5cae15e8e2fe5a0994b\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-30

0.4 / 100
ARC-AGI-3 Public Demoarc_agi

v3 · public-demo · Gemini 3.1 Pro (Preview) · 1 attempt · 5x-human-baseline-median-per-level · arc-agi-3:public-demo · record v3_Semi_Private:google-gemini-3-1-pro-preview

View run source

Source metadata
{
  "provider": "Google",
  "modelType": "CoT",
  "modelsUrl": "https://arcprize.org/media/data/models.json",
  "modelGroup": "google-gemini-3-1-pro-preview",
  "modelNotes": null,
  "modelsEtag": "W/\"bbdf5725ef7a703a7f0970e913262b6f\"",
  "scoreMetric": "relative-human-action-efficiency",
  "evaluationsUrl": "https://arcprize.org/media/data/evaluations.json",
  "evaluationsEtag": "W/\"6529094085f9ca7ce5f3c2e1957d3b9c\"",
  "upstreamModelId": "google-gemini-3-1-pro-preview",
  "environmentCount": 25,
  "modelReleaseDate": "2026-03-05T00:00:00.000Z",
  "upstreamDatasetId": "v3_Semi_Private",
  "actionBudgetPolicy": "5x-human-baseline-median-per-level",
  "technicalReportUrl": "https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf",
  "catalogMatchAliases": [],
  "agentHarnessPublished": false,
  "verificationAuthority": "ARC Prize Foundation",
  "upstreamRunIdPublished": false,
  "actualEvaluationCostUsd": 2213.53,
  "providerModelIdPublished": false
}

Evidence observed 2026-07-24

0.4 / 100
Senior SWE-Bench v2026.06senior_swe_bench

v2026.06 · official-100-task · Gemini 3.1 Pro (high) · harness mini-swe-agent · effort high · 1 attempt · pass@1 · harbor:snorkel-ai/senior-swe-bench-v2026.06 · provider google/gemini-3.1-pro-preview · record gemini-3-1-pro-mini-swe · run e7a084b8bf4fbf7e4723f412d7fc0b621bc5db20fd0e61b5014f602fa22d436e

View run source

Source metadata
{
  "taskCount": 100,
  "agentGroup": "mini-swe-agent:gemini-3.1-pro-preview",
  "basicPassAt1": 9.473684210526317,
  "basicPassAt3": 24.210526315789473,
  "datasetPackage": "snorkel-ai/senior-swe-bench-v2026.06",
  "publicTaskCount": 50,
  "tastefulPassAt1": 2.1052631578947367,
  "tastefulPassAt3": 8.421052631578947,
  "agentDescription": "Google Gemini 3.1 Pro via mini-swe-agent harness (Portkey→Google)",
  "privateTaskCount": 50,
  "upstreamProvider": "google",
  "upstreamRunCount": 50,
  "averageAgentSteps": 107.73684210526316,
  "averageOutputTokens": 17277.894736842107,
  "averageTotalCostUsd": 1.3306002509473684,
  "averageOutputCostUsd": 0.2073347368421053,
  "basicAllThreePassRate": 5.263157894736842,
  "tastefulAllThreePassRate": 0
}

Evidence observed 2026-07-29

2.1 / 100
Senior SWE-Bench v2026.06senior_swe_bench

v2026.06 · official-100-task · Gemini 3.1 Pro (high) · harness mini-swe-agent · effort high · 1 attempt · pass@1 · harbor:snorkel-ai/senior-swe-bench-v2026.06 · provider google/gemini-3.1-pro-preview · record gemini-3-1-pro-mini-swe · run 3018be1dee8e4b352d3aa9260d805ee298040ab383dec290a16294f9afd8472d

View run source

Source metadata
{
  "taskCount": 100,
  "agentGroup": "mini-swe-agent:gemini-3.1-pro-preview",
  "basicPassAt1": 26.136363636363637,
  "datasetPackage": "snorkel-ai/senior-swe-bench-v2026.06",
  "publicTaskCount": 50,
  "tastefulPassAt1": 6.8181818181818175,
  "agentDescription": "Google Gemini 3.1 Pro via mini-swe-agent harness (Portkey→Google)",
  "privateTaskCount": 50,
  "upstreamProvider": "google",
  "upstreamRunCount": 50,
  "averageAgentSteps": 89.52272727272727,
  "averageOutputTokens": 19415.81818181818,
  "averageTotalCostUsd": 0.8407021872727273,
  "averageOutputCostUsd": 0.23298981818181816
}

Evidence observed 2026-06-30

6.8 / 100
SWE-rebenchswe_rebench

vcontinuous-leaderboard · tasks-created:2026-02-01:2026-05-15 · Gemini 3.1 Pro Preview · harness swe-rebench-react@tools · 5 attempts · pass@1 · swe-rebench:standardized:128k-or-model-limit · provider Gemini 3.1 Pro Preview__tools · record Gemini 3.1 Pro Preview__tools:1769904000000:1778803200000

View run source

Source metadata
{
  "passAt5": 69.46107784431138,
  "developer": "google",
  "instanceType": "model",
  "contextPolicy": "128k unless the model supports less",
  "attemptsPerTask": 5,
  "totalTokenUsage": 1502695.5944015582,
  "modelReleaseDate": "2026-02-19",
  "scoreStandardError": 0.9239067816830353,
  "cachedTokenPercentage": 80.60600910081673,
  "modelReleaseTimestamp": 1771459200000,
  "standardContextTokens": 128000,
  "averageInstanceCostUsd": 0.7208578944116585,
  "potentialContamination": true,
  "aggregateTaskRangeTimestamp": {
    "to": 1778803200000,
    "from": 1769904000000
  },
  "publishedTaskRangeTimestamp": {
    "to": 1778803200000,
    "from": 1769904000000
  }
}

Evidence observed 2026-05-15

54.9 / 100
Terminal-Bench 2.1terminal_bench

v2.1 · official-leaderboard · Gemini 3.1 Pro (high) · harness [email protected] · effort high · 5 attempts · harbor:terminal-bench/terminal-bench-2-1 · provider gemini/gemini-3.1-pro-preview · record leaderboard/submissions/2026-05-05-gemini-gemini-3-1-pro-preview-high-gemini-cli.json · run 42cd19c9-42ad-5d79-b033-adf4f879423d

View run source

Source metadata
{
  "passAt2": 0.782,
  "passAt3": 0.8315,
  "passAt4": 0.8584,
  "passAt5": 0.8764,
  "taskCount": 89,
  "actualTokens": {
    "output": 6123596,
    "cachedInput": 366583353,
    "uncachedInput": 44843158
  },
  "datasetPackage": "terminal-bench/terminal-bench-2-1",
  "leaderboardUrl": "https://www.tbench.ai/leaderboard/terminal-bench/2.1",
  "pullRequestUrl": "https://github.com/harbor-framework/terminal-bench-2-1/pull/68",
  "agentDisplayUrl": "https://github.com/google-gemini/gemini-cli",
  "modelDisplayUrl": "https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-pro/",
  "actualTrialCount": 445,
  "agentOrganization": "Google",
  "modelOrganization": "Google",
  "actualTotalCostUsd": 236.49,
  "rewardHacksPercent": 0.22,
  "accuracyStandardError": 1.67,
  "submissionDownloadUrl": "https://raw.githubusercontent.com/harbor-framework/terminal-bench-2-1/main/leaderboard/submissions/2026-05-05-gemini-gemini-3-1-pro-preview-high-gemini-cli.json",
  "disqualifiedTrialCount": 1,
  "expectedAttemptsPerTask": 5,
  "sourceFilterReasoningEffort": "high",
  "actualAverageTrialDurationSeconds": 652.9
}

Evidence observed 2026-05-05

65.8 / 100
Terminal-Bench 2.1terminal_bench

v2.1 · official-leaderboard · Gemini 3.1 Pro (high) · harness [email protected] · effort high · 5 attempts · harbor:terminal-bench/terminal-bench-2-1 · provider gemini/gemini-3.1-pro-preview · record leaderboard/submissions/2026-05-05-gemini-gemini-3-1-pro-preview-high-terminus-2.json · run 42cd19c9-42ad-5d79-b033-adf4f879423d

View run source

Source metadata
{
  "passAt2": 0.7775,
  "passAt3": 0.8225,
  "passAt4": 0.8472,
  "passAt5": 0.8652,
  "taskCount": 89,
  "actualTokens": {
    "output": 12935831,
    "cachedInput": 96199940,
    "uncachedInput": 27525560
  },
  "datasetPackage": "terminal-bench/terminal-bench-2-1",
  "leaderboardUrl": "https://www.tbench.ai/leaderboard/terminal-bench/2.1",
  "pullRequestUrl": "https://github.com/harbor-framework/terminal-bench-2-1/pull/69",
  "agentDisplayUrl": "https://www.tbench.ai/news/terminus",
  "modelDisplayUrl": "https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-pro/",
  "actualTrialCount": 445,
  "agentOrganization": "Terminal-Bench",
  "modelOrganization": "Google",
  "actualTotalCostUsd": 229.99,
  "rewardHacksPercent": 0.45,
  "accuracyStandardError": 1.65,
  "submissionDownloadUrl": "https://raw.githubusercontent.com/harbor-framework/terminal-bench-2-1/main/leaderboard/submissions/2026-05-05-gemini-gemini-3-1-pro-preview-high-terminus-2.json",
  "disqualifiedTrialCount": 2,
  "expectedAttemptsPerTask": 5,
  "sourceFilterReasoningEffort": "high",
  "actualAverageTrialDurationSeconds": 672.8
}

Evidence observed 2026-05-05

65.6 / 100