AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Models

o3

April 16, 2025Last synced Nov 22, 2026, 8:18 AM200K context

Blended Price

$3.50/M

Input Price

$2.00/M

Output Price

$8.00/M

Speed

110 tok/s

TTFT

4.85s

Benchmark Scores

Intelligence Index
—
Coding Index
—
Math Index
—
MMLU Pro
85.3 / 100
GPQA Diamond
82.7 / 100
HLE
20.1 / 100
LiveCodeBench
80.8 / 100
SciCode
— / 100
MATH-500
99.2 / 100
AIME
90.3 / 30
AIME 2025
88.3 / 30
IFBench
71.4 / 100
LCR
74.7 / 100
Terminal-Bench Hard
37.1 / 100
τ²-Bench
80.7 / 100

Performance Radar

o3
Field Average

External Benchmarks

Aider Polyglotaider
76.9 / 100
AlpacaEval 2.0 LCalpacaeval
76.9 / 100
AlpacaEval 2.0 Rawalpacaeval
69.1 / 100
ARC-AGI-1arc_agi
60.8 / 100
ARC-AGI-1 Cost per Taskarc_agi
0.5
ARC-AGI-2arc_agi
6.5 / 100
ARC-AGI-2 Cost per Taskarc_agi
0.8
ARC-AGI-1benchmark_matrix
75.7 / 100
ARC-AGI-2benchmark_matrix
6.5 / 100
AA Long Context Reasoning (Matrix)benchmark_matrix
69.0
AIME 2024benchmark_matrix
96.7
AIME 2025 (Matrix)benchmark_matrix
88.9
Arena-Hard Autobenchmark_matrix
85.9
BrowseCompbenchmark_matrix
49.7
BRUMO 2025benchmark_matrix
95.8
CMIMC 2025benchmark_matrix
79.4
Codeforces Ratingbenchmark_matrix
2706.0
CritPtbenchmark_matrix
1.4
GPQA Diamond (Matrix)benchmark_matrix
87.7
HLE (Matrix)benchmark_matrix
19.8
HMMT Feb 2025benchmark_matrix
77.5
HumanEvalbenchmark_matrix
88.4
IFBench (Matrix)benchmark_matrix
69.3
IFEvalbenchmark_matrix
92.1
IMO 2025benchmark_matrix
16.7
LiveCodeBench (Matrix)benchmark_matrix
75.8
MATH-500 (Matrix)benchmark_matrix
99.2
MMLUbenchmark_matrix
93.3
MMLU-Pro (Matrix)benchmark_matrix
85.9
MMMU-Probenchmark_matrix
75.0
SimpleQAbenchmark_matrix
48.6
SMT 2025benchmark_matrix
87.7
Terminal-Bench 1.0benchmark_matrix
30.2
USAMO 2025benchmark_matrix
21.9
browsecompbenchmark_matrix
49.7
FrontierMathbenchmark_matrix
10.0 / 100
MMMUbenchmark_matrix
80.1 / 100
simpleqabenchmark_matrix
48.6
SWE-bench Verifiedbenchmark_matrix
69.1 / 100
AA-Omniscience AccuracyPredicted
40.9 / 100
AA-Omniscience Hallucination RatePredicted
97.2 / 100
BigCodeBench CompletePredicted
59.9 / 100
BigCodeBench InstructPredicted
55.9 / 100
AA Intelligence Index (Matrix)Predicted
65.3
GSM8KPredicted
96.1
HMMT Nov 2025Predicted
94.2
MathArena Apex 2025Predicted
2.7
MRCR v2Predicted
77.1
OSWorld (Unversioned Matrix Import)Predicted
48.6 / 100
SWE-bench ProPredicted
43.5
Tau-Bench Telecom (Matrix)Predicted
97.6
Terminal-Bench 2.0Predicted
40.4
Video-MMUPredicted
86.5
Apex AgentsPredicted
2.7
BALROGPredicted
0.0
BIG-Bench HardPredicted
3.0
BoolQPredicted
0.7
Chess PuzzlesPredicted
0.2
CyBenchPredicted
0.4
GSM8K (Epoch)Predicted
0.3
HellaSwagPredicted
0.0
PIQAPredicted
0.8
PosttrainbenchPredicted
0.0
The Agent CompanyPredicted
2.8
TriviaQAPredicted
48.1
WinoGrandePredicted
0.8
LegalBenchPredicted
85.4 / 100
LiveBench CodingPredicted
76.0 / 100
LiveBench Data AnalysisPredicted
50.3 / 100
LiveBench LanguagePredicted
68.6 / 100
LiveBench MathPredicted
62.1 / 100
LiveBench OverallPredicted
50.3 / 100
LiveBench ReasoningPredicted
63.2 / 100
LongBench v2 EasyPredicted
56.9
LongBench v2 HardPredicted
47.8
LongBench v2Predicted
43.4 / 100
MathVistaPredicted
59.9 / 100
MT-BenchPredicted
8.3 / 10
NoLiMa (NIAH)Predicted
94.4 / 100
OCRBench v2Predicted
85.0 / 100
SWE-bench LitePredicted
68.6 / 100
tau-bench RetailPredicted
86.9 / 100
Vectara Factual ConsistencyPredicted
89.6 / 100
Vectara Hallucination RatePredicted
10.4 / 100
WebArenaPredicted
1.5 / 100
WildBenchPredicted
45.6
BFCL (Berkeley Function Calling)bfcl
63.0
browsecompbrowsecomp
49.7
BullshitBenchbullshitbench
26.0 / 100
Arena ELO: Creative Writingchatbot_arena
1383.0
Chatbot Arena ELOchatbot_arena
1410.0
arena_elo_searchchatbot_arena
1144.0
arena_elo_visionchatbot_arena
1214.0
Aider Polyglotepoch_ai
76.9
Ale Benchepoch_ai
933.5
Apex Agentsepoch_ai
2.8
Arc Agi 2epoch_ai
0.0
CAD-Evalepoch_ai
0.7
Chess Puzzlesepoch_ai
0.4
Cl Benchepoch_ai
0.2
Critptepoch_ai
0.0
DeepResearchBenchepoch_ai
0.5
Dtbenchepoch_ai
0.8
Eci Scoresepoch_ai
148.5
Enigma Evalepoch_ai
0.1
Epoch Capabilities Indexepoch_ai
146.9
FictionLiveBenchepoch_ai
0.9
Forecastbenchepoch_ai
64.2
Frontiermath Tiers 1 3 V2epoch_ai
0.2
Gdpvalepoch_ai
0.3
GeoBenchepoch_ai
0.7
GSOepoch_ai
0.1
Hleepoch_ai
0.2
Lech Mazur Writingepoch_ai
8.4
Lmcaepoch_ai
39.7
METR Time Horizonsepoch_ai
0.7
Mystery Game Puzzlesepoch_ai
0.2
OSWorld (Epoch Import, Version Unverified)epoch_ai
9.1 / 100
OTIS Mock AIME 2024–2025epoch_ai
0.8
Processed Data For Eciepoch_ai
0.2
SimpleQA Verified (Epoch)epoch_ai
0.5
VPCTepoch_ai
0.5
EQ-Bench 3eqbench
1288.0
FrontierMathfrontiermath
16.9 / 100
GAIA Level 1gaia
36.6
GAIA Level 2gaia
18.2
GAIA Level 3gaia
6.1
GAIAgaia
21.9 / 100
MedQA (USMLE)medqa
96.1
MLE-bench (Original)mlebench
31.6 / 100
MMMUmmmu
82.9 / 100
MMMU-Prommmu
76.4
MMTU Table Understandingmmtu
69.1 / 100
Open LLM Averageopen_llm_leaderboard
4.3 / 100
Open LLM: BBHopen_llm_leaderboard
29.4 / 100
Open LLM: GPQAopen_llm_leaderboard
25.8 / 100
Open LLM: IFEvalopen_llm_leaderboard
15.1 / 100
Open LLM: MATH Level 5open_llm_leaderboard
1.0 / 100
Open LLM: MMLU-PROopen_llm_leaderboard
11.6 / 100
Open LLM: MUSRopen_llm_leaderboard
36.2 / 100
RE-Benchrebench
72.3
EnigmaEvalseal
13.1
PropensityBenchseal
10.5
SimpleBenchsimplebench
53.1 / 100
simpleqasimpleqa
49.4
SWE-bench Verifiedswebench
58.4 / 100
WeirdMLweirdml
52.4 / 100
SWE-rebenchswe_rebench

vcontinuous-leaderboard · tasks-created:2025-05-01:2025-12-01 · o3-2025-04-16 · harness swe-rebench-react@tools · 5 attempts · pass@1 · swe-rebench:standardized:128k-or-model-limit · provider o3-2025-04-16__tools · record o3-2025-04-16__tools:1746057600000:1764547200000

View run source

Source metadata
{
  "passAt5": 49.83818770226537,
  "developer": "openai",
  "instanceType": "model",
  "contextPolicy": "128k unless the model supports less",
  "attemptsPerTask": 5,
  "totalTokenUsage": 0,
  "modelReleaseDate": "2025-04-16",
  "scoreStandardError": 0.4293365424408275,
  "cachedTokenPercentage": null,
  "modelReleaseTimestamp": 1744761600000,
  "standardContextTokens": 128000,
  "averageInstanceCostUsd": 0,
  "potentialContamination": false,
  "aggregateTaskRangeTimestamp": {
    "to": 1764547200000,
    "from": 1746057600000
  },
  "publishedTaskRangeTimestamp": {
    "to": 1764547200000,
    "from": 1746057600000
  }
}

Evidence observed 2025-12-01

38.1 / 100

How o3 Compares

Axes

Filters

$0.00 – $67.5
≥ 1.4

Providers

ai2ai21-labsalibabaanthropicarceeawsazurebaidu
openai
zai
meta
nvidia
xai
upstage
deepseek
alibaba
mistral
nous-research
aws
google
anthropic
xiaomi
stepfun
kimi
reflection
minimax
azure
inclusionai
reka-ai
cohere
bytedance_seed
inception
tencent
thinking-machines
Bubble size = Context Window
8191.00
2000000.00
Compare with other models