Models Agents EvalsIndustry

AgMoDB by @mistakeknot

o1 | AgMoDB

o1

December 5, 2024Last synced Apr 7, 2026, 4:00 PM200K context

Blended Price

$26.25/M

Input Price

$15.00/M

Output Price

$60.00/M

Speed

115 tok/s

TTFT

19.37s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How o1 Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

30.8

Coding Index

20.5

Math Index

—

MMLU Pro

0.8 / 100

GPQA Diamond

0.7 / 100

HLE

0.1 / 100

LiveCodeBench

0.7 / 100

SciCode

0.4 / 100

MATH-500

1.0 / 100

AIME

0.7 / 30

AIME 2025

— / 30

IFBench

0.7 / 100

LCR

0.6 / 100

Terminal-Bench Hard

0.1 / 100

τ²-Bench

0.6 / 100

aider

61.7 / 100

ARC-AGI-1arc_agi

14.0 / 100

ARC-AGI-1 Cost per Taskarc_agi

0.1

ARC-AGI-2arc_agi

0.8 / 100

ARC-AGI-2 Cost per Taskarc_agi

0.2

AA-Omniscience AccuracyPredicted

40.9 / 100

AA-Omniscience Hallucination RatePredicted

96.8 / 100

AIME 2025Predicted

0.8 / 30

AlpacaEval 2.0 LCPredicted

48.6 / 100

AlpacaEval 2.0 RawPredicted

46.8 / 100

BFCL (Berkeley Function Calling)Predicted

49.2

BigCodeBench CompletePredicted

56.4 / 100

BigCodeBench InstructPredicted

45.5 / 100

AA Intelligence Index (Matrix)Predicted

68.6

AA Long Context Reasoning (Matrix)Predicted

72.5

AIME 2024Predicted

88.2

AIME 2025 (Matrix)Predicted

86.4

Arena-Hard AutoPredicted

81.2

BrowseCompPredicted

67.3

BRUMO 2025Predicted

81.7

CMIMC 2025Predicted

88.7

CritPtPredicted

0.4

GPQA Diamond (Matrix)Predicted

74.2

GSM8KPredicted

94.7

HLE (Matrix)Predicted

66.0

HMMT Feb 2025Predicted

68.6

HMMT Nov 2025Predicted

93.7

HumanEvalPredicted

91.2

IFBench (Matrix)Predicted

64.9

IFEvalPredicted

89.4

IMO 2025Predicted

16.6

LiveCodeBench (Matrix)Predicted

75.4

MATH-500 (Matrix)Predicted

97.5

MathArena Apex 2025Predicted

16.1

MMLUPredicted

88.6

MMLU-Pro (Matrix)Predicted

82.9

MMMU-ProPredicted

80.3

MRCR v2Predicted

80.8

OSWorldPredicted

39.7

SimpleQAPredicted

37.4

SMT 2025Predicted

84.0

SWE-bench ProPredicted

35.2

Tau-Bench Telecom (Matrix)Predicted

96.8

Terminal-Bench 2.0Predicted

27.0

Terminal-Bench 1.0Predicted

25.7

USAMO 2025Predicted

14.6

Video-MMUPredicted

86.5

browsecompPredicted

61.0

BullshitBenchPredicted

66.2 / 100

Apex AgentsPredicted

1.8

Arc Agi 2Predicted

1.7

BALROGPredicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.7

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.3

DeepResearchBenchPredicted

0.4

GdpvalPredicted

0.1

GSM8K (Epoch)Predicted

1.6

GSOPredicted

37.3

HellaSwagPredicted

0.2

HlePredicted

0.2

PIQAPredicted

0.8

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.3

The Agent CompanyPredicted

2.3

TriviaQAPredicted

15.6

WinoGrandePredicted

0.8

FrontierMathPredicted

28.8 / 100

LegalBenchPredicted

85.1 / 100

LiveBench CodingPredicted

68.9 / 100

LiveBench Data AnalysisPredicted

45.7 / 100

LiveBench LanguagePredicted

54.3 / 100

LiveBench MathPredicted

60.5 / 100

LiveBench OverallPredicted

48.9 / 100

LiveBench ReasoningPredicted

42.6 / 100

MLE-benchPredicted

33.3 / 100

MMTU Table UnderstandingPredicted

61.4 / 100

MT-BenchPredicted

8.5 / 10

NoLiMa (NIAH)Predicted

94.2 / 100

OCRBench v2Predicted

84.5 / 100

SWE-bench LitePredicted

40.0 / 100

tau-bench RetailPredicted

72.4 / 100

Vectara Factual ConsistencyPredicted

91.4 / 100

Vectara Hallucination RatePredicted

8.6 / 100

WebArenaPredicted

49.2 / 100

WildBenchPredicted

47.7

Aider Polyglotepoch_ai

0.8

CAD-Evalepoch_ai

4.7

Epoch Capabilities Indexepoch_ai

142.9

FictionLiveBenchepoch_ai

0.7

GeoBenchepoch_ai

0.0

Lech Mazur Writingepoch_ai

7.0

METR Time Horizonsepoch_ai

0.6

OTIS Mock AIME 2024–2025epoch_ai

0.7

VPCTepoch_ai

0.4

GAIA Level 1gaia

84.9

GAIA Level 2gaia

66.7

GAIA Level 3gaia

32.7

GAIAgaia

66.8 / 100

LongBench v2 Easylongbench2

38.9

LongBench v2 Hardlongbench2

38.9

LongBench v2longbench2

37.8 / 100

MathVistamathvista

73.9 / 100

MedQA (USMLE)medqa

96.5

MMMUmmmu

78.2 / 100

Open LLM Averageopen_llm_leaderboard

21.9 / 100

Open LLM: BBHopen_llm_leaderboard

52.5 / 100

Open LLM: GPQAopen_llm_leaderboard

31.5 / 100

Open LLM: IFEvalopen_llm_leaderboard

40.6 / 100

Open LLM: MATH Level 5open_llm_leaderboard

7.5 / 100

Open LLM: MMLU-PROopen_llm_leaderboard

31.7 / 100

Open LLM: MUSRopen_llm_leaderboard

46.9 / 100

RE-Benchrebench

60.2

SEAL Tool Use (Enterprise)seal

70.1

SimpleBenchsimplebench

36.7 / 100

simpleqasimpleqa

42.6

SWE-bench Verifiedswebench

58.8 / 100

WeirdMLweirdml

43.8 / 100