Models Agents EvalsIndustry

AgMoDB by @mistakeknot

Claude 3.5 Haiku | AgMoDB

Claude 3.5 Haiku

October 22, 2024Last synced Apr 7, 2026, 4:00 PM200K context

Blended Price

$1.60/M

Input Price

$0.80/M

Output Price

$4.00/M

Speed

0 tok/s

TTFT

0.00s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How Claude 3.5 Haiku Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

18.7

Coding Index

10.7

Math Index

—

MMLU Pro

0.6 / 100

GPQA Diamond

0.4 / 100

HLE

0.0 / 100

LiveCodeBench

0.3 / 100

SciCode

0.3 / 100

MATH-500

0.7 / 100

AIME

0.0 / 30

AIME 2025

— / 30

IFBench

0.4 / 100

LCR

0.2 / 100

Terminal-Bench Hard

0.0 / 100

τ²-Bench

0.2 / 100

aider

28.0 / 100

AA-Omniscience AccuracyPredicted

40.7 / 100

AA-Omniscience Hallucination RatePredicted

93.5 / 100

AIME 2025Predicted

0.2 / 30

AlpacaEval 2.0 LCPredicted

14.5 / 100

AlpacaEval 2.0 RawPredicted

28.3 / 100

ARC-AGI-1Predicted

5.0 / 100

ARC-AGI-1 Cost per TaskPredicted

0.0

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

38.2

AA Intelligence Index (Matrix)Predicted

72.3

AA Long Context Reasoning (Matrix)Predicted

75.6

AIME 2024Predicted

43.0

AIME 2025 (Matrix)Predicted

54.8

Arena-Hard AutoPredicted

62.9

BrowseCompPredicted

74.9

BRUMO 2025Predicted

81.7

CMIMC 2025Predicted

71.1

CritPtPredicted

0.1

GPQA Diamond (Matrix)Predicted

63.3

GSM8KPredicted

94.1

HLE (Matrix)Predicted

14.6

HMMT Feb 2025Predicted

57.1

HMMT Nov 2025Predicted

89.0

HumanEvalPredicted

82.3

IFBench (Matrix)Predicted

35.5

IFEvalPredicted

83.2

IMO 2025Predicted

12.6

LiveCodeBench (Matrix)Predicted

52.8

MATH-500 (Matrix)Predicted

93.8

MathArena Apex 2025Predicted

0.8

MMLUPredicted

83.8

MMLU-Pro (Matrix)Predicted

73.8

MMMU-ProPredicted

80.5

MRCR v2Predicted

75.8

OSWorldPredicted

41.1

SimpleQAPredicted

23.3

SMT 2025Predicted

78.7

SWE-bench ProPredicted

36.6

Tau-Bench Telecom (Matrix)Predicted

95.5

Terminal-Bench 2.0Predicted

19.8

Terminal-Bench 1.0Predicted

15.6

USAMO 2025Predicted

6.7

Video-MMUPredicted

86.2

browsecompPredicted

56.8

Apex AgentsPredicted

1.9

Arc Agi 2Predicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.9

Chess PuzzlesPredicted

0.2

CyBenchPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.4

GdpvalPredicted

0.2

GSM8K (Epoch)Predicted

8.0

GSOPredicted

0.0

HellaSwagPredicted

6.6

HlePredicted

0.1

METR Time HorizonsPredicted

0.5

PIQAPredicted

0.9

PosttrainbenchPredicted

0.0

The Agent CompanyPredicted

1.5

TriviaQAPredicted

3.3

VPCTPredicted

0.4

WinoGrandePredicted

0.8

FrontierMathPredicted

12.4 / 100

GAIA Level 1Predicted

56.4

GAIA Level 2Predicted

39.9

GAIA Level 3Predicted

18.9

GAIAPredicted

43.1 / 100

LegalBenchPredicted

83.6 / 100

LiveBench CodingPredicted

65.1 / 100

LiveBench Data AnalysisPredicted

38.0 / 100

LiveBench LanguagePredicted

55.5 / 100

LiveBench MathPredicted

59.2 / 100

LiveBench OverallPredicted

47.0 / 100

LiveBench ReasoningPredicted

42.0 / 100

LongBench v2 EasyPredicted

44.7

LongBench v2 HardPredicted

41.5

LongBench v2Predicted

37.6 / 100

MathVistaPredicted

55.3 / 100

MedQA (USMLE)Predicted

77.0

MLE-benchPredicted

12.4 / 100

MMMUPredicted

67.3 / 100

MMTU Table UnderstandingPredicted

54.1 / 100

MT-BenchPredicted

8.4 / 10

NoLiMa (NIAH)Predicted

92.5 / 100

OCRBench v2Predicted

85.3 / 100

Open LLM AveragePredicted

36.7 / 100

Open LLM: BBHPredicted

63.7 / 100

Open LLM: GPQAPredicted

33.9 / 100

Open LLM: IFEvalPredicted

70.0 / 100

Open LLM: MATH Level 5Predicted

34.2 / 100

Open LLM: MMLU-PROPredicted

48.7 / 100

Open LLM: MUSRPredicted

44.0 / 100

RE-BenchPredicted

1.1

SimpleBenchPredicted

24.9 / 100

simpleqaPredicted

22.5

SWE-bench VerifiedPredicted

39.9 / 100

tau-bench RetailPredicted

77.3 / 100

Vectara Factual ConsistencyPredicted

91.5 / 100

Vectara Hallucination RatePredicted

8.5 / 100

WebArenaPredicted

18.0 / 100

WildBenchPredicted

50.2

BigCodeBench Completebigcodebench

59.0 / 100

BigCodeBench Instructbigcodebench

46.1 / 100

BullshitBenchbullshitbench

50.0 / 100

Aider Polyglotepoch_ai

0.0

BALROGepoch_ai

0.0

CAD-Evalepoch_ai

4.3

Epoch Capabilities Indexepoch_ai

127.5

GeoBenchepoch_ai

0.0

Lech Mazur Writingepoch_ai

7.3

OTIS Mock AIME 2024–2025epoch_ai

0.0

SimpleQA Verified (Epoch)epoch_ai

0.1

SWE-bench Liteswebench

41.7 / 100

WeirdMLweirdml

30.7 / 100