Llama 3.1 Instruct 405B

July 23, 2024Last synced Apr 7, 2026, 4:00 PM

Blended Price

$3.69/M

Input Price

$2.75/M

Output Price

$6.50/M

Speed

31 tok/s

TTFT

0.47s

Benchmark Scores

Intelligence Index

External Benchmarks

AA-Omniscience Accuracy

How Llama 3.1 Instruct 405B Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

GPQA Diamond

0.5 / 100

LiveCodeBench

0.3 / 100

Terminal-Bench Hard

0.1 / 100

τ²-Bench

0.2 / 100

AA-Omniscience Hallucination RatePredicted

95.3 / 100

Aider PolyglotPredicted

31.4 / 100

AlpacaEval 2.0 LCPredicted

34.1 / 100

AlpacaEval 2.0 RawPredicted

26.6 / 100

ARC-AGI-1Predicted

8.3 / 100

ARC-AGI-1 Cost per TaskPredicted

0.0

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

37.7

BigCodeBench CompletePredicted

52.4 / 100

BigCodeBench InstructPredicted

42.6 / 100

AA Intelligence Index (Matrix)Predicted

65.0

AA Long Context Reasoning (Matrix)Predicted

71.3

AIME 2024Predicted

49.0

AIME 2025 (Matrix)Predicted

63.7

Arena-Hard AutoPredicted

60.4

BrowseCompPredicted

62.9

BRUMO 2025Predicted

87.2

CMIMC 2025Predicted

74.9

GPQA Diamond (Matrix)Predicted

63.9

HLE (Matrix)Predicted

15.0

HMMT Feb 2025Predicted

52.3

HMMT Nov 2025Predicted

89.6

HumanEvalPredicted

86.4

IFBench (Matrix)Predicted

36.9

IFEvalPredicted

83.1

IMO 2025Predicted

14.8

LiveCodeBench (Matrix)Predicted

53.2

MATH-500 (Matrix)Predicted

93.8

MathArena Apex 2025Predicted

1.1

MMLU-Pro (Matrix)Predicted

76.1

MMMU-ProPredicted

80.2

MRCR v2Predicted

75.6

OSWorldPredicted

35.1

SimpleQAPredicted

25.9

SMT 2025Predicted

81.0

SWE-bench ProPredicted

29.9

Tau-Bench Telecom (Matrix)Predicted

96.1

Terminal-Bench 2.0Predicted

23.9

Terminal-Bench 1.0Predicted

17.2

USAMO 2025Predicted

7.4

Video-MMUPredicted

86.0

browsecompPredicted

60.6

BullshitBenchPredicted

16.1 / 100

Aider PolyglotPredicted

0.0

Apex AgentsPredicted

2.3

Arc Agi 2Predicted

0.0

BIG-Bench HardPredicted

3.0

CAD-EvalPredicted

5.7

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.4

GeoBenchPredicted

0.0

GSM8K (Epoch)Predicted

6.3

HellaSwagPredicted

3.4

Lech Mazur WritingPredicted

7.3

METR Time HorizonsPredicted

0.7

OTIS Mock AIME 2024–2025Predicted

0.1

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.3

The Agent CompanyPredicted

1.4

TriviaQAPredicted

3.9

WinoGrandePredicted

0.8

FrontierMathPredicted

14.2 / 100

GAIA Level 1Predicted

39.6

GAIA Level 2Predicted

17.2

GAIA Level 3Predicted

0.6

GAIAPredicted

25.8 / 100

LegalBenchPredicted

80.6 / 100

LiveBench CodingPredicted

66.5 / 100

LiveBench Data AnalysisPredicted

44.1 / 100

LiveBench LanguagePredicted

54.3 / 100

LiveBench MathPredicted

62.6 / 100

LiveBench OverallPredicted

47.1 / 100

LiveBench ReasoningPredicted

41.8 / 100

LongBench v2 EasyPredicted

44.0

LongBench v2 HardPredicted

40.4

LongBench v2Predicted

36.6 / 100

MathVistaPredicted

60.9 / 100

MLE-benchPredicted

22.3 / 100

MMMUPredicted

67.6 / 100

MMTU Table UnderstandingPredicted

52.2 / 100

MT-BenchPredicted

7.9 / 10

NoLiMa (NIAH)Predicted

92.1 / 100

OCRBench v2Predicted

79.1 / 100

Open LLM AveragePredicted

34.1 / 100

Open LLM: BBHPredicted

61.5 / 100

Open LLM: GPQAPredicted

33.2 / 100

Open LLM: IFEvalPredicted

66.4 / 100

Open LLM: MATH Level 5Predicted

29.5 / 100

Open LLM: MMLU-PROPredicted

46.0 / 100

Open LLM: MUSRPredicted

43.3 / 100

RE-BenchPredicted

4.0

SimpleBenchPredicted

27.5 / 100

simpleqaPredicted

24.5

SWE-bench LitePredicted

18.4 / 100

tau-bench RetailPredicted

78.8 / 100

Vectara Factual ConsistencyPredicted

92.8 / 100

Vectara Hallucination RatePredicted

7.2 / 100

WebArenaPredicted

3.4 / 100

WeirdMLPredicted

27.3 / 100

WildBenchPredicted

48.1

Arena ELO: Codingchatbot_arena

271.0

Chatbot Arena ELOchatbot_arena

136.0

SWE-bench Verifiedswebench

40.6 / 100