AgMoDB
ModelsAgentsOrchestratorsEvalsCompositesVisualizeIndustry
AgMoDB by @mistakeknot
All Benchmarks

MedQA (USMLE)

Medical question answering benchmark derived from the US Medical Licensing Examination (USMLE). 1,273 multiple-choice questions covering clinical knowledge across 21 medical specialties. Scraped from Vals.ai.

medicalmedqa

Strengths

Professional-level medical knowledge; ground-truth answers from real licensing exams; widely cited in medical AI literature.

Caveats

Multiple-choice format limits measurement of reasoning; approaching saturation on frontier models (~95%+); US-centric curriculum; public dataset with contamination risk.

How to interpret scores

Contamination risk: highFreshness: static

Higher is better. Percentage of USMLE-style questions answered correctly.

Caveat to keep in mind

Multiple-choice format limits measurement of reasoning.

Leaderboard

Last synced: Nov 22, 2026, 12:18 PM
#ModelProviderScorePrice/M
1
GPT-5.2 Codex (Xhigh)
source benchpress
Evidence observed Apr 13, 2026
OpenAI97.5$4.81
2
Sao10K: Llama 3.1 70B Hanami x1
source benchpress
Evidence observed Apr 13, 2026
Sao10k97.4$3.00
3
Google: Gemini 2.5 Pro Preview 06-05
source benchpress
Evidence observed Apr 13, 2026
Google97.3$3.44
4
GLM-5-Turbo
source benchpress
Evidence observed Apr 13, 2026
Z AI96.8$0.00
5
Qwen3 Max Thinking
source benchpress
Evidence observed Apr 13, 2026
Alibaba96.8$0.00
6
Google: Gemma 2 9B
source benchpress
Evidence observed Apr 13, 2026
Google96.8$0.04
7
Gemini 3 Flash Preview (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Google96.8$1.13
8
Qwen3.5 122B A10B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba96.7$1.10
9
GPT-5.2 (Medium)
source benchpress
Evidence observed Apr 13, 2026
OpenAI96.7$4.81
10
MiMo-V2-Pro
source benchpress
Evidence observed Apr 13, 2026
Xiaomi96.6$0.00
11
OpenAI: GPT-5.4
source benchpress
Evidence observed Apr 13, 2026
OpenAI96.5$5.63
12
o1
source medqa
Evidence observed Apr 10, 2026
OpenAI96.5$26.25
13
Google: Gemma 2 27B
source benchpress
Evidence observed Apr 13, 2026
Google96.4$0.65
14
GPT-5.1 (High)
source medqa
Evidence observed Nov 22, 2026
OpenAI96.4$3.44
15
Gemini 3.1 Pro Preview
source medqa
Evidence observed Nov 22, 2026
Google96.4$4.50
16
Qwen3.5 397B A17B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba96.3$1.35
17
GPT-5 (High)
source medqa
Evidence observed Nov 22, 2026
OpenAI96.3$3.44
18
Gemma 4 31B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Google96.3$0.00
19
Google: Gemini 3.1 Pro Preview Custom Tools
source benchpress
Evidence observed Apr 13, 2026
Google96.2$4.50
20
MiniMax-M2.7
source benchpress
Evidence observed Apr 13, 2026
MiniMax96.1$0.53
21
GPT-5.4 (Xhigh)
source medqa
Evidence observed Nov 22, 2026
OpenAI96.1$5.63
22
Qwen3.5 35B A3B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba96.1$0.69
23
GPT-5 mini (High)
source medqa
Evidence observed Nov 22, 2026
OpenAI96.1$0.69
24
o3
source medqa
Evidence observed Apr 10, 2026
OpenAI96.1$3.50
25
Gemini 3 Pro Preview (High)
source medqa
Evidence observed Nov 22, 2026
Google96.0$4.50
26
o4-mini (High)
source medqa
Evidence observed Nov 22, 2026
OpenAI96.0$1.93
27
GPT-5.3 Codex (Xhigh)
source benchpress
Evidence observed Apr 13, 2026
OpenAI96.0$4.81
28
Qwen3.5 27B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba95.9$0.82
29
Qwen3.5 397B A17B (Non-reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba95.9$1.35
30
MiMo-V2-Flash (Feb 2026)
source benchpress
Evidence observed Apr 13, 2026
Xiaomi95.9$0.00
31
Claude Opus 4.5 (Reasoning)
source medqa
Evidence observed Nov 22, 2026
Anthropic95.9$10.00
32
Google: Gemini 3 Pro Preview
source benchpress
Evidence observed Apr 13, 2026
Google95.8$4.50
33
MiMo-V2-Flash (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Xiaomi95.8$0.15
34
Gemini 3 Flash Preview (Non-reasoning)
source medqa
Evidence observed Nov 22, 2026
Google95.8$1.13
35
KAT Coder Pro V2
source benchpress
Evidence observed Apr 13, 2026
KwaiKAT95.7$0.00
36
Gemini 3 Pro Preview (Low)
source benchpress
Evidence observed Apr 13, 2026
Google95.7$4.50
37
GPT-5.4 mini (Xhigh)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.7$1.69
38
GPT-5.2 (Non-reasoning)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.6$4.81
39
Qwen3.5 9B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Alibaba95.6$0.15
40
GPT-5.1 Codex (High)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.6$3.44
41
DeepSeek V3.2 Speciale
source benchpress
Evidence observed Apr 13, 2026
DeepSeek95.6$0.00
42
Step 3.5 Flash
source benchpress
Evidence observed Apr 13, 2026
StepFun95.5$0.15
43
GPT-5 Codex (High)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.4$3.44
44
Claude Opus 4.6 (Non-reasoning, High)
source medqa
Evidence observed Apr 17, 2026
Anthropic95.4$10.00
45
GPT-5 (Medium)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.4$3.44
46
GPT-5 (Low)
source benchpress
Evidence observed Apr 13, 2026
OpenAI95.4$3.44
47
Gemma 4 26B A4B (Reasoning)
source benchpress
Evidence observed Apr 13, 2026
Google95.3$0.16
48
Qwen Chat 14B
source benchpress
Evidence observed Apr 13, 2026
Alibaba95.2$0.00
49
MiMo-V2-Omni
source benchpress
Evidence observed Apr 13, 2026
Xiaomi95.1$0.00
50
Gemini 3.1 Flash-Lite
source benchpress
Evidence observed Apr 13, 2026
Google95.0$0.56