Benchmarks / Knowledge & Science

MMLU-Pro

Harder, reasoning-focused successor to MMLU.

maintainer unknown · 260 observations on record

Versions

1 version

Version	Tasks	Window	Released	Notes
MMLU-Pro current mmlu-pro	—	—	—

Current leaderboard

MMLU-Pro · Accuracy · top 25 of 259 systems

#	System	Accuracy %	Verification	Source
1	Claude Fable 5max effort Anthropic	91.5%	independent	Vals AI
2	Gemini 3.1 Pro Previewhigh effort Google DeepMind	91.0%	independent	Vals AI
3	Gemini 3 Pro Previewhigh effort Google DeepMind	90.1%	independent	Vals AI
4	Gemini 3 Pro Preview Google DeepMind	90.1%	independent	Vals AI
5	Claude Opus 4.7max effort Anthropic	89.9%	independent	Vals AI
6	Gemini 3 Pro Previewhigh effort Google DeepMind	89.8%	independent	Artificial Analysis
7	Claude Opus 4.8max effort Anthropic	89.6%	independent	Vals AI
8	Gemini 3.5 Flashhigh effort Google DeepMind	89.5%	independent	Vals AI
9	Claude Opus 4.5thinking Anthropic	89.5%	independent	Artificial Analysis
10	Gemini 3 Pro Previewlow effort Google DeepMind	89.5%	independent	Artificial Analysis
11	Qwen3 7 Max Alibaba (Qwen)	89.3%	independent	Vals AI
12	Grok 4.5high effort xAI	89.2%	independent	Vals AI
13	Claude Opus 4.6max effort Anthropic	89.1%	independent	Vals AI
14	GPT 5.6 Solmax effort OpenAI	89.1%	independent	Vals AI
15	Gemini 3 Flash Previewthinking Google DeepMind	89.0%	independent	Artificial Analysis
16	Claude Opus 4.5no reasoning Anthropic	88.9%	independent	Artificial Analysis
17	Muse Spark 1.1xhigh effort Meta AI	88.7%	independent	Vals AI
18	Gemini 3 Flash Previewhigh effort Google DeepMind	88.6%	independent	Vals AI
19	Gemini 3 Flash Previewno reasoning Google DeepMind	88.2%	independent	Artificial Analysis
20	GPT 5.5 (2026-04-22)xhigh effort OpenAI	88.1%	independent	Vals AI
21	Claude Opus 4.1 20250805thinking Anthropic	87.9%	independent	Vals AI
22	Qwen3 6 Plus Alibaba	87.7%	independent	Vals AI
23	Kimi K2 6 Moonshot AI	87.6%	independent	Vals AI
24	Kimi K2 6 Moonshot AI	87.6%	independent	Vals AI
25	Claude Sonnet 5max effort Anthropic	87.5%	independent	Vals AI

One row per evaluated system — reasoning-effort variants rank separately.

Source agreement

where evaluating organizations agree — and don't

MMLU-ProArtificial Analysis vs Vals AI

overlap 27 systemsrank correlation 0.96mean |Δ| 2.17max |Δ| 14.67

most contested: Magistral Small 2509 (+14.67) · Magistral Medium 2509 (+12.84) · GPT OSS 20B (+3.16)

Possible reasons: run setting 'implementation' differs (artificial-analysis vs vals-ai).