Benchmarks / Knowledge & Science

MMLU-Pro

Harder, reasoning-focused successor to MMLU.

maintainer unknown · 245 observations on record

Versions

1 version

Version	Tasks	Window	Released	Notes
MMLU-Pro current mmlu-pro	—	—	—

Current leaderboard

MMLU-Pro · Accuracy · top 25 of 245 systems

#	System	Accuracy %	Verification	Source
1	Claude Fable 5max effort Anthropic	91.5%	independent	Vals AI
2	Gemini 3.1 Pro Previewhigh effort Google DeepMind	91.0%	independent	Vals AI
3	Gemini 3 Pro Previewhigh effort Google DeepMind	90.1%	independent	Vals AI
4	Claude Opus 4.7max effort Anthropic	89.9%	independent	Vals AI
5	Gemini 3 Pro Previewhigh effort Google DeepMind	89.8%	independent	Artificial Analysis
6	Claude Opus 4.8max effort Anthropic	89.6%	independent	Vals AI
7	Gemini 3.5 Flashhigh effort Google DeepMind	89.5%	independent	Vals AI
8	Gemini 3 Pro Previewlow effort Google DeepMind	89.5%	independent	Artificial Analysis
9	Claude Opus 4.5thinking Anthropic	89.5%	independent	Artificial Analysis
10	Qwen3 7 Max Alibaba (Qwen)	89.3%	independent	Vals AI
11	Grok 4.5high effort xAI	89.2%	independent	Vals AI
12	Claude Opus 4.6max effort Anthropic	89.1%	independent	Vals AI
13	GPT 5.6 Solmax effort OpenAI	89.1%	independent	Vals AI
14	Gemini 3 Flash Previewthinking Google DeepMind	89.0%	independent	Artificial Analysis
15	Claude Opus 4.5no reasoning Anthropic	88.9%	independent	Artificial Analysis
16	Muse Spark 1.1xhigh effort Meta AI	88.7%	independent	Vals AI
17	Gemini 3 Flash Previewhigh effort Google DeepMind	88.6%	independent	Vals AI
18	Gemini 3 Flash Previewno reasoning Google DeepMind	88.2%	independent	Artificial Analysis
19	GPT 5.5 (2026-04-22)xhigh effort OpenAI	88.1%	independent	Vals AI
20	Claude Opus 4.1 20250805thinking Anthropic	87.9%	independent	Vals AI
21	Qwen3 6 Plus Alibaba	87.7%	independent	Vals AI
22	Kimi K2 6 Moonshot AI	87.6%	independent	Vals AI
23	Claude Sonnet 5max effort Anthropic	87.5%	independent	Vals AI
24	Minimax M2 1 MiniMax	87.5%	independent	Artificial Analysis
25	Claude 4.5 Sonnetthinking Anthropic	87.5%	independent	Artificial Analysis

One row per evaluated system — reasoning-effort variants rank separately.

Source agreement

where evaluating organizations agree — and don't

MMLU-ProArtificial Analysis vs Vals AI

overlap 27 systemsrank correlation 0.96mean |Δ| 2.17max |Δ| 14.67

most contested: Magistral Small 2509 (+14.67) · Magistral Medium 2509 (+12.84) · GPT OSS 20B (+3.16)

Possible reasons: run setting 'implementation' differs (artificial-analysis vs vals-ai).