BenchAtlas

Rankings / Anthropic

Claude Sonnet 4 20250514

released 2025-05-14

BenchAtlas Index

as of 2026-09-11
56.1
thinking
rank #192
8 families · 4 categories · medium
42.8
base configuration
rank #289
8 families · 4 categories · medium

Benchmark evidence

82 results
external indices
ECI141.7 pointsindependentEpoch AI Benchmarking Hub
ECI2K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI59K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI32K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI16K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI8K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI1K budget141.7 pointsindependentEpoch AI Benchmarking Hub
ECI12K budget141.7 pointsindependentEpoch AI Benchmarking Hub
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
1449.9
1442.5–1457.3
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1428.6
1422.5–1434.7
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1427.6
1420.3–1434.9
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1427.5
1402.3–1452.6
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1422.5
1415.3–1429.8
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1420.0
1407.0–1433.0
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1418.7
1413.1–1424.2
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1409.7
1397.6–1421.8
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1409.7
1401.6–1417.7
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1409.5
1401.5–1417.4
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1405.3
1393.8–1416.8
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1401.0
1387.7–1414.3
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1399.6
1389.1–1410.1
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1398.7
1378.1–1419.2
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1395.6
1390.0–1401.2
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1395.5
1388.9–1402.1
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1393.0
1380.5–1405.4
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1391.3
1372.5–1410.1
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1390.4
1386.0–1394.8
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1389.8
1378.1–1401.6
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1389.7
1382.6–1396.8
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1388.2
1376.2–1400.2
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1387.0
1378.1–1395.8
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1386.8
1378.8–1394.7
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1380.7
1375.4–1385.9
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1374.7
1367.0–1382.5
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1369.6
1363.8–1375.4
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1357.8
1337.4–1378.2
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1355.1
1335.1–1375.1
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond32K budget78.3 %independentEpoch AI Benchmarking Hub
2025-05-22
GPQA Diamond59K budget77.8 %independentEpoch AI Benchmarking Hub
2025-05-26
GPQA Diamond16K budget75.8 %independentEpoch AI Benchmarking Hub
2025-05-22
GPQA Diamond
implementation=vals-ai
thinking75.0 %independentVals AI
GPQA Diamond
implementation=vals-ai
69.4 %independentVals AI
GPQA Diamond66.7 %independentEpoch AI Benchmarking Hub
2025-05-22
MMLU-Pro
implementation=vals-ai
thinking83.9 %independentVals AI
MMLU-Pro
implementation=vals-ai
79.4 %independentVals AI
multimodal
MMMU
implementation=vals-ai
thinking74.9 %independentVals AI
MMMU
implementation=vals-ai
72.4 %independentVals AI
professional
CorpFinthinking61.2 %independentVals AI
CorpFin54.7 %independentVals AI
LegalBench83.0 %independentVals AI
LegalBenchthinking82.1 %independentVals AI
MedQAthinking92.7 %independentVals AI
MedQA90.3 %independentVals AI
TaxEvalthinking72.0 %independentVals AI
TaxEval69.6 %independentVals AI
reasoning math
AIME
implementation=vals-ai
thinking76.3 %independentVals AI
AIME
implementation=vals-ai
38.5 %independentVals AI
ARC-AGI-1older version
split=public_eval · model_type=Base LLM
33.0 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
thinking · 1K budget31.3 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
thinking · 1K budget28.0 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=Base LLM
23.8 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=Base LLM
2.1 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=Base LLM
1.3 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
thinking · 1K budget1.3 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
thinking · 1K budget0.8 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
thinking · 1K budget0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
thinking · 1K budget0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=Base LLM
0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=Base LLM
0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
thinking · 1K budget0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
thinking · 1K budget0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=Base LLM
0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=Base LLM
0.1 usd_per_taskindependentARC Prize Leaderboard
FrontierMath4.1 %independentEpoch AI Benchmarking Hub
2025-07-04
FrontierMath Tier 4older version0.0 %independentEpoch AI Benchmarking Hub
2025-07-01
MATH-500
implementation=vals-ai
thinking93.8 %independentVals AI
MATH-500
implementation=vals-ai
90.3 %independentVals AI
MATH Level 584.4 %independentEpoch AI Benchmarking Hub
2025-05-22
OTIS Mock AIME 2024–202532K budget71.1 %independentEpoch AI Benchmarking Hub
2025-05-22
OTIS Mock AIME 2024–202559K budget68.9 %independentEpoch AI Benchmarking Hub
2025-05-23
OTIS Mock AIME 2024–202516K budget53.3 %independentEpoch AI Benchmarking Hub
2025-05-22
OTIS Mock AIME 2024–202528.9 %independentEpoch AI Benchmarking Hub
2025-05-22

Agent + model results

systems, not bare-model scores
agent + model EPAM AI/Run Developer Agent v20250719 + Claude Sonnet 4 20250514SWE-bench Verified76.8 %unverifiedSWE-bench Leaderboard
agent + model Harness AI + Claude Sonnet 4 20250514SWE-bench Verified74.8 %unverifiedSWE-bench Leaderboard
agent + model Artemis Agent v2 (2025-09-24) + Claude Sonnet 4 20250514SWE-bench Verified57.0 %unverifiedSWE-bench Leaderboard

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.

Related variants

Claude Sonnet 4.5 20250929Claude Sonnet 4.6Claude Sonnet 4.5Claude Sonnet 4Claude Sonnet 5