BenchAtlas

Rankings / Anthropic

Claude 3.7 Sonnet 20250219

released 2025-02-24

BenchAtlas Index

as of 2026-09-11
34.5
base configuration
rank #348
6 families · 4 categories · medium
34.4
64K budget
rank #349
5 families · 3 categories · medium
34.3
16K budget
rank #350
5 families · 3 categories · medium

Benchmark evidence

70 results
external indices
ECI64K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI32K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI15K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI16K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI141.2 pointsindependentEpoch AI Benchmarking Hub
ECI8K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI2K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI12K budget141.2 pointsindependentEpoch AI Benchmarking Hub
ECI1K budget141.2 pointsindependentEpoch AI Benchmarking Hub
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
1430.8
1423.3–1438.2
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1411.4
1403.9–1418.9
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1406.7
1400.7–1412.8
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1405.5
1398.4–1412.7
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1398.4
1392.8–1404.0
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1398.3
1390.6–1406.0
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1395.3
1382.9–1407.8
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1390.0
1376.9–1403.1
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1387.8
1361.6–1414.1
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1385.2
1377.6–1392.8
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1384.8
1378.9–1390.7
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1384.7
1379.7–1389.7
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1381.0
1369.1–1393.0
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1379.0
1365.4–1392.5
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1378.3
1371.7–1384.8
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1376.9
1368.8–1385.0
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1373.4
1363.4–1383.3
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1371.9
1367.9–1375.9
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1371.3
1357.2–1385.4
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1371.2
1363.9–1378.5
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1367.4
1359.1–1375.8
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1364.8
1345.9–1383.7
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1364.3
1339.8–1388.8
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1364.2
1353.8–1374.5
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1363.0
1353.0–1372.9
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1356.3
1351.3–1361.3
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1346.7
1341.4–1352.0
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1333.4
1314.1–1352.7
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1328.2
1304.8–1351.6
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond64K budget78.5 %independentEpoch AI Benchmarking Hub
2025-05-26
GPQA Diamond32K budget76.8 %independentEpoch AI Benchmarking Hub
2025-03-10
GPQA Diamond16K budget76.8 %independentEpoch AI Benchmarking Hub
2025-02-26
GPQA Diamond
implementation=vals-ai
thinking75.3 %independentVals AI
GPQA Diamond
implementation=vals-ai
67.4 %independentVals AI
GPQA Diamond66.0 %independentEpoch AI Benchmarking Hub
2025-02-24
MMLU-Pro
implementation=vals-ai
thinking82.7 %independentVals AI
MMLU-Pro
implementation=vals-ai
80.7 %independentVals AI
multimodal
MMMU
implementation=vals-ai
thinking75.1 %independentVals AI
MMMU
implementation=vals-ai
71.5 %independentVals AI
professional
CorpFinthinking60.4 %independentVals AI
LegalBenchthinking82.5 %independentVals AI
LegalBench80.0 %independentVals AI
MedQAthinking90.2 %independentVals AI
TaxEvalthinking74.0 %independentVals AI
TaxEval72.4 %independentVals AI
reasoning math
AIME
implementation=vals-ai
thinking44.6 %independentVals AI
AIME
implementation=vals-ai
22.3 %independentVals AI
FrontierMath16K budget4.1 %independentEpoch AI Benchmarking Hub
2025-03-06
FrontierMath32K budget3.5 %independentEpoch AI Benchmarking Hub
2025-03-13
FrontierMath64K budget3.1 %independentEpoch AI Benchmarking Hub
2025-03-13
FrontierMath3.1 %independentEpoch AI Benchmarking Hub
2025-03-06
MATH-500
implementation=vals-ai
thinking91.6 %independentVals AI
MATH-500
implementation=vals-ai
76.8 %independentVals AI
MATH Level 564K budget91.2 %independentEpoch AI Benchmarking Hub
2025-03-13
MATH Level 532K budget90.0 %independentEpoch AI Benchmarking Hub
2025-03-12
MATH Level 516K budget86.3 %independentEpoch AI Benchmarking Hub
2025-02-26
MATH Level 568.2 %independentEpoch AI Benchmarking Hub
2025-02-24
OTIS Mock AIME 2024–202564K budget57.8 %independentEpoch AI Benchmarking Hub
2025-03-13
OTIS Mock AIME 2024–202532K budget53.3 %independentEpoch AI Benchmarking Hub
2025-03-12
OTIS Mock AIME 2024–202516K budget46.7 %independentEpoch AI Benchmarking Hub
2025-02-26
OTIS Mock AIME 2024–202521.9 %independentEpoch AI Benchmarking Hub
2025-02-25

Agent + model results

systems, not bare-model scores
agent + model Aime-coder v1 + Claude 3.7 Sonnet 20250219SWE-bench Verified66.4 %unverifiedSWE-bench Leaderboard
agent + model SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench Verified62.4 %communitySWE-bench Leaderboard
agent + model Epoch Inspect harness + Claude 3.7 Sonnet 20250219SWE-bench Verified61.0 %independentEpoch AI Benchmarking Hub
agent + model mini-SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench Verified52.8 %communitySWE-bench Leaderboard
agent + model mini-SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench bash-only52.8 %communitySWE-bench Leaderboard
agent + model SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench Lite48.0 %communitySWE-bench Leaderboard
agent + model SWE-agent 1.0 + Claude 3.7 Sonnet 20250219SWE-bench (full test split)33.8 %communitySWE-bench Leaderboard
agent + model OpenHands-Versa + Claude 3.7 Sonnet 20250219SWE-bench Multimodal31.3 %communitySWE-bench Leaderboard
agent + model Zencoder (2025-04-01) + Claude 3.7 Sonnet 20250219SWE-bench Multimodal30.6 %communitySWE-bench Leaderboard
agent + model mini-SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench bash-only0.4 usd_per_taskcommunitySWE-bench Leaderboard
agent + model mini-SWE-agent + Claude 3.7 Sonnet 20250219SWE-bench Verified0.4 usd_per_taskcommunitySWE-bench Leaderboard

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.

Related variants

Claude 3.7 Sonnet