BenchAtlas

Rankings / Anthropic

Claude 3.5 Sonnet 20241022

released 2024-10-22

BenchAtlas Index

as of 2026-07-14
37.0
base configuration
rank #308
6 families · 3 categories · medium
30.2
base configuration
rank #356
6 families · 3 categories · medium

Benchmark evidence

45 results
coding
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
37.2 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
36.4 %independentLiveCodeBench Leaderboard
external indices
ECI134.3 pointsindependentEpoch AI Benchmarking Hub
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
1434.8
1429.3–1440.3
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1407.4
1402.8–1412.0
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1405.1
1399.8–1410.4
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1398.0
1393.7–1402.4
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1394.1
1388.4–1399.8
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1389.3
1367.9–1410.7
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1388.0
1382.3–1393.8
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1383.7
1369.7–1397.7
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1383.2
1373.3–1393.1
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1382.7
1378.8–1386.6
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1382.4
1373.5–1391.4
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1379.0
1365.5–1392.5
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1376.7
1370.9–1382.4
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1375.8
1370.9–1380.7
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1373.0
1366.1–1380.0
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1373.0
1369.8–1376.1
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1372.3
1367.9–1376.7
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1369.5
1363.3–1375.7
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1368.9
1359.6–1378.3
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1365.8
1357.4–1374.2
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1364.4
1358.8–1370.0
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1363.7
1343.7–1383.6
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1358.9
1352.0–1365.8
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1356.6
1352.7–1360.5
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1354.1
1347.7–1360.4
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1351.0
1344.4–1357.7
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1344.8
1340.4–1349.1
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1308.4
1294.2–1322.6
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1304.0
1285.4–1322.5
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond
implementation=vals-ai
59.3 %independentVals AI
GPQA Diamond55.3 %independentEpoch AI Benchmarking Hub
2025-01-27
MMLU-Pro
implementation=vals-ai
78.4 %independentVals AI
multimodal
MMMU
implementation=vals-ai
68.8 %independentVals AI
professional
CorpFin53.6 %independentVals AI
MedQA83.2 %independentVals AI
TaxEval70.2 %independentVals AI
reasoning math
AIME
implementation=vals-ai
10.0 %independentVals AI
FrontierMath2.1 %independentEpoch AI Benchmarking Hub
2025-03-06
FrontierMath Tier 4older version0.0 %independentEpoch AI Benchmarking Hub
2025-07-01
MATH-500
implementation=vals-ai
72.4 %independentVals AI
MATH Level 557.0 %independentEpoch AI Benchmarking Hub
2025-01-27
OTIS Mock AIME 2024–20258.5 %independentEpoch AI Benchmarking Hub
2025-02-25

Agent + model results

systems, not bare-model scores
agent + model AgentScope + Claude 3.5 Sonnet 20241022SWE-bench Verified63.4 %unverifiedSWE-bench Leaderboard
agent + model EPAM AI/Run Developer Agent v20250219 + Claude 3.5 Sonnet 20241022SWE-bench Verified62.8 %unverifiedSWE-bench Leaderboard
agent + model CodeStory Midwit Agent + Claude 3.5 Sonnet 20241022SWE-bench Verified62.2 %unverifiedSWE-bench Leaderboard
agent + model Learn-by-interact + Claude 3.5 Sonnet 20241022SWE-bench Verified60.2 %unverifiedSWE-bench Leaderboard
agent + model EPAM AI/Run Developer Agent v20241212 + Claude 3.5 Sonnet 20241022SWE-bench Verified55.4 %unverifiedSWE-bench Leaderboard
agent + model devlo + Claude 3.5 Sonnet 20241022SWE-bench Verified54.2 %unverifiedSWE-bench Leaderboard
agent + model Engine Labs (2024-11-25) + Claude 3.5 Sonnet 20241022SWE-bench Verified51.8 %unverifiedSWE-bench Leaderboard
agent + model AutoCodeRover-v2.1 + Claude 3.5 Sonnet 20241022SWE-bench Verified51.6 %unverifiedSWE-bench Leaderboard
agent + model Isea + Claude 3.5 Sonnet 20241022SWE-bench Lite51.3 %unverifiedSWE-bench Leaderboard
agent + model Agentless-1.5 + Claude 3.5 Sonnet 20241022SWE-bench Verified50.8 %unverifiedSWE-bench Leaderboard
agent + model devlo + Claude 3.5 Sonnet 20241022SWE-bench Lite47.3 %unverifiedSWE-bench Leaderboard
agent + model AutoCodeRover-v2.0 + Claude 3.5 Sonnet 20241022SWE-bench Verified46.2 %unverifiedSWE-bench Leaderboard
agent + model KGCompass + Claude 3.5 Sonnet 20241022SWE-bench Lite46.0 %unverifiedSWE-bench Leaderboard
agent + model Kodu-v1 + Claude 3.5 Sonnet 20241022SWE-bench Lite44.7 %unverifiedSWE-bench Leaderboard
agent + model Lingxi + Claude 3.5 Sonnet 20241022SWE-bench Lite42.7 %unverifiedSWE-bench Leaderboard
agent + model OpenHands + Claude 3.5 Sonnet 20241022SWE-bench Lite41.7 %communitySWE-bench Leaderboard
agent + model PatchKitty-0.9 + Claude 3.5 Sonnet 20241022SWE-bench Lite41.3 %unverifiedSWE-bench Leaderboard
agent + model OrcaLoca + Claude 3.5 Sonnet 20241022SWE-bench Lite41.0 %unverifiedSWE-bench Leaderboard
agent + model Agentless-1.5 + Claude 3.5 Sonnet 20241022SWE-bench Lite40.7 %unverifiedSWE-bench Leaderboard
agent + model EPAM AI/Run Developer Agent v20241029 + Claude 3.5 Sonnet 20241022SWE-bench Verified39.6 %unverifiedSWE-bench Leaderboard
agent + model Moatless Tools + Claude 3.5 Sonnet 20241022SWE-bench Lite39.0 %communitySWE-bench Leaderboard
agent + model Patched.Codes Patchwork + Claude 3.5 Sonnet 20241022SWE-bench Lite37.0 %unverifiedSWE-bench Leaderboard
agent + model SWE-agent + Claude 3.5 Sonnet 20241022SWE-bench Verified33.6 %communitySWE-bench Leaderboard
agent + model Kortix AI + Claude 3.5 Sonnet 20241022SWE-bench Lite30.0 %communitySWE-bench Leaderboard
agent + model OpenHands + Claude 3.5 Sonnet 20241022SWE-bench Lite26.7 %communitySWE-bench Leaderboard
agent + model Moatless Tools + Claude 3.5 Sonnet 20241022SWE-bench Lite26.7 %communitySWE-bench Leaderboard
agent + model Agentless Lite + Claude 3.5 Sonnet 20241022SWE-bench Multimodal25.3 %communitySWE-bench Leaderboard
agent + model AutoCodeRover-v2.0 + Claude 3.5 Sonnet 20241022SWE-bench (full test split)24.9 %unverifiedSWE-bench Leaderboard
agent + model SWE-agent + Claude 3.5 Sonnet 20241022SWE-bench Lite23.0 %communitySWE-bench Leaderboard
agent + model SWE-agent + Claude 3.5 Sonnet 20241022SWE-bench (full test split)18.1 %communitySWE-bench Leaderboard
agent + model SWE-agent Multimodal + Claude 3.5 Sonnet 20241022SWE-bench Multimodal11.4 %communitySWE-bench Leaderboard
agent + model Agentless + Claude 3.5 Sonnet 20241022SWE-bench Multimodal6.2 %communitySWE-bench Leaderboard
agent + model RAG + Claude 3.5 Sonnet 20241022SWE-bench Multimodal5.0 %communitySWE-bench Leaderboard

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.

Related variants

Claude 3.5 Sonnet 20240620