BenchAtlas

Rankings / OpenAI

O3 (2025-04-16)

released 2025-04-16

BenchAtlas Index

as of 2026-09-11
60.4
high effort
rank #154
9 families · 4 categories · medium
60.1
high effort
rank #156
9 families · 5 categories · high
59.3
base configuration
rank #166
11 families · 5 categories · high

Benchmark evidence

93 results
agentic coding
τ²-Bench80.7 %independentArtificial Analysis
coding
LiveCodeBench v6
implementation=artificial-analysis
80.8 %independentArtificial Analysis
LiveCodeBench v6
problems=454 · window_start=2024-08-01
high effort75.8 %independentLiveCodeBench Leaderboard
SciCode41.0 %independentArtificial Analysis
external indices
Intelligence Index v4.120.2 pointsindependentArtificial Analysis
AA Math Index88.3 pointsindependentArtificial Analysis
ECImedium effort146.9 pointsindependentEpoch AI Benchmarking Hub
ECI146.9 pointsindependentEpoch AI Benchmarking Hub
ECIlow effort146.9 pointsindependentEpoch AI Benchmarking Hub
ECIhigh effort146.9 pointsindependentEpoch AI Benchmarking Hub
factuality
SimpleQA Verifiedhigh effort53.0 %independentEpoch AI Benchmarking Hub
2025-12-09
SimpleQA Verifiedhigh effort49.4 %independentEpoch AI Benchmarking Hub
2026-08-27
human preference
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1474.9
1463.7–1486.0
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1463.3
1453.2–1473.4
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1462.0
1439.8–1484.2
communityLMArena Leaderboard Dataset
Coding (style control)older version
arena=text · category=coding · style_control=true
1460.1
1453.8–1466.3
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1457.3
1446.2–1468.5
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1454.7
1449.5–1460.0
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1454.4
1447.6–1461.2
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1451.6
1441.3–1461.8
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1449.8
1443.6–1455.9
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1448.7
1437.3–1460.2
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1447.6
1437.6–1457.6
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1447.3
1437.0–1457.7
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1440.6
1435.8–1445.4
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1439.8
1423.8–1455.9
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1439.2
1434.5–1443.9
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1431.4
1427.7–1435.0
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1428.6
1419.8–1437.4
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1426.1
1419.2–1433.0
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1424.6
1419.5–1429.6
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1424.3
1406.8–1441.8
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1422.2
1417.6–1426.7
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1419.9
1413.1–1426.7
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1415.7
1397.4–1433.9
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1410.1
1403.9–1416.3
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1402.1
1396.6–1407.7
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1395.7
1389.7–1401.6
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1394.3
1387.8–1400.8
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1394.1
1375.6–1412.6
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1382.3
1375.0–1389.6
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond
implementation=vals-ai
high effort84.1 %independentVals AI
GPQA Diamond
implementation=artificial-analysis
82.7 %independentArtificial Analysis
GPQA Diamondhigh effort81.8 %independentEpoch AI Benchmarking Hub
2025-04-16
GPQA Diamondmedium effort80.8 %independentEpoch AI Benchmarking Hub
2026-08-07
GPQA Diamondlow effort79.8 %independentEpoch AI Benchmarking Hub
2026-07-15
Humanity's Last Exam
implementation=artificial-analysis
20.1 %independentArtificial Analysis
MMLU-Pro
implementation=vals-ai
high effort85.6 %independentVals AI
MMLU-Pro
implementation=artificial-analysis
85.3 %independentArtificial Analysis
long context instruction
AA-LCR74.7 %independentArtificial Analysis
IFBench71.4 %independentArtificial Analysis
multimodal
MMMU
implementation=vals-ai
high effort80.4 %independentVals AI
professional
CorpFinhigh effort59.7 %independentVals AI
LegalBenchhigh effort83.8 %independentVals AI
MedQAhigh effort96.1 %independentVals AI
TaxEvalhigh effort74.6 %independentVals AI
reasoning math
AIME
implementation=artificial-analysis
90.3 %independentArtificial Analysis
AIME
year=2025 · implementation=artificial-analysis
88.3 %independentArtificial Analysis
AIME
implementation=vals-ai
high effort85.3 %independentVals AI
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort64.3 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort60.8 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort56.7 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort53.8 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort47.6 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort41.5 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort6.5 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort4.5 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort3.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort2.9 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort2.7 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort2.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort0.9 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort0.8 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort0.5 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort0.5 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort0.5 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort0.4 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort0.2 usd_per_taskindependentARC Prize Leaderboard
FrontierMathhigh effort18.7 %independentEpoch AI Benchmarking Hub
2025-11-16
FrontierMathmedium effort16.9 %independentEpoch AI Benchmarking Hub
2025-11-16
FrontierMathlow effort9.7 %independentEpoch AI Benchmarking Hub
2025-11-17
FrontierMath Tier 4older versionhigh effort2.1 %independentEpoch AI Benchmarking Hub
2025-07-01
MATH-500
implementation=artificial-analysis
99.2 %independentArtificial Analysis
MATH-500
implementation=vals-ai
high effort94.6 %independentVals AI
MATH Level 5high effort97.8 %independentEpoch AI Benchmarking Hub
2025-04-16
OTIS Mock AIME 2024–2025medium effort84.4 %independentEpoch AI Benchmarking Hub
2026-08-07
OTIS Mock AIME 2024–2025high effort83.9 %independentEpoch AI Benchmarking Hub
2025-04-16
OTIS Mock AIME 2024–2025low effort60.0 %independentEpoch AI Benchmarking Hub
2026-07-15

Agent + model results

systems, not bare-model scores
agent + model Epoch Inspect harness + O3 (2025-04-16)SWE-bench Verified62.3 %independentEpoch AI Benchmarking Hub
agent + model Codefuse_Pycfuse_SVR + O3 (2025-04-16)SWE-bench Multimodal36.0 %communitySWE-bench Leaderboard
agent + model GUIRepair + O3 (2025-04-16)SWE-bench Multimodal36.0 %communitySWE-bench Leaderboard
agent + model Artificial Analysis harness + O3 (2025-04-16)Terminal-Bench Hard37.1 %independentArtificial Analysis
agent + model terminus-1 + O3 (2025-04-16)Terminal-Bench 1.030.2 %communityTerminal-Bench Leaderboard

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.

Related variants

O3 20250416