BenchAtlas

Rankings / OpenAI

O3 Mini (2025-01-31)

proprietary

released 2025-01-31

BenchAtlas Index

as of 2026-07-14
34.6
high effort
rank #328
15 families · 5 categories · high
34.6
base configuration
rank #330
8 families · 4 categories · medium
32.5
medium effort
rank #341
7 families · 4 categories · medium

Benchmark evidence

128 results
agentic coding
τ²-Benchhigh effort31.3 %independentArtificial Analysis
τ²-Bench28.6 %independentArtificial Analysis
τ²-Bench
subset=banking
high effort5.2 %independentArtificial Analysis
coding
LiveCodeBench v6
implementation=artificial-analysis
high effort73.4 %independentArtificial Analysis
LiveCodeBench v6
implementation=artificial-analysis
71.7 %independentArtificial Analysis
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
high effort69.5 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
high effort67.4 %independentLiveCodeBench Leaderboard
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
medium effort67.4 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
medium effort63.0 %independentLiveCodeBench Leaderboard
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
low effort60.9 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
low effort57.0 %independentLiveCodeBench Leaderboard
SciCode39.9 %independentArtificial Analysis
SciCodehigh effort39.8 %independentArtificial Analysis
external indices
AA Coding Indexhigh effort16.3 pointsindependentArtificial Analysis
Intelligence Index v4.119.0 pointsindependentArtificial Analysis
Intelligence Index v4.1high effort15.6 pointsindependentArtificial Analysis
ECImedium effort141.4 pointsindependentEpoch AI Benchmarking Hub
ECI141.4 pointsindependentEpoch AI Benchmarking Hub
ECIlow effort141.4 pointsindependentEpoch AI Benchmarking Hub
ECIhigh effort141.4 pointsindependentEpoch AI Benchmarking Hub
factuality
MASKlow effort49.7 %
46.5–53.0
independentScale Labs
MASKmedium effort48.9 %
47.7–50.2
independentScale Labs
MASKhigh effort46.8 %
44.2–49.4
independentScale Labs
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
high effort1434.7
1423.2–1446.2
communityLMArena Leaderboard Dataset
Coding (style control)older version
arena=text · category=coding · style_control=true
1416.0
1409.5–1422.4
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
high effort1413.9
1403.0–1424.7
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
high effort1412.3
1395.0–1429.5
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
high effort1412.2
1398.4–1425.9
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
high effort1410.8
1401.6–1420.1
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
high effort1405.5
1392.4–1418.5
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
high effort1401.2
1392.1–1410.3
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
high effort1397.0
1376.6–1417.3
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1391.6
1382.8–1400.5
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1389.5
1384.3–1394.8
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1383.3
1377.3–1389.3
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1381.8
1373.3–1390.3
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
high effort1379.0
1372.5–1385.5
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
high effort1372.5
1362.0–1382.9
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
high effort1372.5
1360.2–1384.7
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1370.2
1365.4–1375.1
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
high effort1367.8
1346.9–1388.7
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
high effort1366.5
1358.8–1374.1
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1365.3
1353.9–1376.6
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
high effort1363.5
1358.2–1368.7
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1363.0
1358.7–1367.3
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1360.4
1350.0–1370.9
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
high effort1358.8
1340.6–1377.0
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1358.4
1351.9–1364.9
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1356.4
1333.2–1379.5
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1349.8
1343.3–1356.4
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
high effort1349.2
1335.5–1362.8
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1347.8
1344.2–1351.3
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
high effort1344.8
1333.1–1356.5
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
high effort1343.5
1336.2–1350.8
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1343.5
1338.4–1348.6
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1342.2
1331.0–1353.3
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
high effort1340.9
1332.1–1349.7
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1340.1
1330.0–1350.2
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1339.7
1333.1–1346.2
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
high effort1329.6
1321.8–1337.3
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
high effort1328.4
1314.9–1341.8
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1327.3
1305.8–1348.7
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1326.7
1310.1–1343.3
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1326.6
1319.5–1333.8
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1326.6
1317.8–1335.3
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1326.4
1321.9–1330.8
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1324.4
1311.9–1337.0
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1324.0
1318.4–1329.6
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
high effort1311.8
1301.0–1322.6
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
high effort1310.3
1300.0–1320.5
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1306.8
1301.9–1311.7
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1301.4
1295.1–1307.6
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1301.0
1294.0–1307.9
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1299.2
1282.3–1316.0
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1293.1
1274.0–1312.1
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond
implementation=artificial-analysis
high effort77.3 %independentArtificial Analysis
GPQA Diamondhigh effort77.0 %independentEpoch AI Benchmarking Hub
2025-02-13
GPQA Diamond
implementation=vals-ai
high effort75.5 %independentVals AI
GPQA Diamond
implementation=artificial-analysis
74.8 %independentArtificial Analysis
GPQA Diamondmedium effort74.3 %independentEpoch AI Benchmarking Hub
2025-01-31
Humanity's Last Exam
implementation=artificial-analysis
high effort12.3 %independentArtificial Analysis
Humanity's Last Exam
implementation=artificial-analysis
8.7 %independentArtificial Analysis
MMLU-Pro
implementation=artificial-analysis
high effort80.2 %independentArtificial Analysis
MMLU-Pro
implementation=artificial-analysis
79.1 %independentArtificial Analysis
MMLU-Pro
implementation=vals-ai
high effort78.7 %independentVals AI
long context instruction
AA-LCRhigh effort39.3 %independentArtificial Analysis
IFBenchhigh effort67.1 %independentArtificial Analysis
professional
CorpFinhigh effort45.3 %independentVals AI
LegalBenchhigh effort71.5 %independentVals AI
MedQAhigh effort94.8 %independentVals AI
TaxEvalhigh effort69.4 %independentVals AI
reasoning math
AIME
implementation=vals-ai
high effort86.5 %independentVals AI
AIME
implementation=artificial-analysis
high effort86.0 %independentArtificial Analysis
AIME
implementation=artificial-analysis
77.0 %independentArtificial Analysis
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort46.6 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort34.5 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort30.6 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort22.3 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort17.4 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort14.5 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort3.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort2.1 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort0.6 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort0.5 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort0.4 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort0.0 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort0.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort0.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort0.0 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort0.0 %independentARC Prize Leaderboard
FrontierMathhigh effort12.4 %independentEpoch AI Benchmarking Hub
2025-11-16
FrontierMathmedium effort8.1 %independentEpoch AI Benchmarking Hub
2025-03-06
FrontierMath Tier 4older versionhigh effort4.2 %independentEpoch AI Benchmarking Hub
2025-07-01
MATH-500
implementation=artificial-analysis
high effort98.5 %independentArtificial Analysis
MATH-500
implementation=artificial-analysis
97.3 %independentArtificial Analysis
MATH-500
implementation=vals-ai
high effort91.8 %independentVals AI
MATH Level 5high effort96.5 %independentEpoch AI Benchmarking Hub
2025-02-13
MATH Level 5medium effort95.2 %independentEpoch AI Benchmarking Hub
2025-01-31
OTIS Mock AIME 2024–2025high effort76.9 %independentEpoch AI Benchmarking Hub
2025-02-27
OTIS Mock AIME 2024–2025medium effort63.9 %independentEpoch AI Benchmarking Hub
2025-02-25

Agent + model results

systems, not bare-model scores
agent + model Agentless Lite + O3 Mini (2025-01-31)SWE-bench Verified42.4 %communitySWE-bench Leaderboard
agent + model Agentless Lite + O3 Mini (2025-01-31)SWE-bench Lite32.3 %communitySWE-bench Leaderboard
agent + model Aegis - o3-mini_1.0 + O3 Mini (2025-01-31)SWE-bench Lite30.3 %unverifiedSWE-bench Leaderboard
agent + model Artificial Analysis harness + O3 Mini (2025-01-31)Terminal-Bench Hard6.8 %independentArtificial Analysis
agent + model Artificial Analysis harness + O3 Mini (2025-01-31)Terminal-Bench Hard6.1 %independentArtificial Analysis
agent + model Artificial Analysis harness + O3 Mini (2025-01-31)Terminal-Bench 2.14.5 %independentArtificial Analysis

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.