BenchAtlas

Rankings / OpenAI

O4 Mini (2025-04-16)

released 2025-04-16

BenchAtlas Index

as of 2026-09-11
49.8
high effort
rank #242
15 families · 5 categories · high
47.8
medium effort
rank #255
6 families · 4 categories · medium
44.8
high effort
rank #276
8 families · 5 categories · high

Benchmark evidence

99 results
agentic coding
τ²-Benchhigh effort55.6 %independentArtificial Analysis
coding
LiveCodeBench v6
implementation=artificial-analysis
high effort85.9 %independentArtificial Analysis
LiveCodeBench v6
problems=454 · window_start=2024-08-01
high effort80.2 %independentLiveCodeBench Leaderboard
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
high effort74.5 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
medium effort74.2 %independentLiveCodeBench Leaderboard
LiveCodeBench v6
problems=454 · window_start=2024-08-01
low effort65.9 %independentLiveCodeBench Leaderboard
SciCodehigh effort46.5 %independentArtificial Analysis
external indices
Intelligence Index v4.1high effort16.7 pointsindependentArtificial Analysis
AA Math Indexhigh effort90.7 pointsindependentArtificial Analysis
ECIhigh effort145.7 pointsindependentEpoch AI Benchmarking Hub
ECI145.7 pointsindependentEpoch AI Benchmarking Hub
ECIlow effort145.7 pointsindependentEpoch AI Benchmarking Hub
ECImedium effort145.7 pointsindependentEpoch AI Benchmarking Hub
factuality
SimpleQA Verifiedhigh effort23.9 %independentEpoch AI Benchmarking Hub
2025-12-09
SimpleQA Verifiedlow effort21.9 %independentEpoch AI Benchmarking Hub
2026-07-11
SimpleQA Verifiedlow effort19.6 %independentEpoch AI Benchmarking Hub
2026-08-27
SimpleQA Verifiedhigh effort18.8 %independentEpoch AI Benchmarking Hub
2026-08-27
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
1432.9
1426.0–1439.8
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1422.1
1416.4–1427.9
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1419.7
1413.0–1426.4
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1417.8
1405.3–1430.2
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1417.1
1406.4–1427.9
communityLMArena Leaderboard Dataset
French (style control)older version
arena=text · category=french · style_control=true
1412.0
1387.3–1436.8
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1411.3
1400.2–1422.3
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1407.6
1395.1–1420.0
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1405.7
1400.5–1410.9
communityLMArena Leaderboard Dataset
Polish (style control)older version
arena=text · category=polish · style_control=true
1405.3
1393.2–1417.5
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1404.7
1399.6–1409.8
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1404.5
1393.4–1415.6
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1403.9
1396.5–1411.3
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1399.0
1387.3–1410.6
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1390.6
1386.6–1394.6
communityLMArena Leaderboard Dataset
Spanish (style control)older version
arena=text · category=spanish · style_control=true
1384.5
1365.1–1404.0
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1381.5
1374.2–1388.7
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1379.1
1371.5–1386.6
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1378.2
1360.6–1395.8
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1374.1
1369.2–1378.9
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1370.9
1361.0–1380.8
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1368.4
1361.4–1375.3
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1368.1
1362.6–1373.6
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1368.1
1362.0–1374.1
communityLMArena Leaderboard Dataset
Korean (style control)older version
arena=text · category=korean · style_control=true
1356.2
1336.2–1376.1
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1354.0
1347.5–1360.6
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1349.1
1342.0–1356.1
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1347.1
1327.8–1366.4
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1336.8
1328.5–1345.0
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamondhigh effort79.6 %independentEpoch AI Benchmarking Hub
2025-04-16
GPQA Diamond
implementation=artificial-analysis
high effort78.4 %independentArtificial Analysis
GPQA Diamondmedium effort77.8 %independentEpoch AI Benchmarking Hub
2026-08-07
GPQA Diamondlow effort75.3 %independentEpoch AI Benchmarking Hub
2026-07-11
GPQA Diamond
implementation=vals-ai
high effort74.5 %independentVals AI
Humanity's Last Exam
implementation=artificial-analysis
high effort16.5 %independentArtificial Analysis
MMLU-Pro
implementation=artificial-analysis
high effort83.2 %independentArtificial Analysis
MMLU-Pro
implementation=vals-ai
high effort80.6 %independentVals AI
long context instruction
AA-LCRhigh effort61.0 %independentArtificial Analysis
IFBenchhigh effort68.7 %independentArtificial Analysis
multimodal
MMMU
implementation=vals-ai
high effort79.7 %independentVals AI
professional
CorpFinhigh effort59.0 %independentVals AI
LegalBenchhigh effort79.2 %independentVals AI
MedQAhigh effort96.0 %independentVals AI
TaxEvalhigh effort74.8 %independentVals AI
reasoning math
AIME
implementation=artificial-analysis
high effort94.0 %independentArtificial Analysis
AIME
year=2025 · implementation=artificial-analysis
high effort90.7 %independentArtificial Analysis
AIME
implementation=vals-ai
high effort83.7 %independentVals AI
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort68.0 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort58.7 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort50.3 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort41.8 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort27.6 %independentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort21.3 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort7.5 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort6.1 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort2.4 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort2.2 %independentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort1.7 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
high effort0.9 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
high effort0.9 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
high effort0.4 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
high effort0.3 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort0.3 %independentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
medium effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
medium effort0.2 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
medium effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
medium effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=public_eval · model_type=CoT
low effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-2
split=semi_private · model_type=CoT
low effort0.1 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=semi_private · model_type=CoT
low effort0.0 usd_per_taskindependentARC Prize Leaderboard
ARC-AGI-1older version
split=public_eval · model_type=CoT
low effort0.0 usd_per_taskindependentARC Prize Leaderboard
FrontierMathhigh effort24.8 %independentEpoch AI Benchmarking Hub
2025-11-13
FrontierMathmedium effort19.0 %independentEpoch AI Benchmarking Hub
2025-11-13
FrontierMathlow effort10.7 %independentEpoch AI Benchmarking Hub
2025-11-16
FrontierMath Tier 4older versionhigh effort6.3 %independentEpoch AI Benchmarking Hub
2025-07-01
FrontierMath Tier 4older versionmedium effort2.1 %independentEpoch AI Benchmarking Hub
2025-08-07
MATH-500
implementation=artificial-analysis
high effort98.9 %independentArtificial Analysis
MATH-500
implementation=vals-ai
high effort94.2 %independentVals AI
MATH Level 5high effort97.8 %independentEpoch AI Benchmarking Hub
2025-04-16
OTIS Mock AIME 2024–2025high effort81.7 %independentEpoch AI Benchmarking Hub
2025-04-16
OTIS Mock AIME 2024–2025medium effort73.3 %independentEpoch AI Benchmarking Hub
2026-08-07
OTIS Mock AIME 2024–2025low effort57.8 %independentEpoch AI Benchmarking Hub
2026-07-13

Agent + model results

systems, not bare-model scores
agent + model PatchPilot-v1.1 + O4 Mini (2025-04-16)SWE-bench Verified64.6 %unverifiedSWE-bench Leaderboard
agent + model GUIRepair + O4 Mini (2025-04-16)SWE-bench Multimodal33.9 %communitySWE-bench Leaderboard
agent + model goose + O4 Mini (2025-04-16)Terminal-Bench 1.027.5 %communityTerminal-Bench Leaderboard
agent + model Codex CLI + O4 Mini (2025-04-16)Terminal-Bench 1.020.0 %communityTerminal-Bench Leaderboard
agent + model terminus-1 + O4 Mini (2025-04-16)Terminal-Bench 1.018.5 %communityTerminal-Bench Leaderboard
agent + model Artificial Analysis harness + O4 Mini (2025-04-16)Terminal-Bench Hard15.2 %independentArtificial Analysis

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.

Related variants

O4 Mini 20250416