BenchAtlas

Rankings / OpenAI

O1 Preview (2024-09-12)

proprietary

released 2024-09-12

BenchAtlas Index

as of 2026-09-11
52.4
base configuration
rank #220
7 families · 3 categories · medium

Benchmark evidence

38 results
coding
LiveCodeBench v5older version
problems=279 · window_start=2024-08-01
42.6 %independentLiveCodeBench Leaderboard
external indices
AA Coding Index34.0 pointsindependentArtificial Analysis
Intelligence Index v4.111.4 pointsindependentArtificial Analysis
AA Math Index79.7 pointsindependentArtificial Analysis
ECI134.8 pointsindependentEpoch AI Benchmarking Hub
human preference
Coding (style control)older version
arena=text · category=coding · style_control=true
1416.0
1406.7–1425.4
communityLMArena Leaderboard Dataset
Hard Prompts English (style control)older version
arena=text · category=hard_prompts_english · style_control=true
1412.0
1402.5–1421.5
communityLMArena Leaderboard Dataset
English (style control)older version
arena=text · category=english · style_control=true
1409.4
1403.1–1415.7
communityLMArena Leaderboard Dataset
Industry Software And It Services (style control)older version
arena=text · category=industry_software_and_it_services · style_control=true
1406.6
1398.7–1414.5
communityLMArena Leaderboard Dataset
Industry Mathematical (style control)older version
arena=text · category=industry_mathematical · style_control=true
1397.8
1387.3–1408.2
communityLMArena Leaderboard Dataset
Hard Prompts (style control)older version
arena=text · category=hard_prompts · style_control=true
1396.2
1388.5–1403.9
communityLMArena Leaderboard Dataset
Industry Legal And Government (style control)older version
arena=text · category=industry_legal_and_government · style_control=true
1391.1
1376.7–1405.5
communityLMArena Leaderboard Dataset
Multi Turn (style control)older version
arena=text · category=multi_turn · style_control=true
1390.2
1380.9–1399.5
communityLMArena Leaderboard Dataset
Industry Life And Physical And Social Science (style control)older version
arena=text · category=industry_life_and_physical_and_social_science · style_control=true
1389.3
1380.1–1398.4
communityLMArena Leaderboard Dataset
Overall (style control)
arena=text · category=overall · style_control=true
1388.3
1383.3–1393.3
communityLMArena Leaderboard Dataset
Chinese (style control)older version
arena=text · category=chinese · style_control=true
1386.7
1374.2–1399.1
communityLMArena Leaderboard Dataset
Math (style control)older version
arena=text · category=math · style_control=true
1386.4
1376.7–1396.1
communityLMArena Leaderboard Dataset
Expert (style control)older version
arena=text · category=expert · style_control=true
1382.1
1367.5–1396.7
communityLMArena Leaderboard Dataset
Longer Query (style control)older version
arena=text · category=longer_query · style_control=true
1381.8
1372.1–1391.4
communityLMArena Leaderboard Dataset
Industry Writing And Literature And Language (style control)older version
arena=text · category=industry_writing_and_literature_and_language · style_control=true
1381.2
1373.2–1389.2
communityLMArena Leaderboard Dataset
Instruction Following (style control)older version
arena=text · category=instruction_following · style_control=true
1380.8
1373.9–1387.7
communityLMArena Leaderboard Dataset
Industry Entertainment And Sports And Media (style control)older version
arena=text · category=industry_entertainment_and_sports_and_media · style_control=true
1371.5
1361.9–1381.0
communityLMArena Leaderboard Dataset
Industry Medicine And Healthcare (style control)older version
arena=text · category=industry_medicine_and_healthcare · style_control=true
1366.2
1350.9–1381.6
communityLMArena Leaderboard Dataset
Exclude Ties (style control)older version
arena=text · category=exclude_ties · style_control=true
1365.7
1358.3–1373.1
communityLMArena Leaderboard Dataset
Creative Writing (style control)older version
arena=text · category=creative_writing · style_control=true
1365.2
1355.3–1375.1
communityLMArena Leaderboard Dataset
Non English (style control)older version
arena=text · category=non_english · style_control=true
1359.3
1352.9–1365.7
communityLMArena Leaderboard Dataset
Russian (style control)older version
arena=text · category=russian · style_control=true
1358.5
1348.8–1368.3
communityLMArena Leaderboard Dataset
German (style control)older version
arena=text · category=german · style_control=true
1353.4
1331.9–1374.9
communityLMArena Leaderboard Dataset
Industry Business And Management And Financial Operations (style control)older version
arena=text · category=industry_business_and_management_and_financial_operations · style_control=true
1352.8
1342.3–1363.3
communityLMArena Leaderboard Dataset
Japanese (style control)older version
arena=text · category=japanese · style_control=true
1348.0
1324.0–1372.1
communityLMArena Leaderboard Dataset
knowledge science
GPQA Diamond
implementation=artificial-analysis
76.5 %independentArtificial Analysis
GPQA Diamond50.3 %independentEpoch AI Benchmarking Hub
2025-01-27
MMLU-Pro
implementation=artificial-analysis
84.8 %independentArtificial Analysis
professional
MedQAhigh effort93.0 %independentVals AI
reasoning math
AIME
year=2025 · implementation=artificial-analysis
79.7 %independentArtificial Analysis
MATH-500
implementation=artificial-analysis
92.4 %independentArtificial Analysis
MATH Level 581.7 %independentEpoch AI Benchmarking Hub
2025-01-27
OTIS Mock AIME 2024–202531.1 %independentEpoch AI Benchmarking Hub
2025-03-07

Agent + model results

systems, not bare-model scores
agent + model W&B Programmer O1 crosscheck5 + O1 Preview (2024-09-12)SWE-bench Verified64.6 %unverifiedSWE-bench Leaderboard

These scores measure the whole agent system (scaffold, tools, budgets) — they are never merged into the bare model’s numbers.