Benchmarks / Reasoning & Math
MATH-500
500-problem subset of the MATH competition dataset.
maintainer unknown · 125 observations on record
Versions
1 version
| Version | Tasks |
|---|---|
| MATH-500 current math-500 | — |
Current leaderboard
MATH-500 · Accuracy · top 25 of 125 systems
| # | System | Accuracy % |
|---|---|---|
| 1 | GPT 5 (2025-08-07)high effort OpenAI | 99.4% |
| 2 | O3 (2025-04-16) OpenAI | 99.2% |
| 3 | GPT 5 (2025-08-07)medium effort OpenAI | 99.1% |
| 4 | Claude 4 Sonnetthinking Unknown | 99.1% |
| 5 | Grok 4 xAI | 99.0% |
| 6 | O4 Mini (2025-04-16)high effort OpenAI | 98.9% |
| 7 | GPT 5 (2025-08-07)low effort OpenAI | 98.7% |
| 8 | O3 Mini (2025-01-31)high effort OpenAI | 98.5% |
| 9 | Qwen3 235B A22b 2507thinking Alibaba | 98.4% |
| 10 | Claude 4 Opusthinking Unknown | 98.2% |
| 11 | Gemini 2.5 Flashthinking Google DeepMind | 98.1% |
| 12 | Qwen3 235B A22b Instruct 2507 Alibaba (Qwen) | 98.0% |
| 13 | GLM 4.5thinking Z.ai | 97.9% |
| 14 | Exaone 4.0 32Bthinking Unknown | 97.7% |
| 15 | O3 Mini (2025-01-31) OpenAI | 97.3% |
| 16 | Kimi K2 Moonshot AI | 97.1% |
| 17 | Gemini 2.5 Pro (2025-06-17) Google DeepMind | 96.7% |
| 18 | GLM 4.5 Air zai | 96.5% |
| 19 | Gemini 3 Pro Previewhigh effort Google DeepMind | 96.4% |
| 20 | Grok 4 0709 xAI | 96.2% |
| 21 | Qwen3 32Bthinking Alibaba | 96.1% |
| 22 | GPT 5 (2025-08-07)high effort OpenAI | 96.0% |
| 23 | Qwen3 30B A3bthinking Alibaba | 95.9% |
| 24 | Qwq 32B Alibaba | 95.7% |
| 25 | Claude Opus 4.1 20250805thinking Anthropic | 95.4% |
One row per evaluated system — reasoning-effort variants rank separately.
Source agreement
where evaluating organizations agree — and don't
MATH-500Artificial Analysis vs Vals AI
overlap 11 systemsrank correlation 0.82mean |Δ| 4.22max |Δ| 6.67
most contested: O3 Mini (2025-01-31) (+6.67) · Deepseek V3 0324 (+5.6) · O4 Mini (2025-04-16) (+4.67)
Possible reasons: run setting 'implementation' differs (artificial-analysis vs vals-ai).