Benchmarks / Long Context & Instruction
MultiChallenge
Multi-turn conversational instruction-following benchmark (Scale Labs).
maintained by Scale (Scale Labs) · 30 observations on record
Versions
1 version
| Version | Tasks |
|---|---|
| MultiChallenge current multichallenge | — |
Current leaderboard
MultiChallenge · Accuracy · top 25 of 30 systems
| # | System | Accuracy % |
|---|---|---|
| 1 | Muse Spark Meta AI | 75.5% 71.5–79.6 |
| 2 | Muse Spark 1.1 Meta AI | 75.3% 74.7–75.9 |
| 3 | Gemini 3.1 Pro Preview Google DeepMind | 71.4% 69.6–73.1 |
| 4 | GPT 5.4 Pro (2026-03-05) OpenAI | 69.2% 66.2–72.3 |
| 5 | Gemini 3 Pro Preview Google DeepMind | 65.7% 63.5–67.9 |
| 6 | GPT 5.1 (2025-11-13)thinking OpenAI | 63.4% 61.3–65.5 |
| 7 | GPT 5 (2025-08-07)thinking OpenAI | 63.2% 61.6–64.8 |
| 8 | O3 Pro 2025 06.10 Reasoninghigh effort OpenAI | 62.4% 59.9–64.9 |
| 9 | Kimi K2 5 Moonshot AI | 61.4% 60.9–61.9 |
| 10 | Gemini 3.1 Flash Lite Preview Google | 60.6% 60.2–61.0 |
| 11 | GPT 5 Mini (2025-08-07)thinking OpenAI | 59.0% 54.9–63.1 |
| 12 | Claude Opus 4.5 20251101thinking Anthropic | 59.0% 58.5–59.4 |
| 13 | Claude 4 Opusthinking Unknown | 58.6% 57.9–59.4 |
| 14 | Claude Opus 4.1 20250805thinking Anthropic | 57.2% 56.3–58.1 |
| 15 | Claude 4 Sonnetthinking Unknown | 57.1% 56.3–57.9 |
| 16 | O3 2025 04.16 Reasoninghigh effort OpenAI | 56.6% 53.3–59.9 |
| 17 | Claude Opus 4.6no reasoning Anthropic | 56.0% 54.9–57.1 |
| 18 | Kimi K2thinking Moonshot AI | 55.4% 53.5–57.3 |
| 19 | Claude Sonnet 4.5 20250929thinking Anthropic | 55.3% 53.9–56.8 |
| 20 | Gemini 2.5 Pro (2025-06-17) Google DeepMind | 53.6% 52.3–54.9 |
| 21 | Claude 3.7 Sonnetthinking Anthropic | 51.6% 48.5–54.6 |
| 22 | GPT 5.1 2025 11.13 Instant OpenAI | 51.2% 49.4–53.0 |
| 23 | Claude Haiku 4.5 20251001thinking Anthropic | 50.5% 48.5–52.5 |
| 24 | Deepseek V3p1 Fireworks AI | 46.1% 45.0–47.2 |
| 25 | GPT OSS 120B OpenAI | 45.3% 43.2–47.5 |
One row per evaluated system — reasoning-effort variants rank separately.