MMLU-Pro explained: the 10-option knowledge exam behind 10% of our League Score.
MMLU-Pro explained in plain terms: a harder successor to the famous MMLU knowledge test, with ten answer options instead of four. Here is what an MMLU-Pro score measures, how the scores in our database were recorded, and why a 10-point raw spread becomes a 0–100 spread inside the League Score.
By AI Pro League · Published
MMLU-Pro explained starts with its place in our own table: it carries 10% of the League Score — the smallest of the five components — and it is the sparsest record we track. Only 8 of the 59 models in our database carry a recorded MMLU-Pro score, against 31 with a LiveBench score. This explainer uses our own MMLU-Pro leaderboard as the worked example throughout, so every number below can be checked against the table itself.
What MMLU-Pro measures
Our benchmarks hub describes MMLU-Pro as “a harder, 10-option successor to MMLU across 14 subjects, ~12,000 questions”. The original MMLU gave four answer options per question, so a model could score a quarter of the points by guessing; ten options cut the guessing floor to a tenth and make the exam a stricter test of what a model actually knows across its subjects — which include dedicated mathematics categories, the reason our best AI for math ranking is built on this benchmark alone. The scores we record come from the BenchLM MMLU-Pro leaderboard and the MMLU-Pro results it mirrors, each with a measured date on the row.
How an MMLU-Pro score is calculated — and what normalisation does to it
On the benchmark itself the score is the share of questions answered correctly. Our League Score then normalises that raw score across the small field we track — today the recorded range runs from just 78.3 to 88.6, a spread of only 10.3 raw points — and weights the result at 10%. The worked example, from the top of our table:
| Model | Raw score | Normalised (0–100) |
|---|---|---|
| Qwen3.8-Max | 88.6 | (88.6 − 78.3) / (88.6 − 78.3) × 100 = 100.0 |
| GPT-5 | 86.5 | (86.5 − 78.3) / (88.6 − 78.3) × 100 = 79.6 |
| GPT-5.6 Luna | 86.0 | (86.0 − 78.3) / (88.6 − 78.3) × 100 = 74.8 |
Read that table carefully, because it is this benchmark’s trap: a raw field compressed into 10.3 points is stretched across the full 0–100 normalised scale, so the lowest recorded score maps to 0.0 and a raw gap of two or three points — well inside the noise between two test runs — becomes a gap of twenty or thirty normalised points. At a 10% weight the League Score itself only moves a little, but anyone reading the component in isolation will over-read it. Full method: how the League Score is built.
What a good MMLU-Pro score looks like
The recorded field is tight and high: the median score is 85.8, the floor is 78.3 and the ceiling is 88.6. In a test with a 10% guessing floor, every model we track sits far above chance — the argument is about the order, not about competence. The top of the table:
| # | Model | Maker | Score | Measured |
|---|---|---|---|---|
| 1 | Qwen3.8-Max | Alibaba Qwen | 88.6 | Oct 14, 2026 |
| 2 | GPT-5 | OpenAI | 86.5 | Oct 14, 2026 |
| 3 | GPT-5.6 Luna | OpenAI | 86.0 | Oct 14, 2026 |
| 4 | Gemini 3.5 Flash-Lite | Google DeepMind | 85.8 | Oct 14, 2026 |
| 5 | Grok 4.3 | xAI (SpaceXAI) | 85.8 | Oct 14, 2026 |
The full table, with a source link on every row, is the MMLU-Pro leaderboard, and the standings this component helps shape are the AI model rankings.
Why so few models have an MMLU-Pro score
Coverage, not quality, is the main reason: 8 recorded scores out of 59 tracked models means most of the field is simply unmeasured on this test in the sources we use, and an absent score contributes nothing — it is not treated as a zero. That is also why MMLU-Pro carries the smallest weight in the League Score: a component most models lack cannot be allowed to dominate a ranking the way the 40%-weighted LiveBench does. As more of the field is measured, the normalisation range above will widen and the component’s swings will calm down.
The limits to keep in mind
- Multiple choice is not knowledge in use. Picking the right option from ten is a narrower skill than explaining, deriving or applying the same fact in open work.
- The field is tiny. Eight models is a leaderboard, not a census — one new measurement redraws the normalisation for everyone.
- Small raw gaps are noise. In a field stretched from 10.3 raw points to 100 normalised points, treat any gap under a raw point or two as a tie.
- It is one component. At 10% of the League Score it is a tiebreaker voice beside LiveBench, SWE-bench Verified, GPQA Diamond and LMArena — by design.
For the companion tests, start with our explainers of GPQA Diamond and LiveBench, or the wider guide to AI benchmarks explained.