GPQA Diamond explained: the science exam built to resist Google.
GPQA Diamond is a 198-question set of graduate-level science questions written so the answers cannot simply be looked up. Here is what a GPQA Diamond score measures, how we turn it into a League Score component, and why the top scores now cluster within a few points of each other.
By AI Pro League · Published
GPQA Diamond carries 15% of the League Score on this site — it is our dedicated reasoning component, the number we read when a model claims to be good at hard science. It is also widely quoted and widely misunderstood: a percentage earned on fewer than two hundred questions, under conditions designed to defeat web lookup. This explainer uses our own GPQA Diamond leaderboard as the worked example throughout.
What GPQA Diamond measures
GPQA is a set of graduate-level multiple-choice questions in biology, chemistry and physics, written by domain experts to be “Google-proof”: a non-expert with web access should still struggle, while an expert in the field should find the question answerable. Diamond is the hardest 198-question subset of that pool — the questions that best separated expert from non-expert performance. The score is the share of those questions the model answers correctly. Our recorded scores come from published leaderboards such as BenchLM’s GPQA leaderboard, always with the source and the date we verified them on the model page.
How a GPQA Diamond score is calculated
The benchmark score itself is a plain accuracy rate: answer 178 of the 198 Diamond questions correctly and the score is about 90. Our League Score then normalises that raw score across the field we track — today the recorded range runs from 73.0 to 96.0 — and weights the result at 15%. The worked example, from the top of our table:
| Model | Raw score | Normalised (0–100) |
|---|---|---|
| GPT-6 Astra | 96.0 | (96.0 − 73.0) / (96.0 − 73.0) × 100 = 100.0 |
| Gemini 3.8 Flash | 95.0 | (95.0 − 73.0) / (96.0 − 73.0) × 100 = 95.7 |
| GPT-5.6 Sol | 94.6 | (94.6 − 73.0) / (96.0 − 73.0) × 100 = 93.9 |
Normalisation matters here more than on any other component: when the whole recorded field sits in a narrow band, a two-point raw gap near the top is worth far more League Score than the same gap would be on a spread-out test. Full method: how the League Score is built.
What a good GPQA Diamond score looks like
Only 19 of the 59 models in our database carry a recorded score, and by the standards of most benchmarks the numbers look extreme: the median recorded score is 92.3, the floor is 73.0 and the ceiling is 96.0. The top of the table:
| # | Model | Maker | Score | Measured |
|---|---|---|---|---|
| 1 | GPT-6 Astra | OpenAI | 96.0 | Oct 14, 2026 |
| 2 | Gemini 3.8 Flash | Google DeepMind | 95.0 | Oct 14, 2026 |
| 3 | GPT-5.6 Sol | OpenAI | 94.6 | Oct 14, 2026 |
| 4 | Gemini 3.1 Pro Preview | Google DeepMind | 94.3 | Oct 5, 2026 |
| 5 | Claude Fable 5.1 | Anthropic | 93.7 | Sep 3, 2026 |
The full table, with a source link on every row, is the GPQA Diamond leaderboard. The shortlist it feeds is our best AI models for reasoning page.
Why the scores cluster — and when to distrust a gap
With only 198 questions, each question is worth about half a percentage point, so small raw gaps can be a handful of questions rather than a real capability difference. Our benchmark page also carries a ledger note that matters: most recorded rows are Diamond, but some are GPQA main-set results — so treat close gaps as directional, not decisive. A model two points ahead on GPQA has not proven it reasons better; it has answered roughly four more questions on one particular test.
The limits to keep in mind
- Coverage is partial. Most tracked models have no recorded score, so absence from the table means untested, not weak.
- It is science-only. Biology, chemistry and physics at graduate level — the score says nothing direct about law, history or everyday reasoning.
- Scores come from different sources. Our rows are a mix of leaderboard, maker-reported and press-reported results, each labelled on the model page; harness and prompting can move a number.
- It is one component. At 15% of the League Score it anchors the reasoning standings, but general capability (LiveBench, 40%) still dominates the overall table.
For the wider map — what SWE-bench, MMLU-Pro and the other famous tests measure — start with our explainer of SWE-bench Verified and the guide to AI benchmarks explained.