// benchmarks
LiveBench 2.0 explained
LiveBench relaunched in October 2026 with a different idea of what to measure: finished work — fixes that pass, browser tasks completed, reports and documents produced — instead of short questions. The categories, the Overall formula and the rankings all changed. Here is what each number now means.
The short version
Seven categories replace the old ones: long-horizon agentic, coding, reasoning, browser, data analysis, research, documents & design. Mathematics, Language and Instruction Following are gone; agentic coding became a broader long-horizon agentic category.
Overall is now a weighted mean — long-horizon agentic ×2, coding ×1.5, documents & design ×1.5. And the field spread out: models that sat within 12 points of each other on v1 now span 42.
The seven categories
What each one tests, how many questions a run includes, its weight in Overall, and who leads it today (release 2026-10-25).
| Category | What it tests | Questions | Weight | Leader |
|---|---|---|---|---|
| Long-horizon agentic | Long-running work: extending applications over several turns of changing requirements, operating an existing codebase over simulated weeks or months, and extended coding and optimisation tasks. Deterministic grading. | 18 | 2× | Claude Fable 5.1 · 77.9 |
| Coding | Bug fixes taken from real merged pull requests on a frozen codebase. Deterministic grading — a fix that breaks existing behaviour scores 0. | 29 | 1.5× | Claude Fable 5.1 · 97.7 |
| Reasoning | Short logic, spatial and theory-of-mind questions, each with one correct answer graded against ground truth. | 202 | 1× | Gemini 4 Argon · 95.1 |
| Browser | Multi-step tasks in working web applications, done through a real browser and scored as the share of tasks passed. A model that cannot accept screenshots scores 0. | 25 | 1× | GPT-6.1 Sol · 93.3 |
| Data analysis | Short questions on small tables — runs of consecutive growth, mapping one table’s columns onto another, converting between formats — each with one correct answer. | 150 | 1× | Gemini 4 Argon · 85.2 |
| Research | Open-ended research briefs that produce a written report, graded against a rubric. | 22 | 1× | Muse Wonder · 88.0 |
| Documents & design | Layout and design work across presentations, PDFs and documents, graded on visual and structural checks of the produced file. | 25 | 1.5× | Claude Opus 5.5 · 86.9 |
Note the question counts. Reasoning and data analysis are hundreds of short, cheap questions; the agentic, coding, browser, research and design categories are a few dozen long tasks each. A single failed agentic task moves its category score by several points, which is one reason to read close results as ties.
How Overall is calculated
Each category score is the mean of its suites' scores, 0–100, from a single run at the model's highest supported effort setting. Overall is the weighted mean of the categories a model has a score for, with long-horizon agentic weighted 2×, coding weighted 1.5× and documents & design weighted 1.5× — so those 3 categories make up 56% of the total.
The release notes and the leaderboard disagree
Who moved: v1 to 2.0
Models with a run in both the final v1 release (2026-06-25) and 2.0, ranked within that shared group. Scores on the two suites are not comparable; positions are.
| Model | v1 Overall | v1 rank | 2.0 Overall | 2.0 rank | Move |
|---|---|---|---|---|---|
| Claude Fable 5.1 | 83.4 | #1 | 85.7 | #1 | — |
| GPT-6 Astra | 82.2 | #3 | 85.6 | #2 | ▲ 1 |
| Claude Opus 5.5 | 83.2 | #2 | 85.1 | #3 | ▼ 1 |
| GPT-6.1 Sol | 81.6 | #4 | 84.1 | #4 | — |
| Claude Sonnet 5.5 | 77.8 | #10 | 80.0 | #5 | ▲ 5 |
| Muse Wonder | 81.2 | #6 | 74.2 | #6 | — |
| SpaceXAI: Grok 4.7 | 77.4 | #11 | 70.1 | #7 | ▲ 4 |
| Claude Haiku 5.5 | 72.1 | #14 | 69.8 | #8 | ▲ 6 |
| GPT-5.6 Terra | 77.9 | #9 | 67.8 | #9 | — |
| SpaceXAI: Grok 4.6 | 78.0 | #8 | 67.8 | #10 | ▼ 2 |
| GPT-6 Luna | 72.0 | #15 | 64.6 | #11 | ▲ 4 |
| Kimi K3 | 79.2 | #7 | 64.6 | #12 | ▼ 5 |
| Muse Spark 1.3 | 81.6 | #5 | 63.0 | #13 | ▼ 8 |
| GLM 5.3 Flash | 71.6 | #18 | 62.1 | #14 | ▲ 4 |
| Gemini 3.8 Flash | 75.8 | #12 | 59.2 | #15 | ▼ 3 |
| Qwen3.8 27B | 75.3 | #13 | 58.4 | #16 | ▼ 3 |
| Inkling | 71.9 | #16 | 47.0 | #17 | ▼ 1 |
| Mistral Large 4 | 71.8 | #17 | 43.9 | #18 | ▼ 1 |
Biggest climbers: Claude Haiku 5.5 (#14 → #8), Claude Sonnet 5.5 (#10 → #5), SpaceXAI: Grok 4.7 (#11 → #7). The new suite rewards finishing long, multi-step work, and these models do it better than their old short-question scores suggested.
Biggest drops: Muse Spark 1.3 (#5 → #13), Kimi K3 (#7 → #12), Gemini 3.8 Flash (#12 → #15). Strong answers to short questions don't carry over to hours-long agentic and browser tasks, where a single unrecoverable mistake fails the task.
The spread matters as much as the order. On v1 this group scored between 71.6 and 83.4; on 2.0 it runs from 43.9 to 85.7. A benchmark that separates models this much is more useful for choosing between them — and a gap of a few points on v1 was often noise.
Cost per task, and what it tells you
LiveBench 2.0 publishes what each run cost and how many questions it answered, per suite. Total spend over total questions gives a measured cost per task; dividing by the score gives what each point of capability costs. LiveBench shows it per successful task (×100); this site shows it per point. Same ranking, different scale.
It is the number list prices hide. Two models at the same price per token can differ several times over per finished task, because one reasons longer or retries more. The leaderboard's value frontier plots score against this measured cost for every category, and best value ranks on it directly.
What it changes if you’re choosing a model
Use the category that matches your work rather than Overall. Coding agents should read long-horizon agentic and coding; browser automation has its own browser ranking; document generation maps to design. If you relied on the old mathematics or instruction-following rankings, there is no 2.0 equivalent — reasoning is the nearest.
Then check what you'd actually pay: each model page lists cheaper alternatives on the new scores, and the coding tools comparison shows which subscriptions include the models that lead.
LiveBench 2.0 FAQ
What is LiveBench 2.0?
LiveBench 2.0 is the October 2026 relaunch of the LiveBench LLM benchmark. It replaces the original short-question suite with seven categories built around finished work — long-horizon agentic tasks, real bug fixes, browser use, research reports and document design — alongside short reasoning and data-analysis questions.
How is the LiveBench 2.0 Overall score calculated?
Overall is a weighted mean of the seven category scores. In the 25 October 2026 release, Agentic counts double and Coding and Design count 1.5×; every other category counts once. The weights are published in the release’s categories file. The release notes still say every suite counts equally, which described the earlier 14 October release.
Why did LiveBench drop Mathematics and Instruction Following?
LiveBench hasn’t published a reason. The 2.0 suite is built mostly around tasks with a finished product to grade — a working fix, a passed browser task, a report, a file — and kept only two short-question categories, reasoning and data analysis. The original suite, including mathematics and instruction following, is frozen and still published under livebench.ai/v1.
Can I compare LiveBench 2.0 scores with v1 scores?
Not directly. The tasks, categories and Overall formula all changed, so an 80 in v1 and an 80 in 2.0 mean different things. Rankings are comparable in a narrower sense: for models with a run in both, you can see how their position moved.
What does LiveBench’s “cost per successful task” mean?
It is the measured dollars LiveBench spent per task, divided by the Overall score and multiplied by 100 — roughly what one fully successful task costs. This site shows the same measure per point of score (the figure divided by 100), so the two numbers differ by a factor of 100 but rank models identically.