LLM model comparison

Live and measured

• Release date — every model carries a publication timestamp, so new launches appear here automatically.
• Price — list price per million input and output tokens. “Blended” is a 3:1 input:output mix, the usual shape of real traffic.
• Context window and max output tokens.
• Capabilities — reasoning, tool calling, and image input, as declared by the provider.
• All seven LiveBench categories — reasoning, coding, agentic coding, mathematics, data analysis, language and instruction following, from LiveBench release —, using their own category map. Overall is the mean of the seven, which reproduces their headline number exactly. Covers 0 models.
• $ / point — LiveBench also publishes the measured dollars spent running each task. Dividing that by the score gives cost per point of capability, which neither a leaderboard nor a pricing page shows on its own.

LiveBench scores a model once per effort setting; the table shows the best published run, and hovering a score names the exact run. They rank models under that setup, not under yours.

Not shown, and why

• A score for every model. LiveBench evaluates a curated set, not the whole catalogue — 0 of 463 models here carry a score. A blank means “not evaluated,” never “bad.”
• Guessed matches. LiveBench rows are per effort setting (“claude-opus-5-xhigh-effort”). We strip the effort suffix and require an exact match to a catalogue id — no fuzzy matching, because in testing it confidently mapped Opus 5 onto Opus 4.5's score. Unmatched rows are dropped rather than guessed.
• A single “best” number. Effort setting moves these scores several points. The table shows each model's strongest published run; hover a score to see which run it was.
• Latency and throughput. The catalogue exposes these fields but returns them empty without an authenticated key.

Price and capability are facts a provider publishes. Quality is a measurement someone has to run — treat the two differently.

What does cost per point mean in an LLM comparison?

Cost per point is the measured dollar cost of running a benchmark divided by the score it earned — dollars per point of capability. A leaderboard tells you which model scores highest; a pricing page tells you what a model charges per token. Neither tells you what a point of capability actually costs, because a reasoning model can burn many times more tokens than its per-token price suggests. LiveBench publishes the measured spend for each run, so dividing that by the score gives a like-for-like value number that token pricing alone hides.

What is the cheapest LLM per million tokens?

The cheapest model in the catalogue by blended price is Mistral: Mistral Nemo at $0.0293 per million tokens, using a 3:1 input:output mix. Blended pricing matters because headline input prices understate the real bill — output tokens typically cost three to six times more than input, and reasoning models emit far more of them. A low sticker price also does not imply low cost per point; a cheap model that needs more attempts can end up costing more per unit of useful work.

How do LiveBench scores compare to price?

They correlate weakly, which is the whole reason this page exists. Sorting by score gives you a ranking; sorting by cost per point usually gives a very different one. This page joins both — LiveBench scores across reasoning, coding, agentic coding, mathematics, data analysis, language and instruction following, against list pricing and the measured spend behind each run — so you can sort by either and see how far apart they are.

How often is this LLM comparison updated?

The model catalogue refreshes every 15 minutes from OpenRouter, so new releases and price changes appear without a redeploy — 36 models were added in the last 30 days. Benchmark scores update when LiveBench publishes a new release; the current one is pending, covering 0 of 463 models.

Why do some models have no benchmark score?

LiveBench evaluates a curated set rather than every model that exists — 0 of 463 models here carry a score. A blank means "not evaluated", never "bad". Scores are also matched to catalogue ids by exact match only, with the effort suffix stripped; fuzzy matching was rejected because in testing it confidently mapped a new model onto an older version's score. Unmatched rows are dropped rather than guessed.