Evaluating LLM & Agent Systems
You can't improve what you don't measure — and you can't ship an agent you can't regression-test. This is how to evaluate LLM and agent systems: the methods, the metrics that matter, and treating evals as CI rather than a launch-day afterthought.
Four ways to measure quality
Use them together — offline sets for regression, judges for scale, humans for ground truth, and online metrics for reality.
Offline eval sets
A fixed, versioned set of inputs with expected properties. Run it on every prompt, model, or retrieval change — the backbone of everything else.
LLM-as-judge
A model scores outputs against a rubric (faithfulness, helpfulness, tone). Cheap and scalable, but calibrate it against human labels and watch for bias.
Human review
Expert spot-checks on a sample, especially for regulated or high-risk actions. The ground truth that keeps automated judges honest.
Online metrics
Production signals: task success, override/edit rate, thumbs, latency, cost, escalation, and complaint rate — real behavior, not a lab.
What to track, by layer
Retrieval (RAG)
- recall@k
- MRR / nDCG
- context precision
Generation
- faithfulness / groundedness
- answer relevance
- toxicity / policy pass
Agents
- task success rate
- tool-call accuracy
- steps to completion
- override rate
Operations
- p95 latency
- cost per task
- escalation rate
Evals as CI, not a launch checklist
Curate an eval set
Real prompts + edge cases + past failures, each with expected properties or a rubric. Version it in the repo.
Score
Run automated judges + metrics on every change; sample a slice for human review.
Gate the release
Block deploys that regress on faithfulness, task success, or policy pass — evals are CI, not a one-off.
Watch production
Capture online metrics and thumbs; the worst cases feed straight back into the eval set.
Evaluation anti-patterns
Shipping on vibes — a good demo is not an eval.
Trusting an LLM judge you never calibrated against humans.
Evaluating generation while retrieval is silently broken.
A static eval set that never absorbs new production failures.