Using an LLM as a Judge
Learn when and how to use a model to grade another model's output at scale, and the failure modes to guard against.
Why use a model to grade a model
Human review is the gold standard for nuanced quality but is slow and expensive to run on every change or every day of production traffic. An LLM judge — a model prompted specifically to evaluate an output against criteria or a reference — can approximate human judgment at a fraction of the cost and latency, making it practical to run continuously rather than occasionally.
Writing a judge prompt that actually discriminates
A vague judge prompt ('rate this response from 1–10') tends to cluster scores near the top and fails to catch real problems. A useful judge prompt gives specific criteria to check, asks for a decision per criterion rather than one holistic number, and — where possible — provides a reference answer or grounding source so the judge isn't grading from its own possibly-wrong beliefs.
Calibrating the judge against humans
Before trusting an LLM judge's scores, validate them: have humans grade a sample of the same outputs and check agreement between judge and human scores. A judge that disagrees with human graders on a meaningful fraction of cases needs a better prompt, a different model, or should be limited to the dimensions where it does agree, rather than trusted wholesale.
Known failure modes of LLM judges
Judges can favor longer or more confidently-worded answers regardless of correctness, can share the same blind spots as the model being graded (missing the same factual gap), and can be inconsistent across runs if temperature isn't controlled. Mitigate with clear per-criterion rubrics, held-out calibration checks, and by combining judge scores with deterministic checks rather than relying on the judge alone for hard constraints.
Position bias and self-preference
When a judge compares two outputs side by side, its verdict can shift depending on which one is shown first (position bias), and a judge from the same model family as one of the outputs being compared can show a measurable preference for its own family's style. Randomize presentation order and, for high-stakes comparisons, use a judge model different from either candidate being evaluated.
Practical exercise
Write a judge prompt with three specific pass/fail criteria for one of your eval examples. Run it against 10 real outputs, then grade the same 10 by hand. Calculate how often the judge and your own grading agreed, and revise the judge prompt to fix the biggest disagreement you find.
Sources and further reading
These primary or specialist references informed the concepts in this guide. Product details can change, so verify current documentation before implementation.