LLM as a judge is an evaluation method in which one language model grades the answers of another, usually against a rubric or reference answer written by people, so that open-ended outputs can be scored at scale.
The method works well on average. The paper that made it standard practice had GPT-4 grade conversational answers and found it agreed with expert human raters on more than 80 percent of non-tie cases, about as often as the experts agreed with each other.
The trouble is what a results page leaves out. A score records how a model answered. It does not record who wrote the key, which model graded, or how those two choices pushed the number.
Three parties shape every score of this kind: the people who wrote the reference answers and criteria, the model that judged, and the model that answered. Each one brings a preference. Some of those preferences can be removed by running the evaluation more carefully, and some cannot be removed at all.
Where an LLM-as-a-judge score comes from
Reference key. The answers and grading criteria that people write before evaluation begins.
Judge model. The model that compares an answer with the key and returns a verdict or score.
Answering model. The model under evaluation, whose output becomes the thing being judged.
Most teams treat the judge as the only moving part. In practice the key and the answering model move the score too, and the three interact. When a team swaps one judge for another and expects the result to become neutral, it changes only one of the three inputs.
The key carries disagreement, not only errors
An error in a key is a wrong label that someone can correct. A disagreement is different: a second, equally qualified person would have written another answer, and there is nothing to fix, only a choice about whose reading counts.
Both matter. When researchers audited ten widely used benchmarks, they found label errors in an average of at least 3.3 percent of test examples, and correcting them changed which models ranked higher. Disagreement is harder to see.
A position paper argued that much of the variation between human labelers is genuine rather than noise, and a study across seven subjective tasks found that modeling each annotator separately matched or beat a majority-vote model. Majority voting keeps the largest group’s reading and drops the rest.
The argument has limits. On tasks with one correct answer, such as physics problems, a re-annotation found the keys nearly error-free. And when a team writes a preference into its rubric on purpose, that preference is a decision, not a hidden bias. The risk sits in subjective tasks where qualified people split and the key quietly picks one side.
Which judge biases can be fixed, and which remain
Some judge biases come from how the evaluation is run. Others come from what the judge is. The first kind has procedural fixes. The second kind does not.
| Bias | What it does | Fixable by procedure? |
|---|---|---|
| Position | Favors the answer shown first | Mostly. Ask twice with answers swapped and discard verdicts that flip. One 2026 preprint still found it severe in two production judges where the fix was not applied |
| Length | Favors longer answers | Unsettled. A length-aware audit found three of five judges preferring longer answers |
| Style | Favors how an answer is written, regardless of correctness | No. The same audit measured it as the dominant bias, from 0.10 to 0.76 on a zero-to-one scale |
| Self-preference | Favors answers that resemble the judge’s own output | No. Judges can recognize their own generations, and preference rises with recognition |
| Lineage | Favors models trained on data the judge generated | No. Rewording answers before judging did not reduce it, and the judges could not even identify those answers (preference leakage study) |
Swapping the order of two answers does not stop a judge from recognizing itself. That is why changing the procedure helps with position, while the bottom three rows stay with the score until someone records where they came from.
How scoring rules teach models to guess
The answering model is not a neutral party either. Language models can express uncertainty, but researchers who examined deployed chat models found that among answers the models marked as confident, 47 percent were wrong. A benchmark built to test when models should decline to answer found that reasoning fine-tuning lowered abstention scores by 24 percent on average.
Part of the cause sits with whoever wrote the scoring rule. When an evaluation scores accuracy alone, a guess that might be right earns more than an honest “I don’t know,” which always earns zero.
Researchers trace hallucination to exactly that incentive and argue for changing how existing benchmarks grade. Whether a benchmark credits abstention is a condition a team can record next to the score, the same way it records the rubric.
The loop can close. Judges are also used for reward modeling, and reward models lean toward confident answers. A judge that grades a model can end up shaping the next one.
What to record with every judged score
A score becomes more useful when the conditions behind it travel with it. Before a team uses a judged score to choose a model, it can check five things:
- Is the task subjective, and if so, does the key keep minority readings or only the majority label?
- Which model judged, and was position bias controlled by swapping answer order?
- Does the judge share a vendor, family, or training data with any model being scored?
- Does the scoring rule give credit for an honest abstention, or only for a correct answer?
- Can someone outside the team see which reference set, judge version, and rubric produced the number?
Most leaderboards cannot answer the third question. The authors of the lineage study note that most leaderboards lack the metadata needed for that comparison. A missing lineage column proves nothing about leakage; it means nobody has examined the question.
This connects to a broader point about evaluation data. A judged score depends on the reference set and the conditions under which it was produced, the same way a benchmark ranking depends on its test set.
When the reference data or rubric changes, earlier scores stop being comparable, which is why two scores are comparable only under matching conditions. If a score drops, attributing the failure starts with knowing which of those conditions moved.
Keep the evaluation set, its version, and the conditions behind each result together so the next comparison starts from the same record. See how Syntitan compares results under the same evaluation conditions.
