What is a known weakness of using an LLM as a judge to score other answers?
Options
- It cannot read text
- It is always fully deterministic
- It can be biased, for example toward longer answers, toward the first option shown or toward text that resembles its own style, so it should be calibrated against human labels (Correct option)
- It is far cheaper than any other method, so it has no downside
Answer
It can be biased, for example toward longer answers, toward the first option shown or toward text that resembles its own style, so it should be calibrated against human labels
Why
Research on LLM judges (for example the MT-Bench study) reports position bias, verbosity bias and self-preference.
Mitigations: randomize the order of answers, use clear rubrics, ask for the reasoning before the score, and check the judge's agreement with human ratings on a sample.
A judge is a useful tool for scale, not a replacement for looking at examples.
Remember
LLM judges are biased: calibrate them with humans.