A RAG answer is correct, but the citation points to a passage that does not support the claim. Which kind of check would catch this?
Options
- Faithfulness (groundedness): does every claim follow from the cited text? (Correct option)
- Retrieval recall@k
- Average answer length
- Latency per request
Answer
Faithfulness (groundedness): does every claim follow from the cited text?
Why
Recall@k tells you whether the right passage was retrieved. It cannot tell whether the answer is supported by what the model was given.
A faithfulness check compares each claim in the answer with the cited passages. A claim that is true but not supported by the sources is still a problem, because the user cannot verify it and it may be luck.
It can be done by humans on a sample, or by a calibrated judge model at scale.
Remember
Check that claims follow from the cited sources.