A support bot invents refund policies that do not exist. Which is the strongest mitigation?
Options
- Add "do not hallucinate" to the prompt
- Ground answers in the retrieved policy text, require citations, and have the bot say it does not know when the sources do not cover the question (Correct option)
- Raise the temperature so it sounds more natural
- Fine-tune it on old chat logs
Answer
Ground answers in the retrieved policy text, require citations, and have the bot say it does not know when the sources do not cover the question
Why
Telling the model not to hallucinate does not give it the facts. Grounding does: the policy text goes in the prompt, and the instruction is to answer only from it.
Requiring a citation per claim makes unsupported statements easy to spot and to check automatically. An explicit "I don't know, I will pass you to a person" path matters as much as the answers.
Fine-tuning on chats can even teach it the old mistakes.
Remember
Ground, cite, and allow "I don't know".