A company wants an assistant that answers from internal documents that change every week. What is the best first approach?
Options
- Fine-tune a model every week on the new documents
- Use the largest available model and hope it knows
- Paste all the documents into every prompt
- RAG: index the documents and refresh the index when they change (Correct option)
Answer
RAG: index the documents and refresh the index when they change
Why
With RAG, a change in a document only needs that document to be re-indexed, which is fast and cheap, and answers can show their source.
Fine-tuning is slow, costly to repeat and is poor at storing exact facts. Pasting everything is too big for the window and for the budget.
A bigger model does not know your private documents at all.
Remember
Changing knowledge: RAG, not fine-tuning.