You change the embedding model of your RAG system for a better one. What must you do to the existing index?
Options
- Nothing: only the queries need the new model
- Reduce the temperature
- Convert the old vectors with cosine similarity
- Re-embed the whole corpus with the new model, because vectors from different models are not comparable (and may have a different size) (Correct option)
Answer
Re-embed the whole corpus with the new model, because vectors from different models are not comparable (and may have a different size)
Why
Each embedding model defines its own space. A vector from one model has no meaningful distance to a vector from another, and the dimensions may even differ.
So the documents and the queries must be embedded with the same model and version. Plan a migration: build a new index in parallel, compare quality with your retrieval evals, then switch.
Store the model name and version with each vector so mixed indexes are easy to spot.
Remember
Change the embedding model, re-embed everything.