Why are documents split into chunks before they are embedded for a RAG system?
Options
- A short, focused chunk gets a more precise vector, and only the relevant passages need to go into the prompt (Correct option)
- Embedding models can only read one word at a time
- Chunks make the model train faster
- Chunking hides private information
Answer
A short, focused chunk gets a more precise vector, and only the relevant passages need to go into the prompt
Why
One vector for a whole 50-page document is a blurry average of everything in it, and the document may not even fit in the model.
With chunks, search returns the few passages that match the question, and the prompt stays short, cheap and focused.
Chunk size is a trade-off: too small loses context, too large mixes topics.
Remember
Chunks make retrieval precise and prompts short.