Users of a parts catalogue search for codes like "XK-4471-B", and vector search ranks the right page poorly. What helps most?
Options
- A higher temperature
- A longer system prompt
- Bigger chunks
- Hybrid search: combine keyword search (such as BM25) with vector search (Correct option)
Answer
Hybrid search: combine keyword search (such as BM25) with vector search
Why
Embeddings capture meaning, not exact strings: codes, names and rare words are often blurred together.
Keyword search matches exact tokens very well but misses paraphrases. Combining both (and merging the two rankings) covers both kinds of query.
Many vector databases and search engines offer hybrid search directly.
Remember
Exact terms need keyword search; combine it with vectors.