Building Hybrid BM25 + Vector Search for RAG
Why vector-only retrieval fails on research papers, and how BM25, Reciprocal Rank Fusion and a CPU cross-encoder fix it.
By Rahul Gupta · RahulX Labs
Closest in vector space is not best answer
Research papers are full of acronyms and notation. Embedding similarity alone misses exact term matches and can surface “right words, wrong answer” chunks.
The Research Paper RAG system retrieves a wide set from Qdrant (top-30), then applies BM25 keyword ranking and Reciprocal Rank Fusion before a MiniLM cross-encoder narrows to top-5.
Why RRF instead of score normalisation
BM25 and cosine live on incompatible scales. Reciprocal Rank Fusion combines rankings without pretending the scores are comparable.
The whole rerank funnel adds ~400ms on CPU — no GPU required — which is acceptable for a research assistant API.
Shared vectors for multi-user products
One PDF becomes one embedding set. Users are appended to a payload user_ids list. Delete is access revocation; vectors are removed only when no owners remain. A DB conditional lock prevents duplicate embedding on concurrent upload of the same hash.