A Multi-Index RAG pipeline

Architecture Overview

• Two search implementations (semantic/vector search and lexical/BM25 search) have nearly identical public APIs
• Both have add_document() and search() methods
• A new Retriever class wraps both implementations into a single search pipeline

Retriever Class Design

• Receives user questions and forwards them to both vector index and BM25 index
• Collects results from both search methods
• Merges results using a combination technique

Reciprocal Rank Fusion (RRF)

• Technique for merging results from different search methodologies
• Process:
• Create a table with all unique results from both searches
• Record the rank (position) each result received from each search method
• Apply formula for each result: sum of (1 / (1 + rank)) for each search method
• Sort results by combined score (highest to lowest)
• Example: A result ranked 1st in vector search and 2nd in BM25 scores higher than one ranked 2nd and 3rd

Benefits of Hybrid Approach

• Combines strengths of semantic and lexical search
• Improves result relevance compared to single search method alone
• Modular design allows easy addition of new search implementations
• Any new search method only needs add_document() and search() methods to integrate

Practical Improvement

• Previous vector-only search for "what happened with incident 2023" returned irrelevant results
• Hybrid search now returns more relevant results in correct order

Takeaways

• Hybrid search combines vector search and BM25 lexical search through a Retriever class that forwards queries to both and merges results
• Reciprocal Rank Fusion (RRF) merges results by calculating a combined score for each result based on its rank position in each search method, then sorting by highest score
• Hybrid approach improves relevance over single search methods by combining semantic and lexical strengths, with modular design allowing easy addition of new search implementations