Contextual retrieval

Contextual Retrieval for RAG Pipeline Improvement

• Core Problem: Standard chunking breaks the context of a source document, meaning individual text chunks lack the necessary surrounding information.
• Solution (Contextual Retrieval): A pre-processing step is added before indexing chunks to enrich them with contextual information.
• Process Flow:
• Take an individual text chunk and the original source document.
• Send both to an LLM (e. g. , Claude) via a prompt.
• The LLM generates additional text to situate or provide context for the chunk within the larger document.
• Combine the generated context with the original chunk text → Contextualized Chunk.
• Index the Contextualized Chunk in the vector index and BM25 index.
• Handling Large Source Documents: If the entire source document is too large for a single prompt, a targeted subset of text is used instead of the whole document.
• Strategy: Include starter chunks (e. g. , abstracts/summaries) $\text{AND}$ chunks immediately preceding the target chunk.
• Goal: Significantly reduce input size while maintaining relevant context.
• Benefit: This technique improves the accuracy of the RAG pipeline, especially when the source document has strong ties or connections between different sections.

Takeaways

• Contextual Retrieval addresses the loss of surrounding information in standard chunking by enriching text chunks before indexing.
• The process uses an LLM to generate additional context, which is then combined with the original chunk to create a Contextualized Chunk.
• For large source documents, targeted subsets (such as starter chunks or preceding chunks) are used to maintain context while reducing input size.
• This technique significantly improves the accuracy of the RAG pipeline, especially for documents with strong internal connections.