Contextual retrieval

Contextual Retrieval for RAG Pipelines

• Problem Addressed: Standard document chunking often isolates text, causing individual chunks to lack the necessary context from the original source document.
• Core Concept: Contextual retrieval is a pre-processing technique designed to enrich individual text chunks before they are indexed in the retriever database.
• Retrieval Process:
• Take an individual text chunk and the source document.
• Send both to a Large Language Model (LLM) via a prompt.
• The LLM generates supplementary text to situate or add context to the chunk.
• Combine the original chunk text with the LLM-generated context to create a "contextualized chunk. "
• Use the contextualized chunk as input for the vector index and BM25 index.
• Handling Large Source Documents (Mitigation):
• If the entire source document is too large for a single prompt, do not use the whole document.
• Instead, provide a curated subset of text to the LLM:
• Starter Chunks: Chunks from the beginning of the document (to provide overall summary or abstract).
• Preceding Chunks: Chunks immediately before the target chunk (to provide local context).
• Goal: To significantly improve retrieval accuracy, especially in complex documents where sections have strong inter-ties.

Takeaways

• Contextual retrieval is a pre-processing technique that enriches individual text chunks before they are indexed in a retriever database.
• The process involves sending a text chunk and the source document to a Large Language Model (LLM) to generate supplementary context.
• The original chunk and the LLM-generated context are combined to create a "contextualized chunk" for use in vector and BM25 indexes.
• To handle large source documents, use curated subsets like "Starter Chunks" or "Preceding Chunks" instead of the entire document.
• The primary goal of this method is to significantly improve retrieval accuracy in complex documents with strong inter-ties between sections.