The full RAG flow

RAG Pipeline Study Notes

**I. RAG Pipeline Overview**

• Retrieval-Augmented Generation (RAG) combines text chunking, text embeddings, and a high-level process to generate informed answers.
• The pipeline involves two main phases: Pre-processing (indexing) and Query Time (retrieval).

**II. Pre-processing Phase (Indexing)**

• Text Chunking: The source document is broken down into smaller, manageable pieces of text (chunks).
• Text Embeddings: Each text chunk is converted into a numerical representation (a vector) using an embedding model.
• The vector captures the semantic meaning of the text.
• Normalization: This step scales the magnitude (length) of each vector to 1. 0.
• Vector Database Storage: The generated embeddings (vectors) are stored in a specialized database optimized for storing, comparing, and looking up long lists of numbers.

**III. Query Phase (Retrieval)**

• Query Embedding: The user's input query is run through the exact same embedding model used during pre-processing to create a query vector.
• Vector Search: The query vector is fed into the vector database to find the stored vectors that are most similar in nature.
• Similarity Calculation: The database uses mathematical calculations to determine the closeness between the query vector and all stored vectors.

**IV. Key Mathematical Concepts**

• Cosine Similarity:
• Calculates the cosine of the angle between the query vector and each stored embedding.
• Result is a number between negative one and one.
• A result close to one (1) indicates high similarity.
• A result close to negative one (-1) indicates low similarity.
• Cosine Distance:
• Calculated as one minus the Cosine Similarity.
• Used to provide an easier-to-interpret number.
• A result close to zero (0) indicates high similarity.

**V. Final Generation**

• Prompt Construction: The most relevant text chunk(s) retrieved from the vector database are combined with the original user query into a single prompt.
• LLM Generation: This final prompt is sent to a Large Language Model (LLM) to generate the final, informed answer.

Takeaways

• RAG pipelines convert source text into numerical vectors (embeddings) and store them in a specialized vector database.
• The user query is also converted into a vector, which is then used to search the database for the most semantically similar stored vectors.
• Similarity is calculated using Cosine Similarity, where a result close to 1 indicates high relevance between the query and the stored text.
• The final answer is generated by combining the retrieved relevant text chunks with the original query and sending them to a Large Language Model (LLM).