The full RAG flow

Core Steps of RAG

• Step 1: Text Chunking – Break source documents into separate pieces of text for processing
• Step 2: Generate Embeddings – Convert each text chunk into numerical vectors using an embedding model
• Step 3: Normalization – Scale the magnitude of each vector to 1. 0 (usually handled automatically by embedding APIs)
• Step 4: Store in Vector Database – Save embeddings in a database optimized for storing and comparing vectors
• Step 5: User Query – Accept user input and convert it to an embedding using the same model
• Step 6: Similarity Search – Find the most relevant stored embeddings using cosine similarity
• Step 7: Generate Response – Combine the user query with the most relevant text chunk and send to an LLM (e. g. , Claude)

Embeddings

• Embeddings are lists of numbers that represent the semantic meaning of text
• Each number in an embedding can represent different aspects of the text (e. g. , medical vs. software engineering topics)
• The embedding model learns these representations during training

Vector Similarity: Cosine Similarity

• Cosine Similarity measures how similar two embeddings are by calculating the angle between them
• Formula: takes the cosine of the angle between two vectors
• Range: -1 to +1
• Values close to 1 = very similar embeddings
• Values close to -1 = very dissimilar embeddings
• Used to rank which stored chunks are most relevant to the user's query

Vector Similarity: Cosine Distance

• Cosine Distance = 1 − cosine similarity
• Alternative metric often seen in vector database documentation
• Range: 0 to 2
• Values close to 0 = high similarity
• Larger values = lower similarity
• Provides an easier-to-interpret number in some contexts

Vector Database

• Specialized database optimized for storing, comparing, and retrieving vectors
• Performs fast similarity searches across many embeddings
• Returns the embedding(s) closest to a query embedding

Takeaways

• RAG converts documents into embeddings, stores them in a vector database, then retrieves the most relevant chunks to augment an LLM's response to user queries
• Embeddings are numerical vectors representing semantic meaning; cosine similarity (ranging from -1 to +1) measures how similar two embeddings are, with values near 1 indicating high relevance
• Vector databases enable fast similarity searches to find the most relevant stored text chunks for a given query embedding