What is the overall function of the chunking step in a RAG pipeline?	To break a source document into smaller, manageable pieces of text that can be retrieved and used in a prompt.
Why is the process of chunking considered one of the most complex steps in a RAG pipeline?	Because the method used to chunk the document has a huge impact on the quality of the final RAG output.
Define size-based chunking.	Dividing a large document into strings of text that are of approximately equal length.
What are the two primary drawbacks of simple size-based chunking?	Chunks often contain cut-off words, and they typically lack the surrounding context.
How does implementing an overlap strategy improve size-based chunking?	It includes a portion of text from neighboring chunks, providing the individual chunk with more context.
Define structure-based chunking.	Dividing text based on the inherent structural elements of a document, such as headers, paragraphs, or sections.
What is a major limitation of structure-based chunking?	It is difficult or impossible to implement reliably when the input documents lack predictable formatting (e.g., plain PDFs).
What is the core principle of semantic-based chunking?	Using natural language processing (NLP) to group consecutive sentences or sections based on how related their meaning is.
What factors determine which chunking strategy should be chosen for a project?	The specific use case and the guarantees available regarding the structure and content of the documents being ingested.
What are the default chunk settings for text processing?	A chunk length of 150 with an overlap of 20.
How can a user improve the quality of text chunks?	By dramatically changing the default settings, such as increasing chunk length and overlap.
What is the "chunk by sentence" strategy?	A strategy that uses a regular expression to split text, typically resulting in five sentences per chunk with one sentence of overlap.
When is the "chunk by section" strategy most effective?	When the document has a guaranteed structure, such as an executive summary or table of contents.
Why might the "chunk by section" strategy fail?	If the document is user-provided and there are no guarantees regarding its structure.
What is the "chunk by character" strategy?	A reliable fallback method that is not guaranteed to produce the best results but works in the vast majority of cases.
What is the primary factor determining which chunking strategy to use?	The nature of the document and the guarantees available regarding its structure.
Why is chunking code by sentence often problematic?	Because code frequently contains periods in unexpected places, leading to incorrect sentence splits.
What is the purpose of the overlap in chunk settings?	To ensure that complex phrases or information spanning multiple chunks are captured.
If a document lacks guaranteed structure, which strategy is generally preferred over "chunk by section"?	Chunk by sentence (or potentially chunk by character).