Building and Evaluating Advanced RAG

Why retrieval quality is the bottleneck

A basic retrieval pipeline often returns passages that look similar to a question but lack the surrounding context an answer needs. This course focuses on two methods built for that gap, sentence-window retrieval and auto-merging retrieval, and on measuring whether they outperform a simple baseline.

Who it is for

Engineers with a retrieval-augmented prototype that gives inconsistent answers will find the most value here. The evaluation half matters as much as the methods, because the course treats pipeline improvement as a loop driven by measurements rather than intuition.

Takeaways

• Chunk size is a trade-off between how precisely you retrieve and how much context reaches the model.
• A retrieval change only counts as an improvement once it beats a baseline on the same test.
• Evaluation works best as a routine step after each change, not as a one-off check.