Overview

0:00 in this course Lance Martin will teach you how to implement rag from scratch Lance is a software engineer at Lang chain and Lang chain is one of the most common ways to implement rag Lance will help you understand how to use rag to combine custom data with llms hi this is Lance Martin I'm a software engineer at Lang chain I'm going to be giving a short course focused on rag or retrieval augmented generation which is one of the most popular kind of ideas and in llms today so really the motivation for this is that most of the world's data is private um whereas llms are trained on publicly available data so you can kind of see on the bottom on the x-axis the number of tokens using pre-training various llms so it kind of varies from say 1.5 trillion tokens in the case of smaller models like 52 out to some very large number that we actually don't know for proprietary models like GPT 4 CLA three but what's really interesting is that the context window or the ability to feed external information into these LMS is actually getting larger so about

1:11 a year ago context windows were between 4 and 8,000 tokens you know that's like maybe a dozen pages of text we've recently seen models all the way out to a million tokens which is thousands of pages of text so while these llms are trained on large scale public data it's increasingly feasible to feed them this huge mass of private data that they've never seen that private data can be your kind of personal data it can be corporate data or you know other information that you want to pass to an LM that's not natively in his training set and so this is kind of the main motivation for rag it's really the idea that llms one are kind of the the center of a new kind of operating system and two it's increasingly critical to be able to feed information from external sources such as private data into llms for processing so that's kind of the overarching motivation for Rag and now rag refers to retrieval augmented generation and you can think of it in three very general steps there's a process of indexing of external data so you can think about

2:22 this as you know building a database for example um many companies already have large scale databases in different forms they could be SQL DBS relational DBS um they could be Vector Stores um or otherwise but the point is that documents are indexed such that they can be retrieved based upon some heuristics relative to an input like a question and those relevant documents can be passed to an llm and the llm can produce answers that are grounded in that retrieved information so that's kind of the centerpiece or central idea behind Rag and why it's really powerful technology because it's really uniting the the knowledge and processing capacity of llms with large scale private external data source for which most of the important data in the world still lives and in the following short videos we're going to kind of build up a complete understanding of the rag landscape and we're going to be covering a bunch of interesting papers and techniques that explain kind of how to do rag and I've really broken it down into a few different sections so starting with a question on the left

3:32 the first kind of section is what I call query trans translation so this captures a bunch of different methods to take a question from a user and modify it in some way to make it better suited for retrieval from you know one of these indexes we've talked about that can use methods like query writing it can be decomposing the query into you know constituent sub questions then there's a question of routing so taking that decomposed a Rewritten question and routing it to the right place you might have multiple Vector stores a relational DB graph DB and a vector store so it's the challenge of getting a question to the right Source then there's a there's kind of the challenge of query construction which is basically taking natural language and converting it into the DSL necessary for whatever data source you want to work with a classic example here is text a SQL which is kind of a very kind of well studied process but text a cipher for graph DV is very interesting text to metadata filters for Vector DBS is also a very big area of study um then there's indexing so that's the process of taking your documents and

4:42 processing them in some way so they can be easily retrieved and there's a bunch of techniques for that we'll talk through we'll talk through different embedding methods we'll talk about different indexing strategies after retrieval there are different techniques to rerank or filter retrieve documents um and then finally we'll talk about generation and kind of an interesting new set of methods to do what we might call as active rag so in that retrieval or generation stage grade documents grade answers um grade for relevance to the question grade for faithfulness to the documents I.E check for hallucinations and if either fail feedback uh re- retrieve or rewrite the question uh regenerate the qu regenerate the answer and so forth so there's a really interesting set of methods we're going to talk through that cover that like retrieval and generation with feedback and you know in terms of General outline we'll cover the basics first it'll go through indexing retrieval and generation kind of in the Bare Bones and then we'll talk through more advanced techniques that we just saw on the prior slide career Transformations routing uh construction