1 00:00:00,000 --> 00:00:01,000 Hello guys. 2 00:00:01,000 --> 00:00:06,000 So in this video we are going to discuss about the entire line chain ecosystem. 3 00:00:06,000 --> 00:00:12,000 Now along with the line chain ecosystem, we first of all we really need to understand this two important 4 00:00:12,000 --> 00:00:12,000 question. 5 00:00:13,000 --> 00:00:17,000 The first question is why lang chain okay. 6 00:00:17,000 --> 00:00:20,000 Why we really need to learn about this framework. 7 00:00:21,000 --> 00:00:22,000 Why lang chain okay. 8 00:00:23,000 --> 00:00:29,000 Along with this, the second thing that we are going to discuss about in this video is about the entire 9 00:00:29,000 --> 00:00:33,000 Lange chain ecosystem, right? 10 00:00:33,000 --> 00:00:38,000 So our entire video we will be discussing about this two important points. 11 00:00:38,000 --> 00:00:46,000 And let me tell you guys, Lange chain right now is the most common framework that is specifically used 12 00:00:46,000 --> 00:00:48,000 to build generative AI application. 13 00:00:48,000 --> 00:00:49,000 Okay. 14 00:00:49,000 --> 00:00:56,000 And in order to make you understand why Lange chain, let me just go ahead and talk about a couple of 15 00:00:56,000 --> 00:00:58,000 years back what had happened, right? 16 00:00:58,000 --> 00:00:59,000 Or one and a half years back. 17 00:00:59,000 --> 00:01:03,000 So I hope everybody knows about this OpenAI company. 18 00:01:03,000 --> 00:01:03,000 Right. 19 00:01:03,000 --> 00:01:04,000 So OpenAI company is there. 20 00:01:04,000 --> 00:01:07,000 And obviously we also had hugging face. 21 00:01:07,000 --> 00:01:07,000 Right. 22 00:01:07,000 --> 00:01:15,000 And these were the first two companies that probably came up with the concept of LLM models, large 23 00:01:15,000 --> 00:01:16,000 language models. 24 00:01:16,000 --> 00:01:16,000 Right. 25 00:01:16,000 --> 00:01:22,000 It was not like uh, OpenAI or Hugging Face came at first, but they were also Google who were working 26 00:01:22,000 --> 00:01:24,000 with open source models. 27 00:01:24,000 --> 00:01:25,000 They were in hugging face. 28 00:01:25,000 --> 00:01:30,000 There are a lot of different, different open source model, but the popularity of LM models and generative 29 00:01:30,000 --> 00:01:35,000 AI apps started when OpenAI came up with its own LM model, right? 30 00:01:35,000 --> 00:01:37,000 Like GPT 3.5 turbo. 31 00:01:37,000 --> 00:01:38,000 All those kinds of models were there. 32 00:01:39,000 --> 00:01:45,000 Now, when, uh, when we started, when when this companies announced about the LM models, uh, many 33 00:01:45,000 --> 00:01:46,000 of the developers started exploring. 34 00:01:46,000 --> 00:01:47,000 Right. 35 00:01:47,000 --> 00:01:52,000 And with respect to the documentation open, I specifically had a different set of code. 36 00:01:52,000 --> 00:01:57,000 And what we do with this LM model, if I'm providing any input, I will be able to get any output in 37 00:01:57,000 --> 00:01:59,000 form of any kind of response. 38 00:01:59,000 --> 00:01:59,000 Okay. 39 00:01:59,000 --> 00:02:04,000 Now when I, when we were using OpenAI hugging face, uh, specifically, if I talk about hugging face, 40 00:02:04,000 --> 00:02:05,000 right. 41 00:02:05,000 --> 00:02:09,000 We use different, different, uh, libraries, some of the libraries that we used, one of the most 42 00:02:09,000 --> 00:02:11,000 common libraries that we use are Transformers. 43 00:02:11,000 --> 00:02:15,000 Now in Transformers we have something called as pipeline, right. 44 00:02:15,000 --> 00:02:19,000 And with the help of pipeline, we are able to call any model that are available in the hugging face, 45 00:02:19,000 --> 00:02:19,000 right? 46 00:02:19,000 --> 00:02:22,000 Call it in a local developer generative AI application. 47 00:02:22,000 --> 00:02:26,000 Similarly, in the case of OpenAI, uh, you know, we really need the API key. 48 00:02:26,000 --> 00:02:30,000 And based on this, OpenAI, I have also created an amazing documentation where you can probably do 49 00:02:30,000 --> 00:02:33,000 fine tuning, you can create a generative AI application and all. 50 00:02:33,000 --> 00:02:37,000 And later on when more companies came into this race, right. 51 00:02:37,000 --> 00:02:40,000 Let's say meta with llama three, right. 52 00:02:40,000 --> 00:02:41,000 Google. 53 00:02:41,000 --> 00:02:43,000 They it came up with this Google Gemini Pro. 54 00:02:44,000 --> 00:02:49,000 Now when this all models were specifically coming up different different set of libraries were getting 55 00:02:49,000 --> 00:02:52,000 created so that we will be able to access this particular models. 56 00:02:52,000 --> 00:02:58,000 Now again so many different models are there, both paid open source, uh, you know, uh, you can 57 00:02:58,000 --> 00:03:01,000 also use these open source for commercial purpose. 58 00:03:01,000 --> 00:03:06,000 Now, what Lang Chen did is that Lang Chen, as a company, it created a common framework. 59 00:03:06,000 --> 00:03:06,000 Right. 60 00:03:06,000 --> 00:03:10,000 And as I said, Lang Chen is like a common framework. 61 00:03:10,000 --> 00:03:11,000 I'll write it down over here. 62 00:03:11,000 --> 00:03:12,000 Okay. 63 00:03:12,000 --> 00:03:13,000 So what did Lang Chen do? 64 00:03:13,000 --> 00:03:17,000 It created this amazing common framework. 65 00:03:17,000 --> 00:03:24,000 It is a framework to develop to develop gen AI application. 66 00:03:25,000 --> 00:03:27,000 Gen AI app. 67 00:03:27,000 --> 00:03:30,000 Okay, now, with the help of this particular framework. 68 00:03:30,000 --> 00:03:38,000 Right, uh, we were able to access like let it be open AI hugging face or Llama three, Google Gemini 69 00:03:38,000 --> 00:03:40,000 Pro, whatever models you specifically want. 70 00:03:40,000 --> 00:03:45,000 So long chain as a library provided all those specific features, right. 71 00:03:45,000 --> 00:03:49,000 So they were libraries getting integrated and this entire long chain, right. 72 00:03:49,000 --> 00:03:54,000 If I probably talk about this long chain, all the modules inside this long chain are open source. 73 00:03:54,000 --> 00:04:00,000 Right now when you have something like open source, definitely it will be very much in ease to create 74 00:04:00,000 --> 00:04:02,000 generative AI application. 75 00:04:02,000 --> 00:04:02,000 Right? 76 00:04:02,000 --> 00:04:07,000 And right now, probably each and every companies, if you even go for interviews, they're specifically 77 00:04:07,000 --> 00:04:12,000 going to ask you about like how you can actually build with the help of generative AI application. 78 00:04:12,000 --> 00:04:13,000 Now this is fine, right? 79 00:04:13,000 --> 00:04:14,000 Long chin was there. 80 00:04:14,000 --> 00:04:18,000 and with the help of Lang Chin we were able to generate generate the applications. 81 00:04:18,000 --> 00:04:22,000 But later on, you know more and more modules were basically getting added. 82 00:04:22,000 --> 00:04:25,000 And that is what we are going to discuss now about the entire Lang chain ecosystem. 83 00:04:25,000 --> 00:04:30,000 Because now the question arises if I probably go ahead and create this application. 84 00:04:30,000 --> 00:04:31,000 Right. 85 00:04:31,000 --> 00:04:36,000 So let's say that if I go ahead and create the application here, we may be using any paid open source 86 00:04:36,000 --> 00:04:40,000 tools open source model, LM model, LM models or multi model. 87 00:04:40,000 --> 00:04:40,000 Right. 88 00:04:40,000 --> 00:04:45,000 But the main task is that after creating this particular model, how do I make sure? 89 00:04:45,000 --> 00:04:49,000 How do I perform some of the activities that we specifically do with LM ops? 90 00:04:49,000 --> 00:04:55,000 Now what all activities we do in this, let's say in this we specifically do something like debugging, 91 00:04:55,000 --> 00:04:55,000 right? 92 00:04:55,000 --> 00:04:58,000 Uh, we want to probably do the evaluation. 93 00:04:58,000 --> 00:04:59,000 Right. 94 00:04:59,000 --> 00:05:01,000 We want to do monitoring. 95 00:05:01,000 --> 00:05:03,000 How do we do specifically do with this. 96 00:05:03,000 --> 00:05:03,000 Right. 97 00:05:03,000 --> 00:05:05,000 So long chain. 98 00:05:05,000 --> 00:05:09,000 What they did is that they started creating this entire ecosystem. 99 00:05:09,000 --> 00:05:10,000 Right. 100 00:05:10,000 --> 00:05:16,000 So for this LM ops activities like debugging, evaluation, playground monitoring and all, it created 101 00:05:16,000 --> 00:05:20,000 a new module which is basically called as Lang Smith write. 102 00:05:20,000 --> 00:05:24,000 And this Lang Smith is specifically used for this purpose. 103 00:05:24,000 --> 00:05:24,000 Right. 104 00:05:24,000 --> 00:05:26,000 The LM ops part. 105 00:05:26,000 --> 00:05:26,000 Right. 106 00:05:26,000 --> 00:05:30,000 So, uh, it's not like we just use the library. 107 00:05:30,000 --> 00:05:31,000 We develop a generative AI application. 108 00:05:31,000 --> 00:05:32,000 But in short, what? 109 00:05:32,000 --> 00:05:38,000 Lang Chen, what is the main thing that it did is that it created that entire ecosystem wherein it is 110 00:05:39,000 --> 00:05:39,000 integrated. 111 00:05:39,000 --> 00:05:43,000 This additional module, which is called as Lang Smith and through which you will be able to probably 112 00:05:43,000 --> 00:05:47,000 do each and every thing which I have written over here, like debugging, playground evaluation, annotation, 113 00:05:47,000 --> 00:05:48,000 monitoring and many more. 114 00:05:48,000 --> 00:05:49,000 Right. 115 00:05:49,000 --> 00:05:53,000 Then came up the next module, which is called as Lang sir. 116 00:05:53,000 --> 00:05:59,000 Now Lang sir, obviously if we are able to do this, finally, we really need to deploy this application. 117 00:05:59,000 --> 00:06:03,000 Now to deploy this application, we really need to convert this into in the form of APIs. 118 00:06:03,000 --> 00:06:09,000 Right now, this APIs can probably be converted by using another module which is called as lang serve. 119 00:06:09,000 --> 00:06:13,000 So lang serve actually helps you to create chain as rest API. 120 00:06:13,000 --> 00:06:14,000 So it is written over here. 121 00:06:14,000 --> 00:06:17,000 It actually helps you to create this chain as Rest APIs. 122 00:06:17,000 --> 00:06:20,000 So now is the main thing right now understand the entire mechanism. 123 00:06:20,000 --> 00:06:22,000 Why I'm saying this as a ecosystem. 124 00:06:22,000 --> 00:06:22,000 Right. 125 00:06:22,000 --> 00:06:24,000 First you had long chain. 126 00:06:24,000 --> 00:06:29,000 Now with the help of long chain by using concepts like chain agents, retrieval strategies which we 127 00:06:29,000 --> 00:06:30,000 will discuss. 128 00:06:30,000 --> 00:06:34,000 Because in this course I'm going to cover each and every thing that is available in this ecosystem. 129 00:06:34,000 --> 00:06:35,000 Right. 130 00:06:35,000 --> 00:06:39,000 And then finally we deploy after creating this particular APIs, we deploy it in different different 131 00:06:39,000 --> 00:06:43,000 clouds like AWS for will focus more on AWS. 132 00:06:43,000 --> 00:06:46,000 Then there is also another cloud, which is called as hugging face space, right? 133 00:06:46,000 --> 00:06:51,000 Which will be available to everyone, probably completely for free for some space for some resources. 134 00:06:51,000 --> 00:06:52,000 Right. 135 00:06:52,000 --> 00:06:58,000 So we'll also be seeing with respect to this particular deployment right now, uh, initially Lang Chain 136 00:06:58,000 --> 00:07:00,000 came we are able to generate generative application. 137 00:07:00,000 --> 00:07:02,000 Now in this Lang chain since we had so many different models. 138 00:07:02,000 --> 00:07:07,000 We have models from OpenAI, we have models from Google Gemini Pro, we have models from hugging face. 139 00:07:07,000 --> 00:07:09,000 That is the reason it came up with this. 140 00:07:09,000 --> 00:07:13,000 Another model which is called as Long Chain Community, which has access to all this kind of models. 141 00:07:13,000 --> 00:07:14,000 Yes. 142 00:07:14,000 --> 00:07:17,000 For the paid one you really need to have a key for open source. 143 00:07:17,000 --> 00:07:21,000 You can directly use hugging face, you know, or any other things like Meta Llama three and all right 144 00:07:21,000 --> 00:07:24,000 then over here prompt engineering is also probably done. 145 00:07:24,000 --> 00:07:27,000 Example selectors is a concept which you will be learning about. 146 00:07:27,000 --> 00:07:31,000 Then you have something like output parser like how should the response come after. 147 00:07:31,000 --> 00:07:36,000 We probably input uh if we give an input to our LLM LM model, how if we really want to customize the 148 00:07:36,000 --> 00:07:37,000 output, how do we do that? 149 00:07:37,000 --> 00:07:38,000 We'll be discussing about that. 150 00:07:38,000 --> 00:07:41,000 Then we'll be discussing about retrieval document loader. 151 00:07:41,000 --> 00:07:43,000 Like they are different different data ingestion techniques. 152 00:07:43,000 --> 00:07:48,000 We can load data from amazing different different data sources which we'll be discussing about this. 153 00:07:48,000 --> 00:07:49,000 Right. 154 00:07:49,000 --> 00:07:51,000 And it will be a detailed videos when we'll be discussing. 155 00:07:51,000 --> 00:07:52,000 Right. 156 00:07:52,000 --> 00:07:55,000 Then we have Vector Store, then we have text splitter. 157 00:07:55,000 --> 00:07:57,000 Now what exactly is this vector store? 158 00:07:57,000 --> 00:08:00,000 See, at the end of the day when we are working with linear models, that basically means we are working 159 00:08:00,000 --> 00:08:01,000 with text, right? 160 00:08:01,000 --> 00:08:08,000 And when we are creating this kind of application called as rag or document Q&A or chat bot Q&A, uh, 161 00:08:08,000 --> 00:08:12,000 there may be scenario where I will be having a huge amount of data, and for that particular data, 162 00:08:12,000 --> 00:08:14,000 I really need to convert into a vectors, right? 163 00:08:14,000 --> 00:08:18,000 For converting into a vectors, we will be specifically using different different embedding techniques. 164 00:08:18,000 --> 00:08:18,000 Right. 165 00:08:18,000 --> 00:08:20,000 And after we convert that into a vectors right. 166 00:08:20,000 --> 00:08:24,000 Because uh, the reason we convert into a vector is because with the help of linear models, whenever 167 00:08:24,000 --> 00:08:29,000 I try to query it right, it tries to apply cosine similarity and it will be able to retrieve the results. 168 00:08:29,000 --> 00:08:34,000 So, uh, after I probably convert into a vectors, how to store it in a vector database will also be 169 00:08:34,000 --> 00:08:35,000 discussing about that. 170 00:08:35,000 --> 00:08:40,000 We'll be discussing about various data vector databases that are available, uh, within the Lang, 171 00:08:40,000 --> 00:08:41,000 Smith Lang chain itself. 172 00:08:42,000 --> 00:08:43,000 We'll be discussing about that. 173 00:08:43,000 --> 00:08:49,000 So as an ecosystem, we'll just not be building generative AI application, but instead we'll be thinking 174 00:08:49,000 --> 00:08:52,000 about the entire life cycle of a gene AI project. 175 00:08:52,000 --> 00:09:07,000 So here our entire focus will be on life cycle, life cycle, life cycle of Gen AI project. 176 00:09:08,000 --> 00:09:13,000 We will be focusing on this entire life cycle which will be including Lange, Smith, Lange, serve 177 00:09:13,000 --> 00:09:16,000 and along with that, some other default templates are also available. 178 00:09:16,000 --> 00:09:18,000 We will be discussing about that. 179 00:09:18,000 --> 00:09:20,000 So I hope I was able to answer this. 180 00:09:20,000 --> 00:09:21,000 Two important things. 181 00:09:21,000 --> 00:09:25,000 While Lange chain and some of the things with respect to Lange chain ecosystem. 182 00:09:25,000 --> 00:09:29,000 Now you'll be seeing that in the upcoming tutorials that we are going to do in this course. 183 00:09:29,000 --> 00:09:31,000 Uh, we'll be discussing about all these modules. 184 00:09:31,000 --> 00:09:33,000 We'll be breaking it down, we'll be developing end to end projects. 185 00:09:33,000 --> 00:09:35,000 We'll be doing many things as such. 186 00:09:35,000 --> 00:09:35,000 Right. 187 00:09:35,000 --> 00:09:37,000 So I hope you like this particular video. 188 00:09:37,000 --> 00:09:38,000 This was it from my side. 189 00:09:38,000 --> 00:09:39,000 I will see you all in the next video. 190 00:09:39,000 --> 00:09:40,000 Thank you.