1
00:00:00,000 --> 00:00:05,000
So guys, in our previous video we have actually done this project till creating this vector store DB.

2
00:00:05,000 --> 00:00:12,000
Now let's go ahead and see that how with the help of similarity search, we will be able to query anything

3
00:00:12,000 --> 00:00:13,000
from that specific page.

4
00:00:13,000 --> 00:00:15,000
Uh, inside this vector DB.

5
00:00:15,000 --> 00:00:16,000
Like let's say that I am a user.

6
00:00:16,000 --> 00:00:19,000
I want any content from that particular website.

7
00:00:19,000 --> 00:00:22,000
So I obviously want to query the vector store DB.

8
00:00:22,000 --> 00:00:24,000
And it should be able to give me the response.

9
00:00:24,000 --> 00:00:26,000
So first of all I'll go ahead and create a query.

10
00:00:26,000 --> 00:00:29,000
So in this query what I will do I will just go over here.

11
00:00:29,000 --> 00:00:34,000
Let's say I will be selecting in this particular page something like this okay.

12
00:00:34,000 --> 00:00:38,000
And uh let's see we will use okay.

13
00:00:38,000 --> 00:00:40,000
Let's go ahead and do this okay.

14
00:00:40,000 --> 00:00:41,000
And extend it something.

15
00:00:41,000 --> 00:00:45,000
I'll just go ahead and ask this particular question from here okay.

16
00:00:45,000 --> 00:00:46,000
Quickly.

17
00:00:46,000 --> 00:00:49,000
So let's see this I will go ahead and select my query.

18
00:00:50,000 --> 00:00:54,000
Uh Lang Lang Smith has two usage limits total traces and extended.

19
00:00:54,000 --> 00:00:57,000
And then we will go ahead and search for the result.

20
00:00:57,000 --> 00:00:59,000
I'm just acquiring it from the vector store DB.

21
00:00:59,000 --> 00:01:02,000
It should be able to provide me the context and provide based on that.

22
00:01:02,000 --> 00:01:06,000
It should be able to retrieve all the information from here itself.

23
00:01:06,000 --> 00:01:07,000
Right.

24
00:01:07,000 --> 00:01:10,000
All this information should be, uh, we should be able to get it.

25
00:01:10,000 --> 00:01:11,000
Okay.

26
00:01:11,000 --> 00:01:11,000
So quickly.

27
00:01:11,000 --> 00:01:12,000
Let's do that.

28
00:01:12,000 --> 00:01:17,000
And here now in the result section, what I am actually going to use, I'm going to just call this particular

29
00:01:17,000 --> 00:01:24,000
vector store DB and uh, probably perform our similarity search and similarity search based on this

30
00:01:24,000 --> 00:01:25,000
particular query.

31
00:01:25,000 --> 00:01:30,000
Once I get the result, uh, I can go ahead and go ahead and write result of Zero.

32
00:01:30,000 --> 00:01:34,000
And inside that we will be having a key which is called as page content.

33
00:01:34,000 --> 00:01:36,000
So let me just quickly go ahead and execute it.

34
00:01:36,000 --> 00:01:38,000
So here you can see use user limit to prevent.

35
00:01:38,000 --> 00:01:44,000
Locksmith has two usage limit total traces and extended retention.

36
00:01:44,000 --> 00:01:46,000
This corresponds to the two metrics.

37
00:01:46,000 --> 00:01:48,000
All those information is specifically be able to retrieve it.

38
00:01:48,000 --> 00:01:49,000
Right.

39
00:01:49,000 --> 00:01:52,000
And that is the same thing that we are able to get from this.

40
00:01:52,000 --> 00:01:53,000
Right.

41
00:01:53,000 --> 00:01:57,000
See this corresponds to two metrics we have been tracking on our usage graph.

42
00:01:57,000 --> 00:02:01,000
We can use this in tandem to have a granular control over spend right.

43
00:02:01,000 --> 00:02:02,000
This is really amazing.

44
00:02:02,000 --> 00:02:08,000
Just from with the help of vector store DB which is having all the content in the form of vectors itself.

45
00:02:08,000 --> 00:02:13,000
From that directly, we are able to query things and any information that you really want to query,

46
00:02:13,000 --> 00:02:18,000
whether it's copy and paste some of the text away and you'll be able to get that particular answer right.

47
00:02:18,000 --> 00:02:21,000
Isn't it just amazing right now?

48
00:02:21,000 --> 00:02:23,000
Uh, this is how you query from a vector store DB.

49
00:02:23,000 --> 00:02:30,000
So I'll just go ahead and write query from a vector store db okay.

50
00:02:30,000 --> 00:02:34,000
And you can probably go ahead with other queries that you are actually looking for.

51
00:02:34,000 --> 00:02:37,000
Now uh, let's talk about retrieval chain.

52
00:02:37,000 --> 00:02:40,000
Now see this querying from a vector store DB.

53
00:02:40,000 --> 00:02:44,000
It will just try to give you, based on the context, all the information that is available near those

54
00:02:44,000 --> 00:02:44,000
vectors.

55
00:02:44,000 --> 00:02:44,000
Right.

56
00:02:44,000 --> 00:02:47,000
It'll just come and it'll display it in front of you.

57
00:02:47,000 --> 00:02:51,000
But uh, let's say I want to ask a question which is much more meaningful.

58
00:02:51,000 --> 00:02:55,000
I really need to also provide context with respect to that particular question.

59
00:02:55,000 --> 00:02:56,000
Right.

60
00:02:56,000 --> 00:02:59,000
So that is the reason why we will be using a retrieval chain.

61
00:03:00,000 --> 00:03:00,000
Okay.

62
00:03:00,000 --> 00:03:03,000
So here we are specifically going to use a retrieval chain.

63
00:03:03,000 --> 00:03:06,000
Now this retrieval chain is quite important.

64
00:03:06,000 --> 00:03:09,000
And in this video I'll be explaining you about that okay.

65
00:03:09,000 --> 00:03:13,000
So multiple things I'm going to talk about one is retrieval chain.

66
00:03:13,000 --> 00:03:17,000
Then we'll talk about documents chain okay document chain.

67
00:03:17,000 --> 00:03:23,000
And then how we can basically combine all these things and get proper response by using the LM also.

68
00:03:23,000 --> 00:03:23,000
Okay.

69
00:03:23,000 --> 00:03:26,000
So that is what I'm actually going to discuss now okay.

70
00:03:26,000 --> 00:03:29,000
So uh, quickly I will just copy this over here.

71
00:03:30,000 --> 00:03:31,000
I'll paste it over here.

72
00:03:31,000 --> 00:03:32,000
Okay.

73
00:03:32,000 --> 00:03:37,000
And uh, understand guys, uh, this retrieval uh, that you will be seeing.

74
00:03:37,000 --> 00:03:37,000
Right.

75
00:03:37,000 --> 00:03:44,000
It is an very important step because wherever we will specifically be working, we have to use this

76
00:03:44,000 --> 00:03:44,000
retriever.

77
00:03:44,000 --> 00:03:50,000
We have to use this specific chain, uh, in most of the Rag application or document QnA chains.

78
00:03:50,000 --> 00:03:51,000
Right.

79
00:03:51,000 --> 00:03:53,000
So that also we will be seeing it.

80
00:03:53,000 --> 00:03:53,000
Okay.

81
00:03:53,000 --> 00:03:59,000
So uh, what we will do is that, uh, first of all, uh, let's let's do something.

82
00:03:59,000 --> 00:03:59,000
Okay.

83
00:03:59,000 --> 00:04:04,000
I will go ahead and import from lang chain dot chains.

84
00:04:04,000 --> 00:04:05,000
Okay.

85
00:04:05,000 --> 00:04:09,000
Now inside this you have something called as chain underscore document.

86
00:04:09,000 --> 00:04:09,000
Okay.

87
00:04:09,000 --> 00:04:13,000
So I will go ahead and write chain underscore documents I hope so.

88
00:04:13,000 --> 00:04:17,000
This will not chain underscore documents I'll not use chain underscore documents here.

89
00:04:17,000 --> 00:04:20,000
Instead I will go ahead and write combine underscore documents okay.

90
00:04:20,000 --> 00:04:25,000
And from here we will go ahead and import create stuff document chain I will talk about this.

91
00:04:25,000 --> 00:04:28,000
What exactly it is create stuff document chain.

92
00:04:28,000 --> 00:04:32,000
But here you can see that create a chain for passing a list of documents to a model.

93
00:04:32,000 --> 00:04:33,000
Right.

94
00:04:33,000 --> 00:04:34,000
So as I said right.

95
00:04:34,000 --> 00:04:41,000
My LM model should also have a context about any documents to answer it much more properly.

96
00:04:41,000 --> 00:04:41,000
Right.

97
00:04:41,000 --> 00:04:45,000
So that is the reason we basically use this create stuff document chain.

98
00:04:45,000 --> 00:04:49,000
Now what I'm actually going to do I'm going to go ahead and create my prompt.

99
00:04:49,000 --> 00:04:55,000
And now with respect to this particular prompt, first of all I will go ahead and uh, you know, go

100
00:04:55,000 --> 00:04:57,000
ahead and import this chat prompt template.

101
00:04:57,000 --> 00:05:01,000
So let me just go ahead and see where that library was present.

102
00:05:01,000 --> 00:05:07,000
So I'll be using from long chain underscore core dot prompts okay I'm going to just import this chat

103
00:05:07,000 --> 00:05:08,000
prompt template.

104
00:05:08,000 --> 00:05:14,000
Now with the help of this chat prompt template I will go ahead and give my own custom prompt.

105
00:05:14,000 --> 00:05:15,000
So I'll say from template.

106
00:05:15,000 --> 00:05:18,000
And this is what is my prompt okay.

107
00:05:18,000 --> 00:05:18,000
Okay.

108
00:05:18,000 --> 00:05:21,000
Now inside this prompt I will go ahead and just define it.

109
00:05:21,000 --> 00:05:25,000
So it will be triple quotes okay okay.

110
00:05:25,000 --> 00:05:25,000
Triple quotes.

111
00:05:25,000 --> 00:05:28,000
And this will be a multi-line prompt okay.

112
00:05:28,000 --> 00:05:32,000
Let's say I will go ahead and say answer the answer the.

113
00:05:32,000 --> 00:05:34,000
So I'm giving some instruction to my LLM model.

114
00:05:34,000 --> 00:05:37,000
Answer the following okay.

115
00:05:37,000 --> 00:05:44,000
Following question based only on the provided context.

116
00:05:44,000 --> 00:05:45,000
Okay.

117
00:05:46,000 --> 00:05:48,000
So I'm just going to ask this particular context.

118
00:05:48,000 --> 00:05:54,000
Uh, I'm I'm I'm I'm just asking say, hey, answer the following question based on the provided context.

119
00:05:54,000 --> 00:05:55,000
Now what is this context?

120
00:05:55,000 --> 00:06:02,000
This context is the information that I will be giving to my LM model regarding the information about

121
00:06:02,000 --> 00:06:03,000
the documents or text.

122
00:06:03,000 --> 00:06:04,000
Okay.

123
00:06:04,000 --> 00:06:06,000
So I will just go ahead and create something like this.

124
00:06:06,000 --> 00:06:08,000
I will close the context tab over here.

125
00:06:08,000 --> 00:06:13,000
And internally I will create a key on this particular name that is context.

126
00:06:13,000 --> 00:06:18,000
Now see let's say that I want to probably search anything okay.

127
00:06:18,000 --> 00:06:25,000
So let's say uh over here I'll take one example over here if I'm so if I give this particular text okay.

128
00:06:25,000 --> 00:06:29,000
Now what it needs to do is that I need to get a context information.

129
00:06:29,000 --> 00:06:34,000
Context information basically means instead of searching the entire page, I can probably go ahead and

130
00:06:34,000 --> 00:06:38,000
give this entire paragraph and I may say him, hey, go and search for this particular text in this

131
00:06:38,000 --> 00:06:39,000
particular paragraph.

132
00:06:39,000 --> 00:06:42,000
So this paragraph becomes my context.

133
00:06:42,000 --> 00:06:45,000
So through this the search will also very happen very much quickly.

134
00:06:45,000 --> 00:06:48,000
Now how do I get this entire paragraph as a context.

135
00:06:48,000 --> 00:06:53,000
And that is where we will be using something called as create stuff document chain okay.

136
00:06:53,000 --> 00:06:58,000
Now inside this create stuff document chain, you will be able to see how we will be able to provide

137
00:06:58,000 --> 00:06:59,000
this context okay.

138
00:06:59,000 --> 00:07:03,000
So this is what is my entire prompt template okay.

139
00:07:03,000 --> 00:07:06,000
Now let me just go ahead and create my document chain okay.

140
00:07:06,000 --> 00:07:12,000
Now this document chain will be responsible in providing my prompt template, this specific context

141
00:07:12,000 --> 00:07:13,000
information.

142
00:07:13,000 --> 00:07:13,000
Okay.

143
00:07:13,000 --> 00:07:16,000
Regarding any input that I am specifically asking.

144
00:07:16,000 --> 00:07:20,000
Now this document chain is basically created by create stop document chain.

145
00:07:20,000 --> 00:07:21,000
And here I have to give two parameters.

146
00:07:21,000 --> 00:07:24,000
One is LM and second one is nothing but prompt.

147
00:07:24,000 --> 00:07:25,000
Okay.

148
00:07:25,000 --> 00:07:28,000
But still I have not defined LM over here.

149
00:07:28,000 --> 00:07:32,000
So let us go ahead and create my LM model okay.

150
00:07:32,000 --> 00:07:39,000
And for creating a LM model I am going to just use the same uh with GPT 4.40.

151
00:07:39,000 --> 00:07:40,000
So I'll be copy this.

152
00:07:40,000 --> 00:07:41,000
Okay.

153
00:07:41,000 --> 00:07:42,000
I'll paste it over here.

154
00:07:42,000 --> 00:07:48,000
So from Langston underscore OpenAI import chat OpenAI I'm going to use this chat OpenAI with model GPT

155
00:07:48,000 --> 00:07:48,000
four.

156
00:07:48,000 --> 00:07:49,000
Oh okay.

157
00:07:49,000 --> 00:07:50,000
Now this is done.

158
00:07:50,000 --> 00:07:53,000
You'll be able to see this I've given my LLM okay.

159
00:07:53,000 --> 00:07:56,000
Now let's go ahead and see my document underscore train.

160
00:07:56,000 --> 00:08:01,000
So here you'll be able to see LLM is not defined because I have not executed it.

161
00:08:01,000 --> 00:08:01,000
No worries.

162
00:08:01,000 --> 00:08:04,000
Now it should be able to execute it okay.

163
00:08:04,000 --> 00:08:04,000
Perfect.

164
00:08:04,000 --> 00:08:07,000
Now see understand what this document chain is.

165
00:08:07,000 --> 00:08:09,000
It is a runnable binding okay.

166
00:08:09,000 --> 00:08:11,000
And the mapper is something like this.

167
00:08:11,000 --> 00:08:12,000
This is my chat prompt template.

168
00:08:12,000 --> 00:08:18,000
The variable the input variable is context message is nothing but human prompt template with prompt

169
00:08:18,000 --> 00:08:21,000
uh input variable is nothing but context information.

170
00:08:21,000 --> 00:08:28,000
And here you will also be able to see that chat open AI chat completion is basically happening in this

171
00:08:28,000 --> 00:08:29,000
memory location.

172
00:08:29,000 --> 00:08:35,000
And you will be able to see the secret key which is hidden along with this chart open IE string output

173
00:08:35,000 --> 00:08:36,000
parser is also there, right?

174
00:08:36,000 --> 00:08:40,000
So step by step, first chart prompt template, then chart open AI, then string output parser.

175
00:08:41,000 --> 00:08:44,000
All has been combined together in the form of chain.

176
00:08:44,000 --> 00:08:51,000
Right after going through all this chain, it should be able to give me some context information.

177
00:08:51,000 --> 00:08:55,000
Okay, some context information about the thing that I am searching for, right?

178
00:08:55,000 --> 00:08:56,000
In short, this is also a chain.

179
00:08:56,000 --> 00:08:59,000
And in this chain you have initially chart prompt template.

180
00:08:59,000 --> 00:09:02,000
Then you have chat open AI, then you have string output parser.

181
00:09:02,000 --> 00:09:05,000
One by one it will go and it will give you some kind of output based on the context.

182
00:09:05,000 --> 00:09:09,000
Right now, right now I have got this entire information.

183
00:09:09,000 --> 00:09:16,000
Now what I'm actually going to do is that I can also go ahead and use this document chain.

184
00:09:16,000 --> 00:09:17,000
So let's go ahead and do this.

185
00:09:17,000 --> 00:09:23,000
So first of all I'll go ahead and just go ahead and write document underscore chain.

186
00:09:23,000 --> 00:09:26,000
And there is an inbuilt function which is called as invoke okay.

187
00:09:26,000 --> 00:09:31,000
Now inside this invoke I will first of all give my input okay.

188
00:09:31,000 --> 00:09:36,000
The input will be the same thing which I gave it earlier let's say over here.

189
00:09:36,000 --> 00:09:39,000
So this will basically be my input okay.

190
00:09:41,000 --> 00:09:43,000
This will be my input Okay.

191
00:09:43,000 --> 00:09:44,000
Now let's see.

192
00:09:44,000 --> 00:09:48,000
Let's execute this and let's see what kind of response we will get okay.

193
00:09:48,000 --> 00:09:52,000
So here it says hey key error context because I have not given the context yet right.

194
00:09:52,000 --> 00:09:56,000
So from where can I get view the context right.

195
00:09:57,000 --> 00:09:58,000
It is very much simple guys here.

196
00:09:58,000 --> 00:10:01,000
What I will do, I will go ahead and create my next field.

197
00:10:01,000 --> 00:10:03,000
And here I'm just going to give my context.

198
00:10:03,000 --> 00:10:06,000
So this is my input which the user is basically giving.

199
00:10:06,000 --> 00:10:09,000
And this is my context which I really need to give.

200
00:10:09,000 --> 00:10:10,000
Right.

201
00:10:10,000 --> 00:10:16,000
And you know, just to let's say over here, if I go ahead and uh, to give the context right over here,

202
00:10:16,000 --> 00:10:18,000
I need to give the entire documents.

203
00:10:18,000 --> 00:10:18,000
Right.

204
00:10:18,000 --> 00:10:24,000
So here I need to probably give this entire documents, whatever documents is basically present in this

205
00:10:24,000 --> 00:10:24,000
vector store.

206
00:10:24,000 --> 00:10:25,000
DB okay.

207
00:10:25,000 --> 00:10:29,000
Now let me do let me just create this document manually for uh, for currently.

208
00:10:29,000 --> 00:10:29,000
Right.

209
00:10:29,000 --> 00:10:37,000
So I will go ahead and write from lang chain from lang chain underscore core dot documents.

210
00:10:37,000 --> 00:10:40,000
I'm going to import the document okay.

211
00:10:40,000 --> 00:10:42,000
So document has been imported.

212
00:10:42,000 --> 00:10:45,000
So now in this particular context I'm just going to give document over here.

213
00:10:46,000 --> 00:10:49,000
And uh it will be in the form of list okay.

214
00:10:49,000 --> 00:10:53,000
Let's say and I will just go ahead and write page underscore content.

215
00:10:53,000 --> 00:10:57,000
And this is what is the format we basically get the document ID right is equal to.

216
00:10:57,000 --> 00:11:00,000
And let's say I will go ahead and write over here.

217
00:11:01,000 --> 00:11:04,000
Uh, let's say this is my context.

218
00:11:04,000 --> 00:11:09,000
Like Smith has two limits, uh, total traces and extended traces.

219
00:11:09,000 --> 00:11:09,000
Okay.

220
00:11:09,000 --> 00:11:12,000
And I will give some more information over here.

221
00:11:12,000 --> 00:11:13,000
Let's go back.

222
00:11:13,000 --> 00:11:15,000
Let's, uh, see over here.

223
00:11:15,000 --> 00:11:18,000
I'll say like this corresponds to something.

224
00:11:18,000 --> 00:11:19,000
Something some more text.

225
00:11:19,000 --> 00:11:20,000
Okay.

226
00:11:20,000 --> 00:11:23,000
So I'm just adding it manually over here okay.

227
00:11:23,000 --> 00:11:25,000
So this is my entire context okay.

228
00:11:25,000 --> 00:11:28,000
And finally I will close this particular bracket over here okay.

229
00:11:28,000 --> 00:11:31,000
So this basically becomes my entire document context.

230
00:11:31,000 --> 00:11:38,000
Now if I'm invoking this based on this particular input it should be able to give me the entire output.

231
00:11:38,000 --> 00:11:39,000
You see this okay.

232
00:11:39,000 --> 00:11:42,000
So if I go ahead and execute it see Laxmidas to this extended.

233
00:11:42,000 --> 00:11:46,000
This corresponds to the two metrics tracked on the user usage graph.

234
00:11:46,000 --> 00:11:48,000
So I've given the context I'm saying.

235
00:11:48,000 --> 00:11:51,000
From this you search this and try to then give me the result okay.

236
00:11:51,000 --> 00:11:56,000
And here you can see just by seeing this particular example document chain works absolutely fabulous

237
00:11:56,000 --> 00:11:58,000
right now.

238
00:11:59,000 --> 00:12:01,000
Anyhow I will put one comment over here.

239
00:12:01,000 --> 00:12:01,000
Okay.

240
00:12:02,000 --> 00:12:09,000
I will put one comment saying that however we want the documents to first come from the retriever we

241
00:12:09,000 --> 00:12:10,000
just set up.

242
00:12:10,000 --> 00:12:16,000
That way we can use the retriever to dynamically select the most relevant documents and pass those in

243
00:12:16,000 --> 00:12:17,000
for a given question.

244
00:12:17,000 --> 00:12:22,000
Now let me make you understand about retrievers.

245
00:12:22,000 --> 00:12:24,000
Okay, what exactly is this retriever?

246
00:12:24,000 --> 00:12:30,000
Right now, guys, I'm just giving you a brief idea of creating this Gemini application where we have

247
00:12:30,000 --> 00:12:34,000
learned about document chain, where we have learned about data ingestion technique.

248
00:12:34,000 --> 00:12:36,000
Now this is the most important thing.

249
00:12:36,000 --> 00:12:37,000
Retrieval.

250
00:12:37,000 --> 00:12:40,000
What exactly is a retrieval okay.

251
00:12:40,000 --> 00:12:41,000
What exactly is a retrieval.

252
00:12:42,000 --> 00:12:45,000
You know right now I have this vector store DB right.

253
00:12:45,000 --> 00:12:53,000
This vector store DB has all the vectors information available in that right now retriever can be considered

254
00:12:53,000 --> 00:12:55,000
as an interface okay.

255
00:12:55,000 --> 00:12:58,000
It can be considered as an interface, and it's responsible.

256
00:12:58,000 --> 00:13:08,000
Responsibility is that if anybody asks any input right, then what it will happen is that this interface

257
00:13:08,000 --> 00:13:18,000
will just be a way of probably getting the data from the vector store DB okay, vector store db, where

258
00:13:18,000 --> 00:13:21,000
we don't even have to do the similarity search.

259
00:13:21,000 --> 00:13:21,000
Right.

260
00:13:21,000 --> 00:13:26,000
So what we do is that after we create this particular vector store db, we convert this vector store

261
00:13:26,000 --> 00:13:27,000
db into a retriever.

262
00:13:28,000 --> 00:13:31,000
Retriever is just like an interface with respect to any input.

263
00:13:31,000 --> 00:13:36,000
I will be able to pass that particular input through this retriever and get the response from the vector

264
00:13:36,000 --> 00:13:37,000
store.

265
00:13:37,000 --> 00:13:42,000
You can just consider like an pathway to probably get the information from the vector store db.

266
00:13:42,000 --> 00:13:43,000
Okay.

267
00:13:43,000 --> 00:13:48,000
And that is where now I will show you how we can go ahead and create a retriever.

268
00:13:48,000 --> 00:13:48,000
Right.

269
00:13:48,000 --> 00:13:51,000
So in order to create a retriever it is very much simple.

270
00:13:51,000 --> 00:13:56,000
I will go ahead and write vector store DB dot as retriever okay.

271
00:13:56,000 --> 00:14:01,000
So as retriever function you can see over here return vector store retriever initialized from vector

272
00:14:01,000 --> 00:14:02,000
store.

273
00:14:02,000 --> 00:14:02,000
Right.

274
00:14:02,000 --> 00:14:05,000
So this basically becomes my entire retriever.

275
00:14:05,000 --> 00:14:09,000
So here you can see this is my vector store retriever tag is fierce OpenAI embedding.

276
00:14:09,000 --> 00:14:16,000
All the information is basically visible over here right this I will save it in my retriever variable

277
00:14:16,000 --> 00:14:16,000
okay.

278
00:14:16,000 --> 00:14:17,000
Now see the magic okay.

279
00:14:17,000 --> 00:14:24,000
Now I will go ahead and say, hey, uh, I will go ahead and import from Lang Chain.

280
00:14:24,000 --> 00:14:24,000
Okay.

281
00:14:24,000 --> 00:14:27,000
Since I have already created the retriever.

282
00:14:27,000 --> 00:14:35,000
So I'll go ahead and right from lang chain dot chains import Import create retriever chain.

283
00:14:35,000 --> 00:14:37,000
Okay, now see this magic.

284
00:14:37,000 --> 00:14:40,000
What will happen with the help of this create retriever chain.

285
00:14:40,000 --> 00:14:49,000
Now this create retrieval chain basically takes the first parameter as retriever okay retriever and

286
00:14:49,000 --> 00:14:51,000
the second parameter as document chain.

287
00:14:51,000 --> 00:14:53,000
Now see see see see the magic over here.

288
00:14:53,000 --> 00:14:55,000
Why I am giving this two parameters over here.

289
00:14:55,000 --> 00:15:01,000
First we need to combine this retriever, which will be taking the information from the vector store

290
00:15:01,000 --> 00:15:02,000
DB.

291
00:15:02,000 --> 00:15:02,000
Right.

292
00:15:02,000 --> 00:15:05,000
And that is the reason we converted it from here.

293
00:15:05,000 --> 00:15:05,000
Right.

294
00:15:05,000 --> 00:15:08,000
Or by just using this as underscore retriever.

295
00:15:08,000 --> 00:15:09,000
And then we are using document chain.

296
00:15:09,000 --> 00:15:15,000
Document chain will be responsible in giving you the context information.

297
00:15:15,000 --> 00:15:16,000
Right.

298
00:15:16,000 --> 00:15:20,000
And then when we are creating this chain along with the document chain we are also going to use the

299
00:15:20,000 --> 00:15:22,000
retriever over here.

300
00:15:22,000 --> 00:15:28,000
When we created this particular prompt, this context information will basically get filled in this

301
00:15:28,000 --> 00:15:34,000
chat prop table chat prop template using this particular document chain that we have defined over here,

302
00:15:34,000 --> 00:15:36,000
which is nothing but create sort of document chain.

303
00:15:36,000 --> 00:15:36,000
Right.

304
00:15:36,000 --> 00:15:39,000
So create sort of document chain only we created over here.

305
00:15:39,000 --> 00:15:39,000
Right.

306
00:15:39,000 --> 00:15:42,000
So here you can see write create stop document chain.

307
00:15:42,000 --> 00:15:47,000
So we are going to combine this chain along with creator retrieval chain right.

308
00:15:47,000 --> 00:15:48,000
Which is nothing but my retriever.

309
00:15:48,000 --> 00:15:52,000
And the context will be spread uh given by this particular document underscore chain.

310
00:15:52,000 --> 00:15:54,000
And it will be able to get the information.

311
00:15:54,000 --> 00:16:01,000
Now here I'm just going to go ahead and define my retrieval underscore chain okay.

312
00:16:01,000 --> 00:16:02,000
Perfect.

313
00:16:02,000 --> 00:16:05,000
Now let's go ahead and execute it.

314
00:16:05,000 --> 00:16:09,000
Now once this is there what is this retriever underscore chain.

315
00:16:09,000 --> 00:16:09,000
Right.

316
00:16:09,000 --> 00:16:11,000
It is very much simple.

317
00:16:11,000 --> 00:16:14,000
See all these things step by step.

318
00:16:14,000 --> 00:16:17,000
Initially it has this runnable binding parameters.

319
00:16:17,000 --> 00:16:22,000
Over here you can see first of all I have this chat prompt template then chat open I then string output

320
00:16:22,000 --> 00:16:22,000
parser.

321
00:16:23,000 --> 00:16:27,000
Inside this you have something called as stuff document chain which will provide you the context.

322
00:16:27,000 --> 00:16:28,000
Right.

323
00:16:28,000 --> 00:16:31,000
All this information is specifically there and you can see it now.

324
00:16:31,000 --> 00:16:34,000
Finally we will get the response.

325
00:16:34,000 --> 00:16:44,000
Let's get the response from the get the response response from the um to get the response from the LM.

326
00:16:44,000 --> 00:16:45,000
Okay.

327
00:16:45,000 --> 00:16:51,000
Now for this, I will go ahead and use my retriever chain dot invoke function.

328
00:16:52,000 --> 00:16:56,000
And this time I will give my invoke with respect to the input.

329
00:16:56,000 --> 00:16:58,000
This input is passed by us, right?

330
00:16:58,000 --> 00:17:00,000
Whatever input we want.

331
00:17:00,000 --> 00:17:05,000
And let's say in this particular input I give the same text lang chain so and so.

332
00:17:05,000 --> 00:17:07,000
And this extended okay.

333
00:17:07,000 --> 00:17:08,000
Some information over here.

334
00:17:10,000 --> 00:17:13,000
So this basically becomes my input right.

335
00:17:13,000 --> 00:17:16,000
That's it I just need to give this particular input.

336
00:17:16,000 --> 00:17:18,000
It will invoke this entire answer.

337
00:17:19,000 --> 00:17:24,000
Now over here what I'm actually going to do, I will go ahead and create my response.

338
00:17:24,000 --> 00:17:28,000
So as soon as I call the retrieval chain, it is going to go to the chat prompt template.

339
00:17:28,000 --> 00:17:32,000
And if you see over here there will be a input which is called as context okay.

340
00:17:32,000 --> 00:17:36,000
This context is going to get filled by what stop document chain.

341
00:17:36,000 --> 00:17:38,000
Then we have this chat open.

342
00:17:38,000 --> 00:17:40,000
I then string output parser.

343
00:17:40,000 --> 00:17:45,000
Finally when I get my response inside this response there will be a parameter.

344
00:17:45,000 --> 00:17:48,000
less response.

345
00:17:48,000 --> 00:17:52,000
There will be a parameter which is basically called as answer okay.

346
00:17:52,000 --> 00:17:55,000
This key I should be able to get my answer.

347
00:17:55,000 --> 00:17:56,000
So let's go ahead and execute this.

348
00:17:56,000 --> 00:18:00,000
So here you can see to set up usage in Langsamt navigate usage billing.

349
00:18:00,000 --> 00:18:04,000
Then you will be able to find the table all with the associated cost.

350
00:18:04,000 --> 00:18:07,000
And all the information is specifically there.

351
00:18:07,000 --> 00:18:07,000
Right.

352
00:18:08,000 --> 00:18:11,000
And this is how you probably get the entire information, right?

353
00:18:11,000 --> 00:18:13,000
I basically created a document Q&A.

354
00:18:13,000 --> 00:18:14,000
Okay, let's do one thing.

355
00:18:14,000 --> 00:18:17,000
Let's go ahead and just print this response also.

356
00:18:17,000 --> 00:18:20,000
Now see in this response here is my input.

357
00:18:20,000 --> 00:18:23,000
Right here is my context.

358
00:18:23,000 --> 00:18:26,000
See this is the context that what what I was actually talking about right.

359
00:18:26,000 --> 00:18:29,000
Document page content use element to this this this more than 10%.

360
00:18:29,000 --> 00:18:31,000
All these are my context right.

361
00:18:31,000 --> 00:18:37,000
And answer it is selected this particular answer usage billing and all based on the context it has.

362
00:18:37,000 --> 00:18:38,000
Right.

363
00:18:38,000 --> 00:18:40,000
So all the information is over here.

364
00:18:40,000 --> 00:18:44,000
And if you also want to probably go ahead and display your context, you can go ahead and write it on

365
00:18:44,000 --> 00:18:47,000
response of context okay.

366
00:18:47,000 --> 00:18:52,000
And if you go ahead and see this, this basically shows you the entire context, right.

367
00:18:52,000 --> 00:18:57,000
All these documents basically from where it has, uh, got the context from.

368
00:18:57,000 --> 00:18:57,000
Right.

369
00:18:57,000 --> 00:19:03,000
And this is only going to get fit where in your prompt template where, uh, we have created this context

370
00:19:03,000 --> 00:19:04,000
variable.

371
00:19:04,000 --> 00:19:05,000
Right.

372
00:19:05,000 --> 00:19:10,000
So I hope you are able to understand this, but at the end of the day, I have completely created an

373
00:19:10,000 --> 00:19:16,000
end to end project, uh, from a simple, uh, like I can say that, okay, this is a simple generative

374
00:19:16,000 --> 00:19:17,000
AI application.

375
00:19:17,000 --> 00:19:21,000
Initially we went with data ingestion, then we loaded the entire documents.

376
00:19:21,000 --> 00:19:26,000
Then we divided our entire data into chunks or documents into chunks.

377
00:19:26,000 --> 00:19:31,000
I will not say text, but I will say document into chunks, document into chunks.

378
00:19:31,000 --> 00:19:32,000
Documents.

379
00:19:32,000 --> 00:19:33,000
Okay.

380
00:19:33,000 --> 00:19:37,000
Then all the text from that document is converted into vectors then embeddings.

381
00:19:37,000 --> 00:19:37,000
Right.

382
00:19:37,000 --> 00:19:40,000
So here we will be able to see this is my documents.

383
00:19:40,000 --> 00:19:43,000
Then we use OpenAI embedding to convert all our text into vectors.

384
00:19:43,000 --> 00:19:45,000
Store it in a vector store db.

385
00:19:45,000 --> 00:19:47,000
Here we have used files.

386
00:19:47,000 --> 00:19:48,000
We can also use chroma db.

387
00:19:49,000 --> 00:19:51,000
Then query from a vector store db.

388
00:19:51,000 --> 00:19:52,000
I used similarity search.

389
00:19:52,000 --> 00:19:53,000
This was one way.

390
00:19:53,000 --> 00:19:55,000
Then we used lm model.

391
00:19:55,000 --> 00:20:00,000
We created create stuff document chain because when we are designing our chat prop template, there

392
00:20:00,000 --> 00:20:02,000
was a context that needed to be used.

393
00:20:02,000 --> 00:20:06,000
And this creates a document chain is going to provide this particular context.

394
00:20:07,000 --> 00:20:09,000
And finally here you can see I have created this.

395
00:20:09,000 --> 00:20:13,000
Then we combined our document chain by seeing one example over here.

396
00:20:13,000 --> 00:20:16,000
And finally we are able to see this vector store DB as a retriever.

397
00:20:16,000 --> 00:20:19,000
And we are able to find out the retrieval chain.

398
00:20:19,000 --> 00:20:23,000
And we are able to invoke based on any question that we have, we will be able to get the response.

399
00:20:23,000 --> 00:20:26,000
Not only that, you will be able to explore the context.

400
00:20:26,000 --> 00:20:30,000
You will be able to explore answers, whatever things you specifically want.

401
00:20:30,000 --> 00:20:36,000
So, uh, this was most of the things that we have actually discussed over here, which is quite amazing.

402
00:20:36,000 --> 00:20:40,000
Uh, and over here, as an end to end project, we have actually developed it.

403
00:20:40,000 --> 00:20:41,000
Right?

404
00:20:41,000 --> 00:20:42,000
Uh, yes.

405
00:20:42,000 --> 00:20:46,000
Uh, in the next video, we are going to discuss still more things that are remaining over here.

406
00:20:46,000 --> 00:20:50,000
And then we are going to also see how with the help of Lama will be able to do it right.

407
00:20:50,000 --> 00:20:56,000
So Lama uh is an another application which actually helps us to access, uh, open source LM models.

408
00:20:56,000 --> 00:20:56,000
Right.

409
00:20:56,000 --> 00:20:57,000
So yes, this was it.

410
00:20:57,000 --> 00:20:59,000
I will see you all in the next video.

411
00:20:59,000 --> 00:20:59,000
Thank.