1
00:00:00,000 --> 00:00:01,000
Hello guys.

2
00:00:01,000 --> 00:00:05,000
So we are going to continue the discussion with respect to tools and agents in search engine.

3
00:00:05,000 --> 00:00:10,000
So first of all, I will try to show you some important functionalities where I will try to create my

4
00:00:10,000 --> 00:00:12,000
own custom tools.

5
00:00:12,000 --> 00:00:16,000
And then I will also try to use the inbuilt tools that are available in the engine.

6
00:00:16,000 --> 00:00:24,000
Now just to talk about some of the inbuilt tools that we have in engine are like Wikipedia are safe

7
00:00:24,000 --> 00:00:27,000
then already we have actually seen about this tools.

8
00:00:27,000 --> 00:00:33,000
Okay, so you can probably go over here and you can see all the tools like Google Finance, Google Places,

9
00:00:33,000 --> 00:00:34,000
let's say Google Search is there.

10
00:00:35,000 --> 00:00:42,000
Uh, if I just consider one example over here, let's say that I want to go ahead with Wikipedia, okay.

11
00:00:42,000 --> 00:00:47,000
So if I just go ahead and click on Wikipedia over here, you'll be able to see how, first of all we

12
00:00:47,000 --> 00:00:49,000
have to install Wikipedia.

13
00:00:49,000 --> 00:00:52,000
Then probably go ahead and implement all these things okay.

14
00:00:52,000 --> 00:00:56,000
So quickly what I will do is that I will just show you in with respect to the code so that you will

15
00:00:56,000 --> 00:00:57,000
also get an idea.

16
00:00:57,000 --> 00:01:01,000
So first of all, let's go ahead with a simple mechanism.

17
00:01:01,000 --> 00:01:03,000
First of all I'll select the kernel over here.

18
00:01:03,000 --> 00:01:06,000
I will say, hey, uh let's go ahead with something called as ourself.

19
00:01:06,000 --> 00:01:08,000
This is for my research paper.

20
00:01:08,000 --> 00:01:14,000
Now this tool will be responsible in interacting with the RC website itself, where I'll be able to

21
00:01:14,000 --> 00:01:15,000
get the information.

22
00:01:15,000 --> 00:01:20,000
Now, first of all, whenever we do this, you know, you have to make sure that we have installed this

23
00:01:20,000 --> 00:01:21,000
library.

24
00:01:21,000 --> 00:01:25,000
So here, if you see I will go ahead and write our CIF okay.

25
00:01:25,000 --> 00:01:27,000
And I will go ahead and install this.

26
00:01:27,000 --> 00:01:30,000
Along with this I'm also going to use one more called as Wikipedia.

27
00:01:30,000 --> 00:01:31,000
Okay.

28
00:01:32,000 --> 00:01:36,000
Uh I have installed both these libraries, but just to show it to you, I'm actually doing it.

29
00:01:36,000 --> 00:01:38,000
So I will just go ahead and open my terminal.

30
00:01:38,000 --> 00:01:40,000
I will open my command prompt.

31
00:01:40,000 --> 00:01:41,000
Sorry, I will delete this.

32
00:01:41,000 --> 00:01:43,000
I will open my command prompt.

33
00:01:43,000 --> 00:01:43,000
Okay.

34
00:01:43,000 --> 00:01:49,000
And here I will just go ahead and write pip install minus r requirements.txt.

35
00:01:49,000 --> 00:01:51,000
Because this is the environment variable that we are using.

36
00:01:51,000 --> 00:01:54,000
So pip install.

37
00:01:55,000 --> 00:01:58,000
So here you can see the installation is taking place.

38
00:01:58,000 --> 00:02:03,000
And already you know once the installation will take place yes it is completed.

39
00:02:03,000 --> 00:02:05,000
So I'll close this I'll start it okay.

40
00:02:05,000 --> 00:02:07,000
So I've installed RCF.

41
00:02:07,000 --> 00:02:10,000
So uh, along with that I've also installed Wikipedia.

42
00:02:10,000 --> 00:02:14,000
So let me go ahead and first of all start creating tools okay.

43
00:02:14,000 --> 00:02:16,000
So I will write tools creation.

44
00:02:17,000 --> 00:02:21,000
So first of all the tool that I'm actually going to create is something called as Wikipedia.

45
00:02:21,000 --> 00:02:24,000
And then uh obviously RC also I'll try to create okay.

46
00:02:24,000 --> 00:02:33,000
So for this I will go ahead and write from login underscore community dot tools I'm going to import,

47
00:02:34,000 --> 00:02:39,000
uh, our save or let's see whether we'll get this RCF query run.

48
00:02:39,000 --> 00:02:42,000
See one tool is this RCF query run.

49
00:02:42,000 --> 00:02:46,000
This is for RCF itself and this is Wikipedia query run.

50
00:02:46,000 --> 00:02:56,000
Then we are going to go ahead and write from long chain underscore community dot utilities import.

51
00:02:57,000 --> 00:02:58,000
Wikipedia.

52
00:03:01,000 --> 00:03:02,000
API wrapper.

53
00:03:02,000 --> 00:03:07,000
Okay, so, uh, this is a kind of wrapper C along with the C.

54
00:03:07,000 --> 00:03:09,000
This is query run okay.

55
00:03:09,000 --> 00:03:15,000
The difference between this tool and this utility is that, um, whenever we really need to run any

56
00:03:15,000 --> 00:03:16,000
of this query.

57
00:03:16,000 --> 00:03:21,000
So whenever we use this library to run a query, this is basically interacting with the tool itself,

58
00:03:21,000 --> 00:03:26,000
with the with the outside world, with the information that is present in arXiv or Wikipedia.

59
00:03:26,000 --> 00:03:26,000
Okay.

60
00:03:26,000 --> 00:03:31,000
Similarly, uh, to run this entire query, we also require some kind of wrapper.

61
00:03:31,000 --> 00:03:31,000
Right.

62
00:03:31,000 --> 00:03:36,000
So here also after this I will just go ahead and write arXiv API wrapper.

63
00:03:36,000 --> 00:03:40,000
So both these are very much important libraries that will be required over here.

64
00:03:40,000 --> 00:03:45,000
Once I execute this here I will go ahead and create my API wrapper.

65
00:03:45,000 --> 00:03:48,000
And this API wrapper will be for my wiki.

66
00:03:48,000 --> 00:03:50,000
Okay, I'll write Wikipedia.

67
00:03:50,000 --> 00:03:53,000
Now I will go ahead and say hey Wikipedia API wrapper.

68
00:03:53,000 --> 00:03:56,000
Now this API wrapper will have some information.

69
00:03:56,000 --> 00:04:02,000
So if I probably see it is a wrapper around Wikipedia API to use, you should have the Wikipedia Python

70
00:04:02,000 --> 00:04:03,000
package installed.

71
00:04:03,000 --> 00:04:06,000
Okay, which we have already done that okay.

72
00:04:06,000 --> 00:04:10,000
This wrapper will use the Wikipedia API to conduct searches and fetch page summaries.

73
00:04:10,000 --> 00:04:16,000
So this is a rapper which is responsible in fetching, in conducting searches and fetching the page

74
00:04:16,000 --> 00:04:17,000
summaries.

75
00:04:17,000 --> 00:04:23,000
Now here we require some parameter some some such as top underscore k underscore results.

76
00:04:23,000 --> 00:04:26,000
So this is basically saying that how many number of results you want.

77
00:04:26,000 --> 00:04:28,000
Let's say that I just want one right now.

78
00:04:28,000 --> 00:04:33,000
And I'll also give one more very important parameter which is called as document content character max.

79
00:04:33,000 --> 00:04:40,000
Like how many max characters you want, I'll say go ahead and do it for 250, okay, you can even increase

80
00:04:40,000 --> 00:04:41,000
this number of characters.

81
00:04:41,000 --> 00:04:45,000
It's up to you now once I've created this particular wrapper.

82
00:04:45,000 --> 00:04:47,000
Right, I need to run this wrapper.

83
00:04:47,000 --> 00:04:52,000
So to run this particular wrapper I will be requiring this Wikipedia query run.

84
00:04:52,000 --> 00:04:57,000
Okay so here I will write Wikipedia query run is equal to whatever wrapper I've actually created my

85
00:04:57,000 --> 00:04:58,000
API wrapper.

86
00:04:58,000 --> 00:05:01,000
This is what is the parameter with respect to that.

87
00:05:01,000 --> 00:05:04,000
And this will basically give my tool.

88
00:05:04,000 --> 00:05:04,000
Right.

89
00:05:04,000 --> 00:05:08,000
So this becomes one tool I have actually created my tool.

90
00:05:08,000 --> 00:05:10,000
And this is the inbuilt tool that we have.

91
00:05:10,000 --> 00:05:15,000
And if I go ahead and just write wiki dot name you'll be able to see that this is a Wikipedia tool.

92
00:05:15,000 --> 00:05:16,000
Right.

93
00:05:16,000 --> 00:05:28,000
So here in this code we have used the used the inbuilt tool of Wikipedia.

94
00:05:28,000 --> 00:05:29,000
Okay.

95
00:05:29,000 --> 00:05:29,000
Perfect.

96
00:05:31,000 --> 00:05:32,000
Done.

97
00:05:32,000 --> 00:05:37,000
Now similarly, if I want to probably go ahead and create our sift tool.

98
00:05:37,000 --> 00:05:42,000
Also how do I write it similarly like first of all we need to create a wrapper.

99
00:05:42,000 --> 00:05:47,000
So I'll write API underscore wrapper underscore or save.

100
00:05:47,000 --> 00:05:48,000
Okay.

101
00:05:48,000 --> 00:05:53,000
And this will basically be nothing but our Sif API wrapper.

102
00:05:54,000 --> 00:05:58,000
And inside this I'm just going to use my top underscore k results.

103
00:05:58,000 --> 00:06:04,000
Again here it will be one and doc content max characters here also it will be 250.

104
00:06:05,000 --> 00:06:06,000
I'm keeping it to 250.

105
00:06:08,000 --> 00:06:10,000
Then I will go ahead and write.

106
00:06:10,000 --> 00:06:15,000
Our Sif is equal to our safe, uh, query run.

107
00:06:16,000 --> 00:06:21,000
I'll use this same API wrapper is equal to API wrapper RCF.

108
00:06:21,000 --> 00:06:23,000
And then I will be calling this.

109
00:06:23,000 --> 00:06:27,000
Let's go ahead and print my RCF dot name.

110
00:06:27,000 --> 00:06:30,000
So this becomes my RCF tool okay.

111
00:06:31,000 --> 00:06:34,000
Now if I really need to combine both this tool.

112
00:06:34,000 --> 00:06:37,000
See at the end of the day there are two tools I want to use in my application.

113
00:06:37,000 --> 00:06:39,000
So I'll combine them in the form of list.

114
00:06:39,000 --> 00:06:40,000
Right?

115
00:06:40,000 --> 00:06:41,000
So first of all I'll say wiki.

116
00:06:41,000 --> 00:06:48,000
Then I will go ahead and ask my chef okay this is fine and this becomes my tools okay.

117
00:06:49,000 --> 00:06:55,000
Now you should know that if you really want to run this tools, how do you run this tools and all okay.

118
00:06:55,000 --> 00:06:59,000
And obviously for that you require a model itself.

119
00:06:59,000 --> 00:07:04,000
Along with that model you specifically require one more very important thing, which is called as,

120
00:07:05,000 --> 00:07:08,000
um, you know, obviously LM model will be there.

121
00:07:08,000 --> 00:07:12,000
Then we have to also create our chart from template, and then we can probably go ahead and create all

122
00:07:12,000 --> 00:07:13,000
the specific tools.

123
00:07:13,000 --> 00:07:13,000
Okay.

124
00:07:14,000 --> 00:07:17,000
Now let me do one more thing over here.

125
00:07:17,000 --> 00:07:20,000
This is the inbuilt tool that we have created right.

126
00:07:20,000 --> 00:07:23,000
Let us go ahead and create our own custom tools also.

127
00:07:23,000 --> 00:07:24,000
Okay.

128
00:07:24,000 --> 00:07:28,000
Now in order to create the custom tools let's take a specific example.

129
00:07:28,000 --> 00:07:30,000
I will import all these libraries.

130
00:07:30,000 --> 00:07:31,000
Okay.

131
00:07:31,000 --> 00:07:34,000
And this will basically be my rag tool.

132
00:07:34,000 --> 00:07:41,000
Let's say okay I'm saying hey from uh lang chain underscore community dot document loaders import web

133
00:07:41,000 --> 00:07:42,000
based loader.

134
00:07:42,000 --> 00:07:44,000
Then from the vector store I've used files.

135
00:07:44,000 --> 00:07:46,000
Then I'm using OpenAI embeddings.

136
00:07:46,000 --> 00:07:47,000
Okay.

137
00:07:47,000 --> 00:07:49,000
You can also use llama embeddings.

138
00:07:49,000 --> 00:07:49,000
No worries.

139
00:07:49,000 --> 00:07:54,000
As such then you have Langston underscore text underscore splitters.

140
00:07:54,000 --> 00:07:58,000
Then you have import recursive character text splitter.

141
00:07:58,000 --> 00:07:59,000
Okay.

142
00:07:59,000 --> 00:08:03,000
So you can also use this with respect to character text splitter.

143
00:08:03,000 --> 00:08:06,000
And with respect to OpenAI embedding.

144
00:08:06,000 --> 00:08:08,000
Instead of opening a embedding, you can also use llama embeddings.

145
00:08:08,000 --> 00:08:12,000
And uh, as we go ahead, I have already shown you how to use Huggingface embeddings.

146
00:08:12,000 --> 00:08:12,000
Okay.

147
00:08:12,000 --> 00:08:17,000
So, uh, if you want I will just go ahead and use this hugging face embeddings.

148
00:08:17,000 --> 00:08:18,000
Let's see.

149
00:08:18,000 --> 00:08:22,000
Uh, let me just go ahead and pick up hugging face embeddings.

150
00:08:22,000 --> 00:08:24,000
So here is my hugging face embeddings.

151
00:08:24,000 --> 00:08:29,000
Now, in order to use hugging face embeddings, I have to use this particular token I have already shown

152
00:08:29,000 --> 00:08:30,000
you.

153
00:08:30,000 --> 00:08:30,000
Okay.

154
00:08:30,000 --> 00:08:33,000
And then we can use this particular hugging face embeddings itself.

155
00:08:33,000 --> 00:08:34,000
Right?

156
00:08:34,000 --> 00:08:35,000
It is up to you whichever you want to use.

157
00:08:35,000 --> 00:08:42,000
I have given so many options, but I want to go ahead with opening embeddings because again, I will

158
00:08:42,000 --> 00:08:47,000
be able this is a I will be able to get the quickly the results, and I'll be able to show you the examples.

159
00:08:47,000 --> 00:08:47,000
Okay.

160
00:08:47,000 --> 00:08:49,000
You can use all of my embeddings for all of my embeddings.

161
00:08:49,000 --> 00:08:50,000
Also, it will take some amount of time.

162
00:08:50,000 --> 00:08:53,000
If not, you just go and directly use Huggingface embeddings.

163
00:08:53,000 --> 00:08:54,000
That is up to you.

164
00:08:54,000 --> 00:08:54,000
Okay.

165
00:08:54,000 --> 00:08:57,000
And then I'm also using recursive character text splitter.

166
00:08:57,000 --> 00:08:58,000
So let's execute this.

167
00:08:59,000 --> 00:09:01,000
So environment agent variable not set consideration.

168
00:09:01,000 --> 00:09:02,000
It's okay.

169
00:09:02,000 --> 00:09:03,000
Uh let this warning come.

170
00:09:03,000 --> 00:09:05,000
So if I execute it again it will go off okay.

171
00:09:05,000 --> 00:09:12,000
Now here what I will actually do is that I will I will have my own one page.

172
00:09:12,000 --> 00:09:12,000
Okay.

173
00:09:12,000 --> 00:09:14,000
One website page.

174
00:09:14,000 --> 00:09:15,000
I will try to load that page.

175
00:09:15,000 --> 00:09:19,000
And based on that I will try to create that into a vector store.

176
00:09:19,000 --> 00:09:21,000
And I'll use that as a separate tool.

177
00:09:21,000 --> 00:09:21,000
Okay.

178
00:09:21,000 --> 00:09:26,000
So here what I'm actually going to do is that I'm going to write web based loader.

179
00:09:28,000 --> 00:09:31,000
Let me give one of the Lang Smith page that I have.

180
00:09:31,000 --> 00:09:32,000
Okay.

181
00:09:32,000 --> 00:09:34,000
So here I'm giving you this URL.

182
00:09:35,000 --> 00:09:37,000
I'm going to use this specific URL.

183
00:09:37,000 --> 00:09:38,000
And then I'll go ahead and write.

184
00:09:38,000 --> 00:09:42,000
Docs is equal to loader dot load.

185
00:09:42,000 --> 00:09:45,000
So after this this will basically give us all the documents.

186
00:09:45,000 --> 00:09:50,000
And then I will go ahead and create my document file documents is equal to.

187
00:09:50,000 --> 00:09:57,000
And here I'm going to use my recursive character text editor the chunk size will be nothing, but it

188
00:09:57,000 --> 00:09:59,000
will be equal to 1000.

189
00:09:59,000 --> 00:10:03,000
And finally you will have your chunk overlap okay.

190
00:10:03,000 --> 00:10:05,000
Which will be equal to 200 okay.

191
00:10:05,000 --> 00:10:10,000
And I will go ahead and split this documents which we basically write in our next line code.

192
00:10:10,000 --> 00:10:13,000
And I will say hey split all these documents.

193
00:10:14,000 --> 00:10:15,000
Okay.

194
00:10:15,000 --> 00:10:20,000
Uh, once this is done then I will go ahead and create my vector store.

195
00:10:20,000 --> 00:10:22,000
So I'll write vector db.

196
00:10:22,000 --> 00:10:27,000
Vector store DB is nothing but files dot from documents.

197
00:10:28,000 --> 00:10:32,000
And from this particular document I'll give my documents over here.

198
00:10:32,000 --> 00:10:36,000
Uh, and then along with that, I'm also going to give my OpenAI embeddings.

199
00:10:36,000 --> 00:10:37,000
Okay.

200
00:10:37,000 --> 00:10:39,000
So this will be the embedding techniques that I will use.

201
00:10:39,000 --> 00:10:42,000
And finally I go ahead and create my retrieval.

202
00:10:42,000 --> 00:10:43,000
Right.

203
00:10:43,000 --> 00:10:47,000
Uh, where I'll write vector db dot as retriever.

204
00:10:47,000 --> 00:10:48,000
Okay.

205
00:10:49,000 --> 00:10:53,000
And then finally you can see the retriever over here.

206
00:10:53,000 --> 00:10:53,000
Okay.

207
00:10:53,000 --> 00:10:56,000
So all these things we are basically doing it over here.

208
00:10:56,000 --> 00:11:00,000
And quickly I'm able to get this entire, uh, vector store retriever.

209
00:11:00,000 --> 00:11:00,000
Okay.

210
00:11:01,000 --> 00:11:02,000
So perfect till here.

211
00:11:02,000 --> 00:11:05,000
Everything looks good right now.

212
00:11:05,000 --> 00:11:10,000
This retriever, you can probably query any information and you can probably get any results from this

213
00:11:10,000 --> 00:11:13,000
particular page, whatever it has a documents okay.

214
00:11:13,000 --> 00:11:19,000
But in order to use it as a tool, I have to convert this retriever into a tool.

215
00:11:19,000 --> 00:11:19,000
Okay.

216
00:11:20,000 --> 00:11:27,000
So in order to convert this retriever into a tool, what I will do, I will say hey from lang chain

217
00:11:27,000 --> 00:11:31,000
dot tools okay.

218
00:11:31,000 --> 00:11:32,000
Dot retriever.

219
00:11:34,000 --> 00:11:38,000
import something called as create retriever tool.

220
00:11:38,000 --> 00:11:40,000
Okay I will use this.

221
00:11:40,000 --> 00:11:46,000
So this will basically be my retriever underscore tool which I will go ahead and use this create retriever

222
00:11:46,000 --> 00:11:51,000
tool and let me write which is my value.

223
00:11:51,000 --> 00:11:52,000
This is my retriever.

224
00:11:52,000 --> 00:11:54,000
I will give my tool name.

225
00:11:54,000 --> 00:11:58,000
I'll say hey this is my Lang Smith search.

226
00:11:58,000 --> 00:12:00,000
Okay, this is my line Smith search.

227
00:12:00,000 --> 00:12:03,000
And I will give some brief info like search.

228
00:12:04,000 --> 00:12:10,000
Let's say I'm just going to write over here, so I'll write.

229
00:12:10,000 --> 00:12:18,000
Hey, search any information about Land Smith okay.

230
00:12:18,000 --> 00:12:19,000
Done.

231
00:12:19,000 --> 00:12:20,000
Perfect.

232
00:12:20,000 --> 00:12:25,000
So this basically becomes my retrieval tool.

233
00:12:25,000 --> 00:12:28,000
And if I go ahead and write dot name similarly I'll be able to get this.

234
00:12:28,000 --> 00:12:31,000
See it is basically coming as Langsamt search okay.

235
00:12:32,000 --> 00:12:35,000
So these are how to probably use the inbuilt tools.

236
00:12:35,000 --> 00:12:37,000
And this is how you can actually create your own tools.

237
00:12:37,000 --> 00:12:38,000
Right?

238
00:12:38,000 --> 00:12:41,000
At the end of the day you just need to have a retriever.

239
00:12:41,000 --> 00:12:43,000
And from that retriever you'll be able to create this.

240
00:12:43,000 --> 00:12:44,000
Okay.

241
00:12:45,000 --> 00:12:47,000
Now let's go to the next step.

242
00:12:47,000 --> 00:12:56,000
And this time when I'm creating these tools right I will go ahead and add this particular Lang uh retriever

243
00:12:56,000 --> 00:12:57,000
tool also.

244
00:12:57,000 --> 00:12:58,000
Okay.

245
00:12:58,000 --> 00:13:00,000
So this basically becomes my tool.

246
00:13:00,000 --> 00:13:01,000
I will go ahead and execute it.

247
00:13:02,000 --> 00:13:05,000
And if you go ahead and see this, it is nothing.

248
00:13:05,000 --> 00:13:09,000
But it is a list of tools which is added with respect to each other.

249
00:13:09,000 --> 00:13:09,000
Right.

250
00:13:09,000 --> 00:13:12,000
So all with respect to the description.

251
00:13:12,000 --> 00:13:13,000
Everything is available over here.

252
00:13:13,000 --> 00:13:14,000
Perfect.

253
00:13:14,000 --> 00:13:19,000
Now, uh, what I'm actually going to do is that I'm going to in the next video.

254
00:13:19,000 --> 00:13:19,000
Right.

255
00:13:19,000 --> 00:13:27,000
We are going to run all this tools with agents and LM models.

256
00:13:27,000 --> 00:13:28,000
Okay.

257
00:13:28,000 --> 00:13:32,000
So this will what we will be discussing in our next video.

258
00:13:32,000 --> 00:13:34,000
So I hope you like this particular video.

259
00:13:34,000 --> 00:13:35,000
Uh, this was it from my side.

260
00:13:35,000 --> 00:13:36,000
I will see you all in the next video.

261
00:13:36,000 --> 00:13:37,000
Thank you.