1
00:00:00,000 --> 00:00:01,000
Hello guys.

2
00:00:01,000 --> 00:00:05,000
So we are going to continue the discussion with respect to the hybrid search.

3
00:00:05,000 --> 00:00:11,000
And uh, here now we are going to implement a complete amazing project with the help of hybrid search

4
00:00:11,000 --> 00:00:12,000
using Lang Chain.

5
00:00:12,000 --> 00:00:18,000
Now, uh, for this, uh, we will be using a vector database, which is called as pine code.

6
00:00:18,000 --> 00:00:23,000
Now, pine cone, uh, vector database provides you this both.

7
00:00:23,000 --> 00:00:23,000
Right.

8
00:00:23,000 --> 00:00:26,000
Like, uh, you will be able to do the keyword search.

9
00:00:26,000 --> 00:00:28,000
You will be able to do the semantic search.

10
00:00:28,000 --> 00:00:29,000
So let's go ahead.

11
00:00:29,000 --> 00:00:31,000
And first of all we will set up our account.

12
00:00:31,000 --> 00:00:35,000
And then with the help of code we will try to write each and every thing.

13
00:00:35,000 --> 00:00:40,000
And we'll try to probably go ahead and create our, you know, the vector database itself.

14
00:00:40,000 --> 00:00:43,000
So first of all just go ahead and sign up over here.

15
00:00:43,000 --> 00:00:44,000
It is completely for free.

16
00:00:45,000 --> 00:00:49,000
Uh, first two first 2 to 3 instances you can create completely for free.

17
00:00:49,000 --> 00:00:51,000
So I have already signed up.

18
00:00:51,000 --> 00:00:52,000
So I will go ahead and log in.

19
00:00:52,000 --> 00:00:54,000
You have to just go to pinecone.io.

20
00:00:54,000 --> 00:00:59,000
Now once you go inside this, you will be able to see that I have already created one pinecone hybrid

21
00:00:59,000 --> 00:01:00,000
search.

22
00:01:00,000 --> 00:01:00,000
Right?

23
00:01:00,000 --> 00:01:04,000
And if you go inside this, there are some kind of data that is already available over here.

24
00:01:04,000 --> 00:01:05,000
Right?

25
00:01:05,000 --> 00:01:11,000
But as usual, uh, we will not probably work with this, but instead we'll go ahead and create our

26
00:01:11,000 --> 00:01:11,000
new index.

27
00:01:11,000 --> 00:01:15,000
Okay, but since I really want to do everything from with the help of code.

28
00:01:15,000 --> 00:01:19,000
So, uh, we will try to create the instance even with the help of code.

29
00:01:19,000 --> 00:01:23,000
So first of all that you actually require is your API key okay.

30
00:01:23,000 --> 00:01:24,000
So just click on API key.

31
00:01:24,000 --> 00:01:28,000
Just make sure that this will be the default API key that will be provided over here.

32
00:01:28,000 --> 00:01:35,000
And uh you can also rotate the key values like you can probably go ahead and change it uh, if somebody

33
00:01:35,000 --> 00:01:36,000
probably sees it.

34
00:01:36,000 --> 00:01:36,000
Right.

35
00:01:36,000 --> 00:01:38,000
So here I will copy this particular key value.

36
00:01:38,000 --> 00:01:41,000
And then I will start my coding over here.

37
00:01:41,000 --> 00:01:42,000
Right.

38
00:01:42,000 --> 00:01:46,000
So uh, here I will be creating my instance.

39
00:01:46,000 --> 00:01:47,000
Everything as such.

40
00:01:47,000 --> 00:01:47,000
Okay.

41
00:01:47,000 --> 00:01:52,000
Now just quickly to go ahead with I have some good requirement dot txt here.

42
00:01:52,000 --> 00:01:54,000
You'll be able to see so many different libraries.

43
00:01:54,000 --> 00:01:58,000
But uh for lang chain obviously you require a lot of libraries.

44
00:01:58,000 --> 00:02:04,000
Uh, but if I probably talk about with respect to pinecone, I will be installing some other libraries

45
00:02:04,000 --> 00:02:04,000
over here.

46
00:02:04,000 --> 00:02:09,000
So let me quickly go ahead and, you know, just paste it over here.

47
00:02:09,000 --> 00:02:13,000
So I'll be installing pinecone client, pinecone text and pinecone notebooks.

48
00:02:13,000 --> 00:02:14,000
Okay.

49
00:02:14,000 --> 00:02:18,000
So uh, these are the initial libraries that we will be requiring.

50
00:02:18,000 --> 00:02:22,000
So I have already done the installation since I don't want to make my video very long.

51
00:02:22,000 --> 00:02:30,000
So here you'll be able to see a green tick and already the entire, uh, you know, the installation

52
00:02:30,000 --> 00:02:31,000
has basically happened.

53
00:02:31,000 --> 00:02:34,000
Then I will go ahead and write my API underscore key.

54
00:02:34,000 --> 00:02:40,000
This API underscore key will basically be replaced with the values over here.

55
00:02:40,000 --> 00:02:40,000
Right.

56
00:02:41,000 --> 00:02:43,000
So I'm just going to create this particular variable.

57
00:02:43,000 --> 00:02:46,000
You can also create an environment variable if you want okay.

58
00:02:46,000 --> 00:02:47,000
Perfect.

59
00:02:47,000 --> 00:02:50,000
Now uh since we are using Lang chain.

60
00:02:50,000 --> 00:02:57,000
So please make sure that we will go ahead and import this from lang chain underscore community dot retrievers

61
00:02:57,000 --> 00:03:06,000
I'm going to import I'm going to import a library which is nothing but pinecone hybrid search retriever.

62
00:03:06,000 --> 00:03:12,000
Now this this class is responsible in doing both the semantic search and syntactic search.

63
00:03:12,000 --> 00:03:13,000
Right.

64
00:03:13,000 --> 00:03:15,000
So keyword search like it.

65
00:03:15,000 --> 00:03:19,000
It has the combination of both the sparse matrix and dense vector search.

66
00:03:19,000 --> 00:03:21,000
And for that we have to create a retriever.

67
00:03:21,000 --> 00:03:23,000
And this is the class that we are going to use.

68
00:03:23,000 --> 00:03:26,000
So let's go ahead and execute this.

69
00:03:26,000 --> 00:03:32,000
And after executing this uh, the next thing will be that we need to make sure that we first of all

70
00:03:32,000 --> 00:03:34,000
create our index in our pinecone.

71
00:03:34,000 --> 00:03:34,000
Right.

72
00:03:34,000 --> 00:03:36,000
Without this, it will be of no use.

73
00:03:36,000 --> 00:03:42,000
So here, uh, quickly, what I will do, I will just go ahead and import import OS, since I have already

74
00:03:42,000 --> 00:03:43,000
imported pinecone.

75
00:03:43,000 --> 00:03:52,000
So I will go ahead and write from pinecone import pinecone comma server list spec.

76
00:03:52,000 --> 00:03:52,000
Okay.

77
00:03:52,000 --> 00:03:56,000
And then I will just go ahead and create my index name.

78
00:03:56,000 --> 00:04:01,000
My index name over here I will say, hey, this will be my hybrid search example.

79
00:04:01,000 --> 00:04:04,000
And please make sure that you give all this in a smaller letter.

80
00:04:04,000 --> 00:04:06,000
Otherwise you'll be getting an error.

81
00:04:06,000 --> 00:04:09,000
So I will say hey hybrid search Lang long chain pine cone.

82
00:04:09,000 --> 00:04:12,000
Okay, so this will basically be my index name.

83
00:04:12,000 --> 00:04:12,000
Okay.

84
00:04:13,000 --> 00:04:21,000
Uh, once I probably create the index name, we have to initialize the pinecone client.

85
00:04:21,000 --> 00:04:22,000
Okay.

86
00:04:22,000 --> 00:04:29,000
Then for initializing the pinecone client, I'll write PC is equal to pinecone, and we will initialize

87
00:04:29,000 --> 00:04:32,000
with this API key that I have actually created.

88
00:04:32,000 --> 00:04:32,000
Right.

89
00:04:32,000 --> 00:04:34,000
So over here is my API key.

90
00:04:34,000 --> 00:04:35,000
You can create your own API keys.

91
00:04:35,000 --> 00:04:37,000
No need to use mine okay.

92
00:04:37,000 --> 00:04:41,000
Because as soon as I upload this video uh, this will get deleted.

93
00:04:41,000 --> 00:04:41,000
Okay.

94
00:04:41,000 --> 00:04:43,000
All I'll rotate the key.

95
00:04:43,000 --> 00:04:47,000
Then, uh, we will go ahead and create the index.

96
00:04:47,000 --> 00:04:56,000
And here I'm going to basically write if index underscore name is not in PC dot list indexes.

97
00:04:56,000 --> 00:04:56,000
Right.

98
00:04:56,000 --> 00:04:57,000
We are listing all the indexes.

99
00:04:57,000 --> 00:05:03,000
If this particular name is not there, then only we should probably go ahead and create this.

100
00:05:03,000 --> 00:05:08,000
So I will write PC dot create index create underscore index.

101
00:05:08,000 --> 00:05:14,000
And here I'm going to give my name is equal to index underscore name whatever index underscore name

102
00:05:14,000 --> 00:05:16,000
we have specifically taken.

103
00:05:16,000 --> 00:05:20,000
Uh, whenever we try to create an index we also have to give our dimension.

104
00:05:20,000 --> 00:05:20,000
Okay.

105
00:05:20,000 --> 00:05:23,000
So here uh, like what will be the vector dimension.

106
00:05:23,000 --> 00:05:26,000
So here I'm going to use the vector dimension is equal to 384.

107
00:05:26,000 --> 00:05:28,000
The reason is very simple.

108
00:05:28,000 --> 00:05:34,000
Uh, the reason why I'm using and this is nothing but this is my dimension of dense model okay.

109
00:05:34,000 --> 00:05:35,000
Dense vector.

110
00:05:36,000 --> 00:05:39,000
The reason is very simple why I'm using 384.

111
00:05:39,000 --> 00:05:44,000
Because, uh, the embedding technique that I'm actually going to use is a hugging face embedding technique,

112
00:05:44,000 --> 00:05:49,000
which is nothing but sentence transformer that by default converts any text into 384 dimensions.

113
00:05:49,000 --> 00:05:53,000
So that is the reason why I'm creating this particular index with this dimension.

114
00:05:53,000 --> 00:05:54,000
Okay.

115
00:05:54,000 --> 00:05:57,000
Uh, along with this, the next thing that I'm going to use is metric.

116
00:05:57,000 --> 00:06:01,000
So for my metric here I will be using a dot product.

117
00:06:01,000 --> 00:06:06,000
So this is all information you specifically get when you are creating an index.

118
00:06:06,000 --> 00:06:12,000
Let's say if I go to over here and create my index so it's asking you for the dimension right over here.

119
00:06:12,000 --> 00:06:14,000
By default you can see metrics.

120
00:06:14,000 --> 00:06:16,000
Also it asks you whether you want to go with dot product cosine.

121
00:06:16,000 --> 00:06:19,000
And all right you can select any of this particular metric.

122
00:06:19,000 --> 00:06:22,000
So that is the reason why I am giving a metric over here.

123
00:06:22,000 --> 00:06:23,000
Right.

124
00:06:23,000 --> 00:06:25,000
So I'm saying hey just go ahead and do the dot product okay.

125
00:06:26,000 --> 00:06:31,000
And uh this dot product specifically I am going to do it for sparse values.

126
00:06:31,000 --> 00:06:32,000
Right.

127
00:06:32,000 --> 00:06:39,000
Sparse values supported only for dot product because we are creating this hybrid search for both of

128
00:06:39,000 --> 00:06:40,000
them.

129
00:06:40,000 --> 00:06:40,000
Right.

130
00:06:40,000 --> 00:06:44,000
And then finally you have the spec is equal to uh.

131
00:06:44,000 --> 00:06:47,000
For this particular spec I will be using serverless spec.

132
00:06:47,000 --> 00:06:54,000
And I have to probably give my cloud information like AWS it is and which region this also you'll be

133
00:06:54,000 --> 00:07:00,000
able to get inside that particular uh, so I will just go ahead and write us East one.

134
00:07:00,000 --> 00:07:01,000
Okay.

135
00:07:01,000 --> 00:07:05,000
So if you remember, uh, just now we saw this particular index also.

136
00:07:05,000 --> 00:07:10,000
So if I go ahead and see this here you have the serverless okay.

137
00:07:10,000 --> 00:07:13,000
All the information is over here let's say.

138
00:07:13,000 --> 00:07:16,000
And here you can select this US East one and all this is locked.

139
00:07:16,000 --> 00:07:17,000
So that is the reason I have written this.

140
00:07:17,000 --> 00:07:19,000
I have selected AWS.

141
00:07:19,000 --> 00:07:22,000
If you go ahead with Google you have to go ahead and write this right.

142
00:07:22,000 --> 00:07:25,000
So this is just a starter plan to go ahead with.

143
00:07:25,000 --> 00:07:27,000
So I'm going to use this free one okay.

144
00:07:27,000 --> 00:07:28,000
Perfect.

145
00:07:28,000 --> 00:07:31,000
So let's see whether this gets executed.

146
00:07:31,000 --> 00:07:37,000
So here I think everything looks good enough I have my index name uh PC dot create.

147
00:07:37,000 --> 00:07:40,000
And here I will just write comma okay.

148
00:07:43,000 --> 00:07:45,000
Oh okay.

149
00:07:45,000 --> 00:07:49,000
Just a second I will just update this entire thing.

150
00:07:51,000 --> 00:07:51,000
Okay, perfect.

151
00:07:51,000 --> 00:07:53,000
Now, this looks good.

152
00:07:53,000 --> 00:07:54,000
Exactly right.

153
00:07:54,000 --> 00:07:55,000
Uh, I have all the details.

154
00:07:55,000 --> 00:07:57,000
Now, let's go ahead and create this index.

155
00:07:57,000 --> 00:08:00,000
So here you will be able to see it will take some amount of time to create this index.

156
00:08:00,000 --> 00:08:04,000
But this is going to get created directly inside our vector database.

157
00:08:04,000 --> 00:08:11,000
Now if I go ahead and click on index, if I reload this I think it should get created okay.

158
00:08:11,000 --> 00:08:13,000
So here is my hybrid search link chain pinecone.

159
00:08:13,000 --> 00:08:15,000
And right now the dimension is 384.

160
00:08:15,000 --> 00:08:16,000
I don't have anything.

161
00:08:16,000 --> 00:08:17,000
The metric is dot product.

162
00:08:17,000 --> 00:08:21,000
The reason of using metric is that I really want to support both.

163
00:08:21,000 --> 00:08:24,000
Uh, that is for my sparse matrix and my vector index.

164
00:08:24,000 --> 00:08:24,000
Right.

165
00:08:24,000 --> 00:08:26,000
So here I have my all details.

166
00:08:27,000 --> 00:08:30,000
Now let's go ahead and further do the more coding over here.

167
00:08:30,000 --> 00:08:31,000
I have already created my API.

168
00:08:31,000 --> 00:08:37,000
Now let's click on my index and I will say hey PC dot index.

169
00:08:37,000 --> 00:08:39,000
Okay PC dot index.

170
00:08:39,000 --> 00:08:42,000
And here I'm just going to give my index underscore name.

171
00:08:42,000 --> 00:08:45,000
So let's go ahead and create see this index over here.

172
00:08:45,000 --> 00:08:49,000
So this is my pinecone index at this particular memory location.

173
00:08:49,000 --> 00:08:54,000
And obviously my index name that you'll be able to see over here that index name based on this index

174
00:08:54,000 --> 00:08:55,000
name I'm trying to take out that details.

175
00:08:55,000 --> 00:08:56,000
Okay.

176
00:08:56,000 --> 00:09:01,000
Um, now first thing first, uh, you know that, right?

177
00:09:01,000 --> 00:09:05,000
Our entire project is a combination of two different.

178
00:09:05,000 --> 00:09:11,000
One, I'll try to create a vector embedding, and the other one, I will try to create a sparse matrix.

179
00:09:11,000 --> 00:09:11,000
Okay.

180
00:09:11,000 --> 00:09:14,000
And that is what hybrid search basically tells you to do it right.

181
00:09:14,000 --> 00:09:17,000
So first of all what I will do I will go ahead and import OS.

182
00:09:17,000 --> 00:09:24,000
And uh here I will write from dot EMV import load underscore dot env okay.

183
00:09:24,000 --> 00:09:31,000
And we will be loading all our environment variables right now just to give you an idea about the environment

184
00:09:31,000 --> 00:09:36,000
variables, since I'm going to use an open source embedding technique from lang chain from Huggingface

185
00:09:36,000 --> 00:09:37,000
itself.

186
00:09:37,000 --> 00:09:42,000
So I will go ahead and import from lang chain underscore Huggingface.

187
00:09:43,000 --> 00:09:47,000
Okay, I'm going to import my hugging face embeddings okay.

188
00:09:47,000 --> 00:09:54,000
And here one of the embedding technique that we regularly use and with respect to the same dimensions,

189
00:09:54,000 --> 00:09:56,000
I'll be using this hugging face embeddings.

190
00:09:56,000 --> 00:10:02,000
And here I'm going to use my model underscore name is equal to all Dash mini.

191
00:10:02,000 --> 00:10:09,000
This is a sentence transformer lm l six and version v2 okay.

192
00:10:09,000 --> 00:10:13,000
So this will specifically be my embeddings.

193
00:10:13,000 --> 00:10:17,000
So here let's go ahead and see the embeddings over here quickly.

194
00:10:19,000 --> 00:10:20,000
Uh yeah.

195
00:10:20,000 --> 00:10:21,000
So my embedding is getting downloaded.

196
00:10:21,000 --> 00:10:23,000
And this will probably take some amount of time.

197
00:10:23,000 --> 00:10:26,000
And here you can see the word embedding is nothing but 384.

198
00:10:26,000 --> 00:10:30,000
Please note uh before doing this, if you directly execute it, it will not work.

199
00:10:30,000 --> 00:10:35,000
Because the reason it is working because I have already imported OS dot environ.

200
00:10:35,000 --> 00:10:38,000
You have to create this huggingface token right?

201
00:10:38,000 --> 00:10:40,000
H f underscore token.

202
00:10:40,000 --> 00:10:42,000
And this is already present in my environment variable.

203
00:10:42,000 --> 00:10:48,000
So I will go ahead and write OS dot get env is nothing but h f underscore token.

204
00:10:48,000 --> 00:10:51,000
You have to execute along with this if you have not.

205
00:10:51,000 --> 00:10:54,000
If you do not have the h f underscore token, this will not get loaded.

206
00:10:54,000 --> 00:10:55,000
Okay.

207
00:10:55,000 --> 00:10:57,000
So please make sure that you have this particular code.

208
00:10:57,000 --> 00:10:57,000
Perfect.

209
00:10:57,000 --> 00:10:59,000
So this was it.

210
00:11:00,000 --> 00:11:02,000
Um now let's go ahead and execute it once again.

211
00:11:02,000 --> 00:11:05,000
And now I will be having my entire huggingface embedding.

212
00:11:06,000 --> 00:11:09,000
Now, this embedding technique is used to create your dense vectors.

213
00:11:09,000 --> 00:11:16,000
Uh, if I go ahead and proceed further, uh, for my sparse matrix, I'm going to use one more.

214
00:11:16,000 --> 00:11:23,000
That is pinecone dot uh, and pinecone underscore text for my sparse.

215
00:11:23,000 --> 00:11:30,000
There is a, there is, there is an encoder which is called as BM two five encoder.

216
00:11:30,000 --> 00:11:35,000
Now this particular encoder uses tf IDF by default okay tf IDF techniques.

217
00:11:35,000 --> 00:11:42,000
So here I'm going to write PAF two five underscore encoder okay is equal to nothing.

218
00:11:42,000 --> 00:11:45,000
But I will just go ahead and initialize this encoder.

219
00:11:45,000 --> 00:11:50,000
And I will say hey use the default one which is nothing but tf IDF okay.

220
00:11:50,000 --> 00:11:53,000
And this basically becomes my BF two five encoder.

221
00:11:53,000 --> 00:11:54,000
Okay.

222
00:11:54,000 --> 00:11:59,000
And this is responsible in doing the sparse matrix conversion for any queries or for any text.

223
00:11:59,000 --> 00:11:59,000
Okay.

224
00:12:00,000 --> 00:12:08,000
Now, uh, I hope uh, you are very much clear with all these things now, and you are able to probably

225
00:12:08,000 --> 00:12:10,000
do all, all these amazing stuffs.

226
00:12:10,000 --> 00:12:14,000
And all right now I will try to create some sentences.

227
00:12:14,000 --> 00:12:16,000
Let's go ahead and create some sentences.

228
00:12:16,000 --> 00:12:19,000
And this sentences will be a list okay.

229
00:12:19,000 --> 00:12:24,000
And just to show you one example, I will just be creating some sentence over here with respect to this

230
00:12:24,000 --> 00:12:25,000
particular list.

231
00:12:25,000 --> 00:12:27,000
I will have this three sentences.

232
00:12:27,000 --> 00:12:35,000
Let's say uh, in 2023 I visited Paris, in 2022, I visited New York, and in 20 and 2021 I visited

233
00:12:35,000 --> 00:12:36,000
Ariana.

234
00:12:36,000 --> 00:12:44,000
Okay, now I will take all this information and quickly I will go ahead and write Bm25 underscore encoder

235
00:12:44,000 --> 00:12:45,000
dot.

236
00:12:45,000 --> 00:12:49,000
We are just going to fit on this entire sentences okay.

237
00:12:49,000 --> 00:12:58,000
So once we do this uh, in short we are just applying tf IDF values on this sentences.

238
00:12:58,000 --> 00:12:59,000
Okay.

239
00:13:00,000 --> 00:13:00,000
Perfect.

240
00:13:00,000 --> 00:13:11,000
So once I do this fit along with this, what I will do I will store the values to a JSON file.

241
00:13:11,000 --> 00:13:11,000
Okay.

242
00:13:12,000 --> 00:13:20,000
So here, uh, I will just go ahead and write p f bm BM 25 encoder dot dump okay.

243
00:13:20,000 --> 00:13:29,000
And I'm going to basically dump my BM 25 underscore values in the form of dot JSON.

244
00:13:29,000 --> 00:13:30,000
Okay.

245
00:13:31,000 --> 00:13:36,000
And here you'll be able to see that I'm actually dumping this entire information.

246
00:13:36,000 --> 00:13:41,000
You can also go ahead and load this just to make sure that you have that entire dump okay.

247
00:13:41,000 --> 00:13:46,000
And you can also load it so has no attribute dumpy okay.

248
00:13:46,000 --> 00:13:47,000
It was dump.

249
00:13:47,000 --> 00:13:47,000
No worries.

250
00:13:47,000 --> 00:13:48,000
Okay.

251
00:13:48,000 --> 00:13:50,000
So now here you can see that I have my encoder.

252
00:13:50,000 --> 00:13:52,000
And this is specifically for sparse matrix.

253
00:13:52,000 --> 00:13:52,000
Okay.

254
00:13:52,000 --> 00:13:52,000
okay.

255
00:13:53,000 --> 00:13:59,000
Um now one more thing that you will be able to see over here is that, uh, once I do this particular

256
00:13:59,000 --> 00:14:03,000
sparse matrix dump, uh, so I've actually created the sparse matrix for this, right.

257
00:14:03,000 --> 00:14:07,000
Now let's go ahead and create my retriever.

258
00:14:07,000 --> 00:14:12,000
And this retriever will be important because with the help of this specific retriever, only I will

259
00:14:12,000 --> 00:14:18,000
have the combination of both, uh, the embedding techniques and, uh, obviously the sparse embedding

260
00:14:18,000 --> 00:14:18,000
techniques.

261
00:14:18,000 --> 00:14:19,000
Right.

262
00:14:19,000 --> 00:14:23,000
So for that, I will go ahead and create this pinecone hybrid search retriever.

263
00:14:23,000 --> 00:14:28,000
And here I'm going to use my embeddings is equal to whatever embedding technique I have used.

264
00:14:28,000 --> 00:14:29,000
That is nothing but huggingface embeddings.

265
00:14:29,000 --> 00:14:37,000
And then my sparse encoder which will be assigned to my b b b b m to five encoder.

266
00:14:37,000 --> 00:14:40,000
And I have to also specify my index right.

267
00:14:40,000 --> 00:14:42,000
So this will basically be my index.

268
00:14:42,000 --> 00:14:45,000
Now once I execute this this is my retriever.

269
00:14:45,000 --> 00:14:50,000
Now with the help of this particular retriever it supports both embedding the vector dense embedding

270
00:14:50,000 --> 00:14:52,000
technique and along with that the sparse matrix.

271
00:14:52,000 --> 00:14:52,000
Okay.

272
00:14:52,000 --> 00:14:54,000
So this is my retriever.

273
00:14:54,000 --> 00:14:58,000
So I will just go ahead and display my retriever okay.

274
00:14:58,000 --> 00:14:59,000
So this is my retriever.

275
00:14:59,000 --> 00:15:02,000
And this is the entire information that you will be able to see.

276
00:15:02,000 --> 00:15:02,000
Right.

277
00:15:02,000 --> 00:15:04,000
So finally uh let's do one thing.

278
00:15:04,000 --> 00:15:08,000
Let's quickly go ahead and add all the text that I want.

279
00:15:08,000 --> 00:15:08,000
Right?

280
00:15:08,000 --> 00:15:13,000
So I will say, hey, we will go ahead and add the text and the text that we are going to add.

281
00:15:13,000 --> 00:15:15,000
Is all my sentences inside my index.

282
00:15:15,000 --> 00:15:15,000
Right.

283
00:15:15,000 --> 00:15:17,000
So what are my sentences?

284
00:15:17,000 --> 00:15:17,000
Here you go.

285
00:15:18,000 --> 00:15:22,000
I will copy this and I will paste it over here okay.

286
00:15:22,000 --> 00:15:24,000
So let's go ahead and execute this.

287
00:15:24,000 --> 00:15:29,000
Now here we'll be able to see that it will get inserted inside that particular index.

288
00:15:29,000 --> 00:15:29,000
Right.

289
00:15:29,000 --> 00:15:30,000
The index.

290
00:15:30,000 --> 00:15:32,000
uh, the pinecone index that we have created.

291
00:15:32,000 --> 00:15:35,000
Now it's time we can probably retrieve any results.

292
00:15:35,000 --> 00:15:38,000
First of all, let's see whether it has got inserted or not.

293
00:15:38,000 --> 00:15:41,000
So if I go over here, here you can see, see it's loading.

294
00:15:41,000 --> 00:15:43,000
And all these three sentences are now available.

295
00:15:43,000 --> 00:15:47,000
And this is all 384 dimensions.

296
00:15:47,000 --> 00:15:47,000
Right.

297
00:15:47,000 --> 00:15:51,000
So this 1 in 2021 I visited context uh Orleans.

298
00:15:51,000 --> 00:15:57,000
In 2023 I visited New York Paris and all right, now let's go ahead and ask some questions.

299
00:15:57,000 --> 00:16:00,000
And this time it is going to use the combination of both right.

300
00:16:00,000 --> 00:16:02,000
Your vector dense search and the other search.

301
00:16:02,000 --> 00:16:10,000
So here I'll say with what city did I visit last okay.

302
00:16:10,000 --> 00:16:14,000
So here you'll be able to see visit last.

303
00:16:15,000 --> 00:16:23,000
So here you can see I in 2021 I visited New Orleans, then New York, then visited Paris.

304
00:16:23,000 --> 00:16:25,000
All this page content is basically able to come over here.

305
00:16:25,000 --> 00:16:32,000
It is used both, uh, because this entire hybrid search retrieval is using both embeddings and the

306
00:16:32,000 --> 00:16:33,000
sparse matrix.

307
00:16:33,000 --> 00:16:33,000
Okay.

308
00:16:35,000 --> 00:16:36,000
If I say recent.

309
00:16:38,000 --> 00:16:41,000
So in 2021 all this information is basically coming up.

310
00:16:41,000 --> 00:16:41,000
Okay.

311
00:16:42,000 --> 00:16:45,000
Which city did I visit first?

312
00:16:45,000 --> 00:16:46,000
Okay.

313
00:16:46,000 --> 00:16:49,000
So here you can see in 2021 I visited neon.

314
00:16:49,000 --> 00:16:52,000
So all the information are basically you are able to see it over here.

315
00:16:52,000 --> 00:16:56,000
But internally it is taking a combination of both hybrid search.

316
00:16:56,000 --> 00:17:01,000
And uh, this is what is an amazing example which you can also go ahead and do it okay.

317
00:17:01,000 --> 00:17:04,000
So I hope you like this particular video.

318
00:17:04,000 --> 00:17:07,000
Uh, I will see you all in the next video.

319
00:17:07,000 --> 00:17:07,000
Thank you.

320
00:17:07,000 --> 00:17:08,000
Take care.

321
00:17:08,000 --> 00:17:08,000
Bye bye.