1
00:00:00,000 --> 00:00:00,000
Hello guys.

2
00:00:00,000 --> 00:00:03,000
So we are going to continue the discussion with respect to text summarization.

3
00:00:03,000 --> 00:00:08,000
Already in our previous video, we have seen the theoretical intuition behind stuff document Chain and

4
00:00:08,000 --> 00:00:10,000
map reduce text summarization technique.

5
00:00:10,000 --> 00:00:15,000
In this video, I will show you the practical implementation how you can specifically implement it.

6
00:00:15,000 --> 00:00:19,000
So first we will go with the stuff document chain text summarization technique.

7
00:00:19,000 --> 00:00:26,000
So here I have written a code of pi load pi pdf loader, where we will be loading a specific PDF and

8
00:00:26,000 --> 00:00:32,000
this PDF that I've actually taken over here you can see it is something called as FPGA speech dot PDF.

9
00:00:32,000 --> 00:00:38,000
Now inside this particular PDF there is an amazing speech that was given by and probably the last speech

10
00:00:38,000 --> 00:00:44,000
that was given by our former president, you know, uh, doctor APJ Abdul Kalam, who is a famous he

11
00:00:44,000 --> 00:00:46,000
who was a famous personality.

12
00:00:46,000 --> 00:00:49,000
And uh, this specific pdf, I will try to read it.

13
00:00:49,000 --> 00:00:50,000
Okay.

14
00:00:50,000 --> 00:00:54,000
So now let's go ahead and do this and we'll try to summarize this entire PDF, okay.

15
00:00:54,000 --> 00:00:57,000
In entire text inside this particular PDF.

16
00:00:57,000 --> 00:00:59,000
So now if I go ahead and just execute this.

17
00:00:59,000 --> 00:01:02,000
So here I'm actually going to get this one.

18
00:01:03,000 --> 00:01:07,000
So as soon as I read this PDF here you can see I'm getting so many number of documents.

19
00:01:07,000 --> 00:01:08,000
Right.

20
00:01:08,000 --> 00:01:12,000
And uh in order to just write pages over here, I'll just go ahead and write documents.

21
00:01:12,000 --> 00:01:17,000
So here are all the documents that I have got from reading from this particular PDF file.

22
00:01:17,000 --> 00:01:22,000
Now I will take up all this particular documents and then uh, we will try to summarize it first of

23
00:01:22,000 --> 00:01:24,000
all by using the stuff document chain.

24
00:01:24,000 --> 00:01:29,000
Now in order to summarize it, uh, with respect to the stuff document chain, first of all, you know

25
00:01:29,000 --> 00:01:33,000
that we need to go ahead and create our prompt template.

26
00:01:33,000 --> 00:01:35,000
So for that we will create our template first of all.

27
00:01:35,000 --> 00:01:37,000
So let's go ahead and write it down over here.

28
00:01:37,000 --> 00:01:40,000
So I'll go ahead and write hey this is my template.

29
00:01:40,000 --> 00:01:49,000
And I will make a, uh, I'll just make a triple quotes for multi-line, uh, multi-line comments or

30
00:01:49,000 --> 00:01:51,000
sentences that I really want to use.

31
00:01:51,000 --> 00:01:58,000
Okay, so inside this template I will go ahead and say, hey, uh, write, uh, concise.

32
00:01:59,000 --> 00:02:07,000
Write a concise and short summary of the following speech.

33
00:02:07,000 --> 00:02:08,000
Okay.

34
00:02:08,000 --> 00:02:11,000
If you remember, uh, in our previous video.

35
00:02:11,000 --> 00:02:12,000
Right.

36
00:02:12,000 --> 00:02:15,000
Uh, what did we basically say?

37
00:02:15,000 --> 00:02:17,000
If I want to use a stuff document chain?

38
00:02:17,000 --> 00:02:22,000
First of all, if this document fits in the LM context window, I am going to, first of all, create

39
00:02:22,000 --> 00:02:26,000
this, uh, prompt, uh, prompt template itself.

40
00:02:26,000 --> 00:02:30,000
I'll say extract a final summary from the documents, and then I'll pass it to the LLM.

41
00:02:30,000 --> 00:02:32,000
And finally I'll get the final summary.

42
00:02:32,000 --> 00:02:37,000
So this is the same, uh, fundamental thing that I'm going to apply, uh, the same chain I'm going

43
00:02:37,000 --> 00:02:39,000
to specifically apply over here.

44
00:02:39,000 --> 00:02:39,000
Okay.

45
00:02:39,000 --> 00:02:41,000
So quickly let's go ahead and continue it.

46
00:02:41,000 --> 00:02:47,000
And here I'm going to basically say write a course concise and short summary of the following speech.

47
00:02:47,000 --> 00:02:50,000
And here I will just go ahead and give my speech itself.

48
00:02:50,000 --> 00:02:53,000
The speech will be nothing but with this.

49
00:02:53,000 --> 00:02:56,000
And I'll use an input variable where I'll say it is a text.

50
00:02:56,000 --> 00:02:57,000
Okay.

51
00:02:58,000 --> 00:02:59,000
Uh, after creating this particular template.

52
00:02:59,000 --> 00:03:02,000
Now let me just go ahead and define my prompt.

53
00:03:02,000 --> 00:03:05,000
So prompt will be nothing, but I will use the same prompt template.

54
00:03:05,000 --> 00:03:08,000
And inside this I will go ahead and use my input variables.

55
00:03:08,000 --> 00:03:13,000
And inside this input variables I will just go ahead and write my input variable as text.

56
00:03:13,000 --> 00:03:14,000
Because this is the information.

57
00:03:14,000 --> 00:03:20,000
This will be the uh text that I am going to pick it from here right from this entire documents.

58
00:03:21,000 --> 00:03:25,000
So here I'll actually go went ahead and probably created my prompt template.

59
00:03:25,000 --> 00:03:28,000
So, uh, if I just go ahead and execute this.

60
00:03:28,000 --> 00:03:34,000
So one error you can probably see speech is equal to text okay I think so.

61
00:03:34,000 --> 00:03:36,000
I have not given my okay.

62
00:03:36,000 --> 00:03:37,000
After this.

63
00:03:37,000 --> 00:03:40,000
The second parameter that we really need to give is our template.

64
00:03:40,000 --> 00:03:40,000
Right.

65
00:03:40,000 --> 00:03:43,000
Because it is expecting template over here.

66
00:03:43,000 --> 00:03:45,000
So I'll just go ahead and write it okay.

67
00:03:45,000 --> 00:03:50,000
So once this is getting executed this has basically created my entire prompt template.

68
00:03:50,000 --> 00:03:56,000
Now the most important thing that we are going to discuss how do we apply this stuff.

69
00:03:56,000 --> 00:04:02,000
Document chain text summarization on this entire text by using this particular prompt template.

70
00:04:03,000 --> 00:04:07,000
So in long chain we have something called as from lang chain.

71
00:04:07,000 --> 00:04:08,000
Let me just go ahead and import this.

72
00:04:08,000 --> 00:04:09,000
And then I'll explain you.

73
00:04:09,000 --> 00:04:12,000
So in long chain you have something called as chains.

74
00:04:12,000 --> 00:04:15,000
And inside this you have a module which is called as summarize.

75
00:04:15,000 --> 00:04:19,000
And whenever you want to create this entire text summarize chain itself.

76
00:04:19,000 --> 00:04:24,000
You can use this import library called as load summarize chain okay.

77
00:04:24,000 --> 00:04:30,000
Load summarize chain is just going to combine this entire thing uh, with respect to the implementation

78
00:04:30,000 --> 00:04:32,000
that we are doing for the text summarization.

79
00:04:32,000 --> 00:04:33,000
Okay.

80
00:04:33,000 --> 00:04:36,000
So I will just go ahead and execute this.

81
00:04:36,000 --> 00:04:39,000
Now, what I will do, I will just go ahead and create my chain.

82
00:04:40,000 --> 00:04:43,000
In this chain I will just use load summarization.

83
00:04:43,000 --> 00:04:46,000
Now understand what we are basically using.

84
00:04:46,000 --> 00:04:47,000
First of all we need our LM model.

85
00:04:47,000 --> 00:04:51,000
The LM model is used as Chad Grok that same LM model that we are going to use.

86
00:04:51,000 --> 00:04:55,000
And then we have our chain type okay.

87
00:04:55,000 --> 00:04:59,000
Now inside this chain type these are the three chain types that I'm going to use.

88
00:04:59,000 --> 00:05:03,000
One is stuff uh one is map reduce and the other one is refine.

89
00:05:03,000 --> 00:05:03,000
Okay.

90
00:05:03,000 --> 00:05:08,000
We have already discussed the theoretical intuition between, uh, you know, we have understood the

91
00:05:08,000 --> 00:05:12,000
theoretical intuition, uh, with respect to stuff document chain and map reduce.

92
00:05:12,000 --> 00:05:15,000
So let me just go ahead and give the parameter as stuff.

93
00:05:15,000 --> 00:05:15,000
Okay.

94
00:05:15,000 --> 00:05:21,000
Then the third parameter that we have to give over here is nothing but my prompt, which will be my

95
00:05:21,000 --> 00:05:22,000
prompt over here.

96
00:05:22,000 --> 00:05:28,000
And then finally I will go ahead and write verbose is equal to true to see all the information whenever

97
00:05:28,000 --> 00:05:30,000
I probably run this particular chain okay.

98
00:05:31,000 --> 00:05:37,000
Now the most important thing what exactly is happening when I keep this particular chain type as stuff?

99
00:05:37,000 --> 00:05:43,000
It is just going to combine all these documents and it is going to pass it to the prompt template along

100
00:05:43,000 --> 00:05:43,000
with the prompt template.

101
00:05:43,000 --> 00:05:46,000
It is going to pass it to the LLM module.

102
00:05:46,000 --> 00:05:50,000
Right now this is the chain that is specifically created.

103
00:05:50,000 --> 00:05:55,000
So let me go ahead and explain what this load underscore summarize chain will basically do.

104
00:05:55,000 --> 00:06:00,000
So here you will be able to see that I have a list of documents.

105
00:06:00,000 --> 00:06:03,000
So whenever we use this chain type is equal to stuff.

106
00:06:03,000 --> 00:06:07,000
What it is going to do is that it is going to combine all these documents.

107
00:06:07,000 --> 00:06:12,000
Once it is going to combine all the documents, then it is going to create a chain wherein this documents

108
00:06:12,000 --> 00:06:14,000
will be passed to the prompt template.

109
00:06:14,000 --> 00:06:17,000
From this prompt template, it will be passed to the LLM model.

110
00:06:17,000 --> 00:06:22,000
And finally we get the entire text summarization of all that particular documents.

111
00:06:22,000 --> 00:06:22,000
Okay.

112
00:06:22,000 --> 00:06:27,000
And this is this will happen when when you go ahead and execute this chain.

113
00:06:27,000 --> 00:06:28,000
Right.

114
00:06:28,000 --> 00:06:32,000
So for that you just need to write chain dot run.

115
00:06:32,000 --> 00:06:35,000
And inside this I will go ahead and pass my documents okay.

116
00:06:35,000 --> 00:06:40,000
Now with respect to this documents I will be getting my output summary okay.

117
00:06:41,000 --> 00:06:43,000
And I will just go ahead and execute it.

118
00:06:43,000 --> 00:06:47,000
Now see the magic okay I'm just going to display the output summary.

119
00:06:47,000 --> 00:06:49,000
So docs was written over here.

120
00:06:49,000 --> 00:06:51,000
Let's see I have not executed this.

121
00:06:52,000 --> 00:06:55,000
Oh now I have executed this docs.

122
00:06:55,000 --> 00:06:56,000
Now it's fine.

123
00:06:56,000 --> 00:06:58,000
We should be able to get the output now.

124
00:06:58,000 --> 00:07:01,000
See, it is entering into the new stuff document chain.

125
00:07:02,000 --> 00:07:08,000
It's entering into the LLM chain that is basically being created with the help of prompt and LLM model.

126
00:07:08,000 --> 00:07:10,000
And here is my entire prompt.

127
00:07:10,000 --> 00:07:13,000
Write a concise and short summary of the following speech.

128
00:07:13,000 --> 00:07:16,000
And this is the speech where you will be able to see APJ Abdul Kalam.

129
00:07:16,000 --> 00:07:18,000
This is the entire speech.

130
00:07:18,000 --> 00:07:22,000
And finally you can see that there are some more content which you are not able to see.

131
00:07:22,000 --> 00:07:24,000
But we get the final summary over here.

132
00:07:24,000 --> 00:07:25,000
Right?

133
00:07:25,000 --> 00:07:28,000
So it shows APJ Abdul Kalam departing speech.

134
00:07:28,000 --> 00:07:35,000
In this departing speech, former uh president APJ Abdul Kalam highlights the remarkable progress made

135
00:07:35,000 --> 00:07:38,000
by uh India and various sectors and emphasizes the potential.

136
00:07:38,000 --> 00:07:39,000
Right.

137
00:07:39,000 --> 00:07:42,000
So this entire information is basically my summary.

138
00:07:42,000 --> 00:07:47,000
So how easy it was basically to do with the help of this stuff.

139
00:07:47,000 --> 00:07:51,000
Document chain type for text summarization okay.

140
00:07:51,000 --> 00:08:00,000
Now similarly what we will do is that we will also go ahead and I'll show you one example for your MapReduce.

141
00:08:00,000 --> 00:08:01,000
Okay.

142
00:08:01,000 --> 00:08:10,000
So MapReduce or to summarize summarize or large documents.

143
00:08:10,000 --> 00:08:16,000
Okay, now here with respect to MapReduce, I have already shown told you like how it is going to work.

144
00:08:16,000 --> 00:08:19,000
First of all, all the document chunks will be made right.

145
00:08:20,000 --> 00:08:21,000
All the document chunks will be named.

146
00:08:21,000 --> 00:08:25,000
Then for each chunk we will try to create a prompt template.

147
00:08:25,000 --> 00:08:29,000
That prompt template can be same and it is going to summarize for that particular chunk.

148
00:08:29,000 --> 00:08:31,000
And finally we will be able to get the output okay.

149
00:08:31,000 --> 00:08:37,000
Once we combine all the chunks and then we send it with another prompt template finally to get the output.

150
00:08:37,000 --> 00:08:43,000
So just to show you what all things we are basically going to do over here, uh, I will also go ahead

151
00:08:43,000 --> 00:08:48,000
and import from Lang chain dot text splitter.

152
00:08:48,000 --> 00:08:54,000
I'm going to import recursive character text splitter because we need to divide that into chunks.

153
00:08:54,000 --> 00:08:54,000
Right.

154
00:08:54,000 --> 00:08:56,000
So we are going to import this okay.

155
00:08:56,000 --> 00:09:00,000
Now you know that uh, this is my documents right.

156
00:09:00,000 --> 00:09:01,000
Or my documents.

157
00:09:01,000 --> 00:09:02,000
I have already read it.

158
00:09:02,000 --> 00:09:07,000
So this is my documents now for this particular documents, what I am actually going to specifically

159
00:09:07,000 --> 00:09:13,000
do is that, uh, I will go ahead and probably create a chat prompt template.

160
00:09:13,000 --> 00:09:18,000
But before that, uh, what I will do is that I will go ahead and use this text splitter.

161
00:09:18,000 --> 00:09:24,000
I will say, hey, create an object of text splitter where I'll say recursive character text splitter

162
00:09:25,000 --> 00:09:34,000
and I'll say, okay, let's go and make this chunk size of 500 or let's say 2000 okay.

163
00:09:34,000 --> 00:09:37,000
And here I'm also going to make a chunk overlap.

164
00:09:37,000 --> 00:09:42,000
So first of all I'm going to divide this documents based on this chunk size and chunk overlap okay.

165
00:09:42,000 --> 00:09:48,000
And I'm considering that uh, the document that is basically present with me, it fits, uh, it is

166
00:09:48,000 --> 00:09:51,000
more than the context size or context window size of the LM models.

167
00:09:51,000 --> 00:09:51,000
Right.

168
00:09:51,000 --> 00:09:54,000
So here I'm going to use a chunk overlap of 100 okay.

169
00:09:54,000 --> 00:09:56,000
So this is what is my text splitter.

170
00:09:56,000 --> 00:10:02,000
And uh here what I will specifically do is that I will take this entire text splitter.

171
00:10:03,000 --> 00:10:03,000
Okay.

172
00:10:03,000 --> 00:10:06,000
I will say from documents okay.

173
00:10:06,000 --> 00:10:15,000
So it should be from underscore documents I hope so it is right.

174
00:10:16,000 --> 00:10:19,000
Um text splitter dot okay.

175
00:10:19,000 --> 00:10:22,000
Or instead of this, you know, if I just use split documents.

176
00:10:22,000 --> 00:10:23,000
Let's see.

177
00:10:24,000 --> 00:10:28,000
So if I just use this function of split documents I will be able to get my final documents.

178
00:10:28,000 --> 00:10:30,000
I can also do this okay.

179
00:10:30,000 --> 00:10:34,000
So I don't want to write much line of code line by line Okay.

180
00:10:34,000 --> 00:10:37,000
Then finally I can see my final documents.

181
00:10:37,000 --> 00:10:39,000
So documents.

182
00:10:39,000 --> 00:10:39,000
Okay.

183
00:10:39,000 --> 00:10:41,000
I have to provide my documents over here.

184
00:10:42,000 --> 00:10:44,000
So let's go ahead and give my docs.

185
00:10:44,000 --> 00:10:45,000
And here you have this.

186
00:10:45,000 --> 00:10:48,000
So here I you have seen that already.

187
00:10:48,000 --> 00:10:52,000
I had this kind of chunks of documents but I still divided this into many more chunks.

188
00:10:52,000 --> 00:10:53,000
Okay.

189
00:10:53,000 --> 00:10:57,000
Now what is going to happen if I use MapReduce?

190
00:10:57,000 --> 00:10:58,000
Okay.

191
00:10:58,000 --> 00:11:00,000
it will first of all take this chunk.

192
00:11:00,000 --> 00:11:05,000
It will pass to the prompt template and then it will summarize this with the help of LM model.

193
00:11:05,000 --> 00:11:11,000
Similarly, it will send the next sentence or next document to the prompt template along with the LM

194
00:11:11,000 --> 00:11:12,000
model in a chain.

195
00:11:12,000 --> 00:11:14,000
And it will summarize this.

196
00:11:14,000 --> 00:11:17,000
So all these documents will get summarized in a separate way.

197
00:11:17,000 --> 00:11:23,000
And finally, whatever summary we get, we can combine them with another prompt template and get the

198
00:11:23,000 --> 00:11:23,000
final summary.

199
00:11:23,000 --> 00:11:26,000
That is what I am planning to do it over here.

200
00:11:26,000 --> 00:11:28,000
Okay, so here you can actually see.

201
00:11:28,000 --> 00:11:32,000
And if I also go ahead and see my length of the final documents.

202
00:11:32,000 --> 00:11:36,000
So this many are like 13 documents are basically there okay.

203
00:11:37,000 --> 00:11:41,000
Now quickly let's go ahead and create our uh prompt template.

204
00:11:41,000 --> 00:11:45,000
And you know that our prompt template, uh, how it is basically created.

205
00:11:45,000 --> 00:11:49,000
Um, over there, we have specifically used something like this.

206
00:11:49,000 --> 00:11:49,000
Right.

207
00:11:49,000 --> 00:11:53,000
So this is my prompt, right, with respect to this particular prompt.

208
00:11:53,000 --> 00:11:53,000
Right.

209
00:11:53,000 --> 00:11:57,000
Um, so what I will do, I will just go ahead and say, okay.

210
00:11:57,000 --> 00:12:02,000
And uh, let's go ahead and create that all prompt over here.

211
00:12:02,000 --> 00:12:04,000
So this will be my first prompt.

212
00:12:04,000 --> 00:12:06,000
Let's say for this I will create another prompt.

213
00:12:06,000 --> 00:12:06,000
Okay.

214
00:12:06,000 --> 00:12:08,000
So here is my chunk underscore prompt.

215
00:12:08,000 --> 00:12:10,000
I'll say please summarize the below.

216
00:12:10,000 --> 00:12:12,000
Speech and speech is equal to text summary.

217
00:12:12,000 --> 00:12:13,000
So much information.

218
00:12:13,000 --> 00:12:15,000
And here I'll get the summary right.

219
00:12:15,000 --> 00:12:17,000
And uh this is my prompt template here.

220
00:12:17,000 --> 00:12:19,000
My input variable is text.

221
00:12:19,000 --> 00:12:22,000
And template is nothing but this chunk prompt okay.

222
00:12:22,000 --> 00:12:23,000
Finally I'll go ahead and execute it.

223
00:12:23,000 --> 00:12:26,000
This is my uh, map prompt okay.

224
00:12:26,000 --> 00:12:31,000
Now understand for all the chunks I'm going to apply this particular prompt okay.

225
00:12:31,000 --> 00:12:35,000
So in order to execute it what I will do I will go ahead and create my summary chain again.

226
00:12:36,000 --> 00:12:39,000
So this will basically be my summary chain.

227
00:12:39,000 --> 00:12:43,000
Okay I'll create a variable and I'll say hey I'll call that load summarize chain again.

228
00:12:43,000 --> 00:12:46,000
Again if you remember the first parameter was nothing but LM.

229
00:12:46,000 --> 00:12:48,000
LM is equal to lm.

230
00:12:48,000 --> 00:12:51,000
The second parameter was something called as chain type, right?

231
00:12:51,000 --> 00:12:53,000
Previously we gave the chain type as stuff.

232
00:12:53,000 --> 00:12:57,000
But in this case we will give it as Map-reduce okay.

233
00:12:57,000 --> 00:13:04,000
And the third one what we will do is that we will give the map prompt template sorry map underscore

234
00:13:04,000 --> 00:13:06,000
prompt okay.

235
00:13:07,000 --> 00:13:10,000
Map underscore prompt is equal to.

236
00:13:10,000 --> 00:13:18,000
And here I'm actually going to give my chunks chunk prompt okay.

237
00:13:18,000 --> 00:13:21,000
So this will specifically be my chunk prompt.

238
00:13:21,000 --> 00:13:26,000
Or instead of giving chunk prompt I'll give this map prompt template because I need to give a prompt

239
00:13:26,000 --> 00:13:27,000
template over here.

240
00:13:27,000 --> 00:13:30,000
Now understand one thing, okay.

241
00:13:30,000 --> 00:13:35,000
This prompt, uh, this prompt template that we have created, this is for the chunks, right.

242
00:13:36,000 --> 00:13:40,000
Finally, when I get the summary of all these things right, separate, separate summary for all these

243
00:13:40,000 --> 00:13:44,000
documents, I also need to create another prompt template.

244
00:13:44,000 --> 00:13:44,000
Right.

245
00:13:44,000 --> 00:13:48,000
And that will basically be my final prompt.

246
00:13:48,000 --> 00:13:55,000
So I'll say, hey, this is my final prompt and this will be for my all the summarized text with respect

247
00:13:55,000 --> 00:13:55,000
to the chunks.

248
00:13:55,000 --> 00:13:55,000
Right.

249
00:13:55,000 --> 00:14:02,000
So here I will just go ahead and write this quickly and let's write it over here.

250
00:14:02,000 --> 00:14:16,000
So what I will say over here is that provide the final summary of the entire speech with these important

251
00:14:16,000 --> 00:14:17,000
points.

252
00:14:17,000 --> 00:14:19,000
I'll just go ahead and write like this.

253
00:14:19,000 --> 00:14:29,000
Okay, first of all, I'll say, hey, go ahead and add a generic or I'll say, hey, go ahead and add

254
00:14:29,000 --> 00:14:30,000
a motivational title.

255
00:14:31,000 --> 00:14:31,000
Okay.

256
00:14:31,000 --> 00:14:36,000
Then start the precise summary.

257
00:14:38,000 --> 00:14:43,000
Summary with an introduction.

258
00:14:43,000 --> 00:14:44,000
I'm just writing my own prompt.

259
00:14:44,000 --> 00:14:44,000
Okay.

260
00:14:45,000 --> 00:14:50,000
And provide the summary in number.

261
00:14:50,000 --> 00:14:55,000
In in number points for the speech.

262
00:14:55,000 --> 00:14:56,000
Okay.

263
00:14:56,000 --> 00:15:01,000
So I basically want the entire summary in the form of numbers.

264
00:15:01,000 --> 00:15:03,000
So that is the reason I'm writing this.

265
00:15:03,000 --> 00:15:03,000
Okay.

266
00:15:03,000 --> 00:15:06,000
Finally I'll go ahead and write my speech.

267
00:15:06,000 --> 00:15:09,000
And this speech will basically be having my text.

268
00:15:09,000 --> 00:15:12,000
So here you can see that this ad has got added.

269
00:15:12,000 --> 00:15:14,000
So let me just go ahead and remove it okay.

270
00:15:14,000 --> 00:15:17,000
So this basically becomes my final prompt.

271
00:15:17,000 --> 00:15:23,000
Now I'm going to combine this uh as my final prompt template okay.

272
00:15:23,000 --> 00:15:25,000
Prompt template.

273
00:15:25,000 --> 00:15:26,000
This will be my final prompt.

274
00:15:26,000 --> 00:15:34,000
Remember when this will get applied, when the entire summary of, uh, the chunks have actually got

275
00:15:34,000 --> 00:15:34,000
right.

276
00:15:34,000 --> 00:15:37,000
So in order to show you the diagram over here.

277
00:15:37,000 --> 00:15:37,000
Right.

278
00:15:37,000 --> 00:15:39,000
Let me just show you the diagram.

279
00:15:39,000 --> 00:15:39,000
Okay.

280
00:15:39,000 --> 00:15:40,000
Quickly.

281
00:15:40,000 --> 00:15:43,000
And then here you will be able to understand it.

282
00:15:43,000 --> 00:15:45,000
So here you can see I'm implementing this.

283
00:15:45,000 --> 00:15:48,000
The first prompt has been created for the chunks right.

284
00:15:48,000 --> 00:15:49,000
I will get all the summarize.

285
00:15:49,000 --> 00:15:53,000
So second will be my final prompt which I am actually implementing.

286
00:15:53,000 --> 00:15:57,000
Okay so let's go back to uh my code okay.

287
00:15:57,000 --> 00:16:03,000
So final prompt template you can see over here I will go ahead and say hey this will basically be my

288
00:16:03,000 --> 00:16:04,000
prompt template.

289
00:16:04,000 --> 00:16:10,000
And this my input variables will be nothing but it will be text okay.

290
00:16:10,000 --> 00:16:12,000
This is how we specifically write.

291
00:16:12,000 --> 00:16:19,000
And here you will be able to see template is equal to final uh final prompt.

292
00:16:19,000 --> 00:16:22,000
Or I can go ahead and write this final prompt.

293
00:16:22,000 --> 00:16:23,000
Okay.

294
00:16:23,000 --> 00:16:24,000
I have to close this.

295
00:16:24,000 --> 00:16:24,000
Right.

296
00:16:24,000 --> 00:16:29,000
So this will basically be my input template okay.

297
00:16:29,000 --> 00:16:30,000
Done.

298
00:16:30,000 --> 00:16:36,000
So if I go ahead and execute this final prompt template this is what is the template I'm going to use

299
00:16:36,000 --> 00:16:37,000
over here in the summarize chain.

300
00:16:37,000 --> 00:16:39,000
Now this became my first template.

301
00:16:39,000 --> 00:16:46,000
The second template parameter that we have in load summarize chain is something called as combine prompt

302
00:16:47,000 --> 00:16:48,000
okay.

303
00:16:48,000 --> 00:16:50,000
So if I go ahead and write combine prompt.

304
00:16:50,000 --> 00:16:57,000
Now what does combine prompt basically says that this map prompt is the first prompt template which

305
00:16:57,000 --> 00:16:58,000
we are going to use for.

306
00:16:58,000 --> 00:17:00,000
And this will give the summary.

307
00:17:00,000 --> 00:17:05,000
This will basically combine with the LM and summary and will give the summary of the smaller chunks.

308
00:17:05,000 --> 00:17:05,000
Right.

309
00:17:05,000 --> 00:17:10,000
Uh, once we get all the summaries, then we can combine them with this specific prompt template.

310
00:17:10,000 --> 00:17:15,000
That basically means all the summary will be combined and sent to this prompt template.

311
00:17:15,000 --> 00:17:15,000
Right.

312
00:17:15,000 --> 00:17:17,000
So here I will go ahead and write this.

313
00:17:17,000 --> 00:17:19,000
And finally I will go ahead and write.

314
00:17:19,000 --> 00:17:23,000
Verbose is equal to true so that you will be able to see the details.

315
00:17:23,000 --> 00:17:27,000
Now quickly let's go ahead and execute this here I'm going to write.

316
00:17:27,000 --> 00:17:34,000
Output is nothing but I'll write summary whatever summary chain.

317
00:17:34,000 --> 00:17:41,000
We are basically going to create our summary chain dot run.

318
00:17:41,000 --> 00:17:47,000
And here what I will be giving I will be giving my docs write whatever docs I have.

319
00:17:47,000 --> 00:17:47,000
Not docs.

320
00:17:47,000 --> 00:17:48,000
Sorry.

321
00:17:48,000 --> 00:17:51,000
Uh, I should be giving this final documents.

322
00:17:51,000 --> 00:17:51,000
Okay?

323
00:17:51,000 --> 00:17:53,000
Not docs, but final documents.

324
00:17:53,000 --> 00:17:53,000
Quickly.

325
00:17:53,000 --> 00:17:55,000
Let's go ahead and execute this.

326
00:18:00,000 --> 00:18:01,000
I'll go ahead and execute this over here.

327
00:18:02,000 --> 00:18:04,000
So this will basically be my output.

328
00:18:04,000 --> 00:18:05,000
Now let's execute this.

329
00:18:05,000 --> 00:18:06,000
See the magic now.

330
00:18:06,000 --> 00:18:11,000
So entering a MapReduce document chain it has entered the LM chain.

331
00:18:11,000 --> 00:18:17,000
First of all it took this prompt template I am getting the summary right.

332
00:18:17,000 --> 00:18:20,000
Then it went into a new chain with the final prompt.

333
00:18:20,000 --> 00:18:21,000
So here you can say find.

334
00:18:21,000 --> 00:18:24,000
Please provide the final summary of the entire speech.

335
00:18:24,000 --> 00:18:25,000
Add a motivational title.

336
00:18:25,000 --> 00:18:31,000
So this is the output that you are getting right and this is how you are displaying your output.

337
00:18:31,000 --> 00:18:32,000
Introduction.

338
00:18:32,000 --> 00:18:33,000
This this this.

339
00:18:33,000 --> 00:18:39,000
So so based on this when I did this verbose is equal to true here you could see that initially we when

340
00:18:39,000 --> 00:18:44,000
we applied this MapReduce documentation, the first prompt template that got applied was this right

341
00:18:44,000 --> 00:18:45,000
for every chunks.

342
00:18:45,000 --> 00:18:47,000
Then we got all the summaries.

343
00:18:47,000 --> 00:18:53,000
And finally we combined all the summaries and passed into a another prompt template, which is nothing

344
00:18:53,000 --> 00:18:54,000
but your final prompt template.

345
00:18:54,000 --> 00:18:58,000
And here you get the complete final response.

346
00:18:58,000 --> 00:19:06,000
So this was the two types of uh text summarization techniques, which we specifically use in load summarization.

347
00:19:06,000 --> 00:19:06,000
Okay.

348
00:19:06,000 --> 00:19:08,000
I'll just make one more change over here.

349
00:19:08,000 --> 00:19:10,000
I'll just make the spelling.

350
00:19:10,000 --> 00:19:10,000
Right.

351
00:19:10,000 --> 00:19:11,000
Okay.

352
00:19:11,000 --> 00:19:14,000
Now in our next video we are going to see about refine.

353
00:19:14,000 --> 00:19:14,000
Right.

354
00:19:14,000 --> 00:19:17,000
And uh, the differences between refine and all.

355
00:19:17,000 --> 00:19:20,000
We will try to discuss, write, refine text summarization technique and all.

356
00:19:20,000 --> 00:19:24,000
There is just a minor changes which we are going to discuss in the next video.

357
00:19:24,000 --> 00:19:26,000
So yes, this was it from my side.

358
00:19:26,000 --> 00:19:27,000
I'll see you all in the next video.

359
00:19:27,000 --> 00:19:27,000
Thank you.