So guys, finally, I'm excited to implement this amazing end to end project related to text summarization.
So I will take the same folder inside this.
I will go ahead and create a API key.
So let me just go ahead and create this.
Sorry not API app.py file.
Okay.
And this is where I'm actually going to write the entire coding.
See the reason why I am creating this kind of folder structure.
You know, because you understand like how every project is basically getting created.
And I'm just putting all the common libraries that may, we may be using in creating an end to end projects.
Right.
And that will be all at one place.
So that is the reason, uh, I'm basically creating all the projects in this same way.
Okay.
So now, uh, let's go ahead and start our coding over here.
So first of all, what we are going to do is that, uh, here we are also going to use some new libraries.
Okay.
So first of all, the library that I really want to use is something called as validators.
Now what exactly is this validator.
Since we are going to use or work with Earl over here, this validator basically is an inbuilt library
which is not an inbuilt library, but it is an external library which actually helps you to validate
the URL.
Okay.
So I will be using one of this particular, uh, version that is 0.281.1.
I think this is the recent version that is going on.
Along with this, I will also be using YouTube transcript.
Transcript underscore API.
Now, with the help of this, whenever we give any YouTube URL okay, we will be able to read the entire
transcript for that particular video.
Okay.
So this is another one library that we are basically going to use.
Uh, let's see whether when we are doing the coding like any more libraries will be required or not.
So now I will go over here, I will open my command prompt.
First of all, I will go ahead and make sure that I do the installation of the requirement dot txt.
So first thing that you really need to validate is that are we in our environment right.
That is V and V.
Yes it shows over here.
We will go ahead and write pip install minus r requirement dot txt.
So once I go ahead and execute it here you can see remaining all the libraries.
It shows requirement already satisfied.
But now YouTube YouTube transcript underscore API validators.
You can see it has got successfully installed.
So now let me just go ahead and close this and let me start writing my code.
Uh, the first thing first, what I really need to do is that I have to import this validators.
So I'll go ahead and write import validators.
And then I'm going to also import Streamlit as st.
Okay.
Uh I'm going to use this then uh from lang chain dot prompts.
Okay I'm going to import my prompt template which I'm going to specifically use.
Okay.
Then uh again from lang chain underscore grok, we are going to import chat grok, which I'm actually
going to use to communicate with my LLM models.
Then from lang chin dot chains dot summarize.
Because we are going to use the summarize for using any of the summarize techniques that we have already
learned.
So here we will be importing this load summarize chain.
Then again uh.
Uh, one more library that I am going to specifically use with respect to YouTube and uh, any other
websites.
Right.
We also require some kind of document loaders.
So if you remember in the initial stages we discussed about document loaders.
Right.
So Langston underscore community dot document loaders I'm going to import one is YouTube loader.
So for this I will be using this YouTube loader.
You can also take the audio if you are really interested.
Okay.
And then we will also go ahead and take a unstructured and structured or unstructured URL road.
Right.
And there are so many like see unstructured HTML is there, CSV is there, email is there, pub is there.
But here we are just going to use a URL.
Let's see.
We'll go ahead and write unstructured URL road.
Okay.
That basically means directly from the URL we will be able to load the content here.
You can also use CSV.
You can also use any other things like let's say you want to upload go ahead and upload CSV.
You can also do that okay as I said guys it is vast.
Lang chain is really really vast.
Okay.
So finally let's go ahead and create our Streamlit app.
Now initially to create the Streamlit app.
What I will do I will go ahead and set up my page config and uh h3 dot title next dot subheader.
Okay, um, then I will quickly go ahead and get the grok API key that I really want to use.
Uh, and I will also create an URL field, the URL field to uh, which URL we really need to summarize,
right.
To be summarized, I will right over here.
Okay.
Um, and, uh, the Earl field.
I'll not say Earl to be summarized.
Whatever.
Earl, we really want to summarize, it can be.
I'll write over here as it can be white or website.
Okay.
So once this is done, then I will go ahead and write with s t dot s sidebar.
Okay.
And here I will go ahead and create the sidebar itself.
Here I will go ahead and first of all get my API key right.
So for this I will go ahead and write TX dot text input.
So I'm just creating a field on the sidebar like text input where I will be giving my grok API key.
So let me just go ahead and write grok grok API key.
Okay.
And here we are basically going to take the value as blank right.
Initially it will be and type will be password.
Okay.
Perfect.
Till here we have done it.
Then let's go ahead and create our URL.
It'll be nothing but PT dot text underscore input.
And here I'm going to basically give my URL.
Okay.
And over here I will keep my initial label.
Visibility is equal to false or collapsed.
Okay.
The reason over here why I'm writing it as collapsed because it should not be highlighted unless until
we go ahead and click it.
Okay, so this will be collapsed.
Uh, this is perfect.
Till here.
We have done this.
Okay.
Um, next, uh, what I will do, I'll go ahead and create my button.
Okay.
So for that, I will go ahead and write if s t dot button, I will go ahead and create my summarize
the content from y t or website.
Okay.
So this will basically have the information in the button.
Okay.
Uh now let's go ahead and first of all validate all the inputs Right.
So we're validating I will go ahead and write.
If not okay.
Grok API key.
If the grok API key is not given okay I will also do a tag dot strip.
Or I'll also go ahead and check for not URL dot strip.
So we will also put a condition with respect to the URL.
I'll say that hey, if the URL is not given okay, dot strip strip basically means removes the empty
characters.
I will just go ahead and give an error saying that SD dot error and I'll guide.
Please provide the information.
Please provide the information okay a generic information like that right.
Otherwise I'll also give an else if condition where I'll say whether my URL that I've actually given
uh we need to validate also that.
Right.
So here I will say if not validators dot URL.
And here I'm going to basically go ahead and give my URL.
So this URL can be your website URL.
It can be a YouTube URL.
So I'll go ahead and just write a generic URL over here.
And here will be my API key.
Here will be my generic URL itself.
Right?
So all this information is over here.
I will also go ahead and make this particular check okay.
So I'll say hey please provide the information to get started okay.
Something like this some errors.
We will try to put it over here.
So now we are checking whether this URL URL is a validator.
You validating your URL or not.
It is a valid URL or not.
So here.
If it is not a valid URL, I will just go ahead and write SD dot error.
Please enter a valid URL okay.
And here I will also go ahead and write.
It can maybe a just go ahead and write this error message.
It can maybe a witty video URL or website URL.
Okay so I'm giving some validation also so that you'll be able to check it out.
Then finally in the else block, uh, what I will do is that if all these conditions are satisfied and
this is where we have to start our coding and whatever happens in the back end.
So I'll say, hey, go ahead and create a try block okay.
And we will also handle the exception with respect to this.
And I'll say hey width width street dot spinner.
Okay.
And I'll say waiting.
Okay.
Waiting till this entire information is basically coming up.
You know, we will first of all load the we will load the URL itself right.
URL data.
So the first step over here is that loading the uh website data okay.
Website or YouTube video data or y t video data.
Okay, so once this is done, we go to the next step.
I'll give a condition in my URL whether youtube.com is present or not.
So if youtube.com is present in this generic URL, then what I'm actually going to do, I'm going to
create a loader.
And inside this I will be using this YouTube loader which I have actually initialized.
As soon as we use the YouTube loader.
All uh, there is an inbuilt function over here, like from YouTube URL where we can give the generic
URL.
Okay.
And uh, we can also say, uh, whether I want to add the video information or not.
So here there will be one more parameter which is called as add ad underscore video underscore info
is equal to true.
I'll keep it as true okay.
So these are the basic information with respect to.
Whenever we upload a YouTube URL I should be able to load this okay.
Otherwise in the else condition I know that if I'm giving a if the generic URL is in the form of website,
I will be using this loader and we can go ahead and use this unstructured unstructured URL road.
Right?
This is also one kind of loader over here.
And here we will just give our URL.
The URLs will be nothing, but it will be a list of.
I can also give multiple URL if I want.
Okay.
So here I will be having this multiple URL.
One more thing, I will also go ahead and make some checks saying that hey uh, whether it should be
SSL verified, you can probably keep it as true or false right now.
I'll just keep it as false just to take any URL.
If I'm given, I will probably go ahead and do this here.
It should be generic underscore URL.
Okay.
And one more information along with this will be that, uh, the kind of header.
See, whenever you work with any um, whenever you work with any kind of let's say you're serving from
any, uh, browser, right.
So in that browser, you basically set up some kind of headers by default so that just to validate your
browsing from the right, uh, right.
Browser itself.
Right.
So I will be keeping this specific information over here.
And uh, this headers you will be giving it over here.
Right.
So headers like user agent Mozilla 5.0 Macintosh, Intel Mac OS, uh Apple WebKit uh like gecko, Chrome,
Safari.
So all these browsers are basically supportable.
So that is the reason we are providing this entire information over here, uh, saying that if that
information if this header is basically there in the website, we should be able to, uh, probably
go ahead and use this.
Okay.
So this is the header that we are setting and unstructured URL loader will be making sure that it's
just like how we browse in the browser.
right?
We are able to browse different, different pages and when we are browser, they're also headers will
be set up with respect to every request.
Now here also when we are using this unstructured URL.
In short this is hitting this particular URL.
So whenever we are hitting this particular URL, the web server will also be demanding the kind of headers
you are putting on top of it with respect to user agent, from which browser it's just like from which
browser you are trying to execute it.
That information, you are able to put it over here.
Okay, now once you are able to do this, the next, the next step, let me just show over here.
So the next step over here will be that, uh, I will get this particular data.
Okay.
And uh, once we get this data so I will say, hey, one, once I have set up the loader, I'll say
loader dot load and I will be getting the data itself.
Okay, so this is done.
This is enough over here with respect to the try block.
Now we are going to initialize.
It is always a good idea that we initialize the grok model outside our, uh, you know, our, our entire,
our, uh, application.
So here I will only initialize it over here.
I'll say, hey, LM is equal to chat grok with all this information.
And let's pick up the grok API.
So grok API is basically coming from here.
Um I will go and set it over here.
Right.
So this is where we are basically setting it up.
I'm not just putting it inside.
I could have loaded my, uh, grok over here, but here will be my grok API.
Okay.
And with respect to this grok, this will be my gamma model that I'm using.
Uh, using grok API right.
Grok API.
Once this is done, uh, we will also go ahead and set up our prompt template.
Okay.
So here I'm just going to go ahead and write my prompt.
Template prompt template.
And for this prompt template I'm going to specifically use a simple prompt template.
Say I will just use this.
Okay.
I'll write.
Hey.
Go ahead and right away provide a summary.
Of the following content of the following content in 300 words.
Okay, so this is what I'm actually going to give over here.
And this will basically be my text, the text info that I'm actually going to put over here.
Right.
So so I'll just go ahead and say this is this and this is my content okay.
The content is equal to text okay.
And then after this what we do, we basically go ahead and define a prompt.
So with respect to the prompt I will go ahead and use my prompt template.
The template will be equal to my prompt template.
And finally my input variables will be nothing but it will be equal to text right.
So I have to basically give it as text.
So perfect I think everything is working fine till here.
So what I did is that instead of writing my prompt over here, I have defined the prompt template and
lm outside.
Okay, now it's time we go ahead and create our chain some chain for summarization okay, so for this
you know what you we really need to use it.
So I'll go ahead and say chain is equal to load summarize chain.
And here I'm going to use my LM chain underscore type.
Let's go ahead and use stuff initially okay.
Because the content may be small or may be bigger.
It is up to you.
Right.
So you can use stuff or map reduce or anything that you want.
Uh, along with this I will just go ahead and define my prompt.
Okay.
So let's go ahead and write.
My prompt is equal to prompt okay.
And here we will go ahead and run the summary dot chain.
Okay.
So once I write uh sorry it should be summary is equal to or I'll just write output underscore summary
is equal to chain dot run.
And here I'm going to give my data okay.
Or the data whatever data we are specifically getting uh over here instead of data.
We will just go ahead and write docs because documents is what we usually get, right.
So here also I will be giving my documents.
Okay.
Perfect.
Till here everything looks good.
And finally I will say TX dot success.
Success right?
And that is the reason you get green color in the demo, right?
Uh, I will just go ahead and display my output summary.
Perfect.
Done.
This is good enough.
Now, in the accept block, we can probably go ahead and write our exception.
So I will go ahead and write accept exception as E.
And then we are just going to write TX dot exception.
As exception.
Colon e.
Right.
Perfect.
So till here.
Everything looks good.
I think, uh, it should run.
It should run.
Okay, let's go ahead and execute it.
So here I'm in my venv environment.
Let's go back to my folder that is seventh text summarization.
So I will go ahead and write seven text summarization.
Here I will go ahead and write Streamlit run Streamlit run app.py.
So let's go ahead and write App.py.
Now I think everything should work fine.
So this is my page.
It looks fine.
Grok API summarize the content from the YouTube website.
Let me go back to my code.
Let me open my env file.
Right.
And let me just pick up this grok API key.
Right.
All the API keys.
You can specifically use it okay.
Just to practice.
So I'll press enter okay.
Now let me do one thing.
Let me use any website docs, Len Smith, whatever website you want I will go ahead and copy this.
Put it over here.
Um, module has no attribute exception.
The reason exception.
Okay, let's see this s t dot exception I think it should be in small letter okay.
Now it should work.
Always run I will summarize.
So it is saying me to install this unstructured because we have used this unstructured over here.
So let's go ahead and install this.
And these are some of the errors that you may get if you missed any requirement.
Dot txt right.
And I want to show you all these errors okay.
So quickly I will go ahead and open my another command prompt okay.
So I'll say hey pip install please make sure that you do the installation in your v env environment
right requirement dot txt once I do this.
So here you'll be able to see this I think one more error.
We may get it but let's see whether we'll get that error or not okay.
So once this unstructured is basically getting installed YouTube transcript API I've already done it.
Validators I have already done it.
I think I do not have to probably get any kind of errors other than this.
So now if I go ahead and summarize it, let's see.
I think it should work.
One more error can come again for the library.
But I think other than that it should work okay.
Why it is taking so much time.
So it's better okay.
Yeah.
Perfect.
So it shows you the output, right.
Uh, let's try some other, uh, URL.
Okay.
So what I will do is that I will quickly go ahead and take some more URLs if I want.
Okay.
I can go and search for this.
So let's take this entire URL.
Okay.
My entire page URL over here.
And with respect to this particular URL, I will just go ahead and open this.
And if you can try it with any YouTube channel video.
Okay, so this is one of the URL.
So it is necessary to properly test each and everything.
So here I have given my YouTube video.
I have to probably go ahead and install Pi tube okay.
And this is one of the requirements I guess with transcript API.
So I will quickly go ahead and copy and paste it over here.
Let's go ahead and do the installation now.
I think it should work okay.
So basic basic requirements.
And that is the reason you have to probably create an another environment for this.
Right.
So this is the version that you are having okay.
So let's summarize.
And now we should be able to get the output.
Here you go.
Right generative I explain.
So yes uh this was it for my side.
I hope you like this particular video and I hope you are liking the entire course.
Uh, so yes, I will see you all in the next video.
Thank you.
Take care.