Hello guys.
So we are going to continue the discussion with respect to text splitters.
Now, uh, we are going to see how we can split by HTML header.
Okay.
So there is an amazing library again which is called as HTML header text splitter.
It is a structured aware chunk chunk that splits text at HTML elements level, and adds metadata for
each header relevant to any given chunk.
It can return chunk elements by element, or combine elements with the same metadata with the objective
of a keeping related text groups more or less semantically pre b preserving context rich information
encoded in the document structure.
Okay, so yes, uh uh, I know, uh, you'll not be able to get much from this specific information,
but don't worry, I will just go ahead and show you this particular code.
Okay.
So first of all, uh, I will go to from lang chain underscore text splitter.
Okay.
We are going to import HTML header text splitter okay.
Now after this what we are going to do is that we'll take one HTML string okay.
So let's go ahead and copy this HTML string.
So here I'll be putting this information.
So here you can see doctype HTML right body for some intro about foo bar main section.
Some some tags we have given like header tag p tag, div tag right.
So different different tags are there.
Even break commands is specifically used right over here b r and the paragraph is there.
Again we are closing the body.
We are closing all the divs and HTML.
Okay.
So let's consider that this is my entire HTML string okay.
Now what I'm actually going to do over here is that, uh, I will just go ahead and create on which
all headers I need to split, right?
So I'll say hey headers underscore two underscore split underscore on okay.
And this will basically be my header splitting here I'm going to write H1 header okay.
So let's say this will basically be my header one okay.
Header one I'm just renaming it over here.
Something like this.
So this will be my H1 tag.
Whatever H1 tag is there.
Then uh let's go ahead and write okay I also want to probably play with my H2 tag.
So I will just go ahead and write over here.
So this will be my header two okay okay I'm giving some custom name over here.
Header two.
And finally uh I'll also go ahead and use h3.
Let's say h3 tag.
And this is nothing but my header three.
Okay so these are my headers tag that I'm using to split okay.
Now in order to use this tag and probably do the splitting, what I will do I will go ahead and write
HTML underscore splitter is equal to HTML text splitter.
And here I'm going to basically use this header to splitter on okay.
So this is the parameter also.
So you can just go ahead and assign this parameter over here.
Also header splitter on is equal to this okay.
Uh otherwise I know I am going to use that.
So I'm just going to use this okay.
Now this is fine.
Now what we are going to do, we are going to basically write HTML underscore splitter dot split underscore
text.
So we are going to probably use split underscore text.
Or there is also options.
Uh yeah.
Split text is what we are going to use over here.
And here I'm going to give my HTML string.
Finally, uh, I will save all these things into my variable which is called as HTML underscore header
underscore splits.
Okay.
So uh, now I can go ahead and write HTML underscore header underscore splits.
And let's go ahead and display this.
So once I probably see this now here you can see I've used this three tags h1 h2 and h3.
So let's see where foo was there h1 tags.
Right.
So based on this H1 tag I was able to get another page document right.
Some intro about foo uh, where it is basically present in the uh p tag.
Right.
Some intro about foo.
So this has also become another tag over here.
Right.
Uh, header one of metadata you can see over here Foo bar main section.
Then you can see some uh, about the first subject.
Something was written.
Right.
So here again with the H3 tag, another uh, division is specifically made.
Right.
So here you can see all the tags have been basically created with the kind of headers that we have actually
used and And, uh, metadata information has also been shared over here.
Uh, based on the tags that we have actually splitted this entire content.
And finally, you can see that when we are splitting this into text, I am able to get in the form of
documents.
Okay.
So this can be handy.
Uh, again, it is up to you, uh, how you specifically want to use it and where you really want to
use it.
Okay.
Um, not only this, uh, let's say that I have some kind of URL.
Okay, so let's, uh, pick up some URL over here.
So I will just open my browser.
Let's see.
So this is my tag.
Let's see.
Okay.
Let me just go ahead and open my browser over here okay.
And with respect to this particular browser.
So this will basically be my browser here I'm going to take one URL.
Let's say this is my URL over here.
I'll just copy and paste it over here.
So once I am uploading this particular URL, uh, this URL pro provides some information over here.
Right.
So here you can see some information is there.
And if you go ahead and inspect all these elements there will be some kind of header tag.
There should be something right a head h1 tag, h2 tag and h3 tag.
So what I will do I will create this particular URL.
Go back to my code okay.
And uh let's create take this URL and I'm saying hey header to split on.
And then I will try to just split that same condition whatever I have actually written over here.
Right.
So I will go ahead and say hey, split this particular URL.
And finally I will be also able to see the splits.
So if I go ahead and write HTML header, splits.
So here you'll be able to see it will take some time because it will be a bigger website.
And here are all my documents that has got separated.
Right.
So this can be really, really handy wherever you specifically want to use it okay.
Whenever you have something like HTML and.
All right.
So, uh, I hope you like this particular video.
Uh, yes.
Uh, this was it.
Uh, again, uh, there are many, many techniques with respect to, uh, character text splitting,
which you should keep on exploring.
Also, make sure that you always check the documentation of the Lang chain to see what all new techniques
are specifically coming.
So I hope, uh, you like this particular video.
Uh, again, guys, uh, we are still in this particular phase.
That is second one.
Now we are going to go into the third one.
Right?
That is nothing but the embedding techniques.
So yes, I will see you all in the next video.
Thank you.