All right, so next thing you know, tokenization we will see in this video are the slicing and indexing
into open eyes.
So just like indexing and slicing, we have learned, you know, string class in a patent.
So that's very much basic about the string classes.
So same way you can apply all those slicing and indexing.
You know, knock objects off.
So if you see here this top object, if I just display not four.
So you'll be able to see V, how to grab the type of this.
Otherwise, it will just display those string only whatever we have supplied.
And it is a kind of dark object, spacy Hawkins Dark Dot.
Now let's see if we want to grab how many tokens are dead inside this dark four object.
We can just simply use land function.
That is a will.
Functions are available now.
Let's say, OK.
And that is a seven.
That means that this particular sentence has been divided into seven tokens.
Let's say if you want to grab the very first token, we can use like a doc for hand, just like we indexing
works for a string class and a list.
Plus it will work on this and the class also.
So first broken can be grab with this indexing.
Zero.
So you can see the first one is like.
We can hear even do the slicing also.
So, like for let's say I want to grab from watch the movie, watch out.
And so this is referring to zero, then one.
So from two, three, four and five.
So two to five.
I can grab it.
Upper bound is all of this textually.
So now you can see what up and moving all the tokens.
It has written based on this indexing value here, the negative index also works.
So you can use like a minus one.
So the very last token return nine not case, the last option is not.
So it does return this.
But you just cannot assigned a new convert because it's like a final thing, you just cannot modify.
So let's say dock four zero.
We already know dock four zero zero.
Like, let's say you fight, right, to assign it to something as, let's say some new.
So immediately we got there that this type of object doesn't support the item assignment.
That means you just cannot publicly cooperate.
So that is all about the tokenization basics.
In the next video, we'll see about some more stuff related to an LP basic technique.