Hello guys.
So we are going to continue the discussion with respect to vector stores.
Uh, in this video we are going to discuss about a chroma DB.
You know, so what exactly is chroma DB?
It is an I native open source vector database focused on developer productivity and happiness.
Chroma is licensed under Apache 2.0.
So let's go ahead and see.
Uh, again I will be showing you with the help of code itself.
So first of all, uh, you know, you need to go ahead and, uh, install chroma.
So in order to install, uh, chroma, you know, you just need to make sure that you update your,
um, you know, requirement dot txt.
So for this, uh, what I'm actually going to do is that I'm going to quickly go ahead and write chroma
over here.
Okay.
Or sorry chroma DB.
And then I'll just go ahead and open my requirement dot txt pip install minus our requirement dot txt.
Let's see.
You'll be able to see that I have I think I may have already done this installation so it has been installed.
Okay.
So this is the first thing, uh, please make sure like one by one you should be able to install all
these packages till now.
Okay.
Now the first step, uh, again we will be building a sample vector db.
Okay.
And then first of all I will go ahead and read from lang chain underscore underscore.
Chroma.
Okay.
Uh import chroma.
Okay.
Now if you go ahead and see the previous version of, uh, chroma DB, right.
Uh, that time we used to directly install chroma DB, and we used to do use it.
Right.
But now in the long chain you have this new library that has come like chain underscore chroma.
So you need to also install this right.
No need to install chroma debris separately.
Now.
Now you have this entirely where you can actually use lang chain underscore chroma db okay.
So what I will do I will again go ahead and uh, I will say, hey, uh, just clear the screen okay.
And then let's go ahead and install this requirement dot txt.
So here you can see now you need to install this lang chain underscore chroma.
Uh, this will be my library that I'll be using.
And right now you can see the recent version 0.11 1.1.
Please make sure that you keep this version also ready so that, uh, you will be able to understand
which version you have actually created.
Okay.
Now from Langston underscore import chroma.
So I'll just go ahead and import this.
Let's see whether it is working fine.
So yes it has got successfully imported okay.
Remaining everything will be almost same like how we used for fires.
So here I'm going to import login underscore community dot document loaders I'm going to use text loader
then use Ola my embedding.
You can also use OpenAI embedding if you want.
Then along with this I'm also going to use login underscore text splitters with recursive character
text splitter okay.
Now uh quickly let's go ahead and read this particular speech dot txt.
And again I'm going to go ahead and write my data dot loader right loader dot loader.
So here you can see text loader.
Is there why it did not get executed.
Let's see I have to execute this.
Then let's go ahead and execute this I think I did not execute this part okay.
So here you have this entire speech dot txt.
And here is my entire data.
Now what we'll do we'll basically do the split.
And for splitting um we have to use recursive character text splitter I've used chunk over size as 500
chunk overlap as zero.
Okay, it is up to you how much overlap you want to use.
Use it.
Then, uh, we will go ahead and write text.
Underscore splitter dot split documents with respect to data is equal to splits okay.
So here uh you can see uh this basically is my splits okay.
Now quickly let's go ahead and create my vector store along with the embedding.
So in order to do that first of all I will go to right.
Embedding is equal to Allama embeddings okay.
And then I'm just going to write vector DB is equal to chroma dot db or sorry from documents okay.
So again it will be like the same thing like how we discussed for files.
And inside this I will go ahead and give my um data that is splits okay comma.
Along with this, I'm going to just go ahead and apply my embeddings, which will be equal to this specific
embedding that I have initialized.
Remember the first parameter is nothing, but it is documents okay.
So this basically becomes my vector store vector db vector store uh vector db over here.
Or you can also say vector store db.
And once I execute it this is basically going to apply to this entire documents okay.
Which we have actually done the split.
It is going to take some amount of time since this is getting executed in the local.
Okay.
Once I do this, let's this get executed, then I'm just going to query this particular, uh DB.
Right.
So for that again how do we query it.
By using the same technique what we used in fast.
So let's say here I say what does the speaker believe is the main reason?
United States should enter the war.
So vector vector db dot similarity search of query.
So this basically becomes my docs of zero.
I will go ahead and print.
Along with that I'll also go ahead and print my page content okay.
Content.
So if I go ahead and execute this you'll be able to see that, hey I'm getting the entire response okay.
Now like how we made sure that we can also save this in the local will also be able to save chroma vector
store DB in the local itself.
Right.
So now let's go ahead and save to the disk okay.
Saving to the disk.
And here again I'll be using the same command like this okay.
This will be my vector store DB okay.
Here what I can do is that, uh, along with this entire documents, uh, that I'm actually providing,
uh, with the embeddings and all, uh, what I will do, I will go ahead and use one persist directory.
So here I have to go ahead and write my persist directory.
And I'll give my directory location.
That is nothing but chroma underscore db okay.
And finally I can use this.
Uh, so if I go ahead and execute this, it's going to take some amount of time.
And again here you'll be able to see a folder is basically getting created.
And this internally creates a SQLite db.
Okay SQLite three.
So this db is specifically used inside.
That basically means every vector is stored inside this particular db.
Now see if you are able to see one db right.
This can be hosted anywhere you like right.
So this is the main fund or importance behind uh, saving it to the disk.
Okay.
Or saving it in, in, uh, in a file.
Uh, that can be reused again.
And that can be called now in order to call this a what you can do is that you can just go ahead and
write this particular code, see Chrome.
I'm just saying, hey, give the persist directory that is uh, dot slash Chrome DB embedding technique
is nothing but my embedding over here.
Right.
Whatever embedding function is there.
and I can again, uh, do the same similarity search.
So let's say if I get my docs.
So this will be my DB two dot similarity search.
You can do the similarity search by vectors.
You can do it by anything as such.
And then we will go ahead and print docs of zero okay.
Dot page underscore content.
So once I execute this here you will be able to get the response okay.
Uh, if you want to probably get the similarity score, uh, you can also go ahead and write this particular
code like this.
Like how we did it for fires.
There is a function which is called as similarity score.
Uh, with score.
And here you will be able to see the docs.
Okay.
Uh, so here again Manhattan distance based on the Manhattan distance, you are able to get this information.
Okay.
Perfect.
Uh, now, finally, uh, again, we should not skip this option.
Also, that is nothing but retriever option.
Okay, now, in the case of retriever option, I will take this vector store DB dot as retriever okay.
And then uh I'm just going to create this particular variable retriever is equal to.
And let's go ahead and invoke this okay retriever dot invoke okay.
And here I'm just going to give my query and get me the first whichever will be the first result.
Right.
So this will basically be my first result.
I can go ahead and also display my page underscore content okay.
Page underscore content okay.
Perfect.
Uh, this was more about it.
You know, uh, where we were discussing about, uh, vector store DBS and all.
Uh, and this is one example with respect to the chroma DB.
We will be using everything like this kind of vector store DB or databases, uh, in our end to end
projects.
Uh, just to give you an example, two of the most common things that we use, which are completely
open source is Faes and Chroma.
Uh, as we go ahead again, uh, when we develop end to end projects, we'll be seeing a new DB like
Cassandra DB.
Astra DB will be talking about different different DBS like pin code, vector store DB.
So that also will be talking about where we will be creating some kind of that will be hosted in some
kind of cloud.
So that is the reason I'm saying that.
We'll discuss about that when we develop our intern projects.
Right.
So yes, uh, this was it.
Uh, I'll see you all in the next video.
Thank you.