Hello guys.
So in this video we are going to discuss about the entire line chain ecosystem.
Now along with the line chain ecosystem, we first of all we really need to understand this two important
question.
The first question is why lang chain okay.
Why we really need to learn about this framework.
Why lang chain okay.
Along with this, the second thing that we are going to discuss about in this video is about the entire
Lange chain ecosystem, right?
So our entire video we will be discussing about this two important points.
And let me tell you guys, Lange chain right now is the most common framework that is specifically used
to build generative AI application.
Okay.
And in order to make you understand why Lange chain, let me just go ahead and talk about a couple of
years back what had happened, right?
Or one and a half years back.
So I hope everybody knows about this OpenAI company.
Right.
So OpenAI company is there.
And obviously we also had hugging face.
Right.
And these were the first two companies that probably came up with the concept of LLM models, large
language models.
Right.
It was not like uh, OpenAI or Hugging Face came at first, but they were also Google who were working
with open source models.
They were in hugging face.
There are a lot of different, different open source model, but the popularity of LM models and generative
AI apps started when OpenAI came up with its own LM model, right?
Like GPT 3.5 turbo.
All those kinds of models were there.
Now, when, uh, when we started, when when this companies announced about the LM models, uh, many
of the developers started exploring.
Right.
And with respect to the documentation open, I specifically had a different set of code.
And what we do with this LM model, if I'm providing any input, I will be able to get any output in
form of any kind of response.
Okay.
Now when I, when we were using OpenAI hugging face, uh, specifically, if I talk about hugging face,
right.
We use different, different, uh, libraries, some of the libraries that we used, one of the most
common libraries that we use are Transformers.
Now in Transformers we have something called as pipeline, right.
And with the help of pipeline, we are able to call any model that are available in the hugging face,
right?
Call it in a local developer generative AI application.
Similarly, in the case of OpenAI, uh, you know, we really need the API key.
And based on this, OpenAI, I have also created an amazing documentation where you can probably do
fine tuning, you can create a generative AI application and all.
And later on when more companies came into this race, right.
Let's say meta with llama three, right.
Google.
They it came up with this Google Gemini Pro.
Now when this all models were specifically coming up different different set of libraries were getting
created so that we will be able to access this particular models.
Now again so many different models are there, both paid open source, uh, you know, uh, you can
also use these open source for commercial purpose.
Now, what Lang Chen did is that Lang Chen, as a company, it created a common framework.
Right.
And as I said, Lang Chen is like a common framework.
I'll write it down over here.
Okay.
So what did Lang Chen do?
It created this amazing common framework.
It is a framework to develop to develop gen AI application.
Gen AI app.
Okay, now, with the help of this particular framework.
Right, uh, we were able to access like let it be open AI hugging face or Llama three, Google Gemini
Pro, whatever models you specifically want.
So long chain as a library provided all those specific features, right.
So they were libraries getting integrated and this entire long chain, right.
If I probably talk about this long chain, all the modules inside this long chain are open source.
Right now when you have something like open source, definitely it will be very much in ease to create
generative AI application.
Right?
And right now, probably each and every companies, if you even go for interviews, they're specifically
going to ask you about like how you can actually build with the help of generative AI application.
Now this is fine, right?
Long chin was there.
and with the help of Lang Chin we were able to generate generate the applications.
But later on, you know more and more modules were basically getting added.
And that is what we are going to discuss now about the entire Lang chain ecosystem.
Because now the question arises if I probably go ahead and create this application.
Right.
So let's say that if I go ahead and create the application here, we may be using any paid open source
tools open source model, LM model, LM models or multi model.
Right.
But the main task is that after creating this particular model, how do I make sure?
How do I perform some of the activities that we specifically do with LM ops?
Now what all activities we do in this, let's say in this we specifically do something like debugging,
right?
Uh, we want to probably do the evaluation.
Right.
We want to do monitoring.
How do we do specifically do with this.
Right.
So long chain.
What they did is that they started creating this entire ecosystem.
Right.
So for this LM ops activities like debugging, evaluation, playground monitoring and all, it created
a new module which is basically called as Lang Smith write.
And this Lang Smith is specifically used for this purpose.
Right.
The LM ops part.
Right.
So, uh, it's not like we just use the library.
We develop a generative AI application.
But in short, what?
Lang Chen, what is the main thing that it did is that it created that entire ecosystem wherein it is
integrated.
This additional module, which is called as Lang Smith and through which you will be able to probably
do each and every thing which I have written over here, like debugging, playground evaluation, annotation,
monitoring and many more.
Right.
Then came up the next module, which is called as Lang sir.
Now Lang sir, obviously if we are able to do this, finally, we really need to deploy this application.
Now to deploy this application, we really need to convert this into in the form of APIs.
Right now, this APIs can probably be converted by using another module which is called as lang serve.
So lang serve actually helps you to create chain as rest API.
So it is written over here.
It actually helps you to create this chain as Rest APIs.
So now is the main thing right now understand the entire mechanism.
Why I'm saying this as a ecosystem.
Right.
First you had long chain.
Now with the help of long chain by using concepts like chain agents, retrieval strategies which we
will discuss.
Because in this course I'm going to cover each and every thing that is available in this ecosystem.
Right.
And then finally we deploy after creating this particular APIs, we deploy it in different different
clouds like AWS for will focus more on AWS.
Then there is also another cloud, which is called as hugging face space, right?
Which will be available to everyone, probably completely for free for some space for some resources.
Right.
So we'll also be seeing with respect to this particular deployment right now, uh, initially Lang Chain
came we are able to generate generative application.
Now in this Lang chain since we had so many different models.
We have models from OpenAI, we have models from Google Gemini Pro, we have models from hugging face.
That is the reason it came up with this.
Another model which is called as Long Chain Community, which has access to all this kind of models.
Yes.
For the paid one you really need to have a key for open source.
You can directly use hugging face, you know, or any other things like Meta Llama three and all right
then over here prompt engineering is also probably done.
Example selectors is a concept which you will be learning about.
Then you have something like output parser like how should the response come after.
We probably input uh if we give an input to our LLM LM model, how if we really want to customize the
output, how do we do that?
We'll be discussing about that.
Then we'll be discussing about retrieval document loader.
Like they are different different data ingestion techniques.
We can load data from amazing different different data sources which we'll be discussing about this.
Right.
And it will be a detailed videos when we'll be discussing.
Right.
Then we have Vector Store, then we have text splitter.
Now what exactly is this vector store?
See, at the end of the day when we are working with linear models, that basically means we are working
with text, right?
And when we are creating this kind of application called as rag or document Q&A or chat bot Q&A, uh,
there may be scenario where I will be having a huge amount of data, and for that particular data,
I really need to convert into a vectors, right?
For converting into a vectors, we will be specifically using different different embedding techniques.
Right.
And after we convert that into a vectors right.
Because uh, the reason we convert into a vector is because with the help of linear models, whenever
I try to query it right, it tries to apply cosine similarity and it will be able to retrieve the results.
So, uh, after I probably convert into a vectors, how to store it in a vector database will also be
discussing about that.
We'll be discussing about various data vector databases that are available, uh, within the Lang,
Smith Lang chain itself.
We'll be discussing about that.
So as an ecosystem, we'll just not be building generative AI application, but instead we'll be thinking
about the entire life cycle of a gene AI project.
So here our entire focus will be on life cycle, life cycle, life cycle of Gen AI project.
We will be focusing on this entire life cycle which will be including Lange, Smith, Lange, serve
and along with that, some other default templates are also available.
We will be discussing about that.
So I hope I was able to answer this.
Two important things.
While Lange chain and some of the things with respect to Lange chain ecosystem.
Now you'll be seeing that in the upcoming tutorials that we are going to do in this course.
Uh, we'll be discussing about all these modules.
We'll be breaking it down, we'll be developing end to end projects.
We'll be doing many things as such.
Right.
So I hope you like this particular video.
This was it from my side.
I will see you all in the next video.
Thank you.