What is LangChain? (AI Crash Course)
Alright! So you’ve decided to dive right into LangChain and want to understand more about how we go about it. Fear not, we will tackle this topic one by one and also take the effort to go through the fundamentals, the core concepts, the features & developments, and why LangChain matters!
01 The Fundamentals of LangChain (Theory)
So, what is LangChain? We know we have LLMs, or Large Language Models that help us build out great text-based AI models. We have models like ChatGPT by Open AI, Llama by Meta, and Gemini by Google. Well, as good as these models are, they are also super expensive to run in production for enterprises. There are also a lot of great open-source LLM models like Llama 2 (Meta AI), Bloom (BigScience), MPT-7B (MosaicML Foundations), Falcon (180B) and OPT-175B (Meta). Each model has its strengths and weaknesses. What if we could get the best of multiple models, based on our use case? Wouldn’t that be great? Well, it's possible and it’s open source. Presenting LangChain.
1.1 What is LangChain?
LangChain is a framework that is designed to help you design and leverage LLMs to make it easier to create custom AI-driven applications.
At the heart of LangChain are the following:
- LLMs: LLMs are AI models that are trained on very, very large datasets. They are trained in a way that the LLMs can generate human-like text, and they are getting really good. For example, ChatGPT
- Chains: Chains in LangChain are sequences of operations that can be performed on the output of the LLMs. Using chains will let us perform complex processing based on our use case and transform the LLMs' output. This will let the AI Application or end-user get more targeted and nuanced text. For instance, when building a chatbot for a car company, we would like for the chatbot to answer questions only pertinent to the specific car company.
- Output Parsing: The ability to parse and tune the output of the LLMs that we string together will make sure that the answer that the LLMs give is only what we want it to be.
LangChain is a very powerful concept that gives us an extremely high level of customization and control over what models we use, how we decide the use the output and what the end user will see in a way that we can optimize resources and ensure that end user gets a relevant answer, depending on the application that we are developing. There are more nuances we will cover in an upcoming blog. In case you want to learn more about setting up VS Code.
1.2 Create a virtual environment & set up your environment
So, now you have VS Code set up. Make sure you have Anaconda, Python, and Visual Studio Code set up on your system.
Now, we will create a virtual environment in our terminal on Visual Studio Code. Let’s say that you want to create an environment called langchain with Python 3.11 using Anaconda. Below is the code to create the virtual environment.
conda create -p langchain python=3.11 anaconda -y
conda activate langchainThe below code is to deactivate the environment. Only use it when you would like to deactivate the environment (not now!).
conda deactivate langchainNow, we will create our requirements.txt file.
requirement.txt file
langchain
openai
huggingface_hub
langchain-openai
python-dotenv
streamlitNow, run the following code to install the required Python packages.
pip install -r requirements.txtGreat, now make a new Python notebook, langchain.ipynb. Your files should look like this.
Bonus tip: One last thing, make a .env file to store the API Keys that we will use.
02 Getting the API Keys
Now, we will get the API keys from Open AI and Hugging Face.
2.1 Open AI API Key
Make sure you are signed up, sign in and then enter the Open AI Website. Go to the OpenAI API site here and select API Keys from the left panel.
Select Create New API Keys and save the key in a separate, secure place. You will not be able to access this key online anymore.
Note: OpenAI provides $18 worth of free tokens for the first three months. Post that, you can load up about $10 worth of credits, that is more than enough.
2.2 Hugging Face API
Go to the HuggingFaceHub website here and log in. Click on your profile in the top right corner and go to the settings.
Go to Access Tokens to access your API token from Hugging Face. Copy the token and next, we will put it into the .env file.
Put the token into the .env file in our workspace.
.env file
OPENAI_API_KEY="sk-LU ..."
HUGGINGFACEHUB_API_TOKEN = "hf_ZB .."03 Using the Open AI API
Don’t worry, I have all the code for you to conveniently use on my GitHub (except the API keys because those are private). First, we will run the initialization scripts to install the required packages. Run this in the terminal.
pip install -r requirements.txtNow, make a new Python notebook called langchain.ipynb. We will import the required packages.
from langchain_community.llms.openai import OpenAI # Importing the LLMs from Open AI
from langchain.llms import OpenAI
from langchain import HuggingFaceHub #Importing the Hugging Face Hub library
from dotenv import load_dotenv, find_dotenv #These are required to load the env file#Command to load API Keys from .env file.
#The .env file is part of .gitignore to not leak secret keys.
load_dotenv(find_dotenv())With this, the API variables from our .env file will get imported. Now, we will load the environment variables.
OPENAI_API_KEY = os.environ["OPEN_API_KEY"]Next, we will set the temperature of the model.
What is the “temperature” parameter in LLMs?
The temperature of the model indicates how creative the model is going to be. It ranges from 0 to 1, with 0 indicating that the model is going to play it safe, and 1 indicating that it is going to be super creative.
llm_openai = OpenAI(openai_api_key=os.environ["OPENAI_API_KEY"], temperature=0.6)Now, we will invoke the Open AI LLM model and ask a simple question.
text = "What is the capital of India"
print(llm_openai.predict(text))
Note: By default, based on the latest model set, the Open AI will choose a default model. You can change the mode by defining a ‘model’ parameter.
04 Using the HuggingFace API
First, we will start by getting the value of the HuggingFace Hub API key from the .env file.
HUGGINGFACEHUB_API_TOKEN = os.environ["HUGGINGFACEHUB_API_TOKEN"]Now, we will select a free model from the hugging face hub to use. Note, that one of the reasons why proprietary models are more in use for enterprise use cases is because they are more conversational and trained on much more data. There will be a positive shift over time with open-source models, and eventually, all models will give a similar result.
The main point of differentiation will come in how fast we can train and update models because in the end any LLM or multi-modal model is simply a snapshot of knowledge. Models are not to be trusted, but can be used with the guidance of humans or to generate more ideas and use cases. Be careful when you put these models into production for any use case. As always, these are my opinions and do not reflect any organization or team that I am a part of. Anyways, feel free to connect with me on LinkedIn.
https://linkedin.com/in/anishmahapatra/
Now, let’s invoke the LLM from HuggingFace — we will use the flan-t5-large model from Google for this use case.
llm_huggingface = HuggingFaceHub(repo_id="google/flan-t5-large", model_kwargs={"temperature":0, "max_length":64})Let’s ask a question to the model now.
output=llm_huggingface.predict("Can you tell me the capital of Norway")
print(output)
Here, we notice the model outputs a single-word answer and is not conversational at all. Let’s try a more complex query.
output=llm_huggingface.predict("Can you write a speech about love?")
print(output)
An average poem. However, remember that the Open AI model is paid and this is a free model. Now, we will get to the fun part — we are going to explore the power of LangChain models.
05 LangChain (Code)
In this section, we will work on some interesting concepts of LangChain, along with the code and explanation.
5.1 Prompt Templates & LLM Chains
We can think of prompt templates as a plug-and-play device, where we are going to design a template for our prompt. Let’s say that we want to know the capital of a country, how would we go about it? We would write a prompt and say, give the prompt of the {country}. If we want to pre-define how the prompt should be asked, then we would fix the format of the template and pass the country as an input variable. Let’s do this with code now.
from langchain.prompts import PromptTemplate
prompt_template = PromptTemplate(input_variables=['country'], template = "Tell me the capital of {country}")
prompt_template.format(country="India")Since we have passed India as the input variable called country, our output would look like the below screenshot.
Now, that we have formatted and decided on our pre-defined template, i.e., prompt template, we need to pass it to our LLM. To invoke it, we will now invoke an LLM Chain, where we decide the LLM we are going to use and then simply pass out the prompt template. To run the chain, we have a chain.run() function.
from langchain.chains import LLMChain
chain = LLMChain(llm=llm_openai, prompt=prompt_template)
print(chain.run("India"))
Now, we defined one simple prompt template and passed it. What if we want to pass multiple variables and form a chain? Let’s learn how to combine multiple chains.
5.2 Combining Multiple Chains Using Simple Sequential Chains
In this section, we will combine two questions.
What is the capital of a {country}? We will get a {capital}.
Now, we would like to pass the output of chain 1 {capital} and ask for more information on the capital.
First Question
- Input: Tell me the capital of a {country}.
- Input key: {country}
country_prompt = PromptTemplate(input_variables=['country'], template="Please tell me the capital of the {country}")
country_chain = LLMChain(llm=llm_openai, prompt=country_prompt)Second Question
- Input: Tell me more about the {capital}.
capital_template = PromptTemplate(input_variables=['capital'], template="Tell me some great places to visit in {capital}, At the start Please mention the {capital} that we are referring to.")
capital_chain=LLMChain(llm=llm_openai, prompt=capital_template)The prompts are now made, let’s string them into a chain so LangChain knows that we want to string the LLM chains together. We will again use the chain.run() function to run our chain function.
from langchain.chains import SimpleSequentialChain
chain=SimpleSequentialChain(chains=[country_chain, capital_chain])
chain.run("India")
So, let’s summarize Sequential chains.
- Linear Flow: The LLM Chains are passed in a series and there is no branching or complex logic.
- No Conditional Logic: Each step is executed without any additional conditions.
- Uniform Processing: All the tasks undergo the same processing, and there is no complex decision-making.
Here, we had two chains, but we only saw the final output. What if we wanted to see the output of the first chain as well? Then, we can use sequential chains as explained in the next section.
5.3 Sequential Chains
The template of the questions will remain more or less the same, except that we will define the output key of the First Question.
First Question
- Input: Tell me the capital of a {country}.
- Input key: {country}
- Output key: {capital}
country_prompt = PromptTemplate(input_variables=['country'], template="Please tell me the capital of the {country}")
country_chain = LLMChain(llm=llm_openai, prompt=country_prompt, output_key="capital")Second Question
- Input: Tell me some great places to visit in the {capital}
- Input key: {capital}
- Output key: {places}
capital_template = PromptTemplate(input_variables=['capital'], template="Tell me some great places to visit in {capital}, At the start Please mention the {capital} that we are referring to.")
capital_chain=LLMChain(llm=llm_openai, prompt=capital_template, output_key="places")Now we will define the Sequential Chain. Our only input will the the {country}, and we would like to see the output of each chain. Read the code carefully.
from langchain.chains import SequentialChain
chain = SequentialChain(chains=[country_chain,capital_chain],
input_variables=["country"],
output_variables=["capital", "places"])The way that we will pass the input this time is through a simple dictionary.
chain({'country':'India'})
5.4 Chatmodels with Open AI
LangChain has a component called ChatOpenAi, which aims to extend the functionalities of LangChain by allowing it to call multiple functions to gather data. Chatmodels by OpenAI supports integration with LangChain, complex tasks, customisation with hyperparameters and is simple to use. In simple words, it is to help our chatbots be more chatty in a way that is grounded in reality.
Let’s learn about the concept of the schema of a prompt:
- HumanMessage: This is the message that is input by the human user
- SystemMessage: This is the message that is generated by the system, that is used for notifications, error messages or system status updates
- AIMessage: This is the message that is generated by the AI model in response to a human message (basically the reply)
Let’s dive into the code then:
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage, AIMessageNext, let’s use the ‘model’ parameter to select our model.
chatllm_openai = ChatOpenAI(openai_api_key=os.environ["OPENAI_API_KEY"], temperature=0.6, model="gpt-3.5-turbo")The System Message is sort of like a pre-preparation step that is sent when the chat thread is started. The model can keep the context of all of the prompts as long as it is within the context window (based on the tokens limit of the model). You can try out ChatGPT’s token counter here to learn more.
chatllm_openai([
SystemMessage(content="You are a comedian AI assistant"),
HumanMessage(content="Please help me with some funny punchlines on AI")
])
We will learn how to perfect System Messages with a solid template in an article soon. Be sure to follow me in case you are interested in AI For Dummies! In the next section, I wanted to show you that depending on the use case, we can decide how to parse our output based on a function or any rules that you prefer.
5.5 Prompt Template & Output Parser
In this section, we will output a comma-separated message (csv) type of answer. Let’s begin!
# Importing the required packages
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import ChatPromptTemplate
from langchain.schema import BaseOutputParserLet’s define a simple function to return comma-separated values called Commaseparatedoutput().
class Commaseparatedoutput(BaseOutputParser):
def parse(self, text:str):
return text.strip().split(",")Now, we will use the ChatPromptTemplate to define our system message and human message.
template="You are a helpful assistant. When the user gives any input, you should generate 5 words synonyms in a comma separated list"
human_template="{text}"
chatprompt=ChatPromptTemplate.from_messages([
("system", template),
("human", human_template)
])So, now we have a chat prompt which is going to take the system template to generate a comma-separated list and the human message in the “text” that we will input. The below code is why LangChain is called Language “CHAIN”. We will chain the below components:
- chat prompt: ChatPromptTemplate, where we have defined the system message and the human template
- chatllm_prompt: The model where we define the model we would like to use, the temperature and the Open AI key
- Commaseparatedoutput(): The function based on which we would like our output to be parsed
chain = chatprompt|chatllm_openai|Commaseparatedoutput()Grand finale time! Let’s showcase what we have learnt by getting the synonyms of the word ‘intelligent’.
chain.invoke({"text":"intelligent"})
The Conclusion
Done! Great job if you made it to the end. In case you would like the code on GitHub, you can find it below.
Connect with me at https://www.linkedin.com/in/anishmahapatra/ and follow me on Medium and hit the claps 👏🏻 if this article helped! Thank you for reading, it was lovely to have you!
Visit us at DataDrivenInvestor.com
Subscribe to DDIntel here.
Have a unique story to share? Submit to DDIntel here.
Join our creator ecosystem here.
DDIntel captures the more notable pieces from our main site and our popular DDI Medium publication. Check us out for more insightful work from our community.
DDI Official Telegram Channel: https://t.me/+tafUp6ecEys4YjQ1
