05 - Chat History and Memory Management

Great, so we have the first response from our chatbot. But quite often, in real-life scenarios, we need to continue with a follow-up question.

With our current code, if we launch the second request to the chain(), it will NOT remember the previous question/answer. Let's fix this.

A straightforward way is to save the history in a local variable and pass it every time:

history = []
 
query = 'Do you have a team plan?'
response = chain.invoke({"question": query, "chat_history": history})
history.append([query, response['answer']])
 
query2 = 'And what about free plan?'
response = chain.invoke({"question": query2, "chat_history": history})
history.append([query2, response['answer']])

But a more structured approach would be to use a specific class for it from LangChain, called ConversationBufferMemory(), passing it as the third parameter to the chain from_llm() function.

Then, the response would automatically contain the item of the chat_history or whatever the memory_key value you provide.

import os
from langchain.document_loaders.csv_loader import CSVLoader
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferMemory 
 
os.environ["OPENAI_API_KEY"] = 'sk-...'
 
loader = CSVLoader(file_path="faq.csv")
documents = loader.load()
 
vectorstore = FAISS.from_documents(documents, OpenAIEmbeddings())
 
memory = ConversationBufferMemory(memory_key='chat_history', return_messages=True) 
 
chain = ConversationalRetrievalChain.from_llm(llm=ChatOpenAI(),
                                              retriever=vectorstore.as_retriever(),
                                              memory=memory) 
 
query = "Do you have a team plan?" 
response = chain.invoke({"question": query}) 
 
query2 = 'And what about free plan?' 
response = chain.invoke({"question": query2}) 
 
print(response['chat_history'])

# Output:
[
  HumanMessage(content='Do you have a team plan?'),
  AIMessage(content='Yes, we do have a team plan. The team plan is available on a yearly basis for up to 7 team members and costs $299 per year. We also offer a lifetime membership for teams with up to 7 members, which is priced at $799.'),
  HumanMessage(content='And what about free plan?'),
  AIMessage(content='Based on the given context, there is no mention of a free plan. Therefore, it can be inferred that there is no free plan available.')
]

That's it, not much to add here. The history of the conversation would be saved throughout the lifetime of the Python script.

Now, if you want to save that history outside of the script, then it's a separate big topic. You would probably need a database to store the messages, assign them to specific users or chat sessions, etc. We will not cover such history management in this tutorial.

So, for now, we are finished with building the back-end of our chatbot. We can have real conversation with the LLM and our documents when we launch the code from above.

But, of course, we need to build the front-end part to use the chatbot more conveniently. Otherwise, how would users interact with our chatbot?

In the next lesson, we will cover exactly that, with two options: a CLI Terminal chatbot or a web-based chatbot in the browser.