Integration Issues with LangGraph, RedisChatMessageHistory, and RunnableWithMessageHistory

[DhavalThkkar]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

contextualize_q_system_prompt = """Given a chat history and the latest user question \
which might reference context in the chat history, formulate a standalone question \
which can be understood without the chat history. Do NOT answer the question, \
just reformulate it if needed and otherwise return it as is."""
contextualize_q_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", contextualize_q_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    llm, retriever, contextualize_q_prompt
)


### Answer question ###
qa_system_prompt = """You are an assistant for question-answering tasks. \
Use the following pieces of retrieved context to answer the question. \
If you don't know the answer, just say that you don't know. \
Use three sentences maximum and keep the answer concise.\

{context}"""
qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", qa_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)

rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

Description

I am currently working on integrating several components into a comprehensive chat application using LangServe and LangChain. Below, I detail the components involved and the specific issues I'm encountering. Any guidance or suggestions would be greatly appreciated.

Components and Setup:

1. History Aware Retriever and Question Answer Chain:
   • I've created a chain that consists of a history-aware retriever and a question-answer chain. These are combined using create_retrieval_chain to form the final rag_chain.

contextualize_q_system_prompt = """Given a chat history and the latest user question \
which might reference context in the chat history, formulate a standalone question \
which can be understood without the chat history. Do NOT answer the question, \
just reformulate it if needed and otherwise return it as is."""
contextualize_q_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", contextualize_q_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    llm, retriever, contextualize_q_prompt
)


### Answer question ###
qa_system_prompt = """You are an assistant for question-answering tasks. \
Use the following pieces of retrieved context to answer the question. \
If you don't know the answer, just say that you don't know. \
Use three sentences maximum and keep the answer concise.\

{context}"""
qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", qa_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)

rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

2. Message History Implementation:

   • The application incorporates RedisChatMessageHistory along with RunnableWithMessageHistory. The intention is to leverage Redis for managing chat message history, tracking conversations by User ID and Conversation ID.

3. LangGraph Integration:

   • I'm attempting to integrate this setup into LangGraph. However, I'm facing challenges because LangGraph documentation suggests using Checkpoints with SQLite, and it's unclear how to integrate RedisChatMessageHistory which is essential for my application.

Issues:

• Integration with LangGraph: How can I integrate RedisChatMessageHistory within LangGraph, given that LangGraph primarily supports SQLite for Checkpoints?
• Consistent Message History: I need to ensure that message history capabilities are maintained across the entire application, allowing tracking of conversations by User ID and Conversation ID.

Resources:

• For the retrieval chain setup, please refer to the LangChain documentation on question answering with chat history: LangChain QA with Chat History.
• For details on managing persistent chat with user IDs and conversation IDs, see this example from LangServe: LangServe Chat with Persistence.

Request:

I am seeking advice or examples on how to properly integrate RedisChatMessageHistory with LangGraph in a manner that maintains full functionality of the message history features. Any insights or pointers towards documentation or similar implementations would be incredibly helpful.

System Info

System Information

OS: Linux
OS Version: #1 SMP Thu Jan 11 04:09:03 UTC 2024
Python Version: 3.11.9 (main, May 14 2024, 08:04:54) [GCC 12.2.0]

Package Information

langchain_core: 0.2.10
langchain: 0.2.6
langchain_community: 0.2.6
langsmith: 0.1.77
langchain_openai: 0.1.10
langchain_text_splitters: 0.2.1
langgraph: 0.1.2
langserve: 0.2.2

[dosubot[bot]]

To integrate RedisChatMessageHistory within LangGraph while ensuring consistent message history capabilities across the entire application, you can follow these steps:

1. Install Dependencies

Ensure you have the necessary dependencies installed:

pip install -U langchain-community redis langchain-openai

2. Set Up RedisChatMessageHistory

Import and set up RedisChatMessageHistory in your application:

from langchain_community.chat_message_histories.redis import RedisChatMessageHistory

3. Create a Chain with History

Define a chain that uses RedisChatMessageHistory to store and retrieve messages:

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_openai import ChatOpenAI

# Define the prompt template
contextualize_q_system_prompt = """Given a chat history and the latest user question \
which might reference context in the chat history, formulate a standalone question \
which can be understood without the chat history. Do NOT answer the question, \
just reformulate it if needed and otherwise return it as is."""
contextualize_q_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", contextualize_q_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)

# Create the history-aware retriever
history_aware_retriever = create_history_aware_retriever(
    llm, retriever, contextualize_q_prompt
)

# Define the QA prompt template
qa_system_prompt = """You are an assistant for question-answering tasks. \
Use the following pieces of retrieved context to answer the question. \
If you don't know the answer, just say that you don't know. \
Use three sentences maximum and keep the answer concise.\

{context}"""
qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", qa_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)

# Create the question-answer chain
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)

# Create the retrieval chain
rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

# Integrate with RunnableWithMessageHistory
chain_with_history = RunnableWithMessageHistory(
    rag_chain,
    lambda session_id: RedisChatMessageHistory(
        session_id, url="redis://localhost:6379"
    ),
    input_messages_key="input",
    history_messages_key="chat_history",
)

# Example configuration
config = {"configurable": {"session_id": "foo"}}

# Example invocations
print(chain_with_history.invoke({"input": "Hi! I'm bob"}, config=config))
print(chain_with_history.invoke({"input": "Whats my name"}, config=config))

4. Using Upstash Redis for Serverless Setup

If you prefer a serverless setup, you can use Upstash Redis. Here’s how to set it up:

from langchain_community.chat_message_histories import UpstashRedisChatMessageHistory

URL = "<UPSTASH_REDIS_REST_URL>"
TOKEN = "<UPSTASH_REDIS_REST_TOKEN>"

history = UpstashRedisChatMessageHistory(
    url=URL, token=TOKEN, ttl=10, session_id="my-test-session"
)

history.add_user_message("hello llm!")
history.add_ai_message("hello user!")

You can then retrieve the messages using:

history.messages

5. Integrate with LangGraph

LangGraph handles custom memory implementations through a concept called persistence, also known as checkpointing. To integrate external memory systems like Redis, you would add a checkpointer to the agent. Here is an example using an in-memory checkpointer:

from langchain_core.messages import SystemMessage
from langgraph.checkpoint import MemorySaver  # an in-memory checkpointer
from langgraph.prebuilt import create_react_agent

system_message = "You are a helpful assistant."
# This could also be a SystemMessage object
# system_message = SystemMessage(content="You are a helpful assistant. Respond only in Spanish.")

memory = MemorySaver()
app = create_react_agent(
    model, tools, messages_modifier=system_message, checkpointer=memory
)

config = {"configurable": {"thread_id": "test-thread"}}
print(
    app.invoke(
        {
            "messages": [
                ("user", "Hi, I'm polly! What's the output of magic_function of 3?")
            ]
        },
        config,
    )["messages"][-1].content
)
print("---")
print(
    app.invoke({"messages": [("user", "Remember my name?")]}, config)["messages"][
        -1
    ].content
)
print("---")
print(
    app.invoke({"messages": [("user", "what was that output again?")]}, config)[
        "messages"
    ][-1].content
)

By following these steps, you can integrate RedisChatMessageHistory within LangGraph, ensuring that message history is consistently managed across your application. The RedisChatMessageHistory class provides methods to store, retrieve, and manage chat messages in a Redis database, ensuring efficient and scalable message history management. Additionally, using Upstash Redis can offer a serverless and scalable solution for managing your chat message history ^[1][2][3].

To continue talking to Dosu, mention @dosu.