docs/tutorials/chatbot/

[giscus[bot]]

docs/tutorials/chatbot/

This guide assumes familiarity with the following concepts:

https://python.langchain.com/docs/tutorials/chatbot/

[paraglondhe098]

Very helpful explanation.

[somendrew]

Explaination made it look easy! Thanks to community.

[Nevertolate83]

Hey guys is this safe to run on my MAC Desktop ?

[somendrew]

I don't think Mac has enough RAM, Use jupyter notebook and t4 kernal , it'll be so smooth and fast

[Murdock135]

It's safe to run on any computer.

[2980897531]

You can use ChatGPT's API to run it. It will not use your Mac's GPU

[shubham-attri]

Loved the easy to understand explanation.

[ruksharahmed7]

Suppose I have a FastAPI backend where this chatbot code is present, the frontend is in react, and the chatbot uses the app.stream method for response streaming. In this case, do I use web-socket to enable streaming of AI response from the FastAPI backend to the React frontend?

[Fritskee]

Followed your exact tutorial, but when making the conversion to async there is an error that comes up:

NotImplementedError: Unsupported message type: <class 'coroutine'>
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE 

Below is the example of the code that throws it:

from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, MessagesState, StateGraph

# Define a new graph
workflow = StateGraph(state_schema=MessagesState)


# Define the function that calls the model
async def call_model(state: MessagesState):
    response = model.ainvoke(state["messages"])
    return {"messages": response}


# Define the (single) node in the graph
workflow.add_edge(START, "model")
workflow.add_node("model", call_model)

# Add memory
memory = MemorySaver()
app = workflow.compile(checkpointer=memory)

config = {"configurable": {"thread_id": "abc123"}}

query = "Hi! I'm Bob."

input_messages = [HumanMessage(query)]
output = await app.ainvoke({"messages": input_messages}, config)
output["messages"][-1].pretty_print()  # output contains all messages in state

[Fritskee]

nevermind, fixed it:

import asyncio
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, MessagesState, StateGraph

# Initialize the model
model = ChatOpenAI(model="gpt-3.5-turbo")

# Define a new graph
workflow = StateGraph(state_schema=MessagesState)

# Define the async function that calls the model
async def call_model(state: MessagesState):
    response = await model.ainvoke(state["messages"])
    return {"messages": response}


workflow.add_edge(START, "model")
workflow.add_node("model", call_model)


memory = MemorySaver()
app = workflow.compile(checkpointer=memory)


config = {"configurable": {"thread_id": "abc123"}}


async def chat():
    query = "Hi! I'm Bob."
    input_messages = [HumanMessage(query)]
    output = await app.ainvoke({"messages": input_messages}, config)
    print("\nBot:", output["messages"][-1].content)

    query = "What's my name?"
    input_messages = [HumanMessage(query)]
    output = await app.ainvoke({"messages": input_messages}, config)
    print("\nBot:", output["messages"][-1].content)


await chat()

[Akshat-Rastogi-6]

Awesome Tutorial!!! Loved it.
I just had this one problem in Managing Conversation History that while I was trying to add human message in the messages list :

input_message = messages + [HumanMessage(content=query)]

it was adding the message at 0 index of the message list shifting the System message to the 1st index; this constantly gave me the following error:

'''
ValueError: Unexpected message with type <class 'langchain_core.messages.system.SystemMessage'> at the position 1.
'''

to solve this, I had to change the trimmer configuration and made

trimmer = trim_messages(
    max_tokens=100,  
    strategy="last",
    token_counter=model,  
    include_system=False, #here
    allow_partial=False,
    start_on="human",
)

then it started working. I guess either the system message needs to be added separately or add the HumanMessage() to the messages, but in some other way.

Please see this and let me know.

[Marsax110]

Very good documentation, but I've encountered a small problem.

In the last example of Streaming, based on the code above, I cannot achieve the effect in the document, that is, streaming output.

This is because when defining the model generation above, invoke was used instead of stream.

As a result, during the subsequent generation, the output is done in whole blocks, and the "|" only appears once, at the end.

# Create a state graph for managing conversation flow
workflow = StateGraph(state_schema=State)

model_OpenAI = ChatOpenAI(model="gpt-4o", api_key=os.getenv("OPENAI_API_KEY"))


def call_model(state: State):
    """Function to generate model responses
    
    Args:
        state (State): Current state containing message history and language settings
        
    Returns:
        dict: Dictionary containing model response messages
    """
    # Use trimmer to trim message history to stay within token limits
    trimmed_messages = trimmer.invoke(state["messages"])
    
    # Generate prompt using template with trimmed messages and language setting
    prompt = prompt_template.invoke(
        {"messages": trimmed_messages, "language": state["language"]}
    )
    
    # Call language model to generate response
    response = model_OpenAI.invoke(prompt, config={"callbacks": [langfuse_handler]})
    
    # Return model response wrapped in messages list
    return {"messages": [response]}


# Add workflow edge connecting START node to model node
workflow.add_edge(START, "model")

# Add model node and bind call_model handler function
workflow.add_node("model", call_model)

# Create memory saver for storing conversation state
memory = MemorySaver()

# Compile workflow with checkpointer for state checkpoints
app = workflow.compile(checkpointer=memory)


config = {"configurable": {"thread_id": "abc789"}}
query = "What company are you a big model for?"
language = "English"

input_messages = [HumanMessage(query)]
for chunk, metadata in app.stream(
    {"messages": input_messages, "language": language},
    config,
    stream_mode="messages",
):
    if isinstance(chunk, AIMessage):  # Filter to just model responses
        print(chunk.content, end="|")

> I am a language model created by OpenAI.|

But if I switch to .stream here, I get the error again

    # Call language model to generate response
    response = model_OpenAI.stream(prompt, config={"callbacks": [langfuse_handler]})

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[119], line 6
      3 language = "English"
      5 input_messages = [HumanMessage(query)]
----> 6 for chunk, metadata in app.stream(
      7     {"messages": input_messages, "language": language},
      8     config,
      9     stream_mode="messages",
     10 ):
     11     if isinstance(chunk, AIMessage):  # Filter to just model responses
     12         print(chunk.content, end="|")

File ~/Library/Caches/pypoetry/virtualenvs/9091-ell-6QbIK8Ul-py3.10/lib/python3.10/site-packages/langgraph/pregel/__init__.py:1655, in Pregel.stream(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, subgraphs)
   1649     get_waiter = None  # type: ignore[assignment]
   1650 # Similarly to Bulk Synchronous Parallel / Pregel model
   1651 # computation proceeds in steps, while there are channel updates
   1652 # channel updates from step N are only visible in step N+1
   1653 # channels are guaranteed to be immutable for the duration of the step,
   1654 # with channel updates applied only at the transition between steps
-> 1655 while loop.tick(input_keys=self.input_channels):
   1656     for _ in runner.tick(
   1657         loop.tasks.values(),
   1658         timeout=self.step_timeout,
   (...)
...
--> 333     raise NotImplementedError(msg)
    335 return _message

NotImplementedError: Unsupported message type: <class 'generator'>
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE 

[HoganHPH]

Hi, I'm facing to the same situation. I think now the returned chunk is a whole sentence. Is it right? Or have you fix this?

[Marsax110]

I didn't solve this problem, and I ended up choosing not to use LangChian, but to use OpenAI's SDK directly to achieve my goal

[ddww]

This is my environment. The model is qwen2.5. It is deployed through vllm. When using the examples in the tutorial, the following error is reported. Can anyone who has encountered this problem help me solve it?

model = ChatOpenAI(model="Qwen2.5-32B-Instruct-AWQ",
openai_api_key="xxxxxx",
openai_api_base="http://localhost:9983/v1",
temperature=0.01)
trimmer = trim_messages(
max_tokens=65,
strategy="last",
token_counter=model,
include_system=True,
allow_partial=False,
start_on="human",
)
messages = [
SystemMessage(content="you're a good assistant"),
HumanMessage(content="hi! I'm bob"),
AIMessage(content="hi!"),
HumanMessage(content="I like vanilla ice cream"),
AIMessage(content="nice"),
HumanMessage(content="whats 2 + 2"),
AIMessage(content="4"),
HumanMessage(content="thanks"),
AIMessage(content="no problem!"),
HumanMessage(content="having fun?"),
AIMessage(content="yes!"),
]
result=trimmer.invoke(messages)
print(result)

The following is the error:
NotImplementedError: get_num_tokens_from_messages() is not presently implemented for model cl100k_base.