Cookbook 03_memories_and_summaries.py example program FAILS with Ollama

[bill-s]

Converting your 03_memories_and_summaries.py program to use remote Ollama with 'llama3.2' as:

agent = Agent(
#model=OpenAIChat(id="gpt-4o"),
model=Ollama(id=llm_model, host="http://192.168.1.197:11434")
and:
memory=AgentMemory(
db=SqliteMemoryDb(
table_name="agent_memory",
db_file="tmp/agent_memory.db",
),
model=Ollama(id=llm_model, host="http://192.168.1.197:11434"),
fails to use Ollama, but instead tries to use the default OpenAI in chat.py:343 return self.get_client().chat.completions.create(

self.get_client() is trying to return OpenAI and gets an OPENAI_KEY_ERROR when not set, or an invalid key error when set (to either Ollama or an old OpenAI key).

I should note that I'm mapping over and testing most of your Cookbook and other example code to Ollama, and testing with a variety of different LLMs. While you have a great framework, you are a little thin with documentation, having to dig into the source code. Do you have any architecture documents for design philosophies to better understand how you've assembled the product.

Thanks.

[manthanguptaa]

Hey @bill-s the AgentMemory class has a classifier param which requires a model. If not given it defaults to OpenAI.

[bill-s]

Thanks for the quick response. I haven’t worked with classifiers and haven’t been successful in setting it for the AgentMemory. While this doesn’t generate an error, it apparently doesn’t set the environment up properly: agent.memory.classifier = Ollama(id=llm_model, host="http://192.168.1.197:11434") Can you give me an example and/or tell me how to adjust your example code in CookbookL 03_memories_and_summaries.py? Thanks, Bill Sumerlin

…

On Jan 2, 2025, at 6:07 PM, Manthan Gupta ***@***.***> wrote: Hey @bill-s <https://github.com/bill-s> the AgentMemory class has a classifier param which requires a model. If not given it defaults to OpenAI. — Reply to this email directly, view it on GitHub <#1688 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMLQSC3UOGRIJTRVW2SCGD2IXWH3AVCNFSM6AAAAABUQRJP46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRYGYYDSNZWHE>. You are receiving this because you were mentioned.

[manthanguptaa]

@bill-s

from phi.agent import Agent , AgentMemory
from phi.memory.classifier import MemoryClassifier
from phi.memory.summarizer import MemorySummarizer
from phi.model.google import Gemini
from phi.memory.db.sqlite import SqliteMemoryDb
from phi.storage.agent.sqlite import SqlAgentStorage

agent = Agent(
    model=Ollama(id=llm_model, host="http://192.168.1.197:11434/"),
    markdown=True,
    debug_mode=True,
        memory=AgentMemory(
        classifier=MemoryClassifier(model=Ollama(id=llm_model, host="http://192.168.1.197:11434/")),
        summarizer=MemorySummarizer(model=Ollama(id=llm_model, host="http://192.168.1.197:11434/")),
        db=SqliteMemoryDb(
            table_name="agent_memory",
            db_file="tmp/agent_memory.db",
        ),
        # Create and store personalized memories for this user
        create_user_memories=True,
        # Update memories for the user after each run
        update_user_memories_after_run=True,
        # Create and store session summaries
        create_session_summary=True,
        # Update session summaries after each run
        update_session_summary_after_run=True,
    ),
    # Store agent sessions in a database, that persists between runs
    storage=SqlAgentStorage(table_name="agent_sessions", db_file="tmp/agent_storage.db"),
    description="You are a helpful assistant that always responds in a polite, upbeat and positive manner.",
)

agent.print_response("My name is bill")

print(agent.memory.summary)

agent.print_response("What is my name?")

Try this out

[bill-s]

I made a couple of adjustments to your code above: defining 'llm_model' and adding an import for Ollama. While this code works better, it is still generating errors and will not complete without crashing.

On the first run, it complained that the sqlite tables hadn't been created, but created them. I would suggest not generating this diagnostic message, but generate an ERROR message if it is unable to create the tables. I can confirm that all of the tables were created.

It also gave errors and wouldn't save anything until I set the OPENAI_API_KEY to ollama.

On my third run, it appeared to be saving things, but it crashed with "openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: ollama..." when executing the 'agent.print_response("My name is bill")' statement.

I should note that the sqlite tables were empty... I would have expected that the data would have been saved before the program crash (i.e., the data flushed to the table after the write) so that the various successful operations could be confirmed.

Thanks for the code sample. I wouldn't have gotten this far without since the MemoryClassifier and MemorySummerizer are only mentioned in 2 places in all your code and sample codes... and I don't believe in any of the supporting documentation.

BTW, I went back and also made the changes on your 03_memories_and_summaries.py program, and that still doesn't work. I/we are closer, but I'm dead in the water until I get this resolved, or figure out and/or create a replacement. I'd much prefer to get your program to work for everyone's benefit. Thanks.

[manthanguptaa]

@bill-s the code I shared works perfectly for me and it's a little difficult for me to help here as I am not sure if you have missed any steps while setting things up. The last error shows that you have set incorrect API key but there is no need for an API key in the code

[bill-s]

I just loaded the latest library that you released a week a go. There is a note in the documentation that says that non-OpenAI models do not work yet for AgentMemory(). The issue is that the settings that you kindly shared with me via classifier and summarizer are NOT being used by the other classes (e.g., AgentMemory model is not set prior to its runtime check, which sets it to use OpenAI). You can confirm this by turning on DebugMode=True and scanning the output. You should NOT see any "*** OpenAI Response Start *** or end. These are being generated by the OpenAI chat.py program. The Ollama chat.py is supposed to be used instead with the AgentMemory functions... Now depending not he LLM you are using, you will get different results since the tool response mechanism is still being refined in Ollama. Depending on those responses, everything can complete without error (when Debug is off), since the response back from the LLMs is NOT to update the database tables. You can view the two tables and see the memory table is defined but empty, and the session table has entries, but the summaries are never written. If a response comes back from the model to do an update, the program will crash...

So far throughout the code that I've looked at (and there is a lot), this is the only place that this type of class initialization is being used. Why? Since the various classes are loosely tied together, and without an architectural overview, it took a while to figure out that the self.models were not initialized when the should have... The BaseModel implementation made things a little challenging single stepping with a debugger through all of the code...

If you do turn on debugging and do NOT have any OpenAI messages in your output stream, then you are using an internal developer release. Thanks for quick responses... You have the makings of great a product, but it needs a lot more testing and better documentation.

Again waiting for your response.... Happy new year.

[bill-s]

Looking at the code again, it appears that instead of running the code that generates "* MemoryManager Start ", it should be running " MemoryClassifier Start " or " MemorySummerizer Start *" depending on the operation.

In phi/agent.py:2628 update_memory() is called, which then calls phi/agent.py:292 update_memory(). One of these needs to be changed to use declared classifier llm. Which one depends on the long-term overall architecture and intent of this section of the main agent.py code. You may have a similar issue in the code when using alternate OpenAI models and code.

Getting all of the memory support functions working properly is important for all of the different LLM environments... and not just Ollama which I'm using across multiple diverse local protected environments.

Thanks.