ComfyUI needs a restart after trying to describe too many images

[serget2]

It works really well, but after two or three images I get this error:
400 Request payload size exceeds the limit: 20971520 bytes. The file size is too large. Please use the File API to upload your files instead. Example: f = genai.upload_file(path); m.generate_content(['tell me about this file:', f])

• I thought the image was too large so I used smaller ones same error
• Then I thought memory issue cleared it same error
• only when I restart it works again (with the same image it claimed too large) but then after three or four images same error

It seems to be hitting a limit, but I ca figure out if it is griptape or Gemini, Ima try Ollama 3.2 vision next to see if it is the same

[shhlife]

Thanks! did you find out if ollama 3.2 vision had the same issue?

[serget2]

Ollama 3.2 suddenly didn't recognize the nodes anymore, after I got them to work again (rebuild the nodes) it worked but I haven't tried long enough I think to reproduce the error, went back to gemini because of better descriptions and less restrictive (Djeez blocking NSFW may be a good thing but if angelic face and similar safe seeming words are linked to CSAM it's a bit too restrictive)

[serget2]

Anyway the problem still arises after about 10-12 different images, Wether I do them 1 by 1 or like 12 images in a row, after a restart of Comfy it works again. Comfy dosn't hang and still wants to work it's just that the agent for "seeing images" doesn't work anymore and just reloading or rebuilding those nodes does not work

[shhlife]

Okay, I ran into this as well.. got a payload size issue like you did:

<RetryCallState 1359710837072: attempt #1; slept for 0.0; last result: failed (InvalidArgument 400 Request payload size exceeds the limit: 20971520 bytes. The file size is too large. Please use the File API to upload your files instead. Example: `f = genai.upload_file(path); m.generate_content(['tell me about this file:', f])`)>

[shhlife]

restarting the server worked - will try now to do the same thing but in pure griptape framework to determine if it's a Griptape issue or a ComfyUI issue.

[shhlife]

Okay, running in Griptape worked just fine.. I ran through 25 images 1 at a time and it worked w/out any errors. Here's the code I used:

import os
from pathlib import Path

from dotenv import load_dotenv
from griptape.drivers import GooglePromptDriver
from griptape.structures import Agent

load_dotenv()
GOOGLE_API_KEY = os.getenv("GOOGLE_API_KEY")
images_path = Path("images")
images = [str(image) for image in images_path.glob("*")]
agent = Agent(
    prompt_driver=GooglePromptDriver(api_key=GOOGLE_API_KEY, model="gemini-1.5-pro"),
    stream=True,
)
if True:
    for i, image in enumerate(images):
        agent.run(f"Describe this image: {image}")
        print(f"Image {i + 1} of {len(images)}")
else:
    agent.run("Describe this image: images\\banana_felt.png")

So you can see it's just going through, 1 at a time.. got all the way to image 25 out of 25 and never hit the issue.

I think ComfyUI must be doing something funky.. I wonder if it's the LoadImage node isn't clearing out memory correctly?

[shhlife]

I don't think there's anything I can do here at the moment. it's a ComfyUI issue. :( It might be something we're able to solve if we can offload this.. going to put it on hold for the moment