Excessive RAM consumption #3733
-
|
I have a problem with excessive RAM consumption?
|
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 1 reply
-
|
does it grow with every api call? |
Beta Was this translation helpful? Give feedback.
-
|
yes |
Beta Was this translation helpful? Give feedback.
-
|
I've noticed the same thing when uploading txt/csv file, it's relatively small compared to when using custom python with chromaDB code, also knowing that everything is running on huggingface with 16Gb of vRAM and relaying on Groq API not a local model! |
Beta Was this translation helpful? Give feedback.
-
|
I detected that graphs are added to a pool, but never removed, except when the prediction is aborted. The pool accumulates the chatflow by session id. Furthermore, this pool is not used as a cache since the nodes need to be reprocessed for each request to resolve the interpolated variables with double braces{{ }}.
I solved the problem by creating a cache and forgoing using variable interpolation with double braces that must be resolved before creating the langgraph node and using state variables instead.
|
Beta Was this translation helpful? Give feedback.
-
|
Resource consumption now
|
Beta Was this translation helpful? Give feedback.
I detected that graphs are added to a pool, but never removed, except when the prediction is aborted. The pool accumulates the chatflow by session id. Furthermore, this pool is not used as a cache since the nodes need to be reprocessed for each request to resolve the interpolated variables with double braces{{ }}.
Also, rebuilding the graph for each request takes a lot of memory and CPU resources
I solved the problem by creating a cache and forgoing using variable interpolation with double braces that must be resolved before creating the langgraph node and using state variables instead.
I did stress testing and it is now working in production.
So far so good.