Error in StreamlitCallbackHandler.on_llm_new_token callback: maximum recursion depth exceeded

[rasmuswissmann]

Summary

I am using the StreamlitCallbackHandler with a LangChain agent in a chatbot style interface.

When agent responses exceed a few hundred tokens, the system throws an error:

Error in StreamlitCallbackHandler.on_llm_new_token callback: maximum recursion depth exceeded

Has anyone come across this?

I can set sys.setrecursionlimit() to more than Python’s default of 1000. That prevents this error but causes other instability and crashes the Streamlit process altogether. It also feels wrong.

Steps to reproduce

cbh = StreamlitCallbackHandler(container,
max_thought_containers=3,
collapse_completed_thoughts=False,
thought_labeler=LLMThoughtLabeler(),
)
agent.run(prompt, callbacks=[cbh])

where agent is a class derived from Agent(BaseSingleActionAgent) and container is a st.container().

I am not aware that my code modifies anything else to uniquely cause this but will keep digging.

Curious if anyone else has seen such a recursion stacking.

[dataprofessor]

Hi @rasmuswissmann

Thanks for the question. In regards to the maximum recursion depth exceeded it may have been caused by an unintentional recursion in the code. There are a few forum post on resolving the issue that you can look into:

Streamlit Recursion

Hello I understand that streamlit rerun script from top-to-bottom. I am using python and after running the streamlit web app for awhile it will throw the Recursion Error: maximum recursion depth exceeded while calling a Python object error. I also understand that I could actually increase python’s recursion limit however it is not advisable. Does the rerun every time causes the recursion error? how can i avoid this recursion error? Please help:) import streamlit as st selected_flavour = None …

Best (fastest) practice to display live 2D data

Waw @w-markus you made an awesome example here! To be honest I did not expect Streamlit and asyncio to play well this way. I don’t know why the slowdown if you store in state, maybe Streamlit tries to do some attribute tracking on elements in a class, did not check . When using asyncio, I really usually prefer to use a producer/consumer method, with your producer pushing images in a queue (could be downloading data from your sensor) and consumer fetching images from the queue and d…

It may also be helpful to experiment with the max_thought_containers parameter to see if fewer value would still cause the error.