I have a function that collects comments from youtube livestreams and the function can take up to 1 hour until itās finished. I noticed that my app stops running after a longer time.
So I was wondering if streamlit has like a maximum length of minutes that a function can run. If not I have to look back into my code.
Edit: Or does it have to do something with the resources, the 1GB space for projects? Because if I reboot the app it works again.
I think itās important to step back and think about the bigger pictureā¦websites generally donāt have a pattern where you wait for an hour for something to happen. So yes, Streamlit does functionally have a timeout (probably 10-15 minutes before the websocket is closed, not sure exactly), but at the same time your app sounds more like something that should be as a cron script/data pipeline.
Of course, if Python is actually doing something reasonably webapp-ish, then you shouldnāt really have timeout issues.
Yeah, i should reconsider how I gather the data. I work with the pytchat module that helps you collecting livestream comments without selenium and beautifulsoup and it needs a longer connection to gather the data.
After around 15 mins the running animation in the top right corner stops. That fits with your explanation.
Hi @randyzwitch, @MaxMnemodo you know if there is a way to extend that timeout - to letās say 30 mins? The processing time of my use case is very close to 10-15 min, and want to avoid any issues. Thanks in advance.
Iām using Streamlit to empower users to run Monte Carlo simulations which take 20-40 mins. Sometimes the processing completes, but the results arenāt displayed. Other times the page just refreshes while the processing continues in the backend.
Iām curious to know if this is recommended. And if not, would it be possible to tweak some settings to work around this?
In the case of 20-40 min long-running processes, I would consider how you can split your code into the Streamlit front-end and a processing backend.
For example, you could move the actual monte carlo simulation to something like FastAPI, so that Streamlit requests a run and then you periodically check to see if the run is complete. Itās slightly more work, but itās much more āwebā like than having a process running 20-40 mins long and hoping nothing happens on the Streamlit frontend.
Hi, thanks for these answers. Iām having a similar issue. I have Streamlit calling my backend function, according to the settings input by each user. Every few minutes it logs something to the Streamlit interface. And then periodically the Streamlit front end resets.
What I donāt understand is: If I moved my backend to FastAPI, and Streamlit resets, how would Streamlit check to see if the run is complete? There could be multiple instances running and I canāt figure out how to know which run this browser window was talking to.
Itās difficult to ācheck if the run is completeā without a logged-in user (and Streamlit doesnāt support user authentication very well).
One solution could be to generate a unique identifier for the job, show it to the user, and store the results in a database linked to the identifier. Then, when the user comes back, they can request the job results using the identifier. Not sure how to handle if the job is still running or has failed though.
I do something similar to the unique identifier. I ask the user for a string that I use as a prefix for his tables that Iāll create. Every step in the job writes a temporary table, using that prefix. Also, when he starts his job, I add a record to a history table with his prefix and other settings.
When the user comes back he can view the history table and select his job. He can see the results, or a list of tables which were created before Streamlit reset. He can click a button to reload his settings and then continue from where it reset, or start from scratch.
Also I keep writing updates to the Streamlit app so until the reset, the user can see whatās going on.
I donāt know if any of that applies to your situation.