Does streamlit has a maximum run time for functions?

šŸŽˆ Using Streamlit
1

I have a function that collects comments from youtube livestreams and the function can take up to 1 hour until itā€™s finished. I noticed that my app stops running after a longer time.

So I was wondering if streamlit has like a maximum length of minutes that a function can run. If not I have to look back into my code.

Edit: Or does it have to do something with the resources, the 1GB space for projects? Because if I reboot the app it works again.

2

Hi @MaxMnemo -

I think itā€™s important to step back and think about the bigger pictureā€¦websites generally donā€™t have a pattern where you wait for an hour for something to happen. So yes, Streamlit does functionally have a timeout (probably 10-15 minutes before the websocket is closed, not sure exactly), but at the same time your app sounds more like something that should be as a cron script/data pipeline.

Of course, if Python is actually doing something reasonably webapp-ish, then you shouldnā€™t really have timeout issues.

Best,
Randy

1 Like
3

Yeah, i should reconsider how I gather the data. I work with the pytchat module that helps you collecting livestream comments without selenium and beautifulsoup and it needs a longer connection to gather the data.

After around 15 mins the running animation in the top right corner stops. That fits with your explanation.

Thanks :pray:

1 Like
4

Hi @randyzwitch, @MaxMnemo do you know if there is a way to extend that timeout - to letā€™s say 30 mins? The processing time of my use case is very close to 10-15 min, and want to avoid any issues. Thanks in advance.

5

@randyzwitch @marduk, Iā€™m also curious to know the answer.

Iā€™m using Streamlit to empower users to run Monte Carlo simulations which take 20-40 mins. Sometimes the processing completes, but the results arenā€™t displayed. Other times the page just refreshes while the processing continues in the backend.

Iā€™m curious to know if this is recommended. And if not, would it be possible to tweak some settings to work around this?

6

Unfortunately I have no solution for that. Maybe try a cloud solution.

Compute the simulation within google cloud and await the results.

7

In the case of 20-40 min long-running processes, I would consider how you can split your code into the Streamlit front-end and a processing backend.

For example, you could move the actual monte carlo simulation to something like FastAPI, so that Streamlit requests a run and then you periodically check to see if the run is complete. Itā€™s slightly more work, but itā€™s much more ā€œwebā€ like than having a process running 20-40 mins long and hoping nothing happens on the Streamlit frontend.

2 Likes
8

Hi, thanks for these answers. Iā€™m having a similar issue. I have Streamlit calling my backend function, according to the settings input by each user. Every few minutes it logs something to the Streamlit interface. And then periodically the Streamlit front end resets.

What I donā€™t understand is: If I moved my backend to FastAPI, and Streamlit resets, how would Streamlit check to see if the run is complete? There could be multiple instances running and I canā€™t figure out how to know which run this browser window was talking to.

Thanks

1 Like
9

Iā€™m facing that same challenge right now.

Itā€™s difficult to ā€œcheck if the run is completeā€ without a logged-in user (and Streamlit doesnā€™t support user authentication very well).

One solution could be to generate a unique identifier for the job, show it to the user, and store the results in a database linked to the identifier. Then, when the user comes back, they can request the job results using the identifier. Not sure how to handle if the job is still running or has failed though.

Any better ideas? :sweat_smile:

10

I do something similar to the unique identifier. I ask the user for a string that I use as a prefix for his tables that Iā€™ll create. Every step in the job writes a temporary table, using that prefix. Also, when he starts his job, I add a record to a history table with his prefix and other settings.

When the user comes back he can view the history table and select his job. He can see the results, or a list of tables which were created before Streamlit reset. He can click a button to reload his settings and then continue from where it reset, or start from scratch.

Also I keep writing updates to the Streamlit app so until the reset, the user can see whatā€™s going on.

I donā€™t know if any of that applies to your situation.

1 Like
11

Hi @enegy,
Did you find a solution for this?
I have a similar use case to you, where I want to run some Monte Carlo simulations for engineering that can run from 30 mins to 48 hours. In this case, the code would run overnight and the user can see the results later. Is this possible on streamlit?

If not streamlit, did you find any alternative websites that might work for this? I had a co-worker suggest https://mecsimcalc.com, which supposedly can run up to 7 days, have you tried this or know if itā€™s any good?

Thanks!