Hi everyone,
Regarding the issue of custom components breaking on Google Cloud Run deployments because of too many requests (viz. “Your app is having trouble loading the X component”), I posted a few questions on a github issue (see #5494) but wanted to ask here too, as this is quite a popular deployment option for Streamlit apps.
Context:
Google Cloud Run offers max. 1000 concurrent requests per instance, so sometimes custom components in Streamlit apps break if/when we reach that limit.
Questions:
-
Without the use of custom components, what exactly counts as 1 request in a Streamlit app deployed to Google Cloud Run? Does 1 refresh of the app = 1 request or is it more complex than that?
-
Do the generated requests just accumulate, or are they cleared when finished? I did some testing with and, even with a high concurrency limit, it seems the custom components break after a while, maybe indicating requests are accumulating.
-
Are there expressions, anti-patterns, or app designs that might cause these requests to “accumulate” (and not be cleared when finished), and that we should therefore avoid? For example, for loops,
st.experimental_rerun, calling functions, spawning threads, updating a widget on each iteration of a for loop, etc. -
When using custom components, is there a way to calculate how many additional requests will be generated? Does it depend on the component, on how the existing v1 template works, or on both?