Loadbalancing in kubernetes issues (cookie help?)

JimJam · March 16, 2023, 6:39pm

Hi, I’m trying to deploy a streamlit app in a kubernetes environment. It works fine when there is one replica serving the app, but if I scale out so that there are several replicas, the app does not display properly - some of the graphs, etc, do not show in the browser (they are replaced by a ‘0’ character).

My current asumption is that this is happening because the requests from a user’s browser are being distributed round-robin among the pods, and that streamlit is websocket based and needs sticky sessions enabled.

If this sounds like the right cause, then I need to find a way of introducing session affinity, so I can route all request from a user to the same pod. However, the nature of our setup is that I don’t have access to the IP of the user. So, either I need to be able to identify users by a cookie or a http header.

Does streamlit set a session cookie I can use for this purpose?

MikeHowells · May 10, 2023, 4:38am

I have a similar scenario and raised this feature request:

github.com/streamlit/streamlit

Bring your own Redis cache

opened 09:54PM - 13 Dec 22 UTC

quantoid

type:enhancement

### Problem Our micro-service hosting platform will auto-scale a service so t…hat multiple instances are created or destroyed to adapt to varying traffic load. A user request might be routed to any one of these instances, as they are assumed to be stateless. However, our Streamlit dashboard service is not stateless due to the way session state and caches are implemented. At the moment each instance of the service has its own Streamlit cache, which means users will often get a cache hit that’s followed immediately by a miss when their next request gets routed to another instance. We also get problems with session state not being mirrored across instances. ### Solution Our platform provides Redis resources that are shared between instances so that to users there appears to be a single cache (rather than one per instance). What I want to do is to tell Streamlit to use this shared Redis resource as the cache, also using it to store session state. ### Additional context We are not using Streamlit Cloud because we have our own PaaS that runs on AWS, so we want our Streamlit services to run there in order to comply with security policies for internal tools like dashboards. --- Community voting on feature requests enables the Streamlit team to understand which features are most important to our users. **If you'd like the Streamlit team to prioritize this feature request, please use the 👍 (thumbs up emoji) reaction in response to the initial post.**

Topic		Replies	Views
Streamlit HA configuration 🚀 Deployment cache	2	765	December 12, 2023
How to really scale a Streamlit App? 🚀 Deployment	5	3078	February 18, 2024
Keep streamlit app running even if pods dies 🚀 Deployment session-state	1	675	April 28, 2023
Streamlit reruns all 30 seconds 🚀 Deployment gcp	5	3094	May 13, 2022
Streamlit deployment - unable to fetch user details 🚀 Deployment	1	320	November 19, 2023

Loadbalancing in kubernetes issues (cookie help?)

Related Topics

Hello there 👋🏻

Cookie settings

Strictly necessary cookies

Performance cookies

Functional cookies

Targeting cookies