First of all, congratulations on having an application with a continuously growing user base. This is a remarkable achievement for an application developed using Streamlit. Although I cannot fully understand your current situation, I can try to provide some suggestions for your reference:
- It is advisable to avoid excessively large Session States. If appropriate external storage is available, you may consider switching to external storage.
- Separating the static files of Streamlit and related components to a CDN may be helpful. In the complex scenarios I have encountered, this approach has indeed been quite beneficial. (However, this requires a certain amount of engineering effort. You can refer to: New Component: Streamlit-CDN allows you to deploy multiple instances in K8S and host all Streamlit dependencies on your own CDN)
- If there are other heavy logic processes on the page, you may consider splitting the logic into separate pages. (New Component: Streamlit-router allows you to create truly production-level multi-page applications)
- You can use the method introduced here to analyze the entire page, which may help identify potential system bottlenecks. (New Component: streamlit-embeded allows easy embedding of an HTML snippet and automatically adjusts the content height)
These suggestions may be beneficial in general situations, but they may not be as effective for applications like chatbots. “Always Async” may be a more effective approach, such as always using AsyncOpenAI Client instead of synchronous methods like requests.
However, these are not the most crucial points. Most importantly, if you already have a stable user base, it may be worth considering rebuilding the application using other frameworks such as Vercel’s AI SDK with React. After all, in more complex application scenarios, Streamlit’s advantages will rapidly diminish. (This is also the reason why I transitioned from extensively using Streamlit to Dash, and then further to Next.js.)