Are you running your app locally or is it deployed? LOCAL
Share the Streamlit and Python versions. Python 3.11 and Streamlit 1.31
How does streamlit handle multiple HTTP requests? In my chatbot, whenever a user queries, a POST API call is made to the cloud which generates an LLM response. This response is returned as an answer to the user.
If there are multiple users querying, would this cause a bottleneck since streamlit is waiting for each response to return before continuing?
Streamlit can wait. It does not do anything stupid while waiting. The bottleneck is on the api host not on streamlit app.
Each session runs in a different thread, so one session waiting doesn’t prevent other sessions from running.
Would this mean if three users submitted a message through chat_input and all three triggered three separate POST requests:
session.post(url, json=data)
Streamlit will process all three POST requests simultaneously and not one at a time?
No, it means it could be waiting on all the three for responses, or waiting on two of them and processing the response of third one.
Okay, as long as they were able to send out the requests. My fear was that the other two wouldn’t even be able to send out the request. Just waiting for the first one to finish before being able to even send the request out.
That is what using threads avoids. While one thread is waiting, another thread can do work.
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.