Are you running your app locally or is it deployed? LOCAL
Share the Streamlit and Python versions. Python 3.11 and Streamlit 1.31
How does streamlit handle multiple HTTP requests? In my chatbot, whenever a user queries, a POST API call is made to the cloud which generates an LLM response. This response is returned as an answer to the user.
If there are multiple users querying, would this cause a bottleneck since streamlit is waiting for each response to return before continuing?
Streamlit can wait. It does not do anything stupid while waiting. The bottleneck is on the api host not on streamlit app.
Each session runs in a different thread, so one session waiting doesn’t prevent other sessions from running.
Would this mean if three users submitted a message through chat_input and all three triggered three separate POST requests:
session.post(url, json=data)
Streamlit will process all three POST requests simultaneously and not one at a time?
No, it means it could be waiting on all the three for responses, or waiting on two of them and processing the response of third one.
Okay, as long as they were able to send out the requests. My fear was that the other two wouldn’t even be able to send out the request. Just waiting for the first one to finish before being able to even send the request out.
That is what using threads avoids. While one thread is waiting, another thread can do work.
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.