Does streamlit is running on a single-threaded development server by default or not?

[korokat]

Hello! I would like to have production setup on AWS EC2 instance. I am able to see that Streamlit does not support the WSGI Protocol for now. So deploying Streamlit with (for example) gunicorn is not currently possible.

Question #1. Does it mean that streamlit run app.py would run the app on a single-threaded development server?

Question #2. I am able to see that Nginx might be configured to forward requests to the port where my Streamlit app is running. I would allow me to run Streamlit app on more common ports such as 80. But does it make any sense from the production load perspective? If the development server is single threaded, then Nginx would not make things better I think. Am I wrong?

[tim]

Hey @korokat, welcome to Streamlit!

1. Yes, Streamlit uses Tornado Web Server under the hood. Tornado is a single-threaded web server, and isn’t based on WSGI. Running it with gunicorn is not currently possible (or at least, likely to be extremely non-trivial).

2. You can use Nginx as a reverse proxy, sitting in front of a Streamlit server. But yeah, reading between the lines here, it sounds like you’re potentially worried about how Streamlit scales with lots of users…

Streamlit scales differently than your typical web app. The big difference is that, when a user connects to Streamlit, we spin up a thread just for them and run your app.py script for that user. And then Streamlit will re-run that script for that user each time they do something interactive (like pressing a button or sliding a slider); and each time the script is re-run, we spin up and down a thread again to run it. This is all to say, Streamlit itself is multi-threaded (so you’ll benefit from running it on a machine with lots of beefy cores), and its scaling challenges come from the CPU/GPU/memory costs of running Streamlit app scripts very frequently.

I’m not a web-scaling expert by any means, but what this means is that traditional scaling solutions (like putting a caching server in front of your app) probably won’t have the same result for your Streamlit app.

To help your Streamlit app scale, you’ll want to focus on having your app.py script run quickly and efficiently. The best tool to start with is probably Streamlit’s built-in caching utility, which is all about not re-running code that doesn’t need to be re-run!

If you’re running into specific scaling issues, please continue posting here - there are lots of users with experience on this sort of thing!