Streamlit backend data flow

[zuberek]

Hi, thank you for your amazing product.

I am making an app where users can upload data and I would like to be explicit in my warnings about the data processing flow.

So there are frontend (user browser running JS) and backend (server running Python).

From my understanding the streamlit backend is hosted on Google servers

Streamlit Trust and Security - Streamlit Docs

When an user uploads a file, then:

1. It is sent to the backend Streamlit Google servers running Python
2. Python reads the file and sends it in a BytesIO format back to the front end (it is not kept on the server whatsoever)
3. The data is then stored in the RAM memory on the user machine (potentially cached so that it’s not lost on a browser reload)

Similarly, when I’m later fitting a model on the data

1. Data is again sent to the server
2. Python fits the model and gets a result
3. The result is sent to the front end

Is that a correct understanding?

Or is the data actually stored in the RAM on the server side? Ii would make sense because then there’s no need to send it back and forth between the server and user, but that would require tons of RAM from streamlit servers?

Or maybe Python is somehow translated to JS and actually simply runs in the user browser.

Thank you for your answers!

Some links with similar questions:

Streamlit Sharing - Fileupload, where does it go?

So, I successfully shared my app via Streamlit Sharing. Even the upload file part works but, my quesion is, does the file persist anywhere (i.e., will it be there in a few days somewhere like in Github ( i didn’t see it)) or, does it reside in memory and clear when the browser/session is terminated? thanx Great work team.

How long are are the files uploaded via file_uploader stored on streamlit's server?

As a follow up, I wanted to understand how I can delete the files as soon as the processing is complete on those files

Where does the data go when using file_uploader? When does it get deleted?

Working on a concept or two here and curious as to what happens to the file once a user uploads a it through streamlit. Say a user uploads a CSV file to process. Once it’s processed, how long does that CSV file live on or when does it get deleted? Thanks!

[mathcatsand]

All of the data is held and processed on the server hosting Streamlit. What is sent to the front end are the visuals for display, whatever that may be. So when a user uploads a file, it goes to the server machine (running Python/Streamlit) and sits on that server in memory until/unless you do something to write that data somewhere. That data only goes back to the front end of you do something to display that data or explicitly provide a download option to the user.

The user’s browser isn’t doing any of the data processing. Only the server is executing the Python computations you specify. Streamlit Cloud has a 1GB resource limit, so if you have a large model, you won’t be able to use it and will need to find another hosting solution.