Options for Streamlit to access local data sources after deployment

Hello. I’ve built a Streamlit app that generates a dashboard of metrics from a log file that is produced by a particular game. For the dashboard to be useful to others (i.e., players of the game), the Streamlit app needs to automatically rescan the log file every few seconds because the game writes new info to the log file in realtime as the game is played.

While the app works as intended when I run it locally, my understanding is that when I deploy the app to Streamlit Community Cloud, or any other cloud service, the app will not be able to programmatically access the log file on a user’s computer (i.e., without manual actions by a user).

Conceptually speaking, what are my options? This is entirely new territory for me.

For those that have faced an analogous situation, how did you solve it?

Thanks!

If your app is reading and writing files, it will be doing so on the Streamlit server, not on the client. You can still have an app that writes files, they’ll just be on the server until you specifically offer them to the user to download with st.download_button. Since Streamlit has a server-client structure which supports multiple, simultaneous users, you may need to use something like tempfile or named directories to keep different players’ data separate (or create some kind of database for it).

Appreciate the quick reply. The problem I’m trying to solve is reading files on the client, which is where the game is running that produces the log file (which is my app’s primary data source). How can this be achieved?

I suppose repackaging my Streamlit app as an exe would enable it to run on the client, which would grant it access to the log file. While I would prefer to deploy the app in the cloud, I’d be open to this approach if it were efficient and reliable. My research on this forum and across other info sources suggests people have a very low success rate with converting Streamlit apps to exe.

Are you generating this file directly from JavaScript? Or is the user “playing” the game by editing a local file? I’m saying that if you are reading and writing to a file in Python, you can still host your app just understanding that all the users’ files will be on the server. What functionally is the reason that the file has to be on the client?

Oh, I just re-read the original post. I misunderstood that the app was the game.

So yes, people will either need to upload their log file themselves, or they’ll need to install Python and Streamlit so they can run the app locally.

Personally, I don’t think it’s a big stretch for people to be told where their log file is so they can select it and upload it. I think that’s generally more convenient than setting up to run an app locally. Also, I wonder how stable the log file is across possible installations. Is this game available on multiple OS’s? Can it be installed in different locations than a single, system default? (I have a separate game partition on one of my computers which changes a lot of file locations. )

Thanks again for digging in, mathcatsand. Correct - the app I built is not the game; rather it’s a dashboard of real-time metrics and insights that complement the gaming experience. Apologies if I was unclear.

That my app delivers real-time metrics and insights is critical to its value proposition. The app would be much less useful if the user playing the game had to break from the action to upload the log file to my app. So, I’m really seeking a way for my app to automatically read the log file every few seconds.

If there were a way for the user to automatically post the log file to cloud storage (e.g., an S3 bucket), then I suppose it would be trivial for my app to automatically access the file. Are there off-the-shelf solutions that my users could install that would automatically upload a specific file to cloud storage? That might be a different way to attack the problem.

As to your question, the game runs solely on Windows and the game’s directory, along with the log file’s location within that directory, is fixed and reliable.

I don’t know of a good solution, unfortunately. This is basic browser security that won’t allow a website to just access your file system without you specifically giving it a file. (It’d be pretty terrifying for sites to just have access to your file system. :grimacing: )

Ha - agreed. I wouldn’t want to live in that world. Conceptually, I feel like there should be a way to authorize a bidirectional connection for a specified purpose.

I suppose I will forego cloud deployment and try to convert my app to an exe using PyInstaller or Auto-Py-to-Exe. Thoughts/learnings on this process are welcome.

Thanks again.