Update the database via Streamlit app

[isksy]

So i have deployed my first Streamlit app to Heroku.

The app is a dashboard where the user can insert certain parameters and data will be presented to the user.
The data is stored in an SQLite3 DB file and requested as a dataframe and shown to the user.
So I would like to update the database on a daily basis with new CSV files. If I do this on my local PC i need to git commit the changes and push the files to Heroku.

My question: Is there a way for me to only update the Database where the dashboard gets the data from without and present this to the user? Without deployment and git commit.

Thanks in advance

[SaintDrei]

Hi isksy,

It seems we have some similarities in our apps. I hope I could help you out with what I know.
Could you please clarify these things to help me understand your situation:

• Your SQLite3 DB file is stored locally?

• where the dashboard gets the data from
  Meaning the local storage?

• How will you be updating the Database? You open a CSV file, add it to the database, and user from another unit will see your data?

[isksy]

Hi SaintDrei,

The SQLite3 DB is a DB file which is created and stored in the app.py folder.
So to elaborate on what I did -
I have made an ‘Input’ folder where I store CSV files that needs to be loaded into the SQLite DB.
After running the app.py file the data in the CSV files are automatically loaded in the SQLite3 database. Then i made a dashboard where the user can give in certain inputs and a table and graph is then presented to the user.

The problem is when i want to update the DB with new CSV files, I have the add the CSV files locally to the Input folder and git push heroku master the changes to the herokuapp. Then i can run the functions to update the database with the new CSV data.

My question: is there a way to update the SQLITE3 DB file, which is stored in the heroku repository, without doing it first locally and commiting the changes to Heroku.

[SaintDrei]

Since your app is already hosted on heroku, I think committing really is redundant on your case. Maybe you could set up an update function on your app, where it asks for the files to be uploaded, then runs mainDB.append(updateCSV, ignore_index=True). Then output the file with to_csv() and overwrite the current mainDB.

This way, you just open your herokuapp and drag your CSV files to a file uploader and run the update function.

[ismet_aksoy]

How about using a cloud database like mongodb or postgresql? If I update the database with CSVs and then run an update function in the streamlit app. Won’t that work?

[SaintDrei]

I think I saw some posts with similar use case before. You get the data from cloud database instead of csv. That would work.