Slow website architecture

🎈 Using Streamlit
1

Im running streamlit deployed on a linux server (private so not able to share it)

I am also using st-pages to hide pages in the side bar. I am reading JSONs from a disk on the same server that streamlit page is running, and I am creating plotly graphs to be then displayed with st.plotly_chart().

So far this has served me well, but as I scaled, it has slowed down a bit. There is no long running/calculation intensive parts of the script, having profiled I see st.plotly_chart takes the most time. Would it be more useful to use parquet instead of JSON, or altogether store the JSON/Parquets in an SQL database?

Happy to investigate both options, but wanted to get an opinion of what would be more worth my time doing, and any other improvements to the architecture would be welcomed! This is far from ideal, I know.

Streamlit version: 1.23
Python Version: 3.9

2

Hi @marjak

There are several ways on improving the app speed as discussed further in the blogs listed below. You may want to go through these posts and see if they can help inspire some code refactoring. For example, if a DataFrame is large, you can selectively read only columns that you need using the usecols parameter of Pandas’ read_csv. Caching may also help to prevent repeated loading of data objects. Any pre-processing may be pre-calculated and exported in the processed form so that the app can readily use those for display.

There are some great blog posts and Docs page on the topic of optimizing the app’s performance:

Hope this helps!