Hitting API each time for data

Using Streamlit
1

Hi all,

My Problem statement:
I am using streamlit for a very large dataset >1,000,000 rows. Due to this, every filtering & change of view takes a lot of time to reload.

I have used caching but as the number combinations in filter are high, it does not improve the performance.

One solution that I was thinking was:

  • Hit my API which gives the filtered data and then display it. I feel like, that should be faster than doing the processing at client side.

Can anyone tell me if there is an option like this or is there any alternative that can be used for this?

Thanks a lot in advance!

2

Hi @vishwaovi, welcome to the Streamlit community!

Returning millions of results will always be a bad experience in terms of performance (as you’ve seen!). To the extent that you can get the API to return a smaller amount of pre-calculated/filtered results, you will have much better performance.

Can you share a repo so we can see what you’re trying to do?

Best,
Randy