Paginate large dataframe ("server side") AND use dynamic widgets for dataframe filtering?

I have two functionalities, and I can do either individually, but I can’t seem to figure out how to do both at once:

1). “Dynamic” widgets for dataframe filtering
I have built a number of widgets and corresponding dataframe filters/masks, and sometimes it’s useful to the user to have widgets impact other widgets. For example, you might have a multi_select for day that’s pre-populated with all seven days, but above it might be a radio with [‘all’, ‘weekend’, ‘weekday’] which then alters the default options value in the multi_select. Allowing for dynamic filter widgets also allows you to warn the user immediately if they have something invalid (like if they remove all days from the multi_select example above), as opposed to having to actually submit the query and then return the error.

As far as I can tell, you can’t do this using the Streamlit “Form” capability, but I’ve emulated this by using my own button and check to see if the form has been “submitted”. This works just fine.

2). Any kind of “server side” Pandas dataframe pagination. I’ve implemented the example here (https://share.streamlit.io/streamlit/release-demos/0.84/0.84/streamlit_app.py?page=headliner), and even ported it to use a slider. So this works great too!

(I’ll note here that solutions like Agrid etc. haven’t worked for me because they just filter the dataframe on the client; they still send the whole thing at once. The files I’m dealing seem to be too large in the worst case to do this, so I’d like to be able to send chunks at a time to increase the user experience)

The problem is, I can’t figure out a way of getting both 1) and 2). to work together. I’ve tried a few things, but haven’t hit the magic combination of session_state, forms, onSubmit, etc. to get it to work.

This feels like a reasonably common use case…anyone experienced anything similar? It’s possible something like this is just antithetical to the Streamlit execution paradigm, but I hope not! It’d hate to have to sacrifice a huge chunk of functionality – being able to offer the user a helpful filter UI – for the simple task of breaking a large dataframe into chunks.

I’ve taken a look at Make st.dataframe more awesome · Issue #1449 · streamlit/streamlit · GitHub, which covers a lot of ground (mostly styling and interactivity, but also efficiency) but there hasn’t been a ton of progress lately. Does anyone have any advice in the meantime?

Thanks so much!