How best to apply a save on_change function to a dynamic filter?

Using Streamlit
, ,
2

Welcome to the Streamlit community, and thanks for sharing your code and a clear description of your challenge! :balloon: You’re right: preserving edits and selections across dynamic filters in Streamlit is tricky, especially with large data and when using st.data_editor and custom filters. The main issue is that when you filter, the edited DataFrame only contains the visible rows, so saving it directly will overwrite or “delete” filtered-out rows. Also, putting on_change on the data editor leads to the “double edit” problem due to Streamlit’s rerun model.

Best Practice:
To persist edits and selections across filters, you should:

  1. Store the full, unfiltered DataFrame in st.session_state.
  2. When the user edits the filtered view, update only the corresponding rows in the full DataFrame.
  3. Avoid using on_change on the data editor; instead, use a callback or button to commit changes, or update the session state immediately after editing.

Here’s a minimal, reproducible example that demonstrates this pattern:

import streamlit as st
import pandas as pd
import numpy as np
from streamlit_dynamic_filters import DynamicFilters

num_rows = 500
np.random.seed(42)
if 'df1' not in st.session_state:
    data = []
    for i in range(num_rows):
        data.append({
            "Preview": f"https://picsum.photos/400/200?lock={i}",
            "Views": np.random.randint(0, 1000),
            "Active": np.random.choice([True, False]),
            "Category": np.random.choice(["🤖 LLM", "📊 Data", "⚙️ Tool"]),
            "Progress": np.random.randint(1, 100),
        })
    st.session_state.df1 = pd.DataFrame(data)

df1 = st.session_state.df1

dynamic_filters = DynamicFilters(df1, filters=['Category', 'Active'])
with st.sidebar:
    dynamic_filters.display_filters()
    progress = st.slider("Progress", 1, 100, 100)

filtered_df = dynamic_filters.filter_df()
filtered_df = filtered_df[filtered_df['Progress'] < progress]

# Show editable table
edited_df = st.data_editor(filtered_df, hide_index=True, use_container_width=True, num_rows="dynamic", key="editor")

# Commit edits back to the full DataFrame
if st.button("Save changes"):
    for idx, row in edited_df.iterrows():
        # Find the corresponding index in the original df
        orig_idx = df1.index[df1['Preview'] == row['Preview']]
        if not orig_idx.empty:
            st.session_state.df1.loc[orig_idx, :] = row

st.write("Full DataFrame (persisted):")
st.write(st.session_state.df1.head())

Key Points:

  • Edits are only committed to the full DataFrame when the user clicks “Save changes”.
  • The original DataFrame in session state is never overwritten by a filtered subset.
  • This avoids the “double edit” bug and preserves edits across filter changes.

For more advanced UX, you could use a custom callback or experiment with Session State patterns to auto-save, but a button is the most robust and user-friendly approach for now.

Sources: