Summary
I’ve created a data_editor that displays data pulled from a dynamo db database. I’ve created a function to capture when a row is added in order to add this new item to the database. This function handles edits and deletions also which work really well with making live changes and deletions to the dynamo db with boto3 library.
The function for handling the changes to the dataframe via the data_editor fires in the “on_change” callback. I started testing figuring out the key creation before sending it to the database. During this testing process, I clicked on the add row button at the bottom of the data_editor and this error popped up:
The addition of the row was captured in the dictionary that captures those, but I can’t do anything else from here.
Steps to reproduce
Here is the function data_editor function that produces the data_editor
def display_table_data():
"""
For displaying the data in the current active table.
This uses Streamlit's data_editor() object.
# TODO - add "save edits" button to become active when changes are made in the data_editor
:return:
"""
# - store the active table object for ease of use in function
active_table = st.session_state['active_table']
# - now get the current page, which will be the "active DataFrame"
if 'current_page' in st.session_state:
current_page = st.session_state['current_page']
# - store the active DataFrame for the active table
st.session_state['active_dataframe'] = st.session_state['active_table_data_list'][active_table][current_page]
else:
current_page = ''
# - generate the data_editor table to display the data
# - store is in the 'edited_df' where any changes are stored as a full df
# - the individual edits made will be stored in st.session_state['work_dict']
st.session_state['edited_df'] = st.data_editor(
data=st.session_state['active_dataframe'],
key='work_dict',
hide_index=False,
num_rows='dynamic',
on_change=live_edits,
use_container_width=True
)
if current_page != '':
# create columns
col1, col2 = st.columns([4, 1])
with col1:
if current_page != 0:
st.button(label="Previous Page:", key="previous_page_button")
with col2:
if current_page != len(st.session_state['active_table_data_list'][active_table])-1:
st.button(label="Next Page:", key="next_page_button")
else:
if "last_key" in st.session_state:
st.button(label="Load Next Page:", key="load_next_page_button")
here is the page where the data_editor is called:
import streamlit as st
import openpyxl
import pandas as pd
import os
import datetime as dt
import re
import zipfile
from tools import client_management_tools as cmt
from tools import dbmanager as db
# ---- START Page Settings and resources
st.set_page_config(
layout="wide",
initial_sidebar_state="collapsed"
)
# - Establish database connection.
db.connect()
# - get table for this page.
if 'active_table' not in st.session_state:
db.get_active_table('client_management')
# - Now get the table data, so it is available to the script
if 'active_table_data_list' not in st.session_state:
db.get_table_data(paginate=True)
# ---- END Page Setting and resources
# ---- START User Interface
# - display new client form
with st.expander("Add New Client"):
with st.form(key="new_client_form"):
st.subheader("Add New Client")
st.text_input(
label="Enter Full Client Name:",
key="full_client_name",
placeholder="Enter Name with First Letter Capitalized"
)
st.date_input(
label="Enter Date of Onboarding:",
key="onboarding_date",
help="Select the date the client started working with BMOC."
)
st.text_input(
label="Enter Client Abbreviation or Short name:",
key="client_abbr",
placeholder="Enter as all caps, example: TUFTS, AU, BMOC"
)
new_client_added = st.form_submit_button(
label="Add Client"
)
if new_client_added:
new_data = {'fields': {
'client_name': st.session_state['full_client_name'],
'client_add_date': dt.datetime.strftime(st.session_state['onboarding_date'],
'%B %d, %Y'),
'client_abbreviation': st.session_state['client_abbr']
}}
db.add_item(add_key=True, item_data=new_data)
# - Display the data editor
db.display_table_data()
zip_file = cmt.show_package_export()
cmt.show_file_download()
# ---- END User Interface
# ---- START Activities of the page
db.get_next_page()
db.get_previous_page()
# ---- END Activities of the page
Here is the function for capturing the changes and sending them to the dynamodb in its draft testing form:
def live_edits():
"""
Function for capturing edits made using the streamlit data_editor widget
This function is set to fire when the dataframe, is edited in the data_editor widget itself.
idea is to get the edited/changed data from the data_editor, and then use the standard
add, edit, delete functions for dynamo db to make the changes.
not necessarily anything passed to this function as parameters due to capturing of changes
in the working dictionary when the data_editor widget of streamlit is in use.
Will need to use this to record changes. Have to figure out a separate change function
to capture changes made and store in a change_tracker table.
Returns
-------
"""
active_table = st.session_state['active_table']
changes = st.session_state['work_dict']
if len(changes['deleted_rows']) > 0:
for deleted in changes['deleted_rows']:
if 'active_table_data_list' in st.session_state:
# - set the current page back to the first of the paginated pages
current_page = st.session_state['current_page']
# - now to replace the existing DF in the 0 current page spot.
adj_df = st.session_state['active_table_data_list'][active_table][current_page]
elif 'active_dataframe' in st.session_state:
# - else, check for "active_dataframe" in st.session_state which is data_editors
adj_df = st.session_state['active_dataframe']
else:
pass
# - capture the item from its original dataframe to get keys.
item = adj_df.reset_index().iloc[deleted].to_dict()
# - Get the keys to pass to the delete function
if 'partition_key' in st.session_state['active_table_keys']:
partition_name = st.session_state['active_table_keys']['partition_key']
# - capture the partition key value from the DF
partition_value = item[partition_name]
else:
st.error('A partition Key and its value could not be captured.')
return
# - get the sort key and val if there is one
if 'sort_key' in st.session_state['active_table_keys']:
sort_key_name = st.session_state['active_table_keys']['sort_key']
# - capture the partition key value from the DF
sort_key_value = item[sort_key_name]
else:
sort_key_name = None
sort_key_value = None
delete_item(
partition_key=partition_name,
partition_val=partition_value,
sort_key=sort_key_name,
sort_key_val=sort_key_value
)
if len(changes['edited_rows']) > 0:
for index, changes_made in changes['edited_rows'].items():
if 'active_table_data_list' in st.session_state:
# - set the current page back to the first of the paginated pages
current_page = st.session_state['current_page']
# - now to replace the existing DF in the 0 current page spot.
adj_df = st.session_state['active_table_data_list'][active_table][current_page]
elif 'active_dataframe' in st.session_state:
# - else, check for "active_dataframe" in st.session_state which is data_editors
adj_df = st.session_state['active_dataframe']
else:
pass
# - capture the item from its original dataframe to get keys.
item = adj_df.reset_index().iloc[index].to_dict()
# - Get the keys to pass to the delete function
if 'partition_key' in st.session_state['active_table_keys']:
partition_name = st.session_state['active_table_keys']['partition_key']
# - capture the partition key value from the DF
partition_value = item[partition_name]
else:
st.error('A partition Key and its value could not be captured.')
return
# - get the sort key and val if there is one
if 'sort_key' in st.session_state['active_table_keys']:
sort_key_name = st.session_state['active_table_keys']['sort_key']
# - capture the partition key value from the DF
sort_key_value = item[sort_key_name]
else:
sort_key_name = None
sort_key_value = None
update_item(
partition_key=partition_name,
partition_val=partition_value,
sort_key=sort_key_name,
sort_key_val=sort_key_value,
changed_data=changes_made
)
if len(changes['added_rows']) > 0:
st.write(changes['added_rows'])
for index, added in enumerate(changes['added_rows']):
st.dataframe(st.session_state['edited_df'])
new_row = changes['added_rows'][index]
if len(new_row) > 0:
# - generate a new key to plug in to the new item
record_key = generate_random_key()
# - Get the name of the partition key for the active table
if 'partition_key' in st.session_state['active_table_keys']:
partition_name = st.session_state['active_table_keys']['partition_key']
else:
st.error('A partition Key and its value could not be captured.')
return
# - Get the name of the sort key if there is one for the current table
if 'sort_key' in st.session_state['active_table_keys']:
sort_key_name = st.session_state['active_table_keys']['sort_key']
else:
sort_key_name = None
# - if there is no sort key, then the partition key is a random key
if sort_key_name is None:
# - if no sort key, then the partition key gets the rando
new_row[partition_name] = record_key
else:
# - if there is a sort key, then it gets the rando
new_row[sort_key_name] = record_key
# - now capture the first
st.session_state['active_dataframe']
#
# item_find = tinydb.Query()
# the_table.upsert(
# new_row,
# item_find['key'] == record_key
# )
Expected behavior:
What I expect to happen at this point is just to start putting in entries into this new row.
Actual behavior:
It simply displays in each cell “Error during cell creation” and when I hover over it, it shows a pop up that says: “This should never happen. Please report this bug. Error: Error: Row index is out of range: 7”
i noted that there is no index provided in the index column of the new row.
Debug info
- Streamlit version: 1.27.0
- Python version: Python 3.11
- Using PyCharm environment with pycharm version 2023.2.1 (professional edition)
- OS version: macOS Catalina version 10.15.7
- Browser version: Chrome, Version 117.0.5938.92 (official building) (x86_64)
Requirements file
pandas>=1.5.1
openpyxl>=3.1.0
tinydb>=4.7.0
datetime>=4.0
streamlit>=1.21.0
pillow>=9.5.0
XlsxWriter>=3.0.8
boto3>=1.24.5
requests>=2.31.0
Links
unable to share
Additional information
none
