Summary
The following intends to subset a dataframe of survey data based on user-selected sample descriptives. Instead of returning the filtered df it returns a df that is empty except for the column names. The .csv file contained both strings and ints, so I convert it to a dataframe with only integers. The ints in the dictionaries correspond with the strings of the multiselect boxes. In an effort to make this functional, I have tried altering the filtering function(using query and logical expressions instead of isin), using select_boxes, and different iterations of the dictionaries, etc. When the I get the alternate methods to be functional, the filtered dataframe still comes up empty. I really appreciate any help! Thank you.
Steps to reproduce
Code snippet:
# Loading in the data and selecting columns to visualize (from the data)
st.set_page_config(layout="wide")
prog_df = pd.read_csv('...')
columns_to_use = ["Age_new","Gender_binary2","Educ_new","Q56_ideology","Q58_Polit_Chg","NE_Index", "NewsUse1_tv_re",
"NewsUse2_radio_re","NewsUse3_print_re","NewsUse4_pod_re","NewsUse5_soc_re",
"NewsUse6_sites_re","NewsUse7_oth_re","Q31_use1_tv","Q31_use2_radio","Q31_use3_print","Q31_use4_pod",
"Q31_use5_soc","Q31_use6_sites","Q31_use7_other",
"Q46","Q47_etrust_g1_local","Q47_etrust_g2_state","Q47_etrust_g3_natl",
"Q48_etrust4_vote","Q49_etrust5_workers","Q50_etrust6_mail", "Q51_itrust1_whouse",
"Q51_itrust2_cong", "Q51_itrust3_court", "Q51_itrust4_mil", "Q51_itrust5_justice", "Q52_itrust6_school",
"Q52_itrust7_corp",
"Q59_1", "Q60",#party registered with
"Q61_1",#economy
"Q61_2", #Health care
"Q61_3", #Education
"Q61_4", #Environment/climate change
"Q61_5", #Gun Control/ Rights
"Q62_1", #Abortion
"Q62_2", #homelessness
"Q62_3", #LGBTQIA+ Rights
"Q62_4", #immigration
"Q62_5", #voting Rights
"C_Theo_Index",
"Q69_ct1_fraud","Q70_ct2_covid","Q71_ct3_ufo","Q72_ct4_911","Q73_ct5_plant",
"Q75","Q79"]
# Processing the data in an easier way
df_database_1 = prog_df.reindex(columns=columns_to_use)
df_database = df_database_1.apply(pd.to_numeric, downcast='integer', errors='coerce')
####################
### INTRODUCTION ###
####################
# Formatting options
row0_spacer1, row0_1, row0_spacer2, row0_2, row0_spacer3 = st.columns((.1, 2.3, .1, 1.3, .1))
with row0_1:
st.title('The Visualizer')
st.subheader("")
with row0_2:
st.title(" ")
st.title(" ")
st.subheader('')
row3_spacer1, row3_1, row3_spacer2 = st.columns((.1, 3.2, .1))
with row3_1:
st.markdown("")
#################
### SELECTION ###
#################
st.sidebar.text('')
st.sidebar.text('')
st.sidebar.markdown("**Please select the attributes of participants you would like to include in the analysis. You can clear the current selection by clicking the corresponding x-button on the right. Press submit when finished:**")
############################
##### SELECTION BOXES #####
############################
education_levels = ["Less than high school","High school graduate","Some college","4-year degree","Graduate degree"]
ages = ["18-25", "26-30", "31-35", "36-40", "41-45", "46-50", "51-55", "56-60", "61-65", "66 or older", "Prefer not to say"]
political_party = ["Democratic Party","Republican Party","Libertarian Party", "Green Party","Constitution Party","Other(Write in", "Not registered with any party"]
household_income = ["Less than $10,000","$10,000 - $19,999","$20,000 - $29,999","$30,000 - $39,999","$40,000 - $49,999","$50,000 - $59,999","$60,000 - $69,999","$70,000 - $79,999","$80,000 - $89,999","$90,000 - $99,999","$100,000 - $149,999", "More than $150,000","Prefer not to say"]
pres_choice_2020 = ["Joe Biden (Democratic Party)","Donald Trump (Republican Party)","Other", "Not sure"]
poli_view = ["Extremely liberal (1)","(2)","(3)","(4)","Extremely conservative(5)"]
view_change = ["Not at all","A little","A lot","A great deal"]
view_change_direction = ["Far more liberal","Moderately more liberal", "Moderately more conservative", "Far more conservative"]
edu_dict = {"Less than high school": 1,
"High school graduate": 2,
"Some college": 3,
"4-year degree": 4,
"Graduate degree": 5}
ages_dict = {"18-25":1, "26-30":2, "31-35":3, "36-40":4, "41-45":5,
"46-50":6, "51-55":7, "56-60":"8", "61-65":9,
"66 or older":10, "Prefer not to say":11}
political_party_dict = {"Democratic Party":1,"Republican Party":2,
"Libertarian Party":3, "Green Party":4,
"Constitution Party":5,
"Other(Write in":6, "Not registered with any party":7}
household_income_dict = {"Less than $10,000":1,"$10,000 - $19,999":2,"$20,000 - $29,999":3,
"$30,000 - $39,999":4,"$40,000 - $49,999":5,
"$50,000 - $59,999":6,"$60,000 - $69,999":7,
"$70,000 - $79,999":8,"$80,000 - $89,999":9,
"$90,000 - $99,999":10,"$100,000 - $149,999":11,
"More than $150,000":12,"Prefer not to say":13}
pres_choice_2020_dict = {"Joe Biden (Democratic Party)":1,
"Donald Trump (Republican Party)":2,
"Other":3, "Not sure":4}
poli_view_dict = {"Extremely liberal (1)":1,"(2)":2,
"(3)":3,"(4)":4,"Extremely conservative(5)":5}
view_change_dict = {"Not at all":1,"A little":2,
"A lot":3,"A great deal":4}
view_change_direction_dict = {"Far more liberal":1,
"Moderately more liberal":2,
"Moderately more conservative":3,
"Far more conservative":4}
with st.sidebar.form(key='my_form'):
selected_education_levels = st.sidebar.multiselect("Education Level:", education_levels, default= education_levels)
selected_ages = st.sidebar.multiselect("Age Ranges:", ages, default=ages)
selected_political_party = st.sidebar.multiselect("Registered political party:", political_party, default=political_party)
selected_household_income = st.sidebar.multiselect("Household Income before taxes:", household_income, default=household_income)
selected_pres_choice_2020 = st.sidebar.multiselect("2020 Presidential candidate choice:", pres_choice_2020, default=pres_choice_2020)
selected_poli_view = st.sidebar.multiselect("Q56 asked: If you had to place yourself somewhere on this scale from 1 (Extremely Liberal) on the left to 5 (Extremely Conservative) on the right, where would it be?", poli_view, default=poli_view)
selected_view_change = st.sidebar.multiselect("Q58 asked: How much have your political views changed, if at all, in the past 10 years or so?:", view_change, default=view_change)
selected_view_change_dir = st.sidebar.multiselect("To the participants who reported political view change, Q59 asked, Thinking about how your views have changed, have you become more liberal or more conservative? ", view_change_direction, default=view_change_direction)
submit_button = st.form_submit_button(label='Submit')
# # initalize the Session State Variables
if 'selected_education_levels' not in st.session_state:
st.session_state.selected_education_levels = []
if 'selected_ages' not in st.session_state:
st.session_state.selected_ages = []
if 'selected_political_party' not in st.session_state:
st.session_state.selected_political_party = []
if 'selected_household_income' not in st.session_state:
st.session_state.selected_household_income = []
if 'selected_pres_choice_2020' not in st.session_state:
st.session_state.selected_pres_choice_2020 = []
if 'selected_poli_view' not in st.session_state:
st.session_state.selected_poli_view = []
if 'selected_view_change' not in st.session_state:
st.session_state.selected_view_change = []
if 'selected_view_change_dir' not in st.session_state:
st.session_state.selected_view_change_dir = []
# Define a function that filters the data based on user input
def filter_data(data, selected_edu, selected_ages, selected_party,
selected_inc, selected_pres, selected_view,
selected_change, selected_dir):
# Convert keys back to original values or representations
converted_edu_list = [level for level, key in edu_dict.items() if level in selected_edu]
converted_ages_list = [age for age, key in ages_dict.items() if age in selected_ages]
converted_a_political_party_list = [party for party, key in political_party_dict.items() if party in selected_party]
converted_household_income_list = [income for income, key in household_income_dict.items() if income in selected_inc]
converted_pres_choice_2020_list = [choice for choice, key in pres_choice_2020_dict.items() if choice in selected_pres]
converted_poli_view_list = [response for response, key in poli_view_dict.items() if response in selected_view]
converted_view_change_list = [response for response, key in view_change_dict.items() if response in selected_change]
converted_view_change_dir_list = [response for response, key in view_change_direction_dict.items() if response in selected_dir]
#edu_values = [edu_dict[key] for key in edu]
#ages_values = [ages_dict[key] for key in ages]
#party_values = [political_party_dict[key] for key in party]
#inc_values = [household_income_dict[key] for key in inc]
# pres_values = [pres_choice_2020_dict[key] for key in pres]
# view_values = [poli_view_dict[key] for key in view]
#change_values = [view_change_dict[key] for key in change]
# dir_values = [view_change_direction_dict[key] for key in dir]
df_select = data.loc[(data['Educ_new'].isin(converted_edu_list))
& (data['Age_new'].isin(converted_ages_list)) &
(data['Q46'].isin(converted_a_political_party_list)) &
(data['Q49_etrust5_workers'].isin(converted_household_income_list)) &
(data['Q48_etrust4_vote'].isin(converted_pres_choice_2020_list)) &
(data['Q56_ideology'].isin(converted_poli_view_list)) &
(data['Q58_Polit_Chg'].isin(converted_view_change_list )) &
(data['Q59_1'].isin(converted_view_change_dir_list))]
return df_select
# Update session state when submit button is pressed
if submit_button:
st.session_state.selected_education_levels = selected_education_levels
st.session_state.selected_ages = selected_ages
st.session_state.selected_political_party = selected_political_party
st.session_state.selected_household_income = selected_household_income
st.session_state.selected_pres_choice_2020 = selected_pres_choice_2020
st.session_state.selected_poli_view = selected_poli_view
st.session_state.selected_view_change = selected_view_change
st.session_state.selected_view_change_dir = selected_view_change_dir
# Convert the selected values to their corresponding dictionary keys
df_select = filter_data(df_database, selected_education_levels, selected_ages,
selected_political_party,selected_household_income,
selected_pres_choice_2020,selected_poli_view,
selected_view_change,selected_view_change_dir)
#Display filtered dataframe
st.dataframe(data=df_select.reset_index(drop=True))
##########################
#### SEE ORIGINAL DATA ###
##########################
row3_spacer1, row3_1, row3_spacer2 = st.columns((.2, 7.1, .2))
with row3_1:
st.markdown("")
see_data = st.expander('You can click here to see the raw data first 👉')
with see_data:
st.dataframe(data=df_database.reset_index(drop=True))
To reproduce the behavior of the empty filtered data frame, one clicks the submit button in the sidebar. I have intermediately checked to see if the multiselect boxes work by adding print(selected_ages) etc under if submit.
Expected behavior:
A filtered data frame should be created which I later pass to bar charts.
Actual behavior:
Here I have hit the submit button, and the empty data frame shown at the top here appears.
Debug info
- Streamlit version: 1.22.0
- Python version: 2.7.16
- OS version: macOS Mojave 10.14.6
- Browser version: Google chrome Version 113.0.5672.92 (Official Build) (x86_64)
Requirements file
Requirement already satisfied: altair==4.2.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (4.2.2)
Requirement already satisfied: attrs==23.1.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (23.1.0)
Requirement already satisfied: blinker==1.6.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (1.6.2)
Requirement already satisfied: cachetools==5.3.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (5.3.0)
Requirement already satisfied: certifi==2023.5.7 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (2023.5.7)
Requirement already satisfied: charset-normalizer==3.1.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (3.1.0)
Requirement already satisfied: click==8.1.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (8.1.3)
Requirement already satisfied: contourpy==1.0.7 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (1.0.7)
Requirement already satisfied: cycler==0.11.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.11.0)
Requirement already satisfied: debugpy==1.6.7 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 10)) (1.6.7)
Requirement already satisfied: decorator==5.1.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 11)) (5.1.1)
Requirement already satisfied: entrypoints==0.4 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 12)) (0.4)
Requirement already satisfied: fonttools==4.39.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 13)) (4.39.3)
Requirement already satisfied: gitdb==4.0.10 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 14)) (4.0.10)
Requirement already satisfied: GitPython==3.1.31 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 15)) (3.1.31)
Requirement already satisfied: idna==3.4 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 16)) (3.4)
Requirement already satisfied: importlib-metadata==6.6.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 17)) (6.6.0)
Requirement already satisfied: Jinja2==3.1.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 18)) (3.1.2)
Requirement already satisfied: jsonschema==4.17.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 19)) (4.17.3)
Requirement already satisfied: kiwisolver==1.4.4 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 20)) (1.4.4)
Requirement already satisfied: markdown-it-py==2.2.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 21)) (2.2.0)
Requirement already satisfied: MarkupSafe==2.1.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 22)) (2.1.2)
Requirement already satisfied: matplotlib==3.7.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 23)) (3.7.1)
Requirement already satisfied: mdurl==0.1.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 24)) (0.1.2)
Requirement already satisfied: numpy==1.24.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 25)) (1.24.3)
Requirement already satisfied: packaging==23.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 26)) (23.1)
Requirement already satisfied: pandas==2.0.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 27)) (2.0.1)
Requirement already satisfied: Pillow==9.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 28)) (9.5.0)
Requirement already satisfied: protobuf==3.20.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 29)) (3.20.3)
Requirement already satisfied: ptvsd==4.3.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 30)) (4.3.2)
Requirement already satisfied: pyarrow==12.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 31)) (12.0.0)
Requirement already satisfied: pydeck==0.8.1b0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 32)) (0.8.1b0)
Requirement already satisfied: Pygments==2.15.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 33)) (2.15.1)
Requirement already satisfied: Pympler==1.0.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 34)) (1.0.1)
Requirement already satisfied: pyparsing==3.0.9 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 35)) (3.0.9)
Requirement already satisfied: pyrsistent==0.19.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 36)) (0.19.3)
Requirement already satisfied: python-dateutil==2.8.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 37)) (2.8.2)
Requirement already satisfied: pytz==2023.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 38)) (2023.3)
Requirement already satisfied: pytz-deprecation-shim==0.1.0.post0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 39)) (0.1.0.post0)
Requirement already satisfied: requests==2.30.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 40)) (2.30.0)
Requirement already satisfied: rich==13.3.5 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 41)) (13.3.5)
Requirement already satisfied: seaborn==0.12.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 42)) (0.12.2)
Requirement already satisfied: six==1.16.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 43)) (1.16.0)
Requirement already satisfied: smmap==5.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 44)) (5.0.0)
Requirement already satisfied: streamlit==1.22.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 45)) (1.22.0)
Requirement already satisfied: tenacity==8.2.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 46)) (8.2.2)
Requirement already satisfied: toml==0.10.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 47)) (0.10.2)
Requirement already satisfied: toolz==0.12.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 48)) (0.12.0)
Requirement already satisfied: tornado==6.3.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 49)) (6.3.1)
Requirement already satisfied: typing_extensions==4.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 50)) (4.5.0)
Requirement already satisfied: tzdata==2023.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 51)) (2023.3)
Requirement already satisfied: tzlocal==4.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 52)) (4.3)
Requirement already satisfied: urllib3==2.0.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 53)) (2.0.2)
Requirement already satisfied: validators==0.20.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 54)) (0.20.0)
Requirement already satisfied: zipp==3.15.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from -r requirements.txt (line 55)) (3.15.0)