I am trying to concatenate pandas dataframes that starts with specific name but getting errors

Using Streamlit
, , ,
1

I am trying to concatenate all the dataframes which are starting with user_ string in the streamlit but have been getting error. Even though the Logic/code works when I do it outside streamlit so may be its a session_state issue.

import streamlit as st
import pandas as pd

st.set_page_config(page_title="Science",
                    layout='wide',
                    initial_sidebar_state="expanded")


# sample dataframes
st.session_state.user_df1 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
st.session_state.user_df2 = pd.DataFrame({'A': [5, 6], 'B': [7, 8]})
st.session_state.df3 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})


# list of dataframes starting with user_
st.session_state.users_list = [name for name in st.session_state if name.startswith('user_')]

st.write(st.session_state.users_list)

# using eval to evaluate string dataframe names
# st.write([eval(st.session_state.name) for st.session_state.name in st.session_state.users_list])

st.session_state.df_final = pd.concat([eval(st.session_state.name) for st.session_state.name 
                                       in st.session_state.users_list], 
                                       ignore_index=True)

st.table(st.session_state.df_final)

This whole logic works without streamlit but I am getting an error in this in streamlit , not sure what’s wrong. Appreciate any help !!

Error:

NameError: name 'user_df2' is not defined

File "C:\Users\xxx\anaconda3\envs\llma_py_3_12\Lib\site-packages\streamlit\runtime\scriptrunner\exec_code.py", line 88, in exec_func_with_error_handling
    result = func()
             ^^^^^^
File "C:\Users\xxx\anaconda3\envs\llma_py_3_12\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 590, in code_to_exec
    exec(code, module.__dict__)
File "V:\1. R & Python work\Python\xxx\Numerology\streamlit_editing_df.py", line 43, in <module>
    st.write([eval(st.session_state.name) for st.session_state.name in st.session_state.users_list])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1, in <module>

System/Version info

Python 3.12.4
streamlit 1.38.0
pandas 2.2.2

This logic/code without streamlit that works:

import pandas as pd

user_df1 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
user_df2 = pd.DataFrame({'A': [5, 6], 'B': [7, 8]})

df3 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})


users_list = ['user_df1','user_df2']

print([eval(name) for name in users_list])

df_final = pd.concat([eval(name) for name in users_list],ignore_index=True)

print('----df_final------')
print(df_final)

I have also posted this on Stackoverflor

2

Hi John!

Looks like you need to explicitly construct the session var as a string for the eval function to work properly:

st.session_state.df_final = pd.concat([eval(f"st.session_state.{name}") for name 
                                       in st.session_state.users_list], 
                                       ignore_index=True)

and here is the full functioning code snippet:



import streamlit as st
import pandas as pd

st.set_page_config(page_title="Science",
                    layout='wide',
                    initial_sidebar_state="expanded")

# sample dataframes
st.session_state.user_df1 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
st.session_state.user_df2 = pd.DataFrame({'A': [5, 6], 'B': [7, 8]})
st.session_state.df3 = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})

# list of dataframes starting with user_
st.session_state.users_list = [name for name in st.session_state if name.startswith('user_')]
st.write('users list')
st.write(st.session_state.users_list)

# using eval to evaluate string dataframe names
# st.write([eval(st.session_state.name) for st.session_state.name in st.session_state.users_list])
st.session_state.df_final = pd.concat([eval(f"st.session_state.{name}") for name 
                                       in st.session_state.users_list], 
                                       ignore_index=True)

st.write('df_final')
st.table(st.session_state.df_final)

Let me know if this is what you were looking for.

3

This works really well and this is all what I needed. Thanks @cgage1 for the code and the explanation.

1 Like
4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.