Pickling and unpickling session state loses dataframes

[altanner]

I want my users to be able to save and load their session, and I have been trying to do this with pickle. But when the pickled file is re-loaded, the structure of the session state looks correct, but the dataframes are empty or null.

Session state in my case is fairly large, with more than 90 items, some of which are dataframes. These dataframes are not huge, a few kilobytes maybe.

I have been trying to replicate this with a trivial example, but of course it works fine when I do that!

Any ideas on what I could try? (Perhaps something other than pickle?) Using the build-in download button doesn’t work for session state, so it looks like it needs to be serialised in some way.

Thank you

[Santhusha_Janana_Mud]

If you are using Streamlit and encountering issues with pickling and loading session state, you can try the following approaches:

1. Use Streamlit’s session_state API: Streamlit provides a built-in mechanism called session_state that allows you to store and retrieve data across multiple user sessions. This eliminates the need for pickling and unpickling the entire session state. You can use it like a regular Python dictionary to store your session data.

   Here’s an example of how to use session_state:

   import streamlit as st
   
   # Set session state
   if 'session_state' not in st.session_state:
       st.session_state.session_state = {}
   
   # Store data in session state
   st.session_state.session_state['data'] = your_data
   
   # Retrieve data from session state
   your_data = st.session_state.session_state['data']
   

   By using session_state, you can save and load specific parts of your session state without the need for pickling and unpickling.

2. Serialize and save dataframes separately: If you still want to serialize and save the session state, you can consider serializing the dataframes separately from the rest of the session state. Instead of pickling the entire session state, pickle the individual dataframes and save them as separate files. When loading the session state, load the dataframes from their respective pickle files and reconstruct the session state as needed.

   This approach can help isolate any issues specific to pickling and unpickling dataframes and allow you to handle them separately.

3. Explore alternative serialization libraries: If pickle is not working as expected for your session state, you can try using alternative serialization libraries like joblib, dill, or cloudpickle. These libraries provide additional features and flexibility for serializing complex objects, including dataframes.

   You can install these libraries using pip and try them out as alternatives to pickle.

Remember to handle any exceptions that may occur during the serialization and deserialization process to help identify and resolve any issues with the dataframes or session state.