Loading a large Graph from networkx

Using Streamlit
1

Hi all,

I have a large (~200MB) graph (from networkx) stored in a pickle file which I would like to load and then use to build and visualize some subgraphs based on certain variables specified on the sidebar.

On my laptop, the graph is read in 30 seconds and takes up about 2GB of RAM. However, when I try to load it with streamlit the process never comes to an end.

This is the code I am using:

import streamlit as st
import pickle

@st.cache
def build_graph(fname):
    with open(fname, 'rb') as f:
        G = pickle.load(f)
    return G

st.title('Network analysis')
st.sidebar.title('Set Network parameters')
option1=st.sidebar.number_input(...)
option2=st.sidebar.slider(...)
option3=st.sidebar.checkbox(...)

graph_state = st.text('Building initial graph...')
G = build_graph(fname)
graph_state.text(f'Building initial graph...done!')

Basically, it never prints the last statement.

Any help is appreciated.

2

Probably the issue is because Streamlit is trying to compute the hash of your 2GB file
(because of st.cache). Read more about streamlit caching here.

Can you edit it as st.cache(allow_output_mutation=True)
and see if that helps?

3

Thanks! st.cache(allow_output_mutation=True) solved the problem.