Use cache_resource to cache database across multiple pages

[Matti_J_Frind]

Summary

I would like to build a multipage streamlit app that uses a database (weaviate with langchain). The db object should be shared across all users and all pages so I only use one db instance (this instance loads a retriever model from langchain so uses a bit of RAM).

My approach

I tried using @st.cache_resource and outsourcing it into another file. So the same function get’s called by all pages:

@st.cache_resource
def get_database() -> EmbeddingDatabase:
    return EmbeddingDatabase()

Unfortunately this doesn’t work. Session state is also not usable because it is replicated for each user. Has anyone another idea what the correct approach would be?

Debug info

• Streamlit version: 1.26.0
• Python version: 3.10.12
• Pipenv
• OS version: Linux

Requirements file

(can append if needed, but I think it is not neccesary)

Thanks for any suggestions!!!

[Goyo]

As far as I can tell the correct approach isthe one you are using and it does work for me.

[rwalde]

Hi, I also cannot get it working in a multi page app.
In the app.py I use:

## initialize connection via snowpark ######################
@st.cache_resource
def get_snowpark_connection():
    conn = st.experimental_connection('snowpark', type='snowpark', max_entries=20, ttl=600)
    return conn

and then:

conn = get_snowpark_connection()

But in all other pages in the pages folder,
conn = get_snowpark_connection() is not working,
because get_snowpark_connection() is not existing there.

Maybe I am doing it wrong.
In that case it would be great to get some snippets to do it right.

KR
Roland

[Goyo]

One straighforward way of having the connection available in all pages is storing it in session_state.

st.session_state.conn = get_snowpark_connection()

[rwalde]

that works fine… thx