Memory used by session state never released?

[Eric_Mitchell]

I have an application that loads several PyTorch models and serves them to the user. I use st.session_state to store models/data across page refreshes. However, after the user closes the tab, if they open the URL of the application again, they get a new session, but the memory used by the old session is not freed! Is there something special I need to do to avoid this continual memory growth? Currently, the streamlit app crashes from OOM continually after ~10 users connect, because the memory from each created session is never released, even after the page is closed.

[snehankekre]

Hi @Eric_Mitchell, welcome to the community!

Session state isn’t intended to cache PyTorch models and datasets. Streamlit has two cache primitives that are precisely for caching singleton objects like PyTorch models (st.experimental_singleton) and data objects (st.experimental_memo):

Experimental cache primitives - Streamlit Docs

Both cache primitives have .clear() commands to clear in-memory and on-disk caches:

• st.experimental_singleton.clear()
• st.experimental_memo.clear()

Best,
Snehan

[Eric_Mitchell]

Snehan,

Thanks a lot for the reply- this looks really useful!

I do still wonder why the session state isn’t released when the user ends their session. Even if we’re not storing huge objects like models in the session state, the memory consumption of the app will continue to grow over time if the session state isn’t released, right?

Eric

[Shawn_Pereira]

Hi @Eric_Mitchell , you could probably try:

del xyz

where xyz = your_session_state_variable_name

Cheers

[snehankekre]

@Eric_Mitchell Good question! I will consult with our Eng team and get back to you.

[snehankekre]

Got more clarity from the team: “each session’s SessionState instance is tied the the app session instance, so it should be released (eventually! because garbage collection is non-deterministic) when the session ends. If not, it’s a bug and we recommended creating a GitHub issue for it.”