Memory leak on linux but not on macos

I’ve noticed a big difference in memory usage when running streamlit on a Mac vs on Ubuntu or in a Docker container. This is on python 3.11 and streamlit 1.30.0.

Using a sample app copied from Memory behavior,

import streamlit as st
import time
import pandas as pd
import numpy as np

def expensive_computation(a, b):
    df = pd.DataFrame(np.random.randn(a, b))
    return df

def main():
    a = 10000
    b = 20
    res = expensive_computation(a, b)
    st.dataframe(res)

if __name__ == "__main__":
    main()

I ran streamlit on macos, Ubuntu 22.04.1 LTS, and a docker container running Debian GNU/Linux 11 (bullseye). I then ran a script that opened a new session every few seconds until 25 were open and then closed them all.

The memory usage on macos looks normal and memory is being released:
image (2)

But on Linux memory is not being released:
Ubuntu 22.04.1 LTS

Docker container (Debian GNU/Linux 11 (bullseye))
image (1)

Also tried adding in garbage collection with gc but that didn’t change the results.

Has anyone else run into this, or know what might be going on?

Hi @chchchchch

That’s a great benchmark comparison to evaluate the memory leak on the different OS.

There’s an active discussion on this topic at: Reddit - Dive into anything

Also, there are some guidelines for optimizing the app in order to improve memory usage in app:

Hi @dataprofessor ,

Just curious, have you previously deployed a streamlit app using Docker or K8s and not run into this issue?

Hi @dataprofessor We have deployed docker containerized app in K8s and we are facing the memory leak problem. We profiled the app as suggested in 3 steps to fix app memory leaks and we found the following streamlit objects from step2. Could you please advise how to proceed further to fix this?

{ β€œ/opt/venv/lib/python3.9/site-packages/streamlit/components/v1/components.py:208”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/components/v1/components.py:218”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/cursor.py:121”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/cursor.py:49”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/delta_generator.py:262”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:754”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:763”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:764”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:307”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:316”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:317”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:105”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:335”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:344”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:345”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:442”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:450”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:451”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:311”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:320”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:321”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:279”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:288”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:289”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:158”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:654”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:663”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:664”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:158”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:747”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:756”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:757”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:91”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:157”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:354”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:355”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:356”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:393”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_data_api.py:641”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py:435”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py:436”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:175”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:203”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:211”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:216”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:223”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:458”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:169”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:178”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:214”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:291”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:388”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:389”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/runtime.py:639”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_requests.py:187”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_requests.py:77”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:87”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:88”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:89”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:90”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:97”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:98”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:138”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:144”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:145”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:148”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:151”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:221”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:267”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:269”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:287”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:294”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:399”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:534”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:535”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:672”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/common.py:211”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/query_params.py:75”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/safe_session_state.py:43”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:138”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:175”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:179”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:189”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:550”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:568”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:577”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:590”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/widgets.py:152”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/widgets.py:153”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/type_util.py:658”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/watcher/local_sources_watcher.py:118”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/web/server/browser_websocket_handler.py:103”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit/web/server/browser_websocket_handler.py:163”: 2,
β€œ/opt/venv/lib/python3.9/site-packages/streamlit_local_storage/init.py:29”: 2 }

Hi @Bala_A

Perhaps you can get some ideas on a fix from this relevant post:

Also, you may try using the mentioned wheel to see if that helps with the memory leak issue.

And it is also mentioned that in version 1.32.0, the memory leak issue would be addressed.

Hope this helps!

1 Like

Thank you @dataprofessor We tried the Streamlit Nightly version which has the mentioned bugfix. But it did not fix the issue. We ended up manually set all the variables to None after the usage. This actually stabilized the memory usage. So looks like the Streamlit built in garbage collector was not working and the manual gc.collect() was also not worked.