Memory leak on linux but not on macos

[chchchchch]

I’ve noticed a big difference in memory usage when running streamlit on a Mac vs on Ubuntu or in a Docker container. This is on python 3.11 and streamlit 1.30.0.

Using a sample app copied from Memory behavior,

import streamlit as st
import time
import pandas as pd
import numpy as np

def expensive_computation(a, b):
    df = pd.DataFrame(np.random.randn(a, b))
    return df

def main():
    a = 10000
    b = 20
    res = expensive_computation(a, b)
    st.dataframe(res)

if __name__ == "__main__":
    main()

I ran streamlit on macos, Ubuntu 22.04.1 LTS, and a docker container running Debian GNU/Linux 11 (bullseye). I then ran a script that opened a new session every few seconds until 25 were open and then closed them all.

The memory usage on macos looks normal and memory is being released:

But on Linux memory is not being released:
Ubuntu 22.04.1 LTS

Docker container (Debian GNU/Linux 11 (bullseye))

Also tried adding in garbage collection with gc but that didn’t change the results.

Has anyone else run into this, or know what might be going on?

[dataprofessor]

Hi @chchchchch

That’s a great benchmark comparison to evaluate the memory leak on the different OS.

There’s an active discussion on this topic at: Reddit - Dive into anything

Also, there are some guidelines for optimizing the app in order to improve memory usage in app:

• 3 steps to fix app memory leaks
• Common app problems: Resource limits

[chchchchch]

Hi @dataprofessor ,

Just curious, have you previously deployed a streamlit app using Docker or K8s and not run into this issue?

[Bala_A]

Hi @dataprofessor We have deployed docker containerized app in K8s and we are facing the memory leak problem. We profiled the app as suggested in 3 steps to fix app memory leaks and we found the following streamlit objects from step2. Could you please advise how to proceed further to fix this?

{ “/opt/venv/lib/python3.9/site-packages/streamlit/components/v1/components.py:208”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/components/v1/components.py:218”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/cursor.py:121”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/cursor.py:49”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/delta_generator.py:262”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:754”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:763”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/button.py:764”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:307”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:316”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/checkbox.py:317”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:105”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:335”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:344”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/multiselect.py:345”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:442”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:450”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/number_input.py:451”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:311”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:320”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/radio.py:321”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:279”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:288”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/selectbox.py:289”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:158”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:654”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:663”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/slider.py:664”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:158”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:747”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:756”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:757”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/elements/widgets/time_widgets.py:91”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:157”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:354”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:355”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:356”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/app_session.py:393”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_data_api.py:641”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py:435”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py:436”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:175”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:203”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:211”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:216”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:223”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py:458”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:169”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:178”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/caching/hashing.py:214”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:291”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:388”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py:389”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/runtime.py:639”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_requests.py:187”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_requests.py:77”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:87”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:88”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:89”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:90”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:97”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_run_context.py:98”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:138”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:144”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:145”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:148”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:151”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:221”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:267”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:269”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:287”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:294”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:399”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:534”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:535”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py:672”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/common.py:211”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/query_params.py:75”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/safe_session_state.py:43”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:138”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:175”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:179”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:189”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:550”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:568”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:577”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/session_state.py:590”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/widgets.py:152”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/runtime/state/widgets.py:153”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/type_util.py:658”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/watcher/local_sources_watcher.py:118”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/web/server/browser_websocket_handler.py:103”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit/web/server/browser_websocket_handler.py:163”: 2,
“/opt/venv/lib/python3.9/site-packages/streamlit_local_storage/init.py:29”: 2 }

[dataprofessor]

Hi @Bala_A

Perhaps you can get some ideas on a fix from this relevant post:

BLOCKING BUG - memory and ephemeral storage leak issue in a Kubernetes deployment

@Ian_B To lower the memory usage, there are two quick things you could try: Use this wheel file which fixes a memory leak. This leak was fixed last week and will be released with 1.32. Deactivate the backend storage of forward messages in config.toml via (requires at least 1.30):[global] storeCachedForwardMessagesInMemory = false This is a bit of an old artifact that most likely doesn’t have any use. And its a bit of a problem if there is a spike of user sessions. These two aspects might he…

Also, you may try using the mentioned wheel to see if that helps with the memory leak issue.

And it is also mentioned that in version 1.32.0, the memory leak issue would be addressed.

Hope this helps!

[Bala_A]

Thank you @dataprofessor We tried the Streamlit Nightly version which has the mentioned bugfix. But it did not fix the issue. We ended up manually set all the variables to None after the usage. This actually stabilized the memory usage. So looks like the Streamlit built in garbage collector was not working and the manual gc.collect() was also not worked.

[lukasmasuch]

We ended up manually set all the variables to None after the usage.

@Bala_A Where these just normal script variables that you reset to None? Or did those variables where attached to any other global object (e.g. session state, cache, …)?

[Bala_A]

@lukasmasuch We set None for all the variables created inside the script including all the streamlit component level variables such as selectors, sliders, database query results, temp variables etc. But nothing on global level objects. The only change we did for Cache was to set ttl and max entries

[lukasmasuch]

The only change we did for Cache was to set ttl and max entries

This is most likely the reason why you don’t see the “memory leak” anymore. Cache entries will be kept forever with ttl or max_entries is not specified.

I assume that the setting of variables to None might not impact the memory consumption.

[_sharad]

@Bala_A Could you please provide more details on how you set the variables (especially the component level ones) to None? I did the same by making all variables None and used gc.collect() at the end of my script, but I’m still facing the same problem. I removed all caching logic as well from my code.