We want to deprecate st.cache! …and need your input ❤️

[jrieke]

Hey all!

Last year, we released two new experimental caching features: st.experimental_memo and st.experimental_singleton (see our docs). Now, we want to de-experimentalize these features and deprecate the old st.cache.

We need your input for that!

• Do you like st.experimental_memo and st.experimental_singleton?
• What do you dislike?
• Are there any killer features in st.cache that your apps can’t live without?
• Do you understand when to use memo and when to use singleton or is this confusing?

Please let me know in the comments! EVERY feedback is helpful for this big step. And if you haven’t tried out memo and singleton yet, now is a good time

-Johannes

[Wally]

Hi Johannes,
I have been using st.experimental_memo and st.experimental_singleton in my recent app thus switching over from st.cache. I like both features and actually prefer them over st.cache because I think using underscores instead of hash_func to exclude unhashable parameters is easier. I also like the distinction between shared singletons and other shared objects.
Performance-wise I cannot contribute much.

As per your questions:

• Do you like st.experimental_memo and st.experimental_singleton? => Yes
• What do you dislike? => While I think st.singleton as a great choice as name, I am not sure about st.memo. st.cache is also much clearer to me.
  If you plan on making these features non-experimental, I think it would be great if the docs had more examples and best practices on how-to implement common scenarios (just like you already did for sharing DB connections using st.singleton for example)
• Are there any killer features in st.cache that your apps can’t live without? => No
• Do you understand when to use memo and when to use singleton or is this confusing? => Yes, absolutely. And I believe implementing a singleton-feature as part of streamlits core functionality is great.

[marduk]

Hey there @jrieke ,

That is great news!

I believe it would help to have more/updated documentation, examples and tutorials ready to go along with the de-experimentalized singleton and memo features. In my case, it did take me a while to understand the differences between the current caching features (st.cache, st.experimental_singleton, st.experimental_memo).

[fredzannarbor]

Honestly, I’m not happy about this, as I have blindly attached st.cache to hundreds of functions over dozens of pages and will now have to work to understand what I was actually trying to accomplish/actually doing, then decide for each and every function which new function it should get.

Can I ask, why is it necessary/beneficial to deprecate something that seemed to be working reasonably well? It seemed to save a lot of reloads even if I didn’t 100% understand what it was doing…

Also, does “deprecate” mean “eventually break”, or just “remove from the documentation”? If the latter, that would be fine. If the former, I don’t see why you would force virtually your entire installed base to modify their code. St.cache is one of the first things I tried out and I am sure most people have done the same.

[dave51c]

Hi. New to Streamlit so the eventual loss of st.cache has no impact for me. Reading the docs, these two new features seem more flexible. I will make use of a singleton for passing around a db connection. Caching other object via memo is going to be essential so happy to see performance increase.

Thanks!

[gagangoku]

I’m new to Streamlit, but my biggest pain with st.cache and st.experimental_memo has been trying to resolve the errors that come when Streamlit isn’t able to hash or pickle something.

st.experimental_memo

@st.experimental_memo(ttl=300)
def initMysqlConnection():
    return mysql.connector.connect(**st.secrets["mysql"])

throws:

TypeError: cannot pickle '_mysql_connector.MySQL' object
Traceback:
File "/Users/gagandeep/work/wfhtoday-st/venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 556, in _run_script
    exec(code, module.__dict__)
File "/Users/gagandeep/work/wfhtoday-st/Home.py", line 174, in <module>
    main()
File "/Users/gagandeep/work/wfhtoday-st/Home.py", line 126, in main
    _conn = initMysqlConnection()

st.cache

Sometimes it’s independent of the parameters being passed to the function, like this:

@st.cache
def initMysqlConnection():
    return mysql.connector.connect(**st.secrets["mysql"])

UnhashableTypeError: Cannot hash object of type _thread.RLock, found in the body of initMysqlConnection().

While caching the body of initMysqlConnection(), Streamlit encountered an object of type _thread.RLock, which it does not know how to hash.

st.experimental_singleton

st.experimental_singleton works for caching mysql connections, but I observed some connection errors in long running apps, and had to clear the cache and refresh. I’m generally wary of singletons in principle

It would be nice if you could come up with an annotation to cache an object as is. Ignoring the function arguments starting with _ should be good enough.

[Mostapha]

@fredzannarbor

It seemed to save a lot of reloads even if I didn’t 100% understand what it was doing…

I think this is a great point. We are using Streamlit as a platform for developing apps for our users. Most of them are not professional programmers but they know how to write a script. The good thing about st.cache is that people can start using it without fully understanding the details. As they get better and more confident then they can start optimizing their solution using memo or singleton. st.cache is not perfect but it gets the job done for a large number of our users. I vote to keep it as an option but clarify the limitations and suggest using the more optimized solutions in the documentation.

[asehmi]

Hi @jrieke,

The separation of semantics make sense. Much more detailed documentation with use cases is key to understand the differences between them. The use cases should explain what combo of arguments is appropriate and why. Please provide a command line migration/upgrade tool which can output a verbose report without making changes, and can also be run making changes in one sweep, or interactively.

Feature requests:

• Add ttl to st.experimental_singleton
• Singleton and memo aren’t really proper caches, are they? Currently it’s not possible to have write-through caching, i.e. where the cache can be updated without first busting (clearing) it. There isn’t the concept of a cache miss which may cause the cache to be dynamically updated. Hence, it’s not that straight-forward to implement eviction strategies like LRU/MFU caching. How could st.experimental_memo be made to support this?

Thanks,
Arvindra

[Seb]

Hi Johannes
I’ve been building streamlit dashboards for years now using classes.
I am taking advantage of the power of classes, derivating many customer dashboards, all based on the same template.

It seems like using @st.experimental_memo on a class method does not work. I get that kind of exception:

streamlit.runtime.caching.cache_errors.UnhashableParamError: Cannot hash argument 'self' (of type `__main__.Dashboard`) in '_load_data'.

Minimal example to reproduce the issue :

import streamlit as st

class Dashboard:
    def run_web_app(self):
        data = self._load_data()
        st.write(data)

    @st.experimental_memo
    def _load_data(self):
        print("Loading data from data source")
        return "my data"


if __name__ == "__main__":
    d = Dashboard()
    d.run_web_app()

The very same code works fine using @st.cache instead.
If you know how to fix that in my example, I’d be very happy to upgrade my version of streamlit and use the latest caching methods.

[knorthover]

I’ve just checked that my issue (#4063) with the two experimental decorators is resolved in v1.12 (possibly earlier). I can now live with the change; but it will be a chore to modify a number of applications as another responder said. st.cache has worked well for me.
Kevin

[asehmi]

@Seb

It seems like using @st.experimental_memo on a class method does not work. I get that kind of exception:

If you reference the instance parameter as _self it’ll work.

[jrieke]

@fredzannarbor @knorthover Just to be clear, we won’t be removing st.cache right away. We’ll show a deprecation warning in the logs if you keep using it in new Streamlit versions. But we’ll only remove it in a future major version of Streamlit (i.e. 2.0 or 3.0 – which we are not even thinking about yet). We absolutely want to make sure we don’t break old apps and this transition is not a pain in the a** for you

[fredzannarbor]

Well, I take deprecation warnings as death sentences! I have to because, like many of us, my natural inclination is to hear the warning and then procrastinate for 36 months until the day after the new version breaks my application.

[fredzannarbor]

Incidentally, I still haven’t seen a good explanation of why this is so beneficial as to be worth the costs. I know st.cache has various problems right now but does your telemetry show that it is causing massive numbers of critical failures? Or will the new functions somehow make life so much easier for professional programmers that they will flock to it even more than they already are.

The big companies are very good about using telemetry to prioritize bug fixes. If something about st.cache is harmfully affecting a large percentage of my fellow users, I would be more supportive of deprecating it.

[Seb]

Thanks a lot @asehmi for your quick help, very appreciated ! If I understand well, this trick (that was clearly stated in the documentation, my bad) would cache the result of my computation (that must be hashable), and not my class object (that is really not hashable). My first tests work fine, this is great, thanks again !

[asehmi]

Glad it worked, and yes, any parameter prefixed with “_” will be ignored by these decorators for hashing purposes. You could also provide a hash_funcs dict to return a custom hash for the Dashboard type/class (which you may wish to do if you have multiple Dashboard instances holding different data sets). If your intention is for Dashboard to be a singleton, then introduce a DashboardFactory static method whose create method returns a singleton instance of Dashboard (using st.experimental_singleton). This may be gold plating (overkill) for your use, so the prefix is sufficient.

A

[Fabio]

I blindly use experimental_memo and it works perfectly for me. I never quite understood what the other one does but I do not do any machine learning so probably do not need it anyway. Great work.

Would be nice to have a way to shut a given cache off and measure whether it helps or not. I use it indiscriminately and the app is fast so I assume it generally helps.

[whitphx]

Similar to We want to deprecate st.cache! …and need your input ❤️ - #6 by gagangoku,
my concern is the lack of unpicklable object support.

My original problem was posted in the previous thread about the new cache primitives:

New experimental primitives for caching

Hi, I’m an author of streamlit-webrtc. I took a glance at these new caching mechanisms though, I think they cannot meet my current needs. I’m using st.cache to keep the object identity returned from a factory function for identical input arguments over reruns. My actual code is here: streamlit-webrtc/factory.py at e351066c5887e7d7ebeadda06c7d12c95b4bac24 · whitphx/streamlit-webrtc · GitHub Example pseudo code is below: obj_a = ... # Assume that the identity of this object is not changed ov…

though this has been solved by using st.session_state instead.
In my original case, my intention was to create a session-specific cache, and the session state was exactly what is for it.
So currently I don’t have specific problems, but I would like just to say that I’m afraid if there are some edge cases where users want to memoize unpicklable objects.

And I recommend to state that st.session_state can also be an alternative of st.cache for some specific cases in the migration guide.

[rcsmit]

You already started deprecating it? from streamlit import caching gives on some scripts errors, and on others it doesnt?

[Caroline]

@rcsmit We haven’t started deprecating it. You don’t need to import caching separately from Streamlit – it’s all in the same library