Session Specific Caching

This might relate to some of the SessionState stuff mentioned here, but is there any way to make caching session specific?

I’ve noticed that cached values are served across all sessions. For example I have a set of demo inputs to my prediction function. If one user runs the demo, the results are cached and then served up to any other user that tries to access the demo. Is there any way to make caching session specific, or serve a fresh session to each new user?

On a related note, does caching have a set lifetime? Are cached values deleted after some time, or do they persist indefinitely? Is there a scenario where cached predictions stored in memory pile up over time until the instance running the app runs out of memory?

Hey Karl! Lots of great questions. I’ll answer one by one:

is there any way to make caching session specific?

Yes, this is possible, but currently a bit hacky. For this, I modified the code in the thread you linked above to make it grab a session-specific ID: https://gist.github.com/tvst/6ef6287b2f3363265d51531c62a84f51

Now you can pass the get_session_id() function from that Gist to make cache session -specific:

user_session_id = get_session_id()

@st.cache
def session_specific_addition(arg1, arg2, user_session_id):
  # Your function body goes here. You don't need to do anything with
  # the user_session_id argument. Just leave it in the function signature.
  return arg1 + arg2 

result = session_specific_addition(10, 20, user_session_id)

# And if you want to hide the user_session_id arg,
# just wrap the function:

def addition(arg1, arg2):
  return session_sepecific_addition(10, 20, user_session_id)

On a related note, does caching have a set lifetime?

Not yet, but thanks for the reminder! We’ve had a ticket about this in our old tracker for a couple of months now, but never got to it. So I created a new issue on the new public tracker, to shame us into doing the right thing :wink:

2 Likes

I elaborated on tvst’s awesome answer above to write a gist defining a @fancy_cache decorator, which is similar to @st.cache, but it add two cool new keyword arguments:

  • ttl : The maximum number of seconds that this item will remain in the cache.
  • unique_to_session : True means that the cache is unique to this session.

for example:

@fancy_cache(ttl=100, unique_to_session=True)
def func(...):
   ...
2 Likes

@Adrien_Treuille’s workaround doesn’t appear to work with versions after 0.53.0…:frowning:

For the TTL option, though, I added a defaulted parameter to my cached function that changes every minute: i.e. func(*args, ttl=math.floor(datetime.datetime.now().timestamp()/60)*60).

Hey @rmartin16,

I’ve posted a fixed version of @Adrien_Treuille’s gist here https://gist.github.com/jrhone/72ae07718f28c5b339ad3f9b14c20fe1

Let me know if you have any trouble with it.

1 Like