Session Specific Caching

Using Streamlit
1

This might relate to some of the SessionState stuff mentioned here, but is there any way to make caching session specific?

I’ve noticed that cached values are served across all sessions. For example I have a set of demo inputs to my prediction function. If one user runs the demo, the results are cached and then served up to any other user that tries to access the demo. Is there any way to make caching session specific, or serve a fresh session to each new user?

On a related note, does caching have a set lifetime? Are cached values deleted after some time, or do they persist indefinitely? Is there a scenario where cached predictions stored in memory pile up over time until the instance running the app runs out of memory?

2

Hey Karl! Lots of great questions. I’ll answer one by one:

is there any way to make caching session specific?

Yes, this is possible, but currently a bit hacky. For this, I modified the code in the thread you linked above to make it grab a session-specific ID: Hack to get a session-specific ID in Streamlit. See https://discuss.streamlit.io/t/session-specific-caching/271 · GitHub

Now you can pass the get_session_id() function from that Gist to make cache session -specific:

user_session_id = get_session_id()

@st.cache
def session_specific_addition(arg1, arg2, user_session_id):
  # Your function body goes here. You don't need to do anything with
  # the user_session_id argument. Just leave it in the function signature.
  return arg1 + arg2 

result = session_specific_addition(10, 20, user_session_id)

# And if you want to hide the user_session_id arg,
# just wrap the function:

def addition(arg1, arg2):
  return session_sepecific_addition(10, 20, user_session_id)

On a related note, does caching have a set lifetime?

Not yet, but thanks for the reminder! We’ve had a ticket about this in our old tracker for a couple of months now, but never got to it. So I created a new issue on the new public tracker, to shame us into doing the right thing :wink:

4 Likes
3

I elaborated on tvst’s awesome answer above to write a gist defining a @fancy_cache decorator, which is similar to @st.cache, but it add two cool new keyword arguments:

  • ttl : The maximum number of seconds that this item will remain in the cache.
  • unique_to_session : True means that the cache is unique to this session.

for example:

@fancy_cache(ttl=100, unique_to_session=True)
def func(...):
   ...
6 Likes
4

@Adrien_Treuille’s workaround doesn’t appear to work with versions after 0.53.0…:frowning:

For the TTL option, though, I added a defaulted parameter to my cached function that changes every minute: i.e. func(*args, ttl=math.floor(datetime.datetime.now().timestamp()/60)*60).

5

Hey @rmartin16,

I’ve posted a fixed version of @Adrien_Treuille’s gist here https://gist.github.com/jrhone/72ae07718f28c5b339ad3f9b14c20fe1

Let me know if you have any trouble with it.

1 Like
6

Hi Jonathan,

thank you for the update. Unfortunately, it doesn’t work im with my Streamlit-version 0.62.
I get the following error:
“AttributeError: ‘Server’ object has no attribute ‘_session_infos’”.
I copied your code in a separate file and imported it. Do you have any suggestions. (I am running on an EC2 Instance on AWS).

Best regards,
Marius

7

Looks like this gist has been updated and should resolve that issue

1 Like
8

It seems to be now much simple:

from streamlit.report_thread import get_report_ctx
ctx = get_report_ctx()
print(ctx.session_id)
4 Likes