Clear cache with a time interval (at least once per hour)?

Using Streamlit
1

Hi all.

I want to clear the cache with a given time interval (say, at least clear once an hour or in every 15min) so my app gets a chance to fetch updated data…

Currently I cache the data access to improve the responsiveness with a simple @st.cache infront of my function…

Is this currently possible?

Cheers,
C

2

The way I, and some others, have handled this is to add another parameter to the data load function. The additional parameter is called with a value that is a timestamp truncated to the refresh interval you want (for our application 5 minutes works well). That will force an update. If the data range is purely historical, not including the current day, I just use None for the timestamp value.
It’s a little bit of extra housekeeping but it works well.
Kevin

3

Appreciate the reply!

Would you have some example code or a link to some GitHub repo where this is used? Having some trouble imagining the code to your explanation :+1:

Cheers,
C

4

Here is a cached function, at_time is the timestamp used to control caching.

@st.cache
def get_phrase_data(start_date, end_date, phrases=None, at_time=None,
                             **kwargs):
    """Wrapper for :func:`load_metadata_for_period` for Streamlit caching
    
    start_date and end_date should be ISO format strings in UTC for date
    
    """
    #print('get_phrase_data', phrases)
    data = load_metadata_for_period(start_date, end_date, phrases=phrases,
                    **kwargs)
    return data

And a function to generate the timestamp value for the interval desired

import datetime
def get_truncated_timestamp(end=None, refresh_interval=5, 
                refresh_interval_units='minutes'):
    """A time truncated down to resolution to control caching

    Parameters:
        end (datetime.datetime):  The time to validate against

    Returns:
        datetime.datetime: a timestamp set to the nearest refresh_interval.

    If end is at or before midnight at the start of the
    current day returns **None** as we'll never want to change that
    cache anyway.

    """
    if end and end <= datetime.date.today():
        #  Fix timestamp for data not updating...
        timestamp = None
    else:
        timestamp = datetime.datetime.now()
        if refresh_interval_units == 'minutes':
            interval = int(refresh_interval)
            timestamp = timestamp.replace(
                            minute=(timestamp.minute // interval) * interval,
                            second=0, microsecond=0, )
    return timestamp

And here is some psuedo code to call it.

    data = get_phrase_data(start, end, phrases, get_truncated_timestamp(end),
                              **kwargs)
3 Likes
5

Wow.

You are my hero for today :love_you_gesture:

Thanks

1 Like
6

@knorthover thnx for your example, can you please explain how data will get refreshed in in every 5 minutes …a little more explanation would be helpful …thnx JB

7

The data isn’t refreshed every 5 minutes. That’s not the way Streamlit works out of the box.

What the code does is ensure that when the page is rerun and the data is more than 5 minutes old, the data gets refreshed.

For my application that was sufficient.

8

@knorthover thnx for your note …thnx JB