Streamlit Autorefresh

Hey everyone!

I created a custom component that triggers Streamlit to rerun on a user-specified interval. I was running across a lot of Streamlit apps using infinite loops, and while we can allow it in a script, I think this custom component will alleviate a lot of the stress of running an infinite loop on a server.

Here’s a trivial example.

import streamlit as st
from streamlit_autorefresh import st_autorefresh

# Run the autorefresh about every 2000 milliseconds (2 seconds) and stop
# after it's been refreshed 100 times.
count = st_autorefresh(interval=2000, limit=100, key="fizzbuzzcounter")

# The function returns a counter for number of refreshes. This allows the
# ability to make special requests at different intervals based on the count
if count == 0:
    st.write("Count is zero")
elif count % 3 == 0 and count % 5 == 0:
    st.write("FizzBuzz")
elif count % 3 == 0:
    st.write("Fizz")
elif count % 5 == 0:
    st.write("Buzz")
else:
    st.write(f"Count: {count}")

Solving the Infinite Loop

Streamlit apps with an infinite loop, utilize the time library to delay between calls.

import time
import streamlit as st

def get_data():
    # Perform some request to get a dataframe
    df = client.get()
    return df
    
table = st.empty()

while True:
     # update every 5 mins
     table.dataframe(get_data())
     time.sleep(300)  

With autorefresh, that is no longer needed. The script will finish executing, and the server will happily wait for the next rerun request.

import streamlit as st
from streamlit_autorefresh import st_autorefresh

# update every 5 mins
st_autorefresh(interval=5 * 60 * 1000, key="dataframerefresh")

def get_data():
    # Perform some request to get a dataframe
    client.get()
    return df

st.dataframe(get_data())

There are a couple features (limiting number of reruns and returning the run recount). Let me know if you have any questions. It’s a simple component so feedback is appreciated.

26 Likes

Awesome :smiley: love the idea of delegating the timer to the frontend!

Could you add it to the Community Tracker page :slight_smile: ?

1 Like

I get following error: To use Custom Components in Streamlit, you need to install PyArrow. Unfortunately, PyArrow does not yet support Python 3.9.

Hi @Ruben_S ! When Python 3.9 was released, PyArrow (which facilitates a serialization of pandas dataframes) did not support it. We added this error to inform users of this. It seems to be addressed, and we have removed it for the next release. You can try out pip install streamlit-nightly if you want to keep working on Python 3.9, or you can try using Python 3.8 using a tool like pyenv

Hope that helps!

Hi @kmcgrady,
this solution saved me a lot of work! Tahnk you so much and keep up the good work :slight_smile: Will you keep maintaing the component?

Hey @fhak Thanks for responding! I hope to keep it maintained where I can. I just realized that the component has two issues. Weird enough I wasn’t notified. I’ll look into them. Feel free to let me know if you have any bugs/recommendations.

Thanks!

1 Like

Hi @kmcgrady, your solution work for me. Thank you very much for your good work.

Hi. I am getting this error message

E0924 17:30:02.857569000 12991352832 backup_poller.cc:124] run_poller: UNKNOWN:Timer list shutdown {created_time:“2022-09-24T17:30:02.852376+08:00”}

This is my streamlit code.

def app():
    page_utils.setup_page("Fractalized Product Template: v{x.y}")
    with open('README.md', 'r') as f:
        readme = f.read()
    st.markdown(readme)

    count = st_autorefresh(interval=5*60*1000, key="counter")

    if count == 0:
        st.write("Count is zero")
    elif count % 6 == 0:
        update_cache()




app()

Is it because of autorefresh?

Also after this error, there was no refresh. So how could i update the data if such an error occurs?

Hey @Bibek_Behera! I assume you are finding this in the Terminal logs and not the browser?

st_autorefresh does not handle any timer events on the server. It’s purely driven by the browser itself. This leads me to think the timer error is unrelated. Curious how frequent this is.

@kmcgrady saw the error once yesterday. In the browser there is a different issue.

Your app is having trouble loading the streamlit_autorefresh.st_autorefresh component.

(The app is attempting to load the component from ****, and hasn’t received its “streamlit:componentReady” message.)

  • If this is a development build, have you started the dev server?
  • If this is a release build, have you compiled the frontend?

For more troubleshooting help, please see the Streamlit Component docs or visit our forums.

I get this error. Does it mean it can stop the counter from initialising. How do i know if the counter has been initialised.

Actually once the component loads, the counter starts ticking. But right now it can take 15-20 mins for the component to load. Secondly I have to keep the browser open always and ensure it doesnt go to sleep. Because then it wont be infinte refresh.

Hmm @Bibek_Behera. Taking 15-20 min for the component to load is rather unusual. Is there anything you are doing that suggests longer times? Is the network connection less strong for example?

It’s possible infinite refresh is not possible as it requires the Streamlit app to maintain its connection, which is not always the case.

Hi,

Actually the component loading is fine. Currently my issue is dependency on the browser. I have to keep it open and hence my laptop can’t shut down as well. Is there a way to open the tab from a terminal background.

I want to update my app on Streamlit every 30 minutes. This update includes calling an API to update the dataset on server side.

For now I should check something like below when user enters the page:

 if (last_update - now > 30*60): do_update()

In that case you would do something like the following

st_autorefresh(interval=30*60*1000, key="api_update")

That way the Streamlit app does not need a loop or time.sleep to keep updating.

1 Like

But I think with this, user should stay on that page. I need the update should be run server side every 30 minutes. Is this possible?

@MAmrollahi

So there’s two solutions I can think of: an easy one, and a harder one.

The easiest one is using st.experimental_memo with a ttl. This will allow every new rerun past a certain time to extract the contents. Depending on how “up to date” the frontend has to be, you can use autorefresh every minute and have the ttl every 30 minutes, so it’s at most a minute delayed…or you can do 15 minutes and it’s at most 15 minutes delayed etc. A well-cached (or memo’ed) information makes a rerun look seamless, so this would be my preferred approach.

The would not work if the effort is extremely time consuming or if you are really looking for very up-to-date information. There’s possible solutions to modify streamlit-autorefresh to have it call on the time marker (every half hour for example) and the ttl and get close and at least sync more clearly. The other option is to run some background job on a thread, but Streamlit is not super well-designed for that.

Hope that helps!

1 Like

I posted something related to this topic here: Best (current) practice for spawing a process to run independent of Streamlit? - #2 by asehmi

Hi @kmcgrady,

I saw this and wanted to use it for my app, which requires a live update every few minutes. However, I get this message when I try to run your trivial example.

Any ideas on how to bypass this? Thanks!

Hi @kmcgrady

I am getting this error when using autorefresh. My frequency is 1 sec.

The autorefresh works well for a while, then it starts giving the following error about “too many open files”, with the file name being the url that I have open. If it matters any, I am using the “pages” feature of Streamlit, and the url is one of the pages in question.

The other strange thing is that, this works fine on my local session of Streamlit, but when I push it to our AWS EC2 instance, the error starts appearing.

Hope you will have time to respond. Thank you very much.

streamlit version 1.17.0

2023-01-24 19:23:05.090 Exception in callback BaseAsyncIOLoop._handle_events(10, 1)
handle: <Handle BaseAsyncIOLoop._handle_events(10, 1)>
Traceback (most recent call last):
File “/usr/lib/python3.7/asyncio/events.py”, line 88, in _run
File “/opt/myapp/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 206, in _handle_events
File “/opt/myapp/lib/python3.7/site-packages/tornado/netutil.py”, line 267, in accept_handler
File “/usr/lib/python3.7/socket.py”, line 212, in accept
OSError: [Errno 24] Too many open files