Best practice for real-time app

Using Streamlit
#1

Hi everyone,

My idea is to develop an app that takes data from mongoDB database. It will pull out from the MongoDB data at every fixed time-step (so far I have fixed every 5 min). Then, I will turn into dataframes, perform some calculations and plot some results. The idea is to present both accomolative results and results related to the real-time data. I would to know if anyone has an example code (with similar characteristics), or recommend best packages to execute that - packages like to plot real-time data.

#2

The ā€˜Plottingā€™ demo from streamlit hello does something similar to what you are describing:

import time
import numpy as np

progress_bar = st.sidebar.progress(0)
status_text = st.sidebar.empty()
last_rows = np.random.randn(1, 1)
chart = st.line_chart(last_rows)

for i in range(1, 101):
    new_rows = last_rows[-1, :] + np.random.randn(5, 1).cumsum(axis=0)
    status_text.text("%i%% Complete" % i)
    chart.add_rows(new_rows)
    progress_bar.progress(i)
    last_rows = new_rows
    time.sleep(0.05)

progress_bar.empty()

# Streamlit widgets automatically run the script from top to bottom. Since
# this button is not connected to any other logic, it just causes a plain
# rerun.
st.button("Re-run")

The key line is chart.add_rows(new_rows), which appends data frame rows to a chart object. When you do this, the app knows that an input has changed and re-renders the graph appropriately.

3 Likes
#3

Awesome!

1 Like
#4

Using this method, the rest of the code wonā€™t get executed until the end of the for loop.
Is there a way to run the loop in the ā€œbackgroundā€ while letting the rest of the code run? Is there where the ā€œasyncā€ technique comes into handle?
Thanks.

#5

This eventually runs for a certain durations and halts.
It can also be achieved with a similar condition for the seamless running of the chart.
is there a way to control the data rows(last_rows) inside the chart?
If I want to restrict the data points inside the chart( to show only last 10 points).

    import time
    import numpy as np
    last_rows = np.random.randn(1, 1)
    
    chart = st.line_chart(last_rows)
    i = 0
    while True:
        i += 1
        new_rows = last_rows[-1, :] + np.random.randn(5, 1).cumsum(axis=0)
        chart.add_rows(new_rows)
        last_rows = new_rows
        time.sleep(0.05)
#8

The above solution doesnā€™t work for live data. In order to make the the live charts. for live time series data.

from bokeh.models import AjaxDataSource, CustomJS

adapter = CustomJS(code= ā€œā€"
const result = {x: , y: }
const x= cb_data.response.x
const y= cb_data.response.x
for (let i=0; i<x.length; i++) {
result.x.push(x[i])
result.y.push(y[i])
}
return result
ā€œā€"
)

    source = AjaxDataSource(data_url='http://localhost:5050/api/data',
                    polling_interval=1000, adapter=adapter, 
                    http_headers={"Access-Control-Allow-Origin":"*"})

Sample:- Ajax API Response in json format

       {"x":[1,3,3,4], 'y':[1,5,6,7]}

}

#9

I was able to create a dynamic radar chart that continuously updates with a stream of data using a ā€˜st.empty()ā€™ placeholder to write my chart to and a while loop to continuously re-run the chart.

You can see a demo app and my source code here:

3 Likes
#10

Oh that is very nice. I have an application for itā€¦

#11

Great demonstration @mkhorasani! Do you have a Streamlit sharing invite? If so, itā€™d be a great example for the community to see liveā€¦if not, send me a direct message and weā€™ll get you set up :100:

2 Likes
#12

Hey @randyzwitch, I just deployed this app on Streamlit sharing as per your recommendation! Please find it here.

1 Like
#13

Hi Randy,

is it possible to use the add_row function also with Plotly?
I didnt archive it. I tried to create a like that: chart = st.plotly_chart(ā€œdataā€) and then chart.add_rows(new_rows). It didnt worked.

#14

I donā€™t believe add_rows works with Plotly, but I think Plotly itself has its own version of appending data into a chart dynamically.

#15

Hi Randy,

I have been on the hunt on the plotly website for a way to append the data into a plotly chart but have not run across the solution yet.

Could you provide a link to what you are referring too? Greatly appreciated!

#16

I donā€™t have a link handy unfortunately, just that Iā€™m sure Iā€™ve seen a Plotly streaming data example.

#17

I have jumped through majority of streamlit forums with keywords relating to graphical updates and have yet to see a good solution. I am quite surprised that this issue has not been formally addressed considering there have been so many questions asked regarding this. Majority if not all of the forum posts have been ignored by streamlit. Seems like a common issue amongst many streamlit users.

#18

Hi @SEVEN,

Hereā€™s the relevant section in the Plotly Python documentation: Plotly Streaming | Python/v3 | Plotly

Best,
Snehan

1 Like
#19

@snehankekre, Thank for the suggestion! Plotly is v5.1+ at this time. That feature does not work unless you run an outdate version with drastically less features. Sorry, not quite the solution, perhaps there is another way to accomplish this?

#20

@SEVEN As this is a limitation / issue with the Plotly Python package, might be best to ask in the Plotly forum or create a new GitHub issue for a feature request?