Fast real-time plot (100Hz)

Using Streamlit
1

Dear all,

I am collecting data from a sensor with a sampling frequency of 100Hz.
I would like to plot the data in real time as I receive them.

I have tried:

Would anybody be so kind to suggest me how to plot a stream of real-time data with streamlit?

Thank you very much!!!

2

Hello,

does anybody have any suggestion?

Thank you again!

3

Hi @blueslevin and welcome to the forum!!
Not sure at all, but maybe this could be of help

regards

1 Like
4

Hi @napoles3d, thank you very much for your suggestion!!
Unfortunately the script that you linked is exploiting “add_rows” function that I have testes already.
This function is very convenient, but it has a couple of drawbacks:

  • Speed
  • There is no function to “remove the rows”. Thus the chart is constantly increasing in size and after some seconds it is impossible to read.

If you have some additional suggestion that would be very much appreciated!
Thank you again

5

You should try a PyQtGraph, it’s much faster real time plot than matplotlib.
https://github.com/ap–/python-live-plotting/blob/master/plot_pyqtgraph.py

6

@Cly1st thank you very much!!

I have tried PyQtGraph.
In terms of speed, it certainly fits the purpose.
However, it is not possible to integrate it within a streamlit app. :frowning:

I hope that you could suggest me an alternative way.
Thank you very much!!

8

So, I am not sure about the compatibility between Streamlit and Pyqtgraph (I will check about this later).
However, for your second problem with keeping visible only recent incoming data, you should be using the Deque data structure. When data appending to maximum (say 1000 sample) the first appended value will remove from the list (Time complexity is O(n) time).
https://www.geeksforgeeks.org/deque-in-python/

9

Here is an example code of deque with matplotlib:

import streamlit as st
import time
import matplotlib.pyplot as plt
import numpy as np
from collections import deque
fig, ax = plt.subplots()

max_samples = 100
max_x = max_samples
max_rand = 100

x = np.arange(0, max_x)
y = deque(np.zeros(max_samples), max_samples)

ax.set_ylim(0, max_rand)
line, = ax.plot(x, np.array(y))
the_plot = st.pyplot(plt)

def animate():  # update the y values (every 1000ms)
    line.set_ydata(np.array(y))
    the_plot.pyplot(plt)
    y.append(np.random.randint(max_x)) #append y with a random integer between 0 to 100

for i in range(200):
    animate()
    time.sleep(0.01)
10

@Cly1st , again thank you very much for your message.

I have tested your code, but matplotlib is extremely slow. I reach an update rate of maximum 2Hz.
Not even 10Hz as per your code.

As I said, I have also tried to use add_rows. An example follows:

import streamlit as st
import time
import numpy as np

progress_bar = st.sidebar.progress(0)
status_text = st.sidebar.empty()
last_rows = np.random.randn(1, 1)
chart = st.line_chart(last_rows)

for i in range(1, 101):
	new_rows = last_rows[-1, :] + np.random.randn(5, 1).cumsum(axis=0)
	new_rows = np.random.randn(1, 1)
	status_text.text("%i%% Complete" % i)
	chart.add_rows(new_rows)
	progress_bar.progress(i)
	last_rows = new_rows
	time.sleep(0.05)

progress_bar.empty()

# Streamlit widgets automatically run the script from top to bottom. Since
# this button is not connected to any other logic, it just causes a plain
# rerun.
st.button("Re-run")

The problem is that the cart becase very quickly unreadable because I cannot remove rows.
Do you think that there is any way to explout deque within the streamlit chart?

Thanks again!

11

Hello,

I have been using PyQtGraph for the past months with success.
However, the graphics of PyQtGraph is not very captivating.
Do you think there is any new python alternative for fast updates of charts? (100Hz)

Thanks a lot!

12

Hi blueslevin,

I am interested in seeing your PyQtGraph implementation as we are also trying to plot sensor data coming in at 100Hz.

Would you be willing to share your implementation or at least what I should do?

Thank you in advance.

13

Adaickalavan/Real-Time-Streaming-Visualization: A web based visualization tool for real time data streaming application in Python. (github.com)