Efficiently Visualizing Multiple Live Data Streams in Streamlit

Hey everyone,

I’m working on visualizing live sensor data in Streamlit, and I’m stuck trying to find a better way to do it. Here’s what I’ve tried so far:

Current Approach: While Loop for Constant Updates

I’m using an infinite loop to keep updating the data in real-time, as shown in this blog post:

import streamlit as st
import random
import time

def read_data():
    """Simulate sensor data reading."""
    return random.random()

with st.empty():
    while True:
        data = read_data()
        st.write(data)
        time.sleep(1 / 30)

The Problem: The loop messes up other Streamlit features. For example, in this code, st.selectbox doesn’t update st.write properly because of the while True loop:

import streamlit as st
import random
import time

@st.dialog('Choose an option')
def option():
    choice = st.selectbox(label='Options', options=['A', 'B'])
    st.write(choice)

def read_data():
    return random.random()

if st.button('Choose an option'):
    option()

with st.empty():
    while True:
        data = read_data()
        st.write(data)
        time.sleep(1 / 30)

Other Things I Tried

1. Using st.write_stream

Here’s another approach using st.write_stream:

import streamlit as st
import random
import time

def read_data():
    while True:
        yield random.random()
        time.sleep(1 / 30)

with st.empty():
    print('1st stream')
    st.write_stream(read_data)

# This will never run
with st.empty():
    print('2nd stream')
    st.write_stream(read_data)

The Problem: Infinite streams keep running without stopping, making it impossible to handle and visualize multiple data streams at the same time.

2. Using st.fragment

I also tried st.fragment for periodic updates:

import streamlit as st
import random

def read_data():
    return random.random()

@st.fragment(run_every='33ms')  # Runs at 30 Hz
def show_data():
    data = read_data()
    st.write(data)

show_data()

The Problem: Fragments freeze when the refresh rate (run_every) is set too high. Even worse, the refresh rate at which a fragment freezes varies depending on the content being rendered. For instance, the simpler example from above runs smoothly at 20 Hz (run_every='50ms') on my machine. However, adding a basic plot causes it to freeze again.

import streamlit as st
import random
import altair as alt
import pandas as pd

def read_data():
    return random.random()

@st.fragment(run_every='50ms')  # Runs at 20 Hz
def show_data():
    data = pd.DataFrame({
        "x": [read_data() for _ in range(5)],
        "y": [read_data() for _ in range(5)]
    })
    chart = alt.Chart(data).mark_circle().encode(x='x:Q', y='y:Q')
    st.write(chart)

show_data()

Trying to use multiple st.fragment instances makes things even worse.

An Idea: st.camera_input

What I really want is a way to render data smoothly without breaking anything else. The st.camera_input feature looks promising. For example:

import streamlit as st

st.camera_input("1st camera stream")
st.camera_input("2nd camera stream")

st.write("Hello World!")

This works great - no endless loops, and both streams update without blocking each other.

The Question

Could something like st.camera_input be adapted for other live sensor data? I watched both tutorials on creating custom Streamlit components, but I’m not sure how st.camera_input works internally to replicate that behavior for other infinite data streams. (Also, I’ve never written custom Streamlit components before or had any experience with web development.)

I’d really appreciate any ideas on how to smoothly visualize multiple data streams at once.

Thanks in advance!

The fragment would be the recommended way to go. I modified your example, though I haven’t built in a cutoff, so it will just keep going. (For starting and stopping streamling, see this tutorial.)

import streamlit as st
import random
import altair as alt
import pandas as pd

def read_data():
    return random.random()

if "df" not in st.session_state:
    st.session_state.df = pd.DataFrame({
        "x": [read_data() for _ in range(5)],
        "y": [read_data() for _ in range(5)]
    })

@st.fragment(run_every='50ms')  # Runs at 20 Hz
def show_data():
    loc = len(st.session_state.df.index)
    st.session_state.df.loc[loc] = [read_data(), read_data()]
    data = st.session_state.df
    chart = alt.Chart(data).mark_circle().encode(x='x:Q', y='y:Q')
    st.write(chart)

show_data()

For the simple, built-in charts, there’s also .add_rows().

Thanks a lot for the reply!

I think the issue I mentioned still applies. With a high refresh rate, st.fragment starts lagging (visually). I’m aiming for a super smooth animation of my data streams. Here’s a GIF running your example with a start button added. And as you can see, it’s very laggy:

laggy_fragment

BTW, the tutorial you linked has the same problem. If you let the refresh rate slider go down to 0.1, the frontend lags again.

The gif appears much choppier than what I get (but who knows with a gif…).

I think there is definitely a limitation here, but this is reasonably smooth for me at 10 frames per second. I don’t think the native components are going to handle 30 or more Hz, but maybe someone else can chime in.

2025-01-02_02-40-42 (1)

import streamlit as st
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
from math import sin


def get_recent_data(last_timestamp):
    """Generate and return data from last timestamp to now, at most 60 seconds."""
    now = datetime.now()
    if now - last_timestamp > timedelta(seconds=6):
        last_timestamp = now - timedelta(seconds=6)
    sample_time = timedelta(seconds=0.05)  # time between data points
    next_timestamp = last_timestamp + sample_time
    timestamps = np.arange(next_timestamp, now, sample_time)
    sample_values = [sin(time.astype("float")/1000000) for time in timestamps]

    data = pd.DataFrame(sample_values, index=timestamps, columns=["A"])
    return data


if "data" not in st.session_state:
    st.session_state.data = get_recent_data(datetime.now() - timedelta(seconds=60))

if "stream" not in st.session_state:
    st.session_state.stream = False


def toggle_streaming():
    st.session_state.stream = not st.session_state.stream


st.title("Data feed")
st.sidebar.slider(
    "Check for updates every: (seconds)", 0.05, 5.0, value=1.0, key="run_every"
)
st.sidebar.button(
    "Start streaming", disabled=st.session_state.stream, on_click=toggle_streaming
)
st.sidebar.button(
    "Stop streaming", disabled=not st.session_state.stream, on_click=toggle_streaming
)

if st.session_state.stream is True:
    run_every = st.session_state.run_every
else:
    run_every = None


@st.fragment(run_every=run_every)
def show_latest_data():
    last_timestamp = st.session_state.data.index[-1]
    st.session_state.data = pd.concat(
        [st.session_state.data, get_recent_data(last_timestamp)]
    )
    st.session_state.data = st.session_state.data[-100:]
    st.line_chart(st.session_state.data)


show_latest_data()

I wish my GIF was choppier than what I get in the browser, but sadly, this is exactly what I’m experiencing. Also, your example code is much laggier for me. :pensive:

From what I’ve observed:

  • st.fragment starts lagging when the refresh rate exceeds what the machine can handle.
  • The lag point depends on what’s rendered in the fragment and the machine it’s running on.
  • More fragments mean hitting this lag point sooner.
  • Only the frontend lags - the backend seems fine, even with “high” refresh rates.

Overall, this makes st.fragment unsuitable for my use case because:

  1. It’s impossible to set a refresh rate that works for all users.
  2. My number of fragments is dynamic (e.g. when adding a new sensor for visualization). A fragment running fine at rate x might break if another fragment is added.