Best (fastest) practice to display live 2D data

[andfanilo]

Waw @w-markus you made an awesome example here! To be honest I did not expect Streamlit and asyncio to play well this way.

@w-markus

What I tried was to put all methods into a class, and upon first run of the script store an instance of that class in st.session_state for reuse and tracking of the asynchronous tasks. But this slowed down things considerably, but why?

I don’t know why the slowdown if you store in state, maybe Streamlit tries to do some attribute tracking on elements in a class, did not check .

When using asyncio, I really usually prefer to use a producer/consumer method, with your producer pushing images in a queue (could be downloading data from your sensor) and consumer fetching images from the queue and displaying them. That way you can control the produce and consumer separately, can scale the number of producers if necessary. Async IO in Python: A Complete Walkthrough – Real Python

Also st.image on a numpy array uses Pillow to convert to an image. Apparently OpenCV is 1.3x faster on this task so I manually create the images from your numpy array.

I’ve created a…a bit of a complex example but here:

import asyncio
from datetime import datetime

import cv2
import numpy as np
import streamlit as st


QUEUE_SIZE = 1000
SIZE_IMAGE = 512


def get_or_create_eventloop():
    try:
        return asyncio.get_event_loop()
    except RuntimeError as ex:
        if "There is no current event loop in thread" in str(ex):
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
            return asyncio.get_event_loop()


async def produce_images(queue, delay):
    while True:
        _ = await asyncio.sleep(delay)
        image = np.random.random((SIZE_IMAGE, SIZE_IMAGE)).astype(np.float32)

        # Add bars depending on state count
        n = st.session_state.produced_images % SIZE_IMAGE
        m = st.session_state.produced_images % SIZE_IMAGE
        image[n : n + 10] = 0
        image[:, m : m + 10] = 1

        _ = await queue.put(cv2.cvtColor(image, cv2.COLOR_GRAY2BGR))
        st.session_state.produced_images += 1


async def consume_images(image_placeholder, queue_size_placeholder, queue, delay):
    while True:
        _ = await asyncio.sleep(delay)
        image = await queue.get()
        image_placeholder.image(
            image,
            caption=f"Consumed images: {st.session_state.consumed_images}, {str(datetime.now())}",
        )
        queue_size_placeholder.metric(
            f"In queue (queue size is {QUEUE_SIZE})", st.session_state.queue.qsize()
        )
        st.session_state.consumed_images += 1
        queue.task_done()


async def run_app(
    image_placeholder, queue_size_placeholder, queue, produce_delay, consume_delay
):
    _ = await asyncio.gather(
        produce_images(queue, produce_delay),
        consume_images(image_placeholder, queue_size_placeholder, queue, consume_delay),
    )


##### ACTUAL APP

if __name__ == "__main__":
    st.set_page_config(
        layout="wide",
        initial_sidebar_state="auto",
        page_title="Asyncio test",
        page_icon=None,
    )

    if "event_loop" not in st.session_state:
        st.session_state.loop = asyncio.new_event_loop()
    asyncio.set_event_loop(st.session_state.loop)

    # if "queue" not in st.session_state:
    #    st.session_state.queue = asyncio.Queue(QUEUE_SIZE)
    # if "produced_images" not in st.session_state:
    #    st.session_state.produced_images = 0
    # if "consumed_images" not in st.session_state:
    #    st.session_state.consumed_images = 0
    st.session_state.queue = asyncio.Queue(QUEUE_SIZE)
    st.session_state.produced_images = 0
    st.session_state.consumed_images = 0

    st.title("Hello random image!")
    produce_delay = 1 / st.sidebar.slider(
        "Produce images Frequency (img / second)", 1, 100, 10
    )
    consume_delay = 1 / st.sidebar.slider(
        "Display images Frequency (img / second)", 1, 100, 10
    )
    c1, c2 = st.columns(2)
    image_placeholder = c1.empty()
    queue_size_placeholder = c2.empty()

    asyncio.run(
        run_app(
            image_placeholder,
            queue_size_placeholder,
            st.session_state.queue,
            produce_delay,
            consume_delay,
        )
    )

This is not perfect yet but hopefully can help:

• if no elements in queue or too much the app will bug
• I’m showing 50 fps on the gif, which is almost way too much for st.image to handle, at about 700 images it looks like the InMemoryFileManager is in pain, and something else goes awry. 10 fps should be fine but I’ll let you experiment and tell me, I did not dive deep into it.

Hope it helps,
Fanilo