Display spatio-temporal data with st.video?

🎈 Using Streamlit
1

I have 2D spatio-temporal data that I would like to display as a video. The data is stored as a 3D numpy array and I am trying to display it with st.video whose documentation says it accepts numpy arrays.
However, all I see is a black screen that does not play.

Below is sample code. What is the best way to display this data with Streamlit?

import numpy as np
import streamlit as st

height, width, n_frames = 20, 20, 100
data = np.linspace(0, 255, height*width*n_frames).reshape(height,width,n_frames)
st.video(data)
2

I doubt that that really works for videos but I wish I’m wrong and the docs are right :sweat_smile:

I would encode the numpy array data using OpenCV, then pass it to ffmpeg to make the video compatible to the HTML5 player that streamlit uses (see this thread about the codec’s issue: Processing video with OpenCV and write it to disk (to display in st.video) - #2 by edsaac).

numpy_to_video

Code: 
import numpy as np
import streamlit as st
import cv2
import subprocess
from tempfile import NamedTemporaryFile

height, width, n_frames = 360, 200, 30
frames_per_second = 12

# Generate some data
data = np.linspace(0, 256, height * width * n_frames, dtype=np.uint8).reshape(
    n_frames, width, height
)

data[:, 50:100, 50:100] = 255  # A white square

"### `st.video(np.ndarray)`"
st.video(data) ## This will fail

temp_files_kwargs = dict(suffix=".mp4", dir=".")

with (
    NamedTemporaryFile(**temp_files_kwargs) as temporal_video,
    NamedTemporaryFile(**temp_files_kwargs) as converted_video,
):
    out = cv2.VideoWriter(
        temporal_video.name,
        cv2.VideoWriter_fourcc(*"mp4v"),
        frames_per_second,
        (height, width),
        False,
    )

    for frame in data:
        out.write(frame)

    out.release()

    ## Call ffmpeg to convert to a web-playable codec.
    subprocess.call(
        args=f"ffmpeg -y -i {temporal_video.name} -c:v libx264 {converted_video.name}".split(),
        stdout=subprocess.DEVNULL,
    )

    ## This will show a video
    """### `st.video("encoded_data.mp4")`"""
    st.video(converted_video.name)

:warning: You will need ffmpeg (apt install ffmpeg) and OpenCV ( pip install opencv-python-headless) installed.