Processing video with OpenCV and write it to disk (to display in st.video)

🎈 Using Streamlit
2

TLDR: you need to get the cv2.VideoWriter_fourcc(*'h264') codec to work if you want your mp4 video compatible with streamlit’s web player.

This was a fun puzzle!. Turns out it’s more of an OpenCV issue, rather than a streamlit problem but here it goes.

  • To generate an mp4 file you should use an appropriate codec, such as “mp4v” or “h264”. In your case, it is probably falling back to mp4v as “xvid” expects a .avi file.
  • The MIME type error comes from the “mp4v” codec as MPEG-4 is not supported by streamlit’s html5 web player. You would need to set the “h264” (or “x264”) codec to make the video compatible, however, that support seems to not come by default with OpenCV (see issue).

Here is a little demo using ffmpeg to reencode to H264 the video written by OpenCV:

opencvvideo

The code:

import streamlit as st
import cv2
import subprocess

video_data = st.file_uploader("Upload file", ['mp4','mov', 'avi'])

temp_file_to_save = './temp_file_1.mp4'
temp_file_result  = './temp_file_2.mp4'

# func to save BytesIO on a drive
def write_bytesio_to_file(filename, bytesio):
    """
    Write the contents of the given BytesIO to a file.
    Creates the file or overwrites the file if it does
    not exist yet. 
    """
    with open(filename, "wb") as outfile:
        # Copy the BytesIO stream to the output file
        outfile.write(bytesio.getbuffer())

if video_data:
    # save uploaded video to disc
    write_bytesio_to_file(temp_file_to_save, video_data)

    # read it with cv2.VideoCapture(), 
    # so now we can process it with OpenCV functions
    cap = cv2.VideoCapture(temp_file_to_save)

    # grab some parameters of video to use them for writing a new, processed video
    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    frame_fps = cap.get(cv2.CAP_PROP_FPS)  ##<< No need for an int
    st.write(width, height, frame_fps)
    
    # specify a writer to write a processed video to a disk frame by frame
    fourcc_mp4 = cv2.VideoWriter_fourcc(*'mp4v')
    out_mp4 = cv2.VideoWriter(temp_file_result, fourcc_mp4, frame_fps, (width, height),isColor = False)
   
    while True:
        ret,frame = cap.read()
        if not ret: break
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) ##<< Generates a grayscale (thus only one 2d-array)
        out_mp4.write(gray)
    
    ## Close video files
    out_mp4.release()
    cap.release()

    ## Reencodes video to H264 using ffmpeg
    ##  It calls ffmpeg back in a terminal so it fill fail without ffmpeg installed
    ##  ... and will probably fail in streamlit cloud
    convertedVideo = "./testh264.mp4"
    subprocess.call(args=f"ffmpeg -y -i {temp_file_result} -c:v libx264 {convertedVideo}".split(" "))
    
    ## Show results
    col1,col2 = st.columns(2)
    col1.header("Original Video")
    col1.video(temp_file_to_save)
    col2.header("Output from OpenCV (MPEG-4)")
    col2.video(temp_file_result)
    col2.header("After conversion to H264")
    col2.video(convertedVideo)

PDs:

  • Don’t forget to close the video files you are writing to and reading from. The files should be closed automatically after the script terminates, but in streamlit the file you are writing to would never released. Just add cap.release() and out_mp4.release() after finish reading and writing.
  • When writing a grayscale video, don’t forget to add that flag (isColor=False) to the VideoWriter.
  • It’s better to use relative file paths like ./myvideo.mp4, rather than the windows-only C:/myvideo.mp4

edit:

  • Instead of running ffmpeg as a subprocess, you could try the ffmpy package to reencode the video :vhs:.
4 Likes