How to show stream of frames as video?

Using Streamlit
,
1

I am running streamlit locally(still developing) and after doing the inference on the video stream I have a stream of frames that I am getting and want to display as a video using streamlit
how can this be done?

Here is the sample code

def main(
    weight_path: str,
    rtsp_url: str,
    zone_configuration_path: str,
    model_id: str,
    confidence: float,
    iou: float,
    classes: List[int],
) -> None:
    sink = CustomSink(weights_path=weight_path ,zone_configuration_path=zone_configuration_path, classes=classes)

    pipeline = InferencePipeline.init_with_custom_logic(
        video_reference=rtsp_url,
        on_video_frame=sink.infer,
        on_prediction=sink.on_prediction
    )

class CustomSink:
 def __init__(self, weights_path: str, zone_configuration_path: str, classes: List[int]):
 // initialization etc

def on_prediction(self, result: dict, frame: VideoFrame) -> None:
       // doing some frame processing here
        annotated_frame = frame.image.copy()

        annotated_frame = sv.draw_text(
            scene=annotated_frame,
            text=f"{fps:.1f}",
            text_anchor=sv.Point(40, 30),
        )
         labels = [
                f"#{tracker_id} {int(time // 60):02d}:{int(time % 60):02d}" if class_id != 2 else f"#{tracker_id}"
                for tracker_id, time, class_id in zip(detections_in_zone.tracker_id, time_in_zone,detections_in_zone.class_id)
            ]

          annotated_frame = LABEL_ANNOTATOR.annotate(
                scene=annotated_frame,
                detections=detections_in_zone,
                labels=labels,
                custom_color_lookup=custom_color_lookup,
            )

        st.image(annotated_frame)
      
        try:
        # replace this with streamlit 
        # Resize the frame to fit the window size
            resized_frame = cv2.resize(annotated_frame, (width, height))
        except cv2.error as e:
            print(f"Error resizing frame: {e}")
    
        # Display the resized frame in the window
        cv2.imshow('Resizable Window', resized_frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            raise SystemExit("Program terminated by user")

I am using the inference pipeline to do the inference on the video stream. To display the video I just pass the annoated_frame to the open cv and I get the video. I want to do the same thing but with the streamlit.
I tried the st.image(annotated_frame) but I am getting a series of images on the web page instead of video since it is inside the on_prediction callback
I tried st.video as well but not working

so my question is how to display a stream of processed frames as video on the web suing streamlit?

You can read about the inference pipeline here: Inference Pipeline - Roboflow Inference

2

Hello from Roboflow :slight_smile:

This thread adds extra bits of information and explains what was already tried.

3

Check my github ccl-private/streamlit_rtsp, I do not know how to share my github link. One can change rtsp stream to video frame simply.