Session State does not update using streamlit-webrtc

🎈 Using Streamlit
,
1

I am trying to implement a system where an object must be detected using live camera feed and it should update a text component if a particular object is detected. The session_state update is not really working.

Below is the code:

import streamlit as st
import av
import cv2
from streamlit_webrtc import webrtc_streamer, VideoTransformerBase

class VideoProcessor(VideoTransformerBase):
    def __init__(self) -> None:
        self.detect = False
        self.model = # some model

    def recv(self, frame: av.VideoFrame) -> av.VideoFrame:
        img = frame.to_ndarray(format="bgr24")

        # Face detection
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        faces = # return objects

        # Check for object
        for (x,y,w,h) in faces:
            roi_gray = gray[y:y+h, x:x+w]
            required_obj = # predicted number of detected objects
            if len(required_obj) > 0:
                st.session_state.is_recog= True
                st.experimental_rerun()

        return av.VideoFrame.from_ndarray(img, format="bgr24")

# Start the webcam feed
webrtc_ctx = webrtc_streamer(key="smile", video_processor_factory=VideoProcessor, media_stream_constraints={"video": True, "audio": False})


if 'is_recog' not in st.session_state:
    st.session_state.is_recog= False


if st.session_state.is_recog:
    webrtc_ctx.stop_all()
    st.experimental_rerun()

if not st.session_state.is_recog:
    st.write("Show objects")
else:
    st.title("Welcome!")
    st.write("New text.")
2

Hello @tf123,

Here’s an idea that might work better with Streamlit’s model:

  • Instead of directly setting the session state within recv, you could accumulate detection statuses in a global or class-level variable within VideoProcessor. This accumulation could be a simple boolean flag or a counter that tracks the number of frames where the object was detected.
  • Instead of trying to perform an immediate rerun from within recv, consider having a button or a periodic timer (using st.button and some form of manual refresh) that checks this custom state and updates the UI accordingly. This requires user interaction but fits better with Streamlit’s execution model.
class VideoProcessor(VideoTransformerBase):
    def __init__(self) -> None:
        self.detect = False
        # Assuming self.model is properly initialized here

    def recv(self, frame: av.VideoFrame) -> av.VideoFrame:
        img = frame.to_ndarray(format="bgr24")
        # Assume your object detection logic is correctly implemented here

        # Instead of modifying session state, you might update a class-level attribute
        # For example:
        # if object_detected:
        #     self.detect = True

        return av.VideoFrame.from_ndarray(img, format="bgr24")

# This part remains mostly unchanged
webrtc_ctx = webrtc_streamer(key="example", video_processor_factory=VideoProcessor)

# You could have a button to check the detection status
if st.button("Check Detection Status"):
    if webrtc_ctx.video_processor:
        if webrtc_ctx.video_processor.detect:
            st.write("Object Detected!")
        else:
            st.write("No Object Detected Yet.")

Hope this helps!

Kind Regards,
Sahir Maharaj
Data Scientist | AI Engineer

P.S. Lets connect on LinkedIn!

➤ Want me to build your solution? Lets chat about how I can assist!
➤ Join my Medium community of 30k readers! Sharing my knowledge about data science and AI
➤ Website: https://sahirmaharaj.com
➤ Email: sahir@sahirmaharaj.com
➤ 100+ FREE Power BI Themes: Download Now

3 Likes
3

The solution works for that particular use case, but I want to automate the process so the user does not need to click the second button again. Is there any way to call the click function repeatedly?