Need Help with Streamlit: Updating/Displaying Counter Value from VideoTransformerBase

[infernoyt213]

Summary

Issue Description

I’m working on a Streamlit app that involves real-time video processing using the VideoTransformerBase class. The app is designed to track hand movements and count repetitions. I’ve successfully implemented the counting logic within the transform method of the VideoTransformerBase class, and I’m trying to display the updated counter value on the Streamlit app interface.

Problem

The counter value (self.right_counter) is correctly updating within the transform method, but I’m having trouble displaying this updated value on the Streamlit interface. I’ve tried using various Streamlit display functions like st.text, st.write, and even direct HTML rendering using st.markdown, but none of them seem to work within the transform method.

What I’ve Tried

1. Using st.text, st.write, and st.markdown to display the counter value.
2. Placing the Streamlit display code both inside and outside the transform method.
3. Using st.session_state to store the counter value and attempting to display it.

Reproducible Code Example

import streamlit as st
from streamlit_webrtc import VideoTransformerBase, webrtc_streamer, WebRtcMode, VideoHTMLAttributes

class PoseEstimationTransformer(VideoTransformerBase):
    def __init__(self):
        self.right_counter = 0

    def transform(self, frame):
        # Your pose estimation logic here
        # ...

        # Update right counter
        if right_condition:
            self.right_counter += 1

        return frame

def main():
    transformer = PoseEstimationTransformer()
    st.title("Workout Buddy")

    # Display the right curl counter value
    st.text(f"Right Curl Counter: {transformer.right_counter}")

webrtc_ctx = webrtc_streamer(
    key="object-detection",
    mode=WebRtcMode.SENDRECV,
    video_processor_factory=PoseEstimationTransformer,
    media_stream_constraints={"video": True, "audio": False},
    video_html_attrs=VideoHTMLAttributes(
        autoPlay=True, controls=False, style={"width": "100%"},
    ),
)

if __name__ == "__main__":
    main()

Steps To Reproduce

1. Open the Streamlit app.
2. Allow the app to access your webcam for video input.
3. Observe the video feed displayed on the app.
4. Perform specific hand movements that should trigger the counter update according to the app’s functionality.
5. Monitor the Streamlit app’s interface to see if the updated counter value (self.right_counter) is being displayed correctly.
6. Check if the counter value updates as expected with each repetition of the hand movement.
7. Observe any changes or lack of changes in the displayed counter value.

Expected Behavior

I expect that when I perform the specific hand movements that trigger the counter update, the updated counter value (self.right_counter) should be displayed on the Streamlit app’s interface in real-time. The counter value should increase with each repetition of the hand movement, and I should be able to see the changes reflected on the app.

Current Behavior

Currently, when I perform the hand movements to trigger the counter update, the self.right_counter value updates correctly within the transform method. However, I am facing difficulty in displaying this updated counter value on the Streamlit app’s interface. I have tried using various Streamlit display functions like st.text, st.write, and even direct HTML rendering using st.markdown, but none of them seem to work within the transform method. As a result, I’m unable to see the real-time changes of the counter value on the app interface.

Debug info

• Streamlit version: 1.25.0
• Python version: 3.8.0
• Operating System: Windows 11 22h22
• Browser: Google Chrome Version 116.0.5845.97 (Official Build) (64-bit)

Requirements file

streamlit==1.25.0
streamlit-webrtc==0.44.7

[dataprofessor]

Hi @infernoyt213

Thanks for a very detailed and well written question. Looking at the code, it seems that your counter was defined as a local variable inside the defined function. Could you try saving the counts to a global variable such as the Streamlit’s session state.

More details and code snippets in the following Docs page:

Session State - Streamlit Docs

Add statefulness to apps - Streamlit Docs

Consider the following example code snippet (from Example 1 of the above mentioned Docs page: Add statefulness to apps - Streamlit Docs) as a starting point for adapting for your app:

import streamlit as st

st.title('Counter Example')
if 'count' not in st.session_state:
    st.session_state.count = 0

increment = st.button('Increment')
if increment:
    st.session_state.count += 1

st.write('Count = ', st.session_state.count)

Hope this helps!