New Component: streamlit-webrtc, a new way to deal with real-time media streams

@Joe_Tay Hi, I found that this is a problem having been discussed in the issue tracker of aiortc, which streamlit-webrtc depends on: How to change video stream size written by MediaRecorder? · Issue #183 · aiortc/aiortc · GitHub
I will investigate this and want to find a solution, but currently have no idea.

I created an issue, Unable to change the frame size · Issue #438 · whitphx/streamlit-webrtc · GitHub to track this.

New answer here: New Component: streamlit-webrtc, a new way to deal with real-time media streams - #74 by whitphx

1 Like

hey @whitphx welcome

Hello, Good day.

I tried to set up with my Apple M1 device however, I have errors regarding with Failed to build aiortc av pylibsrtp

Hope someone can help me to understand this error (: Thank you

Thank you

@Joe_Tay @nerilou.delacruz
My previous answer was wrong. Being unable to set the frame size is for the case of recorder, but not processor.

You can configure the input resolution as below:

        "video": {
            "width": 1280,

or for finer controls:

        "video": {
            "width": {"min": 800, "ideal": 1200, "max": 1920 },

media_stream_constraints parameter is passed to navigator.mediaDevices.getUserMedia() on the frontend code to configure the media devices.
This page, for example, explains the details of that API. If you want more controls, this page can be a hint.

Note that the media_stream_constraints setting is a instruction for the browser to configure the media devices, but the browser sometimes does not follow it.

@nerilou.delacruz It’s simply a problem of these libraries related to M1 chip, not this component, IMO.
These libs contain C code and have to be built at installation, so may have problems on M1 platform.

Just for a reference, I paste my recipe on M1 mac here restored from my memory and some documents.
pyenv installed via homebrew is required.

$ arch -arch x86_64 env PATH=${PATH/\/opt\/homebrew\/bin:/} pyenv install 3.8.7
$ pip install -U pip

The point is to install the python in an x86-emulated environment. arch command does it.
Setting the PATH envvar is to use the x86 version of the Python dependency libraries installed via homebrew, during pyenv install.
Using the latest version of pip may also be a point.

Note that I haven’t tested this instruction from top to bottom. There can be a missing part.
And I don’t support it. Please use this info just as a clue.

1 Like

I just installed streamlit v.1.0 and want to use simply the webcam. So I tried the provided snippet from the newest blog entry:

from webcam_component import webcam

captured_image = webcam()
if captured_image is not None:

It says: “ModuleNotFoundError: No module named ‘webcam_component’”. I installed your package with pip install streamlit-webrtc. It still does not work somehow…

This is the blog post: Announcing Streamlit 1.0! 🎈 right?

Anyway, the sample code in it is not about this streamlit-webrtc component, but just shows an example of a usage of a third-party component.

Forwarding to @Adrien_Treuille who is probably the author of that blog post.

1 Like

Then it is a bit misleading because it looks like that a webcam can be used with just those lines of code. Anyway, I just used your repo :wink: Just another question, is it possible to access the streamlit session state from inside of the callback function of VideoProcessor?

is it possible to access the streamlit session state from inside of the callback function of VideoProcessor?

It’s impossible now.
However, it’s also an interesting point - I will investigate and think of the way to make it possible in the future versions.

1 Like

After some survey, I think it won’t be implemented because it seems to require not intuitive hacks to create.
However, as inspired from this conversation, I’m thinking of introducing a new state object like session_state shared both by inside and outside the callbacks.

For example, we will be able to use the shared state like below.

from streamlit_webrtc import shared_state

class VideoProcessor:
    def recv(self, frame):
        img = frame.to_ndarray(...)
        threshold = shared_state["threshold"]
        return av.VideoFrame.from_ndarray(img, ...)


shared_state["threshold"] = st.slider("Threshold", ...)

Or, if such a shared_state is introduced, class-based video processors will no longer be needed and callables might be in that place as below.

from streamlit_webrtc import shared_state

def process_frame(frame):
    img = frame.to_ndarray(...)
    threshold = shared_state["threshold"]
    return av.VideoFrame.from_ndarray(img, ...)


shared_state["threshold"] = st.slider("Threshold", ...)

Do you think such design is helpful for your needs?
Can you tell me your use case where you want to pass the session_state directly to the callbacks?

And not only him but also everyone, please give me your opinion!

@ whitphx Thanks a lot for the great work, keep it up.

I was wondering if I can use the async def recv_queued function to provide a batch of frames to be processed by the model at once, i.e i will delay the stream by n secs, to be able to batch process it

is this applicable, and if not, what is your recommendation for providing the DL model a batch of frames at once instead of one frame at a time?

@ahmedanis03 I think it’s possible though I haven’t actually tested it.
It would be a kind of extended versions of the delayed echo example.

For example,

  • Assume the input video is 10 FPS.
  • When recv_queued() is called first time, the input frames contain the first frame (probably len(frames) == 1).
    If you call await asyncio.sleep(1) inside recv_queued() and returns the frames from the callback, these frames are displayed with 1 sec delay.
  • When recv_queued() is called next time, the input frames contain frames that have arrived after the previous frames. Probably len(frames) == 10 because the previous recv_queued() took 1sec in which 10 frames should have arrived.
    So, now you can process these 10 frames at one time.
    If this batch processing takes 1 sec again, the same thing continues in the successive calls…
1 Like

@whitphx Just, maybe a simple question, when I am deploying the app on streamlit cloud it does not render the output of the webcam. Instead I get only a black screen. On localhost it works smoothly but on remote host it returns the black screen output.


import streamlit as st
import av
import cv2
from streamlit_webrtc import webrtc_streamer
from PIL import Image


How can I fetch the video output when I am deploying the app on e.g. streamlit cloud?

I think it’s because WebRTC connection cannot be established in your network environment due to for example network topology, firewall, etc.
This issue comment might be a solution.

BTW, this is a frequently asked question, so I’m planning to write a document about it.GitHub - whitphx/streamlit-webrtc: Real-time video and audio streams over the network, with Streamlit.


I published a new tutorial for real-time video app development using streamlit-webrtc (on my new blog :slight_smile: ):

( Crosspost to Developing web-based real-time video/audio processing apps quickly with Streamlit - DEV Community )

1 Like

It’s now working, thank youuuu :clap: :raised_hands:

Hi everyone I need to mute and unmute the audio of the video streaming after making an action.
Any suggestions on how can I do this ?

How about this?

import streamlit as st
from streamlit_webrtc import webrtc_streamer, VideoHTMLAttributes

muted = st.checkbox("Mute")

        autoPlay=True, controls=True, style={"width": "100%"}, muted=muted

You can configure the <video> element’s attribute via the video_html_attrs argument,
and the muted attribute switches mute/unmute on the client-side.

Note that other attributes such as autoPlay and style are specified too in the example above.
This default set of the video attribute values is used when nothing is provided for the video_html_attrs argument,
but if you override it, you have to explicitly provide these necessary attributes.

Thank for the reply @whitphx!
Any chance to do it without using checkboxs
Like calling it from a function or something like that
Thanks in advance