I made streamlit-webrtc, which sends and receives video (and audio, but it’s only partially supported now) streams between frontend and backend via WebRTC.
This should be helpful for creating, for example, performant real-time computer vision WebApps.
This week I tried Steamlit, found it interesting and useful, and then wanted to run real-time computer vision apps on Streamlit, on which users can try some computer vision models with video input from their webcams.
So I tried to develop a component to achieve it using WebRTC. Fortunately, there is a great WebRTC library for Python, aiortc.
One interesting thing about this component is that the input video streams are sourced from users’ webcams and transferred to the server-side Python process via the network. Therefore, the server does not need the access to the camera, unlike the usual approach using OpenCV (
cv2.VideoCapture). It means the Python code can be hosted on a remote server (actually, I hosted a sample app on Heroku and it worked, as stated below).
In addition, WebRTC provides good performance with sending and receiving video/audio frames.
Here is a demo movie:
(A full version is hosted on YouTube)
You can see
- The app consumes, processes, and renders video frames in real time.
- Streamlit’s interactive controls are still working well in combination with the code using WebRTC. For example, the threshold for object detection is changed interactively during execution.
You can try out the sample app using the following commands.
$ pip install streamlit-webrtc opencv-python $ streamlit run https://raw.githubusercontent.com/whitphx/streamlit-webrtc-example/main/app.py
I also deployed it to Heroku, https://streamlit-webrtc-example.herokuapp.com/ and confirmed it worked; however, it’s running on a very small instance on a free plan, and may not work well if multiple users access it at the same time.
I recommend you to run it on your own environment.
It’s still a prototype and the API is not finalized, and the documentation has not been written. Please refer to the sample code when you use the component.
I welcome any feedback!