Technical note:
One technical challenge was to combine WebRTC and real-time processing with Streamlit’s execution model. This problem has already been stated in Access Webcam Video from a hosted Streamlit application.
In case of streamlit-webrtc , additionally, aiortc does not work in normal Streamlit scripts, which are executed from top to bottom and terminated at each user interaction (more fundamentally, it’s because of WebRTC’s async nature).
This library approaches this problem by creating threads independent of Streamlit execution.
When the component is used for the first time, it forks a thread, launches aiortc’s code on an event loop in the thread, and maintains the thread over all the script executions until the WebRTC session is closed.
As a result of this approach, developers should alter their programming style when using this library, from Streamlit’s declarative way to an event-and-callback-based one. streamlit-webrtc/app.py at 477416da84e77d76201d97269589fe5ff5d17d37 · whitphx/streamlit-webrtc · GitHub is an example. In this code, the computer vision part resides in a callback function, which is called from the forked thread triggered by the media stream regardless of Streamlit’s execution timing.
To bridge this forked execution of WebRTC and normal Streamlit’s controls, streamlit-webrtc exposes a context object, which is an interface to the objects running in the WebRTC’s async world.
By using it, values from Streamlit’s controls can be passed to the computer vision code running in the forked thread, like streamlit-webrtc/app.py at 477416da84e77d76201d97269589fe5ff5d17d37 · whitphx/streamlit-webrtc · GitHub.