streamlit_webrtc supports multiple connections as long as the server has enough CPU cores.
The video chat sample below is the example about it - multiple users can access the server at the same time.
Apart, I have an experience that multiple sessions do not work when the streamlit_webrtc server uses MediaPipe and I suspect that only single MediaPipe object can exist in a single process. I solved this problem by using multiprocessing to create a process-isolated MediaPipe object for each user as below.