Combine video input and chatbot

tf123 · January 22, 2024, 10:43am

Let me give a brief about my app. I am trying to build a small app that can take a live image and a text as an input and apply some prediction based on that.

The app is supposed to be deployed on my university server. As of now, I am still developing it, so I am running the program on a server and by port forwarding, and testing on my system.

I have the models ready, but I need to develop the front-end part to take user text and frame as input.

I have come across st.camera_input, but I am not able to integrate it such that the live video feed is not shown and it should directly capture frame as soon as the user types something in the textbox and presses enter.

Can someone please help me out on this? I am very new to streamlit and just started learning and using it. I am using streamlit=1.30.0 and python=3.9.18

Caroline · February 2, 2024, 3:07pm

Hi @tf123,

From the description of your use case, it sounds like the streamlit-webrtc component might be helpful – check it out here

tf123 · February 3, 2024, 3:57am

Is it possible to capture a single frame when the user clicks in “Enter” in the chat? I am thinking in this direction to avoid computational overhead by live streaming the entire feed.

dataprofessor · February 3, 2024, 7:43am

Hi @tf123

You could use the streamlit-webrtc library that @Caroline had mentioned to accept the video feed, and see if you can limit the time allocation (perhaps using the time module) for this component so that it consume less bandwidth.

tf123 · February 10, 2024, 5:05pm

I tried creating a very simple app as given on the github page and I am getting this error.

The code for app

from streamlit_webrtc import webrtc_streamer

webrtc_streamer(key="sample")

The error:

Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.679631956 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.795399997 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.815933422 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.836423066 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.856991508 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.877526022 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.898057376 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
Exception in callback Transaction.__retry()
handle: <TimerHandle when=4778449.918592512 Transaction.__retry()>
Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1054, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/stun.py", line 312, in __retry
    self.__protocol.send_stun(self.__request, self.__addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/site-packages/aioice/ice.py", line 266, in send_stun
    self.transport.sendto(bytes(message), addr)
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 1064, in sendto
    self._fatal_error(
  File "/home/cs20btech11063/anaconda3/envs/honors_env/lib/python3.9/asyncio/selector_events.py", line 711, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'```

system · February 15, 2024, 2:27pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
New Component: streamlit-webrtc, a new way to deal with real-time media streams streamlit-webrtc real-time , computer-vision , video	192	66519	March 13, 2025
Live webcam feed into the web app Using Streamlit video	8	21782	March 23, 2022
Developing a streamlit-webrtc component for real-time video processing Show the Community!	5	3275	February 10, 2023
Streamlit WebRTC questions streamlit-webrtc	2	564	May 16, 2023
Access Webcam Video from a hosted Streamlit application Using Streamlit computer-vision , opencv , video	19	19963	September 14, 2022

Combine video input and chatbot

Related topics

Hello there 👋🏻

Cookie settings

Strictly necessary cookies

Performance cookies

Functional cookies

Targeting cookies