New Component: Streamlit camera live input

Custom Components
1

Inspired by @Data_Dam’s question Streamlit Cloud + QR code - #3 by Data_Dam, I thought it would be nice to have a much-dumbed-down version of the incredible streamlit-webrtc component New Component: streamlit-webrtc, a new way to deal with real-time media streams, that just returned an image every N milliseconds, without any need for callbacks, for use in very basic cases like this. So, I created a new component to do just that. Hope some of you find it helpful!

Repo: GitHub - blackary/streamlit-camera-input-live: Alternative version of st.camera_input which returns the webcam images live, without any button press needed
Package: streamlit-camera-input-live · PyPI
Example app: https://camera.streamlitapp.com/

4 Likes
2

:balloon: awesome, and you added it to the tracker! I really should clean and organize that tracker too…

Have a nice day,
Fanilo

3

Hi @blackary,

Thanks for putting in the work to create something like this!

I do have two things to say about it:

  • The camera works extremely slow. Sometimes it doesn’t work at all (it freezes). And I try it with the fastest wifi, newest MacBook Pro/iPhone. So I think there is something wrong with the way the component is build
  • It is very important that the component uses the smartphones back camera in stead of the front camera. In that way it can be used as a code scanner.

Like to hear what you think about this!

Greetings,

Kris

4

Hi @Data_Dam,

This component is set to update every 1s by default, but if you build your own app you can make it update faster by doing camera_input_live(debounce=100) (which will make it update every 100ms instead of every 1000ms. Can you figure out a way to reproduce the camera freezing entirely (other than pressing the pause button)? I haven’t seen that happen yet, but it’s definitely possible. To be clear, I’m not intending this to be the optimal QR Code reader app, I mostly wanted to make a replacement to st.camera_input that allows you to get the images back right away, rather than having to press “save”, so that you, and others, can use it to build interesting apps.

Adding the ability to switch to back camera is a great idea. I’ll see if I can figure out a way to add that option.

5

Hi blackary,

Very nice that you are also excited about making a QR scanner.

About the frame rate: Is it possible that the frame rate is as high as this example? → https://whitphx-streamlit-webrtc-example-app-fi5ecr.streamlitapp.com

Even though I try to do this “camera_input_live(debounce=100)”, it still seems lagging.

And about the back camera: Would be really great if we could use the back camera as the standard camera, so without having to manually switch to back camera!

Greetings,

Kris

6

No, you won’t easily to be able to get similar performance to webrtc with this component. If you’re looking for performance, that one will always beat this one. This is meant to be a much simpler (and non-callback-based) component to use (see New Component: streamlit-webrtc, a new way to deal with real-time media streams - #103 by lorenzweb for an example of what the code for a bar code detector looks like, as opposed to streamlit-camera-input-live/streamlit_app.py at main · blackary/streamlit-camera-input-live · GitHub.

So, it’s simpler to use, but if you want to build a better performing, more-real-time app, webrtc is your best bet.

Once I get around to adding switching cameras, I’m planning to add an argument that lets you default to the back camera.

1 Like
7

Great camera live input, I’m developing an automatic catalog for a library and it looks like a good starting!
Two questions: Is there a way to freeze the image automatically when the QR is already recognized? I see there’s some code in ini.py (startLabel=start_label and stopLabel=stop_label) but I dont know how to make them work, maybe something like this?


    if data:
        st.write(data)
        camera_input_live(stopLabel)

or even increasing the debounce

    if data:
        st.write(data)
        camera_input_live(debounce=50000)

Well I have tried a lot of things but nothing seems to work for me.
About the rest:

update every 100ms
–just fine, its a QR code reader, not Netflix :wink:

smartphones back camera
–Great idea! looking forward to it!

8

Right now if you pass show_controls=True it will show a stop button, but it won’t be called automatically.

Here’s one way to do it using st.session_state to keep track of whether a QR code has been found, and to save the image once one is found. You could also add a “reset” button that clears those items from session state.

import cv2
import numpy as np
import streamlit as st
from camera_input_live import camera_input_live

"# Streamlit camera input live Demo"
"## Try holding a qr code in front of your webcam"

if "found_qr" not in st.session_state:
    st.session_state.found_qr = False

if "qr_code_image" not in st.session_state:
    st.session_state.qr_code_image = None

if not st.session_state["found_qr"]:
    image = camera_input_live()
else:
    image = st.session_state.qr_code_image

if image is not None:
    st.image(image)
    bytes_data = image.getvalue()
    cv2_img = cv2.imdecode(np.frombuffer(bytes_data, np.uint8), cv2.IMREAD_COLOR)

    detector = cv2.QRCodeDetector()

    data, bbox, straight_qrcode = detector.detectAndDecode(cv2_img)

    if data:
        st.session_state["found_qr"] = True
        st.session_state["qr_code_image"] = image
        st.write("# Found QR code")
        st.write(data)
        with st.expander("Show details"):
            st.write("BBox:", bbox)
            st.write("Straight QR code:", straight_qrcode)
1 Like
9

Works great! When I’ve got the QR it stops and let me fill a st form (library stuff).
One more question: Is there a way to hide the “start/stop” button?

10

Yes! Just pass show_controls=False when you call camera_input_live

11

While waiting for the author’s update, I set the rear camera as default by editing this function in main.js

function onRender(event) {
  // Only run the render code the first time the component is loaded.
  if (!window.rendered) {
    // You most likely want to get the data passed in like this
    //const {height, width, debounce, showControls, startLabel, stopLabel} = event.detail.args
    var {height, width, debounce, showControls, startLabel, stopLabel} = event.detail.args;

    if (showControls) {
      Streamlit.setFrameHeight(45)
    }

    var device = "destop";
    const ua = navigator.userAgent;
    if (/(tablet|ipad|playbook|silk)|(android(?!.*mobi))/i.test(ua)) {device = "tablet";}
    if (/Mobile|iP(hone|od)|Android|BlackBerry|IEMobile|Kindle|Silk-Accelerated|(hpw|web)OS|Opera M(obi|ini)/.test(ua)) {device="mobile";}

    if (device == "desktop"){height = 3*width / 4;}
    if (device == "mobile" || device == "tablet"){height = 16*width / 9;}
    

    let video = document.getElementById('video');
    let canvas = document.getElementById('canvas');
    let button = document.getElementById('button');

    let stopped = false;

    video.setAttribute('width', width);
    video.setAttribute('height', height);
    canvas.setAttribute('width', width);
    canvas.setAttribute('height', height);

    function takepicture() {
      if (stopped) {
        return;
      }
      let context = canvas.getContext('2d');
      canvas.width = width;
      canvas.height = height;
      context.drawImage(video, 0, 0, width, height);

      var data = canvas.toDataURL('image/png');
      sendValue(data);
    }


    function stopVideo() {
      video.pause();
      video.srcObject.getTracks()[0].stop();
      stopped = true;
    }

    function startVideo() {
      navigator.mediaDevices.getUserMedia({video: {width: 1280, height:  720, facingMode:'environment'}})
        .then(function(stream) {
          video.srcObject = stream;
          video.play();
        })
        .catch(function(err) {
          console.log("An error occurred: " + err);
        });
    }

    function toggleVideo() {
      if (stopped) {
        startVideo();
        stopped = false;
      } else {
        stopVideo();
        stopped = true;
      }
      // Toggle the button text
      button.textContent = stopped ? startLabel : stopLabel;
    }

    if (navigator.mediaDevices.getUserMedia) {
      navigator.mediaDevices
        .getUserMedia({ video: {width: 1280, height: 720, facingMode:'environment' }})
        .then(function (stream) {
          video.srcObject = stream;
        })
        .catch(function (error) {
          console.log("Something went wrong!");
          console.error(error);
        });
    }

    button.addEventListener('click', toggleVideo);
    button.textContent = stopped ? startLabel : stopLabel;

    takepicture();
    setInterval(takepicture, debounce);
    window.rendered = true
  }
}
1 Like
12

Hello. May I ask where would you place the main.js file so that it defaults to the back camera?

13

In frontend streamlit-camera-input-live/src/camera_input_live/frontend at main · blackary/streamlit-camera-input-live · GitHub

Streamlit_back_camera_input with default rear camera:
Github: GitHub - phamxtien/streamlit_back_camera_input: Streamlit camera input with back camera as default
Pypi: streamlit-back-camera-input · PyPI
Demo: https://phamxtien-stream-scrsrtreamlit-back-camera-input--init---era7wa.streamlit.app/

QRCode scanner for phones with default rear camera:
Github: GitHub - phamxtien/streamlit_qrcode_scanner
Pypi: streamlit-qrcode-scanner · PyPI
Demo: https://phamxtien-streamlit-qrcode-scanner---init---ibk12k.streamlit.app/

1 Like