Drawable canvas

Do you like Quick, Draw ?

Well what if you could train/predict doodles drawn inside Streamlit ?

Have fun :upside_down_face: . Oh and do tell me about possible improvements or bugs you encountered.

pip install streamlit-drawable-canvas
app.py
import streamlit as st
from streamlit_drawable_canvas import st_canvas

st.title("Drawable Canvas")
st.markdown("""
Draw on the canvas, get the image data back into Python !
* Doubleclick to remove the selected object when not in drawing mode
""")
st.sidebar.header("Configuration")

# Specify brush parameters and drawing mode
b_width = st.sidebar.slider("Brush width: ", 1, 100, 10)
b_color = st.sidebar.beta_color_picker("Enter brush color hex: ")
bg_color = st.sidebar.beta_color_picker("Enter background color hex: ", "#eee")
drawing_mode = st.sidebar.checkbox("Drawing mode ?", True)

# Create a canvas component
image_data = st_canvas(
    b_width, b_color, bg_color, height=150, drawing_mode=drawing_mode, key="canvas"
)

# Do something interesting with the image data
if image_data is not None:
    st.image(image_data)
7 Likes

Hi,

Thanks for the component. This is actually what I was looking for!
I was planning to experiment with a handwriting recognition tool with Streamlint.

I have an issue with the component, only 1/4 top-left section of the canvas appears on the image.
I pasted your code in this post (I’ve tried your code in the github as well), output is same.

When I inspect the canvas code, I saw the double sizes in the code for canvas element as follows:
It should be 600x150 but the in the html below it is 1200x300.

<div id="root"><div class="canvas-container" style="width: 600px; height: 150px; position: relative; user-select: none;"><canvas id="c" width="1200" height="300" class="lower-canvas" style="position: absolute; width: 600px; height: 150px; left: 0px; top: 0px; touch-action: none; user-select: none;"></canvas><canvas class="upper-canvas " width="1200" height="300" style="position: absolute; width: 600px; height: 150px; left: 0px; top: 0px; touch-action: none; user-select: none; cursor: crosshair;"></canvas></div></div>

Hey @fatihkurtoglu

It looks like an issue with the CSS pixel ratio (or devicePixelRatio) that I have yet to implement.
If your screen has a pixel ratio of 2:1 (like an apple screen) then I think I’m expecting this to be the behavior, I did not yet use this ratio to resize the output image from the component. Are you able to you confirm the devicePixelRatio of your screen ?

I’ve also updated the code in the post to be the same as the Github project.

EDIT : oh I can simulate a Retina display with the browser, i get the same behavior as yours :wink: I’ll try to debug it in the next days

Yes, you’re right. My device’s window.devicePixelRatio is 2.

Hey @fatihkurtoglu, I released a new build which disables Retina scaling, I don’t know if it will make things blurry on your side, I’d be grateful if you could test this !

1 Like

Hi, thank you a lot for the component!

Hope i did not miss anything, but do you think it might be possible to have a “default” background image to manipulate / draw on?

This would be very helpful in creating human-in-the-loop models, e.g. where a model outputs some predicted segmentation map for unlabeled data, the user interactively cleans the segmentation map and retrains the model with the additional data.

I already tried to pass an encoded image to background_color, because of this answer but sadly i could not get it to work.

Kind regards and thank you for your awesome work again.

Hey @Tobias_Schiele, welcome to the community :slight_smile:

This is also something I have in mind and tracked here. Don’t hesitate to track it! I’ll be giving it a shot this week.

Thanks for explaining your usecase, it helps design the API and maybe change the underlying canvas library XD. Right now my 2 thoughts on this:

  • What would be the ideal Python type of the background image you want to send ?
  • I think right now, if I implemented the setBackgroundImage, when you draw on the canvas it would send you the image + the drawing, I’m not sure that’s what we want. What’s your opinion on this ?

Hi, sorry for coming back so late on this.

  • The ideal python type of the background image would be maybe a PIL.Image? If we send a raw numpy array, the component would have to guess the image type.
  • The ultimate solution would be a layered approach. Along the lines of add_layer(img=None). For drawing, one would need to select the active layer Defaulting to the last layer or first layer would be enough in most use cases. The compent then always returns the active layer as a numpy array, like currently implemented. Not sure though if this layered approach is feasible to add retrospectively.

Thanks for your quick reply and your time!
Cheers.

Just released version 0.3.0 with:

  • Straight lines and rectangles on the canvas. I guess labeled bounding box is next to enable image annotation then…
  • Add a background image and draw over it @Tobias_Schiele
  • Return the JSON representation of objects on the canvas, so you get get the coordinates of your objects in Streamlit

Try it: pip install -U streamlit-drawable-canvas

Release 0.4.0 :

  • Circles !
  • Argument to choose when to send back data back to Streamlit.

Have fun drawing in Streamlit!

3 Likes

Hi @andfanilo ,
I was trying out the Drawable canvas and tried out the code for the example shown above.
I have two questions:

  1. canvas_result.image_data -> Is there a streamlit function to save this array as an image file (.jpg, png etc) ?
  2. The numpy array’s shape is (100,100,4) where, 100 -> canvas size
    i.e. print(canvas_result.image_data.shape)
    Shouldn’t the shape be 100,100,3 , i.e. for 3 channels? I didn’t understand why this value is 4.

Hi @arindom! Sure !

The image_data is a numpy array with 3 color channel + 1 alpha channel…even though I haven’t implemented any opacity channel yet in the brush coloring :slight_smile: so I guess the 4th channel should be 1 everywhere.
The HTML canvas sends me back a byte array of height x width x RGBA so by default I kept the Alpha channel. You can dump this channel in your application.

Pillow is installed by the package so :

from Pillow import Image
im = Image.fromarray(img_data)
im.save(file_path, "JPEG")

should work (though I haven’t tested :wink: )

Image fromarray
Image save

Fanilo

1 Like

Awesome! Thanks a lot.