Drawable canvas

[andfanilo]

Do you like Quick, Draw ?

Well what if you could train/predict doodles drawn inside Streamlit ? Meet Drawable Canvas

Have fun . Oh and do tell me about possible improvements or bugs you encountered.

pip install streamlit-drawable-canvas

app.py

import streamlit as st
from streamlit_drawable_canvas import st_canvas

st.title("Drawable Canvas")
st.markdown("""
Draw on the canvas, get the image data back into Python !
* Doubleclick to remove the selected object when not in drawing mode
""")
st.sidebar.header("Configuration")

# Specify brush parameters and drawing mode
b_width = st.sidebar.slider("Brush width: ", 1, 100, 10)
b_color = st.sidebar.beta_color_picker("Enter brush color hex: ")
bg_color = st.sidebar.beta_color_picker("Enter background color hex: ", "#eee")
drawing_mode = st.sidebar.checkbox("Drawing mode ?", True)

# Create a canvas component
image_data = st_canvas(
    b_width, b_color, bg_color, height=150, drawing_mode=drawing_mode, key="canvas"
)

# Do something interesting with the image data
if image_data is not None:
    st.image(image_data)

[fatihkurtoglu]

Hi,

Thanks for the component. This is actually what I was looking for!
I was planning to experiment with a handwriting recognition tool with Streamlint.

I have an issue with the component, only 1/4 top-left section of the canvas appears on the image.
I pasted your code in this post (I’ve tried your code in the github as well), output is same.

When I inspect the canvas code, I saw the double sizes in the code for canvas element as follows:
It should be 600x150 but the in the html below it is 1200x300.

<div id="root"><div class="canvas-container" style="width: 600px; height: 150px; position: relative; user-select: none;"><canvas id="c" width="1200" height="300" class="lower-canvas" style="position: absolute; width: 600px; height: 150px; left: 0px; top: 0px; touch-action: none; user-select: none;"></canvas><canvas class="upper-canvas " width="1200" height="300" style="position: absolute; width: 600px; height: 150px; left: 0px; top: 0px; touch-action: none; user-select: none; cursor: crosshair;"></canvas></div></div>

[andfanilo]

Hey @fatihkurtoglu

It looks like an issue with the CSS pixel ratio (or devicePixelRatio) that I have yet to implement.
If your screen has a pixel ratio of 2:1 (like an apple screen) then I think I’m expecting this to be the behavior, I did not yet use this ratio to resize the output image from the component. Are you able to you confirm the devicePixelRatio of your screen ?

I’ve also updated the code in the post to be the same as the Github project.

EDIT : oh I can simulate a Retina display with the browser, i get the same behavior as yours I’ll try to debug it in the next days

[fatihkurtoglu]

Yes, you’re right. My device’s window.devicePixelRatio is 2.

[andfanilo]

Hey @fatihkurtoglu, I released a new build which disables Retina scaling, I don’t know if it will make things blurry on your side, I’d be grateful if you could test this !

[Tobias_Schiele]

Hi, thank you a lot for the component!

Hope i did not miss anything, but do you think it might be possible to have a “default” background image to manipulate / draw on?

This would be very helpful in creating human-in-the-loop models, e.g. where a model outputs some predicted segmentation map for unlabeled data, the user interactively cleans the segmentation map and retrains the model with the additional data.

I already tried to pass an encoded image to background_color, because of this answer but sadly i could not get it to work.

Kind regards and thank you for your awesome work again.

[andfanilo]

Hey @Tobias_Schiele, welcome to the community

This is also something I have in mind and tracked here. Don’t hesitate to track it! I’ll be giving it a shot this week.

Thanks for explaining your usecase, it helps design the API and maybe change the underlying canvas library XD. Right now my 2 thoughts on this:

• What would be the ideal Python type of the background image you want to send ?
• I think right now, if I implemented the setBackgroundImage, when you draw on the canvas it would send you the image + the drawing, I’m not sure that’s what we want. What’s your opinion on this ?

[Tobias_Schiele]

Hi, sorry for coming back so late on this.

• The ideal python type of the background image would be maybe a PIL.Image? If we send a raw numpy array, the component would have to guess the image type.
• The ultimate solution would be a layered approach. Along the lines of add_layer(img=None). For drawing, one would need to select the active layer Defaulting to the last layer or first layer would be enough in most use cases. The compent then always returns the active layer as a numpy array, like currently implemented. Not sure though if this layered approach is feasible to add retrospectively.

Thanks for your quick reply and your time!
Cheers.

[andfanilo]

Just released version 0.3.0 with:

• Straight lines and rectangles on the canvas. I guess labeled bounding box is next to enable image annotation then…
• Add a background image and draw over it @Tobias_Schiele
• Return the JSON representation of objects on the canvas, so you get get the coordinates of your objects in Streamlit

Try it: pip install -U streamlit-drawable-canvas

[andfanilo]

Release 0.4.0 :

• Circles !
• Argument to choose when to send back data back to Streamlit.

Have fun drawing in Streamlit!

[arindom]

Hi @andfanilo ,
I was trying out the Drawable canvas and tried out the code for the example shown above.
I have two questions:

1. canvas_result.image_data -> Is there a streamlit function to save this array as an image file (.jpg, png etc) ?
2. The numpy array’s shape is (100,100,4) where, 100 -> canvas size
   i.e. print(canvas_result.image_data.shape)
   Shouldn’t the shape be 100,100,3 , i.e. for 3 channels? I didn’t understand why this value is 4.

[andfanilo]

Hi @arindom! Sure !

The image_data is a numpy array with 3 color channel + 1 alpha channel…even though I haven’t implemented any opacity channel yet in the brush coloring so I guess the 4th channel should be 1 everywhere.
The HTML canvas sends me back a byte array of height x width x RGBA so by default I kept the Alpha channel. You can dump this channel in your application.

Pillow is installed by the package so :

from Pillow import Image
im = Image.fromarray(img_data)
im.save(file_path, "JPEG")

should work (though I haven’t tested )

Image fromarray
Image save

Fanilo

[Tobias_Schiele]

Awesome! Thanks a lot.

[Tobias_Schiele]

Just wanted to come back and congrat you on the latest realse of drawable-canvas. I could sucessfully build a labeling pipeline for our internal projects. Thank you so much.

[andfanilo]

Wow good to know! I’d love to hear more about your pipeline if that’s not too confidential

[andfanilo]

And I did not talk about the latest release here so here

Release 0.5.0:

• Undo/redo and clear canvas !
• If realtime_update is disabled, there’s a “send to Streamlit” button now

(welp I’ve lost the GIF so here is the GIF in the original tweet)

Fanilo Andrianasolo (andfanilo)

[banji007]

@andfanilo , how can I predraw some shapes on @streamlit canvas such as square, rectangle, traingle or some icon(say smartphone icon). I do not want the user to use his mouse to create these shapes on the canvas. Basically I want to draw these shapes automaticaly at random places in the canvas based on some other actions by user such as his age, gender, etc.

[Alexander_Hexemer]

Thanks for much for your code. I had a quick question. When I look at the output of the rectangular (json), it doesn’t seem to update the width and height if I use the transform function. The rectangular on the images is updated but not the value in the json.

[andfanilo]

Hey @Alexander_Hexemer,

Interesting could you bring a new issue (with some code snippet would be perfect!) to the Github repo? Issues · andfanilo/streamlit-drawable-canvas · GitHub

Thanks a lot!
Fanilo

[Alexander_Hexemer]

Hi Fanilo,
Thanks for the super-fast response. I just added a small snippet of code to show my issue. Thanks again for the help.
Cheer,s
Alex