Drawable canvas

Hey everyone, this should be fixed with Streamlit 0.84.1 streamlit/streamlit#3507 (comment). Iā€™ve tested on the demo app and it works well, so it should work on your side too!

Fanilo

Hey @andfanilo
I have a small issue I m trying to draw a polygon and use it as the ROI for detections in a video but I am not sure how exactly can I extract the points from the table as I am getting the following error

for this code st.dataframe(pd.json_normalize(canvas_result.json_data["objects"])) in drawing a polygon

ArrowTypeError: ("Expected bytes, got a 'int' object", 'Conversion failed for column path with type object')

2 Likes

Iā€™ve also had the same issue with the st.dataframe function.

Hey @Pavankunchala @aswincandra

Sorry for taking so long to answer, holidays :slight_smile:

Looks like a Streamlit Arrow upgrade problem, does it work if you downgrade Streamlit before version 0.85?

Fanilo

1 Like

TypeError: a bytes-like object is required, not ā€˜CanvasResultā€™

I am using 0.88 version

# Create a canvas component
image_data = st_canvas(
    b_width, b_color, bg_color, height=150, drawing_mode=drawing_mode, key="canvas"
)

# Do something interesting with the image data
if image_data is not None:
    st.image(image_data)

and the code is:

1 Like

I have a small issue I m trying to draw a polygon and use it as the ROI for detections in a video but I am not sure how exactly can I extract the points from the table as I am getting the following error for this code st.dataframe(pd.json_normalize(canvas_result.json_data["objects"])) in drawing a polygon ArrowTypeError: ("Expected bytes, got a 'int' object", 'Conversion failed for column path with type object')
I think the explanation is https://discuss.streamlit.io/t/all-in-on-apache-arrow/15342/3:

        if canvas_result.json_data is not None:
            objects = pd.json_normalize(canvas_result.json_data["objects"])
            for col in objects.select_dtypes(include=['object']).columns:
                objects[col] = objects[col].astype("str")
            st.dataframe(objects)
  • or change the serialization from arrow to legacy, by putting in .streamlit/config.toml the following:
[global]
dataFrameSerialization = "legacy"

@BeyondMyself the image is stored in image_data.image_data, the image_data variable contains 2 properties: image_data for the image and json_data for the canvas JSON representation.

Hi, for some reason when I use the drawable canvas, a noticeable lag occurs while updating in real time.

ezgif-2-7937ef0ff193

Could I optimize the code in a way so that such a lag is not noticed?

Hey @watchdog welcome to the forums :slight_smile:

  • Are you using this locally or on the demo app (https://share.streamlit.io/andfanilo/streamlit-drawable-canvas-demo/master/app.py)? Every drawing executes a full Streamlit script rerun which on larger apps can take some time depending on the specs of your machine (I see about half the lag on Streamlit Cloud and a third of the lag on my personal machine). You can disable the Streamlit rerun for every stroke by passing update_streamlit=False and updating by pushing the arrow in the toolbar if the lag really puts you off (actually you could even run custom code with Session State to pass update_streamlit to True every 5-10 strokes if you want).
  • Also right now I donā€™t know how exactly I could optimize the code more, maybe thereā€™s an iframe remounting I can optimize or some other advanced React useEffect that is not well implemented, but Iā€™d need to dig deep which I wonā€™t be able to do immediately :frowning:

Happy drawing :balloon:
Fanilo

Hi man, thank you for your awesome work!
When Iā€™m trying to run your default demo code, I got exception info like this:
StreamlitAPIException : (ā€œExpected bytes, got a ā€˜floatā€™ objectā€, ā€˜Conversion failed for column path with type objectā€™)
and it occurred here:
st.dataframe(pd.json_normalize(canvas_result.json_data[ā€œobjectsā€]))
Plus, when I uncheck update in realtime, thereā€™s no button called ā€œsend to streamlitā€.
Can you take a look?
Btw, Iā€™m using Steamlit 1.1.0

Hello @Sunix_Liu ! Thanks for the kind words

  • On the StreamlitAPIException, I think the correct code is
if canvas_result.json_data is not None:
    objects = pd.json_normalize(canvas_result.json_data["objects"]) # need to convert obj to str because PyArrow
    for col in objects.select_dtypes(include=['object']).columns:
        objects[col] = objects[col].astype("str")
    st.dataframe(objects)

because Streamlit uses Arrow to serialize dataframes, everything has to be cast out of objects. Is there a location I forgot to change this?

  • For the update in realtime part, the button is circled in red in the following

image

Do tell me if you have a better idea for a clearer icon :slight_smile:

Happy drawing :balloon:
Fanilo

@andfanilo, thank you for your prompt response.

I used the demo here:

I guess itā€™s yet to be updated.

Btw, that icon makes me think itā€™s download button :grinning_face_with_smiling_eyes:, so I never try it.

1 Like

Hi @andfanilo , actually I use it locally. By modifying the

update_streamlit=False

flag
I do not see a significant difference. I just find it strange that it has significant lag in contrast with the your demos. Regarding the Session State, is there any example showcasing its use for the canvas?

Hello! Thank you for providing Streamlit and guidance on how to use it, this package has really been a game-changer!
My question is about the centering of the images drawn in the drawing pad:
When I use the simple canvas method:

canvas_result = st_canvas(
fill_color=ā€œrgba(255, 165, 0, 0.3)ā€,
stroke_width=10,
stroke_color=ā€œBlackā€,
background_color=ā€œWhiteā€,
# width = 150,
height= 150,
key=ā€œcanvasā€,
)

I draw the image in the center; however, when I go to look at the image which I drew, it looks like this:

I should also mention I am using keras to reshape the data into to the size (600,600).

The goal was to make an app in which you could draw faces, and predict the emotions of the drawings; however, I donā€™t want the image to be split like this if the user decides to draw the image in the center of the drawing pad, which is the most likely place they will draw it.

If I draw slightly off to the right or left I get a centered image:

ā†’

1 Like

Hello @Paul_Fentress,

Hmmm that is odd.

  • When you st.image(canvas_result.image_data) what do you see?
  • Can you write the method you are using to reshape the image?
  • If you want you can put width=600 and height=600 to the st_canvas call, so your image already has the correct size.

That looks super cool, looking forward to the result!

Have a nice Streamlitinā€™ :balloon:
Fanilo

Hey Fanilo,
Thanks for the quick response!
I set the width and height to an already square shape like you suggested:

canvas_result = st_canvas(
stroke_width=10,
background_color=ā€œWhiteā€,
width = 300,
height= 300,
)

By doing this I am no longer reshaping the incoming data, and I get nice square images. Here are some examples of what I got after the images were augmented for training purposes, and this is exactly what I was trying to get:

This issue is resolved.

Thank you!

1 Like

image
Hi @andfanilo , Iā€™m using st_canvas but i cant display correct image (1080x1920). Please help me !!!

Thank you!! ^^

Is there a way I can override a drawing with a numpy array? It might be possible to structure it in the initial_drawing JSON form and then use it like this but that seems really complicated.

ty a lot for your awesome work btw! :slight_smile:

1 Like

Hey @foersterrobert !

Uhhh that looks a bit harder than I expected (I thought rastering an image to SVG).
What type of drawing are you envisioning to import, or type of use case you want to solve?

Fanilo

I created an MNIST-Drawer application once using your component. Now I also added the feature to generate MNIST-Digits using GANs and I would like to ā€œdrawā€ the result on the canvas. So technically itā€™s just a 28x28 tensor/array/image that I would like to resize to 280x280 pixels.

Thank you a lot btw.

1 Like

Hi, I am seeing a weird behavior when using the polygon tool. I ask users to draw a polygon on the background image in order to create a mask later on and there seem to be issues with the toolbar:

  1. if i close a polygon and then use ā€˜backā€™ arrow to remove a few vertices, it would be reasonable to expect that one could resume the polygon. Instead a new polygon is created, so now we have one that is not closed and one that is closed.
  2. some users tried to use ā€˜backā€™ arrow before they closed the polygon, which resulted in quite a mess - at first it looks like the vertices are removed, but when you try to add new ones, all the old ones reappear.
    any idea how difficult it is to solve this?