Drawing boxes on images, capturing clicks, and capturing key strokes for labelling/cleaning an image dataset

[Joshuaalbert]

I’m trying to make an interface for labelling/cleaning an image dataset, and would like to use streamlit to built a simple responsive interface for rapidly cycling through images and interacting with them.

I’d like to load in a image and then draw boxes on it, after each box is drawn to push a key to label that box. Is this possible from streamlit?

[PeterT]

Hi @Joshuaalbert

I’m fairly certain this is possible using the excellent drawable canvas by @andfanilo - see the forum thread and the github page.

Best,
Peter

[andfanilo]

Hey @Joshuaalbert

As @PeterT said, the drawing box + label can be approximated from this demo https://share.streamlit.io/andfanilo/streamlit-drawable-canvas-demo/master/app.py in the “Color based image annotation” page (here’s a link to source code streamlit-drawable-canvas-demo/app.py at d4293636f89861c7f4d85e5a9c73f8575e6303e9 · andfanilo/streamlit-drawable-canvas-demo · GitHub).

For the pagination part you will probably need a mix of Streamlit button + Streamlit Session State + callback to store the page number in it and increment it on button click, then clear the drawing, load the correct image as background to the canvas and start drawing again.

So yes it is possible read the docs carefully, try to bring features little by little and you should be able to build it!

Good luck! Keep us posted!
Fanilo

[Joshuaalbert]

Thanks both for the quick feedback. I’m going to try this out and get back to you.