A component to annotate images (bounding boxes, keypoints, coordinates)

A specific component for this would make streamlit a really powerful annotation tool.
Standard annotation tools work fine but aren’t easy to customize, and customizing your workflow is the key to efficient data annotation.

Unfortunately I don’t know enough JS to help build this, but would gladly help anyone with testing and feedback.

1 Like

Hey @rjtavares, welcome to the community,

I was thinking about this recently and tried to look for JS librairies to help, tell if one of them answers your need :slight_smile: :

I’m not into the image annotation domain so I’m wondering : what kind of custom workflow are you looking for ?

Those are great. MarkerJS is basically what I need.

I collect sports data from broadcast footage. A lot of that is identifying pitch lines for perspective transformation and detecting objects (the players).

For that, I need to integrate automatic detection with manual input. E.g. suggesting pitch lines but allowing you to make corrections.

So, I made a tkinter GUI for that. But Streamlit would be so much easier to improve and adapt (especially a bunch of dedicated scripts for parts of the workflow).

For example, now I’m working with a different camera perspective and the line angles are different than usual. I could add a parameter for that.

Wrapping the Labelstudio frontend in a bidirectional component would be great as well - it’s quite customizable for various annotation tasks (keypoints, classification, bounding boxes ,segmentation, pairwise etc …). I gave creatign the component a try but am failing quite miserably - not really a frontend guy :stuck_out_tongue:. Any help would be appreciated!

In the meantime I’ve tried adding background image, bounding box / lines and send back coordinates of objects on canvas as JSON to Streamlit Drawable Canvas. It’s a first step, when I have some time I’ll add a labeled bounding box. See if it answers part of your needs and what other use cases you would want to be added :slight_smile:

I wrote a big component tutorial here. Did you try this? I’m open to feedback on this too!

Awesome! I think that’s enough for my use case. I’ll try it out and let you know if it works.

My test worked perfectly (https://twitter.com/lastrowview/status/1300789058113863688). A few tweaks and I’ll publish the app.

Ahahah I love it, this is so unexpected yet so great :rofl: so I guess I’ll have to add circles, loading a drawing and point snapping one day

Considering the interest my tweet generated (including from a lot of professionals in the sport), now I got to release it, so I’ll be watching closely the developments in your component. I’ll also try to contribute to it, but my JS skills are non existent, so it’s probably going to be limited help.

1 Like

You know, just putting new features in the issues tracker is very helpful :wink: will be waiting for your post in Show the community then !

Wait no more!


Hey @rjtavares I added a tool to draw circles (felt bad for those players crossed by lines ahah) and an argument to prevent Streamlit update everytime you release the mouse, which greyed the canvas a bit too much in my opinion when you draw on the canvas with a heavy image.

Check the updated README :wink: and download the latest version!