A component to annotate images (bounding boxes, keypoints, coordinates)

[rjtavares]

A specific component for this would make streamlit a really powerful annotation tool.
Standard annotation tools work fine but aren’t easy to customize, and customizing your workflow is the key to efficient data annotation.

Unfortunately I don’t know enough JS to help build this, but would gladly help anyone with testing and feedback.

[andfanilo]

Hey @rjtavares, welcome to the community,

I was thinking about this recently and tried to look for JS librairies to help, tell if one of them answers your need :

• marker.js - JavaScript image annotation, rotation, cropping, and diagram libraries
• Annotorious has moved
• http://easyannotation.com/index.html
• GitHub - Secretmapper/react-image-annotation: An infinitely customizable image annotation library built on React

@rjtavares

Standard annotation tools work fine but aren’t easy to customize, and customizing your workflow is the key to efficient data annotation.

I’m not into the image annotation domain so I’m wondering : what kind of custom workflow are you looking for ?

[rjtavares]

Those are great. MarkerJS is basically what I need.

@andfanilo

I’m not into the image annotation domain so I’m wondering : what kind of custom workflow are you looking for ?

I collect sports data from broadcast footage. A lot of that is identifying pitch lines for perspective transformation and detecting objects (the players).

For that, I need to integrate automatic detection with manual input. E.g. suggesting pitch lines but allowing you to make corrections.

So, I made a tkinter GUI for that. But Streamlit would be so much easier to improve and adapt (especially a bunch of dedicated scripts for parts of the workflow).

For example, now I’m working with a different camera perspective and the line angles are different than usual. I could add a parameter for that.

[fungidick]

Wrapping the Labelstudio frontend in a bidirectional component would be great as well - it’s quite customizable for various annotation tasks (keypoints, classification, bounding boxes ,segmentation, pairwise etc …). I gave creatign the component a try but am failing quite miserably - not really a frontend guy . Any help would be appreciated!

heartexlabs/label-studio-frontend

[andfanilo]

In the meantime I’ve tried adding background image, bounding box / lines and send back coordinates of objects on canvas as JSON to Streamlit Drawable Canvas. It’s a first step, when I have some time I’ll add a labeled bounding box. See if it answers part of your needs and what other use cases you would want to be added

@fungidick

I gave creatign the component a try but am failing quite miserably

I wrote a big component tutorial here. Did you try this? I’m open to feedback on this too!

[rjtavares]

Awesome! I think that’s enough for my use case. I’ll try it out and let you know if it works.

[rjtavares]

My test worked perfectly (https://twitter.com/lastrowview/status/1300789058113863688). A few tweaks and I’ll publish the app.

[andfanilo]

Ahahah I love it, this is so unexpected yet so great so I guess I’ll have to add circles, loading a drawing and point snapping one day

[rjtavares]

Considering the interest my tweet generated (including from a lot of professionals in the sport), now I got to release it, so I’ll be watching closely the developments in your component. I’ll also try to contribute to it, but my JS skills are non existent, so it’s probably going to be limited help.

[andfanilo]

You know, just putting new features in the issues tracker is very helpful will be waiting for your post in Show the community then !

[rjtavares]

Wait no more!

BirdsPyView - an app built for football data collection

While I was experimenting the Drawable canvas component, I posted a small video online which generated a lot of interest, so here’s a working app: BirdsPyView. Basically, you have to draw the lines of the penalty box on the original image and then identify the players in a converted top-down view of the image. All of these steps can be automated, of course (that’s the idea, but I make no promises). If you want to try it out, here’s an example image you can use: …

[andfanilo]

Hey @rjtavares I added a tool to draw circles (felt bad for those players crossed by lines ahah) and an argument to prevent Streamlit update everytime you release the mouse, which greyed the canvas a bit too much in my opinion when you draw on the canvas with a heavy image.

Check the updated README and download the latest version!

[milcent]

This is great! Does it have bounding boxes (squares) too, so as to annotate the coordinates for the intended sub-images?

[andfanilo]

Have a look at the demo. You can draw and get back rectangle coordinates. Is that ok ?

[milcent]

Beautiful!!

[fungidick]

Have a look here

This component does exactly what you want (annotate images with keypoints, bounding boxes, polygons, classifications, …), but should also support annotation for timeseries, audio, etc.

This is my first go at is, so certainly not complete, but let me know what you think!

[robmarkcole]

Also just discovered https://github.com/lit26/streamlit-img-label