New tiny component: streamlit-image-coordinates

[blackary]

After seeing a few different posts and issues related to getting the clicked coordinates from an image, I decided to make a new mini component called streamlit-image-coordinates.

GitHub: GitHub - blackary/streamlit-image-coordinates: Get the coordinates of clicks on an image in your streamlit app
PyPI: streamlit-image-coordinates · PyPI
Example app: https://image-coordinates.streamlit.app/

Hope this is useful!

[Egos]

Hello
and thanks for this
is it possible to update the same image after a click event ?
is it possible to use this component on PIL Image ?

thanks

[blackary]

Hi @Egos, I just released a new version (0.1.2) which supports PIL images. Here is an example of updating an image on click https://image-coordinates.streamlit.app/dynamic_update using st.session_state

[Egos]

hello blackary

thank you very much for this new version the PIL lib works perfectly and it’s a big plus

However on the update we arrive at the same problem as on similar components like plotly_events we are forced to use st.experimental_rerun() which is not a good solution
I am not a specialist in custom components on streamlit but I think the problem here is that your component is not a container in the sense of streamlit

I made an example with st.image
we see that the widget is stored in ShowImage which allows to update it specifically further after each modification without going through a rerun

import streamlit as st
from PIL import Image, ImageDraw
from streamlit_image_coordinates import streamlit_image_coordinates
import numpy as np

def get_ellipse_coords(point: tuple[int, int]) -> tuple[int, int, int, int]:
    center = point
    radius = 10
    return (
        center[0] - radius,
        center[1] - radius,
        center[0] + radius,
        center[1] + radius,
    )

img0 = np.full((256, 256), 3)
img = Image.fromarray(img0, 'RGB')
draw = ImageDraw.Draw(img)

c1, c2 = st.columns(2)
with c1:
    value = streamlit_image_coordinates(img)  
ShowImage = c2.image(img, width = 256, output_format = 'PNG')    
st.write(value)

print(ShowImage)
if value is not None: 
    point = value["x"], value["y"]
    coords = get_ellipse_coords(point)
    draw.ellipse(coords, fill="red")
    ShowImage.image(img, width = 256, output_format = 'PNG')

is it possible to make this modification on streamlit_image_coordinates? it will be a great add

[blackary]

Hi @Egos, thanks for this example script – that inspired me to dig into the JS code a bit more, and I realized that I could make some small changes that should give this this package the same behavior as st.image. You can see the difference now at https://image-coordinates.streamlit.app/dynamic_update (or if you install version 0.1.3 of the package). The app code hasn’t really changed, but the library doesn’t recreate the image element each time, so it looks much nicer.

[Egos]

hello blackary

thank you for this new version, indeed it goes faster to refresh the image, we can already imagine uses
however I think that with a larger app the rerun will cause problems

To push a little more the reflection on the custom components is it possible to modify the original st image
indeed the output of st image ==> ShowImage as in the example above is:

DeltaGenerator(_root_container=0, _provided_cursor=LockedCursor(_root_container=0, _index=0, _parent_path=(1, 1), _props={'delta_type': 'imgs', 'last_index': None}), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1, 1), _index=1), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1,), _index=2), _parent=DeltaGenerator(_root_container=0, _provided_cursor=None, _parent=None, _block_type=None, _form_data=None), _block_type='horizontal', _form_data=FormData(form_id='')), _block_type='column', _form_data=FormData(form_id='')), _block_type=None, _form_data=None)

we could imagine adding in this output the coordinates of the click event?
I’m not very confident since all the custom components have the same limitation, that’s a shame

In any case, it’s always a pleasure to exchange with you on this subject.
and to follow up on your first post the click event feature should be basic on all streamlit & table figures I think, So , thank you again for trying to improve streamlit

[blackary]

Hi Egos,

No, I don’t think there is any was (at least with the current API for components) to actually directly modify an existing component like st.image. That certainly could be done through monkey-patching the streamlit code, but that seems like a bad idea in this case, since it’s simple to swap it out with a custom component instead.

Is there particular behavior you would like to see in this component that isn’t there already, or you would just like to have the option of modifying a built-in component in general, rather than building up a new component with javascript?

As for on-click working for everything, I think that would be great for many use-cases, and it’s definitely coming for things like dataframes. See https://roadmap.streamlit.app/#selections-on-dataframes

[Egos]

@Egos

blackary

hello blackary

I was thinking more of a way to reproduce the operation of st image to avoid the use of st.rerun

From my point of view there is no need for another feature, with the PIL and numpy tools we can already do a lot with an image
my goal is to make a grid of manipulable objects

I look forward to the evolution of streamlit widgets even if I am still surprised that the ability to follow the coordinates of the mouse is not present, maybe it is due to a limitation of the JS lib

thanks again for your help

note : you can store PIL image in session state

[Henrik_Schiller]

Hey blackary ,
Thank you so much for this beautiful streamlit component. I wonder how it is possible that the width of the image is set to the standard width of the streamlit app. That would be really helpful for me that the feature is displayed in a good size on the mobile as well as on the website. Do you have any idea about this?

Thanks!,
Henrik

[blackary]

Hi @Henrik_Schiller, good question. Currently no, but it would probably be doable by adding something like use_container_width as an argument, and setting the width to be “100%”. That’s not currently supported by the library, but I’ve added an issue for myself (or anyone else who wants to contribute) to add it eventually Add use_container_width argument · Issue #3 · blackary/streamlit-image-coordinates · GitHub

[Aleix_Cambray]

Hi blackary!

Thanks a lot for this component, I’m finding it useful for medical imaging.

I am finding that the image shown does not display the whole extent of the original image compared to e.g. st.image(). See an example below. Are you aware of this?

I can see the same things in the images in https://image-coordinates.streamlit.app/

Thanks, Aleix

[blackary]

@Aleix_Cambray Nope, I had never noticed that! Thanks so much for pointing that out! I’ll look into that and see if I can come up with a fix.

[blackary]

This should be fixed with the new 0.1.6 release – let me know if you continue to see this issue.

Release v0.1.6 · blackary/streamlit-image-coordinates

[The_Court]

Hi blackary,

Thank you so much for this. I just wonder Can I make it so the coordinate become X X2 Y Y2. X2 and Y2 in here are results of action when we click drag cursor in image area and record the last position once it’s undrag. This will so helpful if I want to use it for record an action event of Football Match. Thank you

[blackary]

Feel free to add a new issue to suggest adding this new functionality. Issues · blackary/streamlit-image-coordinates · GitHub

I’m not sure how hard that will be, but it might be doable.