New tiny component: streamlit-image-coordinates

🧩 Streamlit Components
1

After seeing a few different posts and issues related to getting the clicked coordinates from an image, I decided to make a new mini component called streamlit-image-coordinates.

GitHub: GitHub - blackary/streamlit-image-coordinates: Get the coordinates of clicks on an image in your streamlit app
PyPI: streamlit-image-coordinates · PyPI
Example app: https://image-coordinates.streamlit.app/

Hope this is useful!

4 Likes
2

Hello
and thanks for this
is it possible to update the same image after a click event ?
is it possible to use this component on PIL Image ?

thanks

3

Hi @Egos, I just released a new version (0.1.2) which supports PIL images. Here is an example of updating an image on click https://image-coordinates.streamlit.app/dynamic_update using st.session_state

4

hello blackary

thank you very much for this new version the PIL lib works perfectly and it’s a big plus

However on the update we arrive at the same problem as on similar components like plotly_events we are forced to use st.experimental_rerun() which is not a good solution
I am not a specialist in custom components on streamlit but I think the problem here is that your component is not a container in the sense of streamlit

I made an example with st.image
we see that the widget is stored in ShowImage which allows to update it specifically further after each modification without going through a rerun

import streamlit as st
from PIL import Image, ImageDraw
from streamlit_image_coordinates import streamlit_image_coordinates
import numpy as np

def get_ellipse_coords(point: tuple[int, int]) -> tuple[int, int, int, int]:
    center = point
    radius = 10
    return (
        center[0] - radius,
        center[1] - radius,
        center[0] + radius,
        center[1] + radius,
    )

img0 = np.full((256, 256), 3)
img = Image.fromarray(img0, 'RGB')
draw = ImageDraw.Draw(img)

c1, c2 = st.columns(2)
with c1:
    value = streamlit_image_coordinates(img)  
ShowImage = c2.image(img, width = 256, output_format = 'PNG')    
st.write(value)

print(ShowImage)
if value is not None: 
    point = value["x"], value["y"]
    coords = get_ellipse_coords(point)
    draw.ellipse(coords, fill="red")
    ShowImage.image(img, width = 256, output_format = 'PNG')

is it possible to make this modification on streamlit_image_coordinates? it will be a great add

5

Hi @Egos, thanks for this example script – that inspired me to dig into the JS code a bit more, and I realized that I could make some small changes that should give this this package the same behavior as st.image. You can see the difference now at https://image-coordinates.streamlit.app/dynamic_update (or if you install version 0.1.3 of the package). The app code hasn’t really changed, but the library doesn’t recreate the image element each time, so it looks much nicer.

6

hello blackary

thank you for this new version, indeed it goes faster to refresh the image, we can already imagine uses
however I think that with a larger app the rerun will cause problems

To push a little more the reflection on the custom components is it possible to modify the original st image
indeed the output of st image ==> ShowImage as in the example above is:

DeltaGenerator(_root_container=0, _provided_cursor=LockedCursor(_root_container=0, _index=0, _parent_path=(1, 1), _props={'delta_type': 'imgs', 'last_index': None}), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1, 1), _index=1), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1,), _index=2), _parent=DeltaGenerator(_root_container=0, _provided_cursor=None, _parent=None, _block_type=None, _form_data=None), _block_type='horizontal', _form_data=FormData(form_id='')), _block_type='column', _form_data=FormData(form_id='')), _block_type=None, _form_data=None)

we could imagine adding in this output the coordinates of the click event?
I’m not very confident since all the custom components have the same limitation, that’s a shame :wink:

In any case, it’s always a pleasure to exchange with you on this subject.
and to follow up on your first post the click event feature should be basic on all streamlit & table figures I think, So , thank you again for trying to improve streamlit

7

Hi Egos,

No, I don’t think there is any was (at least with the current API for components) to actually directly modify an existing component like st.image. That certainly could be done through monkey-patching the streamlit code, but that seems like a bad idea in this case, since it’s simple to swap it out with a custom component instead.

Is there particular behavior you would like to see in this component that isn’t there already, or you would just like to have the option of modifying a built-in component in general, rather than building up a new component with javascript?

As for on-click working for everything, I think that would be great for many use-cases, and it’s definitely coming for things like dataframes. See https://roadmap.streamlit.app/#selections-on-dataframes

8

hello blackary

I was thinking more of a way to reproduce the operation of st image to avoid the use of st.rerun

From my point of view there is no need for another feature, with the PIL and numpy tools we can already do a lot with an image
my goal is to make a grid of manipulable objects

I look forward to the evolution of streamlit widgets even if I am still surprised that the ability to follow the coordinates of the mouse is not present, maybe it is due to a limitation of the JS lib

thanks again for your help

note : you can store PIL image in session state :wink:

1 Like