After seeing a few different posts and issues related to getting the clicked coordinates from an image, I decided to make a new mini component called streamlit-image-coordinates.
GitHub: GitHub - blackary/streamlit-image-coordinates: Get the coordinates of clicks on an image in your streamlit app
PyPI: streamlit-image-coordinates · PyPI
Example app: https://image-coordinates.streamlit.app/
Hope this is useful!
Hello
and thanks for this
is it possible to update the same image after a click event ?
is it possible to use this component on PIL Image ?
thanks
Hi @Egos, I just released a new version (0.1.2) which supports PIL images. Here is an example of updating an image on click https://image-coordinates.streamlit.app/dynamic_update using st.session_state
hello blackary
thank you very much for this new version the PIL lib works perfectly and it’s a big plus
However on the update we arrive at the same problem as on similar components like plotly_events we are forced to use st.experimental_rerun() which is not a good solution
I am not a specialist in custom components on streamlit but I think the problem here is that your component is not a container in the sense of streamlit
I made an example with st.image
we see that the widget is stored in ShowImage which allows to update it specifically further after each modification without going through a rerun
import streamlit as st
from PIL import Image, ImageDraw
from streamlit_image_coordinates import streamlit_image_coordinates
import numpy as np
def get_ellipse_coords(point: tuple[int, int]) -> tuple[int, int, int, int]:
center = point
radius = 10
return (
center[0] - radius,
center[1] - radius,
center[0] + radius,
center[1] + radius,
)
img0 = np.full((256, 256), 3)
img = Image.fromarray(img0, 'RGB')
draw = ImageDraw.Draw(img)
c1, c2 = st.columns(2)
with c1:
value = streamlit_image_coordinates(img)
ShowImage = c2.image(img, width = 256, output_format = 'PNG')
st.write(value)
print(ShowImage)
if value is not None:
point = value["x"], value["y"]
coords = get_ellipse_coords(point)
draw.ellipse(coords, fill="red")
ShowImage.image(img, width = 256, output_format = 'PNG')
is it possible to make this modification on streamlit_image_coordinates? it will be a great add
Hi @Egos, thanks for this example script – that inspired me to dig into the JS code a bit more, and I realized that I could make some small changes that should give this this package the same behavior as st.image. You can see the difference now at https://image-coordinates.streamlit.app/dynamic_update (or if you install version 0.1.3 of the package). The app code hasn’t really changed, but the library doesn’t recreate the image element each time, so it looks much nicer.
hello blackary
thank you for this new version, indeed it goes faster to refresh the image, we can already imagine uses
however I think that with a larger app the rerun will cause problems
To push a little more the reflection on the custom components is it possible to modify the original st image
indeed the output of st image ==> ShowImage as in the example above is:
DeltaGenerator(_root_container=0, _provided_cursor=LockedCursor(_root_container=0, _index=0, _parent_path=(1, 1), _props={'delta_type': 'imgs', 'last_index': None}), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1, 1), _index=1), _parent=DeltaGenerator(_root_container=0, _provided_cursor=RunningCursor(_root_container=0, _parent_path=(1,), _index=2), _parent=DeltaGenerator(_root_container=0, _provided_cursor=None, _parent=None, _block_type=None, _form_data=None), _block_type='horizontal', _form_data=FormData(form_id='')), _block_type='column', _form_data=FormData(form_id='')), _block_type=None, _form_data=None)
we could imagine adding in this output the coordinates of the click event?
I’m not very confident since all the custom components have the same limitation, that’s a shame 
In any case, it’s always a pleasure to exchange with you on this subject.
and to follow up on your first post the click event feature should be basic on all streamlit & table figures I think, So , thank you again for trying to improve streamlit
Hi Egos,
No, I don’t think there is any was (at least with the current API for components) to actually directly modify an existing component like st.image. That certainly could be done through monkey-patching the streamlit code, but that seems like a bad idea in this case, since it’s simple to swap it out with a custom component instead.
Is there particular behavior you would like to see in this component that isn’t there already, or you would just like to have the option of modifying a built-in component in general, rather than building up a new component with javascript?
As for on-click working for everything, I think that would be great for many use-cases, and it’s definitely coming for things like dataframes. See https://roadmap.streamlit.app/#selections-on-dataframes
hello blackary
I was thinking more of a way to reproduce the operation of st image to avoid the use of st.rerun
From my point of view there is no need for another feature, with the PIL and numpy tools we can already do a lot with an image
my goal is to make a grid of manipulable objects
I look forward to the evolution of streamlit widgets even if I am still surprised that the ability to follow the coordinates of the mouse is not present, maybe it is due to a limitation of the JS lib
thanks again for your help
note : you can store PIL image in session state
Hey blackary 
,
Thank you so much for this beautiful streamlit component. I wonder how it is possible that the width of the image is set to the standard width of the streamlit app. That would be really helpful for me that the feature is displayed in a good size on the mobile as well as on the website. Do you have any idea about this?
Thanks!,
Henrik 
Hi @Henrik_Schiller, good question. Currently no, but it would probably be doable by adding something like use_container_width as an argument, and setting the width to be “100%”. That’s not currently supported by the library, but I’ve added an issue for myself (or anyone else who wants to contribute) to add it eventually Add use_container_width argument · Issue #3 · blackary/streamlit-image-coordinates · GitHub
Hi blackary!
Thanks a lot for this component, I’m finding it useful for medical imaging.
I am finding that the image shown does not display the whole extent of the original image compared to e.g. st.image(). See an example below. Are you aware of this?
I can see the same things in the images in https://image-coordinates.streamlit.app/
Thanks, Aleix
@Aleix_Cambray Nope, I had never noticed that! Thanks so much for pointing that out! I’ll look into that and see if I can come up with a fix.
This should be fixed with the new 0.1.6 release – let me know if you continue to see this issue.
Hi blackary,
Thank you so much for this. I just wonder Can I make it so the coordinate become X X2 Y Y2. X2 and Y2 in here are results of action when we click drag cursor in image area and record the last position once it’s undrag. This will so helpful if I want to use it for record an action event of Football Match. Thank you 
Feel free to add a new issue to suggest adding this new functionality. Issues · blackary/streamlit-image-coordinates · GitHub
I’m not sure how hard that will be, but it might be doable.
Hello @blackary ,
Thank you for this component!
I’m trying to built an app that displays a grid of images.
Users can then click on any image, and then take an action that impacts the selected image.
This is my first Python app and I’ve only started learning about a month ago.
I think I narrowed the problem down to: not being able to reset the st.session_state.clicked image, but I’m not sure.
Update:
I think I can confidently confirm that somehow, the keys of past clicks on streamlit_image_coordinates() appear at rerun, even after using “del” or setting them to None.
Is there a way to disable this behavior?
I’ve updated the code:
import os
import streamlit as st
from streamlit_image_coordinates import streamlit_image_coordinates as st_ic
st.set_page_config(layout="wide")
e1, col_states, col_grid, col_select, e2 = st.columns([1, 2, 2, 1, 2], gap="small")
col_states.subheader("FIRST")
col_states.write(st.session_state)
def first_run_setup():
# Initialize st.session_state.cell_dict with default image
if "cell_dict" not in st.session_state:
st.session_state.cell_dict = {}
for row_index, row_local in enumerate(layout):
for col_index, value in enumerate(row_local):
if value == 1:
cell_key_local = f"{row_index}_{col_index}"
st.session_state.cell_dict[cell_key_local] = image_default
# Initialize st.session_state.clicked_cell with default None
if "clicked_cell" not in st.session_state:
st.session_state.clicked_cell = None
if "choice_img" not in st.session_state:
st.session_state.choice_img = None
if "first_run" not in st.session_state:
st.session_state.first_run = False
print("Finished Setup")
layout = [[0, 1, 0],
[1, 1, 1],
[0, 1, 0]]
image_list = [f"data/sprites/{f}" for f in os.listdir("../proto/data/sprites")]
image_default = "data/sprites/Empty_Module.png"
if "first_run" not in st.session_state:
first_run_setup()
# Attempt to clean image_coords from st.session_state
# This works a bit, but clicking on the same cell, results in a switch between that cell id and the highest id
# Example: If all cells are clicked, clicking cell 01 repeatedly, alternates clicked_cell between 01 and 21
if st.session_state.clicked_cell:
for idx in st.session_state.cell_dict.keys():
if st.session_state.clicked_cell == idx:
st.session_state[f"{idx}"] = None
col_states.subheader("BEFORE LOOP")
col_states.write(st.session_state)
# If any cell is clicked, store the clicked cell and reset the click
for idx in st.session_state.cell_dict.keys():
if f"{idx}" in st.session_state and st.session_state[f"{idx}"]:
st.session_state.clicked_cell = f"{idx}"
st.session_state[f"{idx}"] = None
# DELTE THE COORDS FROM EXISTENCE
for idx in st.session_state.cell_dict.keys():
if f"{idx}" in st.session_state:
del st.session_state[f"{idx}"]
col_states.subheader("AFTER LOOP")
col_states.write(st.session_state)
with col_grid:
for row_idx, row in enumerate(layout):
cells = st.columns(len(row), gap="small")
for cell_idx, cell in enumerate(cells):
if row[cell_idx] != 0:
with cell:
cell_key = f"{row_idx}_{cell_idx}"
st_ic(source=st.session_state.cell_dict[cell_key],
key=str(cell_key),
use_column_width="always")
with col_select:
if st.session_state.clicked_cell == cell_key:
st.write(f"Editing cell: {cell_key}")
col_states.subheader("AFTER CELL GEN")
col_states.write(st.session_state)
I found a solution, but i had to modify the main.js of this package and add a unix_timestamp to this function:
function clickListener(event) {
const {offsetX, offsetY} = event;
const img = document.getElementById("image");
const unixTime = Date.now();
sendValue({x: offsetX, y: offsetY, width: img.width, height: img.height, unix_time: unixTime});
}
I use the latest timestamp to identify the clicked cell.
I’m not sure if this is a good solution, but I’m also not sure exactly where to go from here.
That seems like a great addition to the package – would you be interested in making a PR to add that timestamp to the returned values?
Hi
do you think there is a way to implement this into the st.plotly_chart somehow? For my ML project i do like mapping and somehow i want to retrieve the x and y coordinates when you click on it. Sadly, the built in on_select method doesnt work, since technically there are no datapoints to it, just colors.
import streamlit as st
import numpy as np
import plotly.express as pxdef main():
st.title(‘Click Coordinates on Random Image’)# Generate a random 100x100 image with red and blue colors width, height = 100, 100 image = np.zeros((height, width, 3), dtype=np.uint8) for i in range(width): for j in range(height): # Randomly assign red or blue color if np.random.rand() > 0.5: image[j, i] = [255, 0, 0] # Red else: image[j, i] = [0, 0, 255] # Blue # Display the random image using Plotly fig = px.imshow(image) fig.update_layout(coloraxis_showscale=False) fig.update_layout( margin=dict(l=0, r=0, t=0, b=0), height=800 ) # Configure hover template to show pixel coordinates hovertemplate = "Pixel: X %{x}, Y %{y}<extra></extra>" fig.data[0].hovertemplate = hovertemplate # Display Plotly chart & capture click coordinates st.plotly_chart(fig, use_container_width=True)st.set_page_config(layout=“wide”)
if name == ‘main’:
main()
I am not really that good in programming, so any help would be appreciated.
@Robert2, that’s a very excellent question – I have wondered about that myself, but found the same limitation you did. Someone suggested adding an invisible grid of pixels under the image, but that seems pretty impractical. My colleague was inspired by this to create a new streamlit extra: 🤠 Image Selector - streamlit-extras. Unfortunately, as you found, you can’t just click a point, but you can nicely select an area with this method.
Thank you very much for the fast response. It should work for the purpose i think 
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.