I am working on creating a text based annotation tool to help fine tune a LLM model.
The generated answer through ollama is displayed as st.markdown. User should be able to select a sentence on the displayed text and provide a feedback.
To capture the feedback, I need a popover to get trigger once a mousedown and mouseup event is done on a markdown area.
I have tried all the text annotation tools in streamlit community. None of them does something like this.
Below is a representation of what I want to see in the UI.
Here’s some example code that gives behavior similar to what you’re asking for, I think:
import streamlit as st
from text_highlighter import text_highlighter
if "highlight_key" not in st.session_state:
st.session_state["highlight_key"] = 0
if "feedback" not in st.session_state:
st.session_state["feedback"] = []
def submit_feedback(text: str, rating: int | None, detailed_feedback: str):
print(f"Text: `{text}`")
print(f"Rating: {rating}")
print(f"Detailed feedback: `{detailed_feedback}`")
st.session_state["feedback"].append(
{"text": text, "rating": rating, "detailed_feedback": detailed_feedback}
)
@st.dialog("Feedback")
def give_feedback():
text = st.session_state[st.session_state["highlight_key"]][0]["text"]
st.write("Text: ", text)
rating = st.feedback()
detailed_feedback = st.text_area("More detailed feedback")
if st.button("Submit"):
submit_feedback(text, rating, detailed_feedback)
# Change the key to force the selection to be cleared
st.session_state["highlight_key"] += 1
st.rerun()
if st.button("Cancel"):
st.session_state["highlight_key"] += 1
st.rerun()
generated_text = "John Doe is the founder of MyComp Inc. and lives in New York with his wife Jane Doe."
result = text_highlighter(
text=generated_text,
labels=[("SELECTION", "yellow")],
key=st.session_state["highlight_key"],
)
with st.expander("Feedback", expanded=True):
for feedback in st.session_state["feedback"]:
st.write(feedback)
# Show the results (as a list)
if not result:
st.stop()
give_feedback()
This pops open a dialog window whenever text is selected, and lets you add feedback, and then when you hit submit, the feedback is added to a list, the popup closes, and the highlighting is undone.
You’ll probably want to tweak this for your own workflow, but I hope this is helpful in at least getting you started.
Thanks for stopping by! We use cookies to help us understand how you interact with our website.
By clicking “Accept all”, you consent to our use of cookies. For more information, please see our privacy policy.
Cookie settings
Strictly necessary cookies
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
Performance cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
Functional cookies
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
Targeting cookies
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.