Streamlit-trubrics Platform show user score_feedback=None after submitting

Using Streamlit
1

Hello, am running into a issue while using Trubrics plateform for my LLM based chatbot . So basically, this is a streamlit application that runs locally, my goal is to gather user feedback when they got an answer. Trubrics seems to work fine for storing the prompt and answer text, the timing and some other infos, but not the score. From my understanding my code seems to not even submit the user optional text or the button that he clicked(thumb up/down).
ps: I can’t share my github rep.

@st.cache_data
    def init_trubrics(email, password):
            collector = FeedbackCollector(email=email, password=password, project="default")
            return collector
    collector = init_trubrics(email, password)

    if "messages" not in st.session_state:
        st.session_state["messages"] = [{"role": "assistant", "content": "How can I help you?"}]
    if "session_id" not in st.session_state:
        st.session_state["session_id"] = str(uuid.uuid4())
    if "chain" not in st.session_state: 
        st.session_state["chain"] = chain
    if "feedback_key" not in st.session_state:
        st.session_state["feedback_key"] = 0

    messages = st.session_state.messages
    for msg in messages:
        st.chat_message(msg["role"]).write(msg["content"])

    if prompt := st.chat_input("Ask your question"):
        messages.append({"role": "user", "content": prompt})
        st.chat_message("user").write(prompt)

        with st.chat_message("assistant"):
            with st.spinner("chatbot thinking ..."):
                    response =st.session_state["chain"]({'question': prompt,'chat_history': chat_history})
            msg = {'role':'assistant','content':response['answer']}
            messages.append(msg)
            chat_history.append(msg)
            message_placeholder = st.empty()
            message_placeholder.markdown(msg['content'])
            feedback_key = st.session_state["feedback_key"]
            st.session_state[f"feedback_key"]=st.session_state["feedback_key"]+1
            feedback = collector.st_feedback(
                    component="default",
                    feedback_type="thumbs",
                    open_feedback_label="[Optional] Provide additional feedback",
                    model=model,
                    metadata={"prompt":prompt, "answer":response['answer']},
                    key=f"feedback_{feedback_key}",
                    user_id=email,
                )
            if feedback:
                score = feedback["score"]
                text = feedback["text"]
                with st.sidebar:
                    st.write(score)
                    st.write(text)

            logged_prompt = collector.log_prompt(
                config_model={"model": model},
                prompt=prompt,
                generation=response['answer'],
                session_id=st.session_state.session_id,
                user_id=email,
            )

python version== 3.11
streamlit==1.27.2

the “if feedback:” block doesn’t work, nothing gets displayed after submitting the feedback, and as I said in the table of trubrics web page, the column “score_feedback”=None.
I am new to trubrics ans streamlit, so please feel free to suggest me solutions if u have.
Thank you.

2

Anyone!!

4

I could not get the callback to work to process the feedback…

Here’s what worked for me…
Initialization:

    if 'mythumbs' not in st.session_state:
        st.session_state.mythumbs = {}
    if 'feedback_submitted_mythumbs' not in st.session_state:
        st.session_state.feedback_submitted_mythumbs = None

In the chatbot code, right after the msg from the LLM is displayed to the user:

if question := st.chat_input():
[...code snipped]
        st.chat_message("assistant").write(msg)

        streamlit_feedback(
            feedback_type="thumbs",
            optional_text_label="[Optional] Please provide any comments about your choice.",
            align="flex-start",
            key = "mythumbs",
            )

Right after the chat input block:

    if len(st.session_state.mythumbs) > 2 :
            print("Feedback available") #console diagnostic
            score = {"👍": 1, "👎": -1}[st.session_state.mythumbs["score"]]                    
            # Here I ingest the data... you'll probably do something else
            last_index = persistent_chat_hist.index[-1]
            persistent_chat_hist.loc[last_index,'rating_text'] = st.session_state.mythumbs["text"]
            persistent_chat_hist.loc[last_index,'rating'] = score
            st.session_state.mythumbs = {} #Reset to empty for next pass

This is a trial and error hack… hopefully this works for someone else…

–OC

5

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.