Chatbot message appears twice

LLMs and AI
1

Hello! Do you kwnow why the responses of the chatbot/llm are appearing twice when it is thinking for the next answer?

I mean, when I asked a second question, the response of the first question appears twice. Then it disappears when the second answer is delivered.

image
image846×332 16.4 KB

2

We need to see the relevant code.

3

That just happens when using StreamlitCallbackHandler

Here the code:

Start the chat conversation

if “messages” not in st.session_state:
st.session_state.messages = [{“role”: “assistant”, “content”: “How may I assist you today?”}]

for message in st.session_state.messages:
with st.chat_message(message[“role”]):
st.markdown(message[“content”])

user_input = st.chat_input(“Write your message here…”)
if user_input:
st.session_state.messages.append({“role”: “user”, “content”: user_input, “save”: True})
with st.chat_message(“user”):
st.markdown(user_input)

try:
    with st.chat_message("assistant"):
        with st.spinner("Thinking..."):
            
            st_callback = StreamlitCallbackHandler(
                parent_container = st.container(),
                max_thought_containers = 4,
                expand_new_thoughts = False)

            def response_generator():
                response = agent_conversational.invoke({"input": user_input, "chat_history": message}, {"callbacks": [st_callback]})
                output_message = response["output"]
                for word in output_message.split(" "):
                    yield word + " "
                    time.sleep(0.1)
            output_message = st.write_stream(response_generator())
        st.session_state.messages.append({"role": "assistant", "content": output_message})
4

Try to remove that,

time.sleep(0.1)

See if the ghost disappears.

5

That is not working. I think its because of the StreamlitCallbackHandler containers/functions. But I do not know how to fix it

6

I have encountered this issue too and I did not use StreamlitCallbackHandler @RomeoSe. Unsure why is this though, did not see any problem in the code.

7

Try:

# Your code above...
with st.chat_message("assistant"):
   st.empty()
# Your code below...
8

It works perfectly. Thanks @asehmi

9

Great… this is a general technique you can use to address these kinds of “ghost” trails.

The OP where this solution came from is here: Ghost double text bug