Spinner doesn't show and then everything updates once the task in the spinner completes

Using Streamlit
1

I am creating a chat app using Streamlit that will be connected to an LLM to respond to the user.

While the LLM is generating a response, I would like a spinner to show until the response can be printed.

Currently, I am mocking the LLM’s response generation with a simple time.sleep(5). However, the spinner doesn’t show for these 5 seconds, and then the UI updates with the response.

The Streamlit app:

import streamlit as st

from sensei.ui import text, utils

st.chat_input("Your response...", key="disabled_chat_input", disabled=True)

if "messages" not in st.session_state:
    st.session_state["messages"] = [
        {"name": "Sensei", "avatar": "🥷", "content": message, "translated": True, "printed": False}
        for message in text.ONBOARDING_START_MESSAGES[st.session_state.source_language]
    ]


for message in st.session_state.messages:
    with st.chat_message(name=message["name"], avatar=message["avatar"]):
        if message["name"] == "Sensei" and not message["printed"]:
            utils.stream_message(message=message["content"])
            message["printed"] = True
        else:
            st.markdown(body=message["content"])

if st.session_state.messages[-1]["name"] == "user":
    with st.spinner("Thinking..."):
        sensei_response = utils.temp_get_response()
        st.session_state.messages.append(
            {"name": "Sensei", "avatar": "🥷", "content": sensei_response, "translated": True, "printed": False}
        )
        st.rerun()

if user_response := st.chat_input(placeholder="Your response...", key="enabled_chat_input"):
    st.session_state.messages.append({"name": "user", "avatar": "user", "content": user_response})
    st.rerun()

The temp_get_response function:

def temp_get_response() -> str:
    """Get a response from the user."""
    time.sleep(5)
    return "Well isn't that just wonderful!"

The stream_message function (this isn’t the issue as the behaviour is the same if I write normally without streaming):

def stream_message(message: str) -> None:
    """Stream a message to the chat."""
    message_placeholder = st.empty()
    full_response = ""
    for chunk in message.split():
        full_response += chunk + " "
        time.sleep(0.1)
        message_placeholder.markdown(body=full_response + "▌")
    message_placeholder.markdown(body=full_response)
2

Hi @k3vob

Have you tried to start the spinner and within it add the code block for response generation:

with st.spinner(‘Processing …’):
   # Response generation goes here
3

Seems to be working for me. I can see the “Thinking…” spinner for 5 seconds each time I sent a response.

4

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.