Markdown rendering issue with chat "streaming"

[kyouens]

Hi all,

I’ve got an interesting rendering issue. I’m working on a chatbot application. I am simulating a streaming chat response using a delay. The issue is that markdown formatting (spacing, line breaks, etc.) is not rendering in the current assistant message during “streaming” or during final rendering of the message, even after the “streaming” is complete. Historical messages in the chat string are rendered correctly. I have tried emptying the message_placeholder prior to re-rendering, but that doesn’t seem to work. Any ideas? Here’s the relevant code:

# Display chat messages from history on app rerun
# Custom avatar for the assistant, default avatar for user
for message in st.session_state.messages:
    if message["role"] == 'assistant':
        with st.chat_message(message["role"], avatar=logo):
            st.markdown(message["content"])
    else:
        with st.chat_message(message["role"], avatar=user_logo):
            st.markdown(message["content"])

# Chat logic
if query := st.chat_input("Ask me a question!"):
    # Add user message to chat history
    st.session_state.messages.append({"role": "user", "content": query})
    # Display user message in chat message container
    with st.chat_message("user", avatar=user_logo):
        st.markdown(query)

    with st.chat_message("assistant", avatar=logo):
        message_placeholder = st.empty()
        # Send user's question to chain
        result = chain({"question": query})
        response = result['answer']
        sources = [doc.metadata.get("source") for doc in result["source_documents"]]
        full_response = ""

        # Simulate streaming reponse
        for chunk in response.split():
            full_response += chunk + " "
            time.sleep(0.05)
            message_placeholder.markdown(full_response + "▌")
        message_placeholder.markdown(full_response)

[dataprofessor]

Hi @kyouens

To help the community in understanding the question more, could you help with a screenshot or example of the intended formatting/ spacing versus the actual formatting/spacing that your code snippet is getting.

[kyouens]

Thanks for the suggestion! You are right. I should have done that initially. Here is the formatting of the immediate chatbot response. There is no spacing and indentation of lists, etc.

Here is the chatbot response after subsequent responses are submitted and the message becomes part of the chat history. You can see, the spacing is correct.

Somehow, the markdown is not rendered correctly until the message is re-drawn, and I can’t figure out how to force that to happen.

[Bas_Jaburg]

Hi, was this ever solved?

[blazing]

I also faced this problem. Would like to know if this can be solve?

[Reguton_Labs]

Was this ever resolved? I’m also on the same boat here with this …Thanks in advance!

[blackary]

If you change the code to response.split(" "), then it won’t remove all the newlines when streaming the data. response.split() and then joining with ' ' will end up replacing all the new lines with spaces, so it all gets merged together.

Here’s an example using the new st.write_stream, which makes this much easier

import time
import streamlit as st

TEXT = (
    """
Blah, blah, blah

* Item 1

* Item 2

* Item 3
"""
    * 5
)


def stream_data():
    for word in TEXT.split(" "):
        yield word + " "
        time.sleep(0.02)


if st.button("Stream data"):
    st.write_stream(stream_data)

[Jessica_Smith]

Hey all,

As @blackary mentioned, there’s a new command, st.write_stream, out in the latest 1.31.0 release to conveniently handle generators and streamed responses for your chat apps!

Check it out in the docs.

Looking forward to hearing what you think! Feel free to Show the Community your finished apps.