Chat Message Reload Behaviour

Using Streamlit
,
1

Hi all. I am building a basic ChatGPT-style interface using Streamlit’s chat elements, where each assistant message has the length of the message displayed beneath it.

However, I notice that when I submit a new message, the previous assistant message’s word count is temporarily duplicated while the next assistant message is being written. Please see the below code and the attached photo showing this behaviour.

import streamlit as st
from openai import AzureOpenAI
from os import environ
import tiktoken
import re

st.set_page_config(page_title="Bid Writer", page_icon="🖋️")

st.markdown("# 🖋️ Bot")

if "openai_client" not in st.session_state:
    st.session_state["openai_client"] = AzureOpenAI(
        api_key=environ["OPENAI_API_KEY"],
        azure_endpoint=environ["OPENAI_API_ENDPOINT"],
        api_version="2023-05-15"
    )

if "openai_model" not in st.session_state:
    st.session_state["openai_model"] = "gpt-35-turbo"

if "messages" not in st.session_state:
    st.session_state.messages = []

if "encoder" not in st.session_state:
    st.session_state.encoder = tiktoken.get_encoding("cl100k_base")

system_prompt = """You are a helpful assistant."""

for i, message in enumerate(st.session_state.messages):
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt := st.chat_input("Ask a question"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        messages= [{"role": "system", "content": system_prompt}]
        for m in st.session_state.messages[-5:-1]:
            messages.append({"role": m["role"], "content": m["content"].split("\n\n*(")[0]})
        final_message = {"role": st.session_state.messages[-1]["role"], "content": st.session_state.messages[-1]["content"]}
        messages.append(final_message)

        stream = st.session_state["openai_client"].chat.completions.create(
            model=st.session_state["openai_model"],
            messages=messages,
            stream=True
        )
        response = st.write_stream(stream)
        response_words = len(response.split(" "))
        st.markdown(f"\n\n*({response_words} words)*")
        response_tokens = len(st.session_state.encoder.encode(response))
    st.session_state.messages.append({"role": "assistant", "content": response + f"\n\n*({response_words} words)*"})

duplication
duplication807×765 51 KB

This duplicated text disappears once the assistant message is generated. This only affects the previous message. I believe this is something to do with the reloading of the line: st.markdown(f"\n\n*({response_words} words)*"), but I am wondering if there is anything I can do to remove this duplication?

Thanks!

1 Like
2

I am having same issue.