St.chat_message irresponsive

Using Streamlit
, , ,
1

Hello Community,

this is my first post to this awesome community so please be patient! :slight_smile:

I face an issue with a streamlit app deployed as a docker container on a azure app service(linux). The app is a ChatGPT like app for internal use.
Python: 3.9
Streamlit: Tested with 1.37/1.39

The issue im facing is that after a use puts the first message into st.chat_message, the chat gets irresponsive after a short a mount of time. Sometimes instant after the first message, sometimes after a few minutes. The problem doesnt occur if i run the app locally.

if prompt := st.chat_input(placeholder=st.session_state.locale.chat_placeholder):
    # set role for Chatbot in conversation
    if not st.session_state.gpt_messages: 
        ai_role = f"{st.session_state.locale.ai_role_prefix} helpfull assistant. {st.session_state.locale.ai_role_postfix}"
        st.session_state.gpt_messages.append({"role": "assistant", "content": ai_role})

    #Render user question
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        message_placeholder = st.empty()
        full_response = ""

        #call api for answer geneartion
        completion = client.chat.completions.create(
            model="gpt4o",
            messages=[
            {"role": m["role"], "content": m["content"]}
            for m in st.session_state.gpt_messages
            ],
            stream=True
        )

        #Render answer
        for chunk in completion:
            if chunk.choices:
                full_response += (chunk.choices[0].delta.content or "")
            message_placeholder.markdown(full_response + "▌")
        message_placeholder.markdown(full_response)

        st.session_state.gpt_messages.append({"role": "assistant", "content": [
                {
                    "type": "text",
                    "text": full_response
                }
            ]})

I found a similiar post but without any solution. https://discuss.streamlit.io/t/st-chat-message-enter-does-not-post/57370

Thanks for any ideas, input or help!
Lucas

2

Seems to me this is a network issue with websockets (look at other forums for potential issues with your environment).

In short, streamlit is good for prototyping but if you are looking to build a product out of it, suggest looking at open source chatbots (plenty out there)