Hi dears! I have an issue, let’s start from mandatory answers and then I will explain context further.
1. Are you running your app locally or is it deployed?
Both ways. On local - no problems, problem appears when I use deployed app
2. If your app is deployed:
a. Is it deployed on Community Cloud or another hosting platform?
Google Kubernetes Engine
b. Share the link to the public deployed app.
It’s private
3. Share the link to your app’s public GitHub repository (including a requirements file).
Its private, but I’ll send you a code.
4. Share the full text of the error message (not a screenshot).
DEBUG:urllib3.connectionpool:http://host:port "POST /route HTTP/1.1" 200 279
INFO:main:LLM answer: ANSWER
2024-02-07 09:50:07.180 Removing orphaned files...
2024-02-07 09:50:07.181 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.ENQUEUE_FORWARD_MSG
2024-02-07 09:50:07.271 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.SCRIPT_STOPPED_WITH_SUCCESS
2024-02-07 09:50:07.272 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.SHUTDOWN
2024-02-07 09:50:30.102 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
5. Share the Streamlit and Python versions.
streamlit 1.30.0
python 3.10.6
So, the context is:
I running this application
# flake8: noqa
# mypy: ignore-errors
import logging
import os
from typing import Any, Optional
from uuid import uuid4
import streamlit as st
from api_requests import (
create_chat,
create_chat_member,
create_user,
send_message,
)
# poetry run streamlit run --server.port 8080 --server.address 0.0.0.0 frontend/streamlit_chat_app.py
from dotenv import load_dotenv
# Retrieve API credentials from environment variables
load_dotenv(dotenv_path=".env.local", override=True)
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
environ = os.environ
session_state = st.session_state
def setup(session_state: Any) -> None:
if 'session_id' not in st.session_state:
session_state.session_id = str(uuid4())
# Initialize session state variables if they don't exist
if 'chat_id' not in session_state:
session_state.chat_id = create_chat()
if 'user_id' not in session_state:
session_state.user_id = create_user(str(uuid4()))
# Ensure chat member is created only once per session
if 'member_created' not in session_state:
create_chat_member(session_state.chat_id, st.session_state.user_id)
session_state.member_created = True # Set a flag to indicate member creation
session_state.messages = session_state.get('messages', [])
setup(session_state)
st.caption(
f"User Id: {session_state.user_id}, Chat Id: {session_state.chat_id}",
)
st.title(":car: Digital Pilot")
for message in st.session_state.messages:
if message["role"] != "system":
with st.chat_message(message["role"]):
st.markdown(message["content"])
prompt = st.chat_input("Ask me...")
if prompt:
user_message = {"role": "user", "content": prompt}
st.session_state.messages.append(user_message)
with st.chat_message("user"):
st.markdown(prompt)
with st.chat_message("assistant"):
message_placeholder = st.empty()
logger.info(f"Going to send LLM prompt: {prompt}")
try:
response = send_message(
prompt,
session_state.chat_id,
session_state.user_id,
)
except Exception as e:
logger.error(e)
message_placeholder.markdown(
f"Something went wrong. Please try again later. {type(prompt)}",
)
else:
logger.info(f"LLM answer: {response}")
message_placeholder.markdown(response) # Display the response
st.session_state.messages.append(
{"role": "assistant", "content": response},
)
Locally and have a nice chat with LLM (thank you for the great simplicity).
But, when I deploy application on K8S, I got the next behavior:
After sending random message to LLM, UI shows for a time that it’s waiting for response, but then - reloads it’s state, and no reply message shows up.
As a result - I have the user message shown, but no answer from the server.
In parallel, DEBUG level streamlit logs shows that streamlit backend receives message from the app backend, but it’s going kinda in old ScriptRunner thread.
Detailed logs of whats going on failure (here two messages wasn’t delivered to frontend):
2024-02-07 09:49:51.687 Beginning script thread
2024-02-07 09:49:51.688 Running script RerunData(widget_states=widgets {
id: "$$WIDGET_ID-ff3562a5a0f41a6e59e0deb75451175e-None"
string_trigger_value {
data: "UserContent1"
}
}
, page_script_hash='07943488615f26beea8fdf20ec391b09')
2024-02-07 09:49:51.688 Disconnecting files for session with ID aefc9bf0-4d01-48dd-b00e-a62bd9ec0703
2024-02-07 09:49:51.688 Sessions still active: dict_keys([])
2024-02-07 09:49:51.688 Files: 0; Sessions with files: 0
INFO:main:Displaying message: {'role': 'user', 'content': 'Hi!'}
INFO:main:Displaying message: {'role': 'assistant', 'content': 'LLMContent1'}
INFO:main:Sending LLM prompt: UserContent1
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): host:8000
2024-02-07 09:49:59.544 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
2024-02-07 09:50:00.103 Watcher created for /app/streamlit_chat_app.py
2024-02-07 09:50:00.103 Runtime state: RuntimeState.NO_SESSIONS_CONNECTED -> RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED
DEBUG:watchdog.observers.inotify_buffer:in-event <InotifyEvent: src_path=b'/app/streamlit_chat_app.py', wd=1, mask=IN_OPEN, cookie=0, name='streamlit_chat_app.py'>
2024-02-07 09:50:00.293 Received the following back message:
rerun_script {
widget_states {
}
page_script_hash: "07943488615f26beea8fdf20ec391b09"
}
2024-02-07 09:50:00.294 Beginning script thread
2024-02-07 09:50:00.295 Running script RerunData(widget_states=, page_script_hash='07943488615f26beea8fdf20ec391b09')
2024-02-07 09:50:00.295 Disconnecting files for session with ID aefc9bf0-4d01-48dd-b00e-a62bd9ec0703
2024-02-07 09:50:00.296 Sessions still active: dict_keys([])
2024-02-07 09:50:00.296 Files: 0; Sessions with files: 0
INFO:main:Displaying message: {'role': 'user', 'content': 'Hi!'}
INFO:main:Displaying message: {'role': 'assistant', 'content': 'LLMContent1'}
INFO:main:Displaying message: {'role': 'user', 'content': 'UserContent1'}
2024-02-07 09:50:00.303 Removing orphaned files...
DEBUG:watchdog.observers.inotify_buffer:in-event <InotifyEvent: src_path=b'/app/api_requests.py', wd=1, mask=IN_OPEN, cookie=0, name='api_requests.py'>
2024-02-07 09:50:00.487 Watcher created for /app/api_requests.py
2024-02-07 09:50:00.489 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
DEBUG:urllib3.connectionpool:http://host:port "POST /route HTTP/1.1" 200 279
INFO:main:LLM answer: ANSWE?
2024-02-07 09:50:07.180 Removing orphaned files...
2024-02-07 09:50:07.181 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.ENQUEUE_FORWARD_MSG
2024-02-07 09:50:07.271 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.SCRIPT_STOPPED_WITH_SUCCESS
2024-02-07 09:50:07.272 Ignoring event from non-current ScriptRunner: ScriptRunnerEvent.SHUTDOWN
2024-02-07 09:50:30.102 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
DEBUG:watchdog.observers.inotify_buffer:in-event <InotifyEvent: src_path=b'/app/streamlit_chat_app.py', wd=1, mask=IN_OPEN, cookie=0, name='streamlit_chat_app.py'>
2024-02-07 09:50:30.692 Watcher created for /app/streamlit_chat_app.py
2024-02-07 09:50:30.692 Runtime state: RuntimeState.NO_SESSIONS_CONNECTED -> RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED
And how things going on local stage:
digital_pilot-streamlit-app-1 | INFO:__main__:Going to send LLM prompt: Hi!
digital_pilot-streamlit-app-1 | DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): digital_pilot-app-1:8090
digital_pilot-streamlit-app-1 | DEBUG:urllib3.connectionpool:http://digital_pilot-app-1:8090 "POST /chats/65c49d340ad777a0b1675b78/messages HTTP/1.1" 200 237
digital_pilot-streamlit-app-1 | INFO:__main__:LLM answer: Answer
digital_pilot-streamlit-app-1 | 2024-02-08 09:22:12.656 Removing orphaned files...
digital_pilot-streamlit-app-1 | 2024-02-08 09:22:12.705 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
As I can see from logs, remote app have some iterative (ReRun script launches, and local - do not have. And each that ReRun makes all previous requests obsolete)
The one reason I can suggest for know is that K8S running healthchecks, and local env do not - so that healthcheck re-runs script, but why does it affects the sessions of real users? This is not the proper way to run Web App. I think, I don’t understand something here, but I wrote code just like in guides. Will this ignoring happen if, for example, two users will interact with streamlit backend?
Finally: do you have Ideas, how to fix it?
Thank you!