Schedule streamlit app re-run on external event
I have developed a streamlit app that displays outputs of CrewAI agents, tasks and tools. CrewAI runs in a separate thread and I use a queue to pass messages from CrewAI to Streamlit (using agent and task callbacks in CrewAI). In the streamlit app I’m attempting to consume the messages from the above mentioned queue.
However, in order to show a new message in the queue in streamlit, I need to trigger a rerun / refresh. The only working way of achieving this so far was to modify the streamlit python source file upon insertion of a new message from the CrewAI thread. This indeed triggers a rerun, but first of all it sometimes interrupts a running streamlit executing, and I also very often get import errors for other modules in the application.
As already mentioned, I have done some research and tried using other approaches to trigger a rerun, to no avail
The ideal solution I’m looking for is to schedule a Streamlit app rerun, but without stopping the current run (as st.rerun() does). I would really appreciate if someone could help me with this.