Hi,
I was able to reproduce something similar with :
from time import sleep
from joblib import Parallel, delayed
import streamlit as st
def get_data(*args):
sleep(0.5)
return args
request_ids = ((str(i),) for i in range(5000))
data = Parallel(n_jobs=-1)(
st.write(get_id) or delayed(get_data)(*get_id) for get_id in request_ids
)
From my understanding so far, it’s not directly related with stqdm
but more with using streamlit from the main thread of joblib.Parallel (the job provider).
I am not versed enough in joblib.Parallel, but you need to use streamlit.scriptrunner.add_script_run_ctx
or your streamlit version equivalent on the thread.
See : How to run a subprocess programs using thread inside streamlit? - #2 by tim .