STqdm : a tqdm-like progress bar for streamlit

Custom Components
21

I have the missing ReportContext error with the following code:

import streamlit as st
from stqdm import stqdm
from joblib import Parallel, delayed

data = Parallel(n_jobs=-1)(delayed(get_data)(*get_id) \
           for get_id in stqdm(request_ids.itertuples(index=False), \
                               total=request_ids.shape[0])

where my get_data function gets some data using some requests.
the code runs fine without streamlit.
Any ideas ?

22

A hero much.

1 Like
23

Hi,

I was able to reproduce something similar with :

from time import sleep

from joblib import Parallel, delayed
import streamlit as st


def get_data(*args):
    sleep(0.5)
    return args


request_ids = ((str(i),) for i in range(5000))

data = Parallel(n_jobs=-1)(
    st.write(get_id) or delayed(get_data)(*get_id) for get_id in request_ids
)

From my understanding so far, itā€™s not directly related with stqdm but more with using streamlit from the main thread of joblib.Parallel (the job provider).

I am not versed enough in joblib.Parallel, but you need to use streamlit.scriptrunner.add_script_run_ctx or your streamlit version equivalent on the thread.

See : How to run a subprocess programs using thread inside streamlit? - #2 by tim .

1 Like
24

Hi,

Thank you for responding.

I donā€™t know where exactly to add add_report_ctx(thread).

I searched a bit in joblib code and Iā€™ve found two places where this could be placed: in the process executor or in the Queue

but in both cases I donā€™t know how I could apply this to my code after importing joblib, maybe by subclassing it and overriding these two classes ?

25

@Wirg This was fun to play with. Hereā€™s a little test app.

1 Like
26

Hi @asehmi,

Nice to see you successfully using stqdm with multiprocessing !

27

Hi @Abdelgha_4,

To be honest, I tried to poke a bit around with joblib and streamlit but was not really successful.

My last attempt was to use manager,Namespace(). Unfortunately the context provided by streamlit add_report_ctx was not pickable and I could not really spend more time on it.

I think the question would be best asked to streamlit runner or a joblib expert.

Did you try to create an issue on either of the repos ? I could support you on this.

30

Hi @Wirg, it seems you didnā€™t understand was @Bhargav_Choithwani was trying to say. Let me try to demonstrate it with a small reproducible code.

from time import sleep
from stqdm import stqdm
import streamlit as st

submit = st.button('Click me')
if submit:
    for _ in stqdm(range(50)):
        sleep(0.5)

The problem is, if you stop the execution while a for loop enclosed in stqm is running, in the next re-run the code enclosed in the stqdm-ed for loop becomes non-functional. It reaches executes upto the point of the for loop but no further and just the running indication by streamlit perpetually stays in the top right corner of the screen. It is demonstrated below.

stqdm_stop_bug

Hope this helps to understand the bug!

Edit: Just wanted to mention, this makes debugging the code enclosed in stqdm very annoying.

31

Hi, I have exactly the same problem. The progress bar looks awesome, but if you stop it and try to run it again, the bar will just not show up again, just like @PSP and @Bhargav_Choithwani reported. Any updates on this, @Wirg ???

thanks for your work!

32

Awesome work! Can it apply in the while loop also?

33

+1, the same issue for me.

34

Hi @Wirg,
I am not familiar with either tqdm nor STqdm, so this question might be a bit naive: Is it possible to use STqdm (tqdm) to display some kind of progress bar for a running SQL query that is fired upon clicking a st.button?
I am asking because the tqdm readme mentions in section IPython/Jupyter Integration also a mode for a progress bars without ETA (even though it is not shown in the included images).
Iā€™d imagine that SQL queries wonā€™t have any ETA, since you wonā€™t know the number of result records in advance, but still you might be able to display some kind of progress bar thus indicating to the user, that the query is running and retrieving records at the speed of x/sec?
Thank you.

35

Hi,

Thank you for clarifying this with an example and sorry for the late response.

I am failing to reproduce this. And I am curious on how you are stopping the execution during the for loop and the versions you are using.

Could you please fill an issue here ? Issues Ā· Wirg/stqdm Ā· GitHub

Thanks again for your help.

36

By stopping, I mean pressing the ā€œStopā€ icon/text/button that is present in the top right corner of any Streamlit app (at least by default). You can see my cursor pressing ā€œStopā€ in the gif in my previous post.

My versions:
Python - 3.7.9
Streamlit - 1.12.2
STqdm - 0.0.4

Sure, I can open an issue regarding the same.

2 Likes
37

Yes, it should work with something like this for example.

In general, every tqdm example should work with stqdm.
There are a few limitation mainly with some weird interaction between multithreading and streamlit. But it works in general.

Note, somebody reported that progress without a total or a length is not working in stqdm v0.0.4.
There will be in v0.0.5.

38

I created and account in Streamlit only to say thankyou.

1 Like
39

@Wirg this is great and I have been trying to use it in my streamlit app but canā€™t figure out how to incorporate it in a for loop of user-input URLs. Iā€™ve asked an open question on this on Stackoverflow if you or someone here is looking to beef up their reputation :slight_smile:

40

Hi @jrodriguez5909 ,

As answered on SO, I donā€™t think itā€™s streamlit / stqdm related. Probably a typo.

Thank you for using stqdm :slight_smile:

Have a nice day!

1 Like
41

so cooooool!

42

me too

1 Like