How to run a subprocess programs using thread inside streamlit?

Using Streamlit
1

I am trying to run python scripts on button click using subprocess and thread. I could able to run it using the following script

def PopenCall(onExit, PopenArgs):
        def runInThread(onExit, PopenArgs):
                script_ID = PopenArgs[1]
                proc = subprocess.Popen(PopenArgs)
                proc.wait()
                onExit(script_ID)
                return
   
        thread = threading.Thread(target=runInThread, args=(onExit, PopenArgs))
        thread.start()
        
        return thread

def onExit(script_ID):
       st.write("Done processing", script_ID + ".")
 
PopenArgs = [
                        "python", os.path.join("src", "models" , fn),
                        json_path, cycle_name
                        ]
                        print ("Running {} in background.......".format(PopenArgs))
                        PopenCall(onExit, PopenArgs)

When the python script completes the job, I am getting a message in terminal “Thread ‘Thread-7’: missing ReportContext”. This same piece of code worked in Jupyter widgets.

Can someone advise me what is the right way to use subprocess and threads in streamlit using above example?

2

Hey @ITMSEC17, welcome to Streamlit!

Yeah, this isn’t well-documented in Streamlit. If you want to perform Streamlit calls from within threads you create, you need to attach a Streamlit “ReportContext” to those threads. (This is a bit of state that lets Streamlit know which user session the thread belongs to.)

Here’s how to attach a ReportContext to your thread. You should do this after creating the thread, and before starting it.

from streamlit.ReportThread import add_report_ctx

# Your thread creation code:
thread = threading.Thread(target=runInThread, args=(onExit, PopenArgs))
add_report_ctx(thread)
thread.start()

I’ve opened a feature request to improve the error message!

4 Likes
3

Thanks @tim, now my code runs without any error.
If you don’t mind can you let me know how to return to streamlit once the subprocess gets done. For instance, in my previous post you can notice onExit(). Basically, what I was expecting is once the subprocess script is done, will print a message in streamlit app that the program finished running. But currently am not getting it.

4

You probably want to use thread.join() in your main thread, after creating the job thread. Something like:

job_thread = PopenCall(onExit, PopenArgs)
job_thread.join()  # this will block until the thread exits
1 Like
5

Thanks @tim it works.