Multiprocessing while updating a text and graph

[Vivek_Goel]

Hello Everyone.
I am very new to Streamlit and while building a small webpage i need a bit of help.

import matplotlib.pyplot as plt
import streamlit as st
import random
import time 
import multiprocessing

x = []
y = []
x2 = []
y2 = []
i = 0
x_max = 0
y_max = 0

def function1(x_max,y_max,i):
    with st.empty():
        while True:

            with open('data.txt','a') as f:
                f.write(str(random.randint(1+i,10+i)) + ',' + str(random.randint(1+i,10+i)) + '\n')

            with open("data.txt") as f:
                data = f.readlines()[-1]
                print(data)
                a,b = data.split(',')
                a = int(a)
                b = int(b)
                x.append(a)
                y.append(b)
                x2.append(a + 2)
                y2.append(b + 2)
                if x_max<a:
                    x_max = a
                if y_max<b:
                    y_max = b


            fig, ax = plt.subplots()
            ax.plot(x,y)
            ax.plot(x2,y2)
            st.pyplot(fig)

            i += 1
            time.sleep(0.1)


def function2():
    with st.empty():
        st.write("Max x:",x_max)
        st.write("Max y:",y_max)
        time.sleep(0.1)


if __name__ == "__main__":
    p1 = multiprocessing.Process(target=function1,args=(x_max,y_max,i))
    p2 = multiprocessing.Process(target=function2)

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print("Done")

I am trying to read from a file and plotting the data read form the file in a graph.
Although i wanted to draw 2 graphs but i not able to find a way to do so, so i embedded 2 line graphs in one graph.

function 1 will update the graph on regular interval.
while the function 2 should print the maximum value of x and y and update them accordingly.
and over it i am using multiprocessing module to run both this function at the same time.
But when i went to see the output on the browser it just keeps on running and nothing shows up.
Please help.
Thank you…

[arafeek]

I think streamlit does not support multiprocessing, I have a similar case where a function captures some data and generate a df, then pass the df to the streamlit function, everything works ok, except that streamlit does not show any output, even if you try the simplest example like the one I am posting below does not work, I am not sure why is it or nobody is clarifying the situation

import streamlit as st
import multiprocessing as mp
 
def fun():
    st.write('Testing streamlit with multiprocessing')

if __name__ == "__main__":
    p1 = mp.Process(target=fun)
    p1.start()
    p1.join()

[ferdy]

Here is a sample case to use multiprocessing in streamlit.

We are given 3 chess positions. The task is to analyze these positions in parallel with multiprocessing using the stockfish engine at 30 sec per position. The engine is only run on a single thread. Analyzing a chess position with the engine is cpu-intensive so the use of multiprocessing here is suitable.

Code

import multiprocessing
import time

import streamlit as st
import chess
import chess.engine


if 'disabled' not in st.session_state:
    st.session_state.disabled = False


def cb():
    st.session_state.disabled = True


def rerun():
    st.session_state.disabled = False


def worker(epd, ats, output):
    # https://stockfishchess.org/download/
    engine = chess.engine.SimpleEngine.popen_uci("stockfish_15.1.exe")

    board = chess.Board(epd)
    info = engine.analyse(board, chess.engine.Limit(time=ats))
    engine.quit()

    score = info['score'].relative.score(mate_score=32000)
    output[epd] = score


if __name__ == "__main__":
    manager = multiprocessing.Manager()
    output = manager.dict()

    epds = [
        'rnbqkbnr/pppp1p1p/8/6p1/4Pp2/5N2/PPPP2PP/RNBQKB1R w KQkq -',
        'rnbqkb1r/pppn1ppp/4p3/3pP3/3P4/8/PPPN1PPP/R1BQKBNR w KQkq -',
        'r1bqk1nr/pppp1pbp/2n3p1/4p3/2P5/2N3P1/PP1PPPBP/R1BQK1NR w KQkq -'
    ]

    st.write('## Positions')
    st.write(epds)

    ats = st.number_input('analysis time sec', min_value=1, max_value=60, value=10, on_change=rerun)

    # Disable the button while positions are still analyzed.
    if st.button('analyze positions', on_click=cb, disabled=st.session_state.disabled):
        t1 = time.perf_counter()

        jobs = []

        for i in range(3):
            p = multiprocessing.Process(target=worker, args=(epds[i], ats, output,))
            jobs.append(p)
            p.start()

        for j in jobs:
            j.join()

        t2 = time.perf_counter()

        st.write(f'elapse sec: {round(t2 - t1)}')

        st.write('## Analysis Output')
        st.write(output)

        # Restore the button disability once the processes are done.
        if len(output) and st.button('enable analyze button', on_click=rerun):
            pass

Output

[arafeek]

Thanks, but I am not sure how does this answer the question, I am not an expert in this to analyse it, but I have a task to do using MTing, and it kept failing, otherwise everything else works fine, so I started narrowing things until I reached this very simple code that does not work. If you can tell me why this is not working, or what mistake I am making, this will solve my issue, and I will appreciate it
Thanks again

[ferdy]

@arafeek

def fun():
    st.write('Testing streamlit with multiprocessing')

if __name__ == "__main__":
    p1 = mp.Process(target=fun)
    p1.start()
    p1.join()

From your sample code, the fun is run in another process. It is obvious that streamlit widget is out of it as it is in another process. In my example, I use multiprocess for calculation and display its output using streamlit after the task is done.