Max concurrent access or processable users in streamlit cloud plan

Community Cloud
1

hi, first of all thankyou for providing a wonderful framework and a hosting service :+1:

I’m currently interested in providing a simple image analysis service via pytorch in streamlit cloud. since one analysis takes only few seconds including model loading, and the loaded model and variables are immediately deleted after analysis, i’ve thought the following code will run fine with multiple users even if accessing at the same time.

simplified code is as bellow.

def main():
    img_file = st.file_uploader(label='xxxx', type=["JPG"])
    button = st.button("analyze")
    if button and img_file:
        model = load_pytorch_model()
        file_bytes = np.asarray(bytearray(img_file.read()), dtype=np.uint8)
        image = cv2.imdecode(file_bytes, 1)
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
        inputs = preprocess(image)
        prediction = model(inputs)
        
        st.image(image)
        st.write(prediction)
        
        del model, image, inputs, prediction

if __name__ == '__main__':
    main()
    gc.collect()

I’ve tested whether the program runs fine with concurrent accesses by launching 5~10 secret mode chrome browsers and pushing the analyze button nearly all at the same time. It went well on “Community Plan”. So the questions are,

  1. What does the maximum “3 users for Community and 20 users for Teams” mean in the cloud plan? As per above, it definately exceeded 3 concurrent users at the same time.Is it a rough guideline?
  2. What happens it exceeeds the real user limit (or perhaps reaches memory limit, will not happen but suppose if 100 users press the button at the same time. pytorch model cosumes not a lot but some memory.)
  3. If something kind of that happens and triggers a server error, is there any workarounds to just make the users wait upon access (via process queueing)?

Considering production service, currently, not so many concurrent access is expected but by chance over 20 users may press the button at the same time. So would like to know whether should upgrading to Teams plan sollves the potential issues or something different must be cosidered.

thankyou

1 Like
2

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.