Error running app while loading model from Nvidia

[suryadheeshjith]

The app simply uses a GAN to generate images that don’t exist. A click of any of the buttons in the app would load a pre-trained pickle from a web link and generate an image(s). It works fine when I run it locally but once deployed it does not work as expected.

App Link : https://share.streamlit.io/suryadheeshjith/stylegan3/main/app.py
GitHub Link : GitHub - suryadheeshjith/StyleGAN3: Streamlit app that displays images of faces that don't exist

[Franky1]

I suspect that this GAN model is quite huge and therefore fails on streamlit cloud.
If you run it locally, how large is the model?

[suryadheeshjith]

Yes, it is quite big. It takes a few minutes to run locally on my macbook pro 2017.

[snehankekre]

That’s precisely the issue. Streamlit Cloud isn’t designed to host large, resource hungry deep learning models that ideally require a GPU for training and inference.

Your best bet would be to host your model as an inference server on a cloud provider like GCP, AWS, etc with an NVIDIA GPU and make REST API / gRPC calls to the inference endpoint from your Streamlit app.

Best,
Snehan