What's your preferred deployment platform for GPU-enabled applications?

Deployment
1

Hi guys,

I’ve got a handful of Streamlit GPU apps I wanted to deploy but I’ve not decided yet on where to deploy them. So I’m curious about your preferred deployment platform(s) for GPU aplications.

Ease of deployment is important, yet the key factor for me would be cost, as these would be personal projects. :slight_smile:

Thanks in advance!
Charly

1 Like
2

That’s a tough one Charly, as “GPU” and “low cost deployment” tend to be mutually exclusive, just due to the nature of the hardware and cloud pricing. I’m interested to hear what people respond with, as I’m not familiar with instant-on/fractional GPU solutions, just getting your own cloud image and running 24/7

1 Like
3

Thanks Randy!

Besides the usual Azure/GCP offerings, I’ve stumbled upon a handful of GPU solutions which I’ll have to try, such as:

Seems like Paperspace has some Streamlit integration in the pipeline :star_struck:


image
image1151×683 30 KB

I may report on my findings here. :slight_smile:

Thanks,
Charly

2 Likes
4

I have had success deploying a Voila application to Google Kubernetes Engine with GPU nodes using Knative (see this repo for my Knative config files: https://gitlab.com/proteinsolver/dashboards).

If you are ok with a 1-2 min cold start, you can even spin down the number of GPU nodes to zero, which is nice. In my case, the biggest cost is running the 3 nodes for the Kubernetes machinery, but if you have a Kubernetes cluster anyway, than this may be a cost-effective solution.

The following StackOverflow posts describes how we set up a Kubernetes cluster with Knative support and a GPU node pool on Google Kubernetes Engine: https://stackoverflow.com/q/59850010/2063031, https://stackoverflow.com/q/63251231/2063031.

1 Like