I’m working on a recommender system in which I’m using some NLP models that are greater than 4 GB. In previous versions I was using Heroku but when I changed my pipeline and added these models (developed with StarSpace) the small 500mb from slugsize in Heroku was simply not enough. Which other way would yo recommend me on deploying my Streamlit application in order to handle these filesizes?
Host the models themselves elsewhere so you can make an api call with your data and get back the model output?
Using something like glcoud or torchservce version. might be overkill but sometimes its easier to treat the model totally independently of the consuming app, and it makes the streamlit app smaller/easier.
1 Like
This is a great point. While we talk about Streamlit as being a self-contained thing for application code and presentation code, for something like this where the models are static, it doesn’t seem like it matters that the NLP models are actually present with the app. Of course, once you start talking about paid options, there are any number of ways to solve this issue.