New streamlit app: football data!

[dcaribou]

Hi streamlit community ,

I’d like to share a small app I’ve built to demonstrate one of my projects, transfermakt-datasets.

transfermarkt-datasets is a full automated data pipeline that scrapes, curates and publishes data from Transfermarkt, a well known football (soccer) website. You can find out out more about it in the streamlit app itself (I chose Heroku for the hosting)

I’ll be sharing some my learnings and the patterns I used in the thread. I hope you find it useful.

[dcaribou]

Service Hosting

When looking for options to host this app, I mainly considered two factors

• Service hosting cost
• Data transfer cost (mostly S3 egress costs)

For a demo app like this with almost no traffic, it wouldn’t have been reasonable for the costs to be more than a few cents per month.

Streamlit Cloud

Being specifically designed to host streamlit projects, Streamlit Cloud was the first option I considered. By simply connecting the project’s github repo to streamlit cloud I was able to quickly deploy my app in there. There were, however, a couple of drawbacks that eventually made me consider other options

• The level of customization for the build, deploy and run process was limited
• It is GCP hosted, which meant that I would’ve eventually incurred in noticeable egress costs

AWS App Runner

App Runner seemed like an interesting alternative, considering the app uses S3 as its storage service. However it proved to be a dead end for me, mainly because of two reasons

• App Runner does not support scale down to zero capabilities, which was a very much desired feature for a demo app like this
• I was unable to get the past the Please wait... screen, no matter how many combinations from the troubleshooting steps I tried

Heroku (winner)

Finally, I looked at Heroku and realised that it seemed quite a fit

• As long as I could use the free tier I would not need to worry about hosting costs
• Other users had successfully deployed streamlit apps in there, so hopefully I could get past this annoying Please wait... screen if I used the service
• It enabled a lot of customization for deployment process

What was even better was that Heroku is hosted in AWS, so provided I was able to deploy the app to the right AWS region I could cut off most of those egress costs. Luckily for me, the generic Heroku eu region maps to eu-west-1, which was exactly what I needed

curl -n -X GET https://api.heroku.com/regions/eu -H "Accept: application/vnd.heroku+json; version=3"
{
  "country": "Ireland",
  "created_at": "2013-09-19T01:29:12Z",
  "description": "Europe",
  "id": "ed30241c-ed8c-4bb6-9714-61953675d0b4",
  "locale": "Dublin",
  "name": "eu",
  "private_capable": false,
  "provider": {
    "name": "amazon-web-services",
    "region": "eu-west-1"
  },
  "updated_at": "2016-08-09T22:03:28Z"
}

In the end I was able to deploy the app by doing some little adjustments to the project’s Dockerfile and conveniently using Herkou CLI to build and release the image.

docker build --tag registry.heroku.com/transfermarkt-datasets/web
docker push registry.heroku.com/transfermarkt-datasets/web
heroku container:release web

[dcaribou]

Service Hosting (Update)

Given the removal of Heroku’s free plan I started exploring for alternatives. This is a quick summary of the options I considered.

Name	Docker support*	Elastic pricing**	AWS Ireland	Custom domains	Test successful
Heroku Eco	Yes	No	Yes	Yes
Heroku Basic	Yes	No	Yes	Yes
Streamlit Cloud	No
Cyclic	No
Deta	No
Render	Yes	Yes	No	Yes	No
Railway	Yes	Yes	us-west-1 only	-	No
Fly.io	Yes	Yes	No	Yes	Yes

Eventually I gave up on the requirement to co-locate the app within the AWS region that hosts the data, and settled on fly.io as my new provider. Check it out here.

More in the related issue.

Docker support means we have the ability to create a custom Docker image for the app, and define a launch command
Elastic pricing means either free hosting or a cheap pay as you go option with scale down to zero