New streamlit app: football data!

Hi streamlit community :balloon:,

I’d like to share a small app I’ve built to demonstrate one of my projects, transfermakt-datasets.

transfermarkt-datasets is a full automated data pipeline that scrapes, curates and publishes data from Transfermarkt, a well known football (soccer) website. You can find out out more about it in the streamlit app itself (I chose Heroku for the hosting)

I’ll be sharing some my learnings and the patterns I used in the thread. I hope you find it useful.

1 Like

Service Hosting

When looking for options to host this app, I mainly considered two factors

  • Service hosting cost
  • Data transfer cost (mostly S3 egress costs)

For a demo app like this with almost no traffic, it wouldn’t have been reasonable for the costs to be more than a few cents per month.

Streamlit Cloud

Being specifically designed to host streamlit projects, Streamlit Cloud was the first option I considered. By simply connecting the project’s github repo to streamlit cloud I was able to quickly deploy my app in there. There were, however, a couple of drawbacks that eventually made me consider other options

  • The level of customization for the build, deploy and run process was limited
  • It is GCP hosted, which meant that I would’ve eventually incurred in noticeable egress costs

AWS App Runner

App Runner seemed like an interesting alternative, considering the app uses S3 as its storage service. However it proved to be a dead end for me, mainly because of two reasons

Heroku (winner)

Finally, I looked at Heroku and realised that it seemed quite a fit

What was even better was that Heroku is hosted in AWS, so provided I was able to deploy the app to the right AWS region I could cut off most of those egress costs. Luckily for me, the generic Heroku eu region maps to eu-west-1, which was exactly what I needed

curl -n -X GET https://api.heroku.com/regions/eu -H "Accept: application/vnd.heroku+json; version=3"
{
  "country": "Ireland",
  "created_at": "2013-09-19T01:29:12Z",
  "description": "Europe",
  "id": "ed30241c-ed8c-4bb6-9714-61953675d0b4",
  "locale": "Dublin",
  "name": "eu",
  "private_capable": false,
  "provider": {
    "name": "amazon-web-services",
    "region": "eu-west-1"
  },
  "updated_at": "2016-08-09T22:03:28Z"
}

In the end I was able to deploy the app by doing some little adjustments to the project’s Dockerfile and conveniently using Herkou CLI to build and release the image.

docker build --tag registry.heroku.com/transfermarkt-datasets/web
docker push registry.heroku.com/transfermarkt-datasets/web
heroku container:release web