Support for deploying multiple dashboards from a Monorepo to Streamlit Sharing

Hi all,

Currently, we host 8 dashboards on Streamlit and plan to add tens if not hundreds in the coming months. Some of these dashboards are pretty simple but some are complex enough depending on the topic. Here is the link for the landing page that shows all the dashboards: https://dashboards.factly.in/

We are hosting all these dashboards as a Monorepo which contains multiple dashboards and are deployed on Kubernetes. The monorepo structure is as follows:

.
├── README.md
├── projects
│ ├── atm-stats
│ ├── electoral-bonds
│ ├── gst-collection
│ ├── nobel-prize-winners
│ ├── nota-stats
│ ├── president-rule
│ ├── republic-day-guests
│ └── women-council-of-ministers
├── tools
│ └── cookiecutter
└── ui
├── README.md
├── dashboards.json
├── gatsby-config.js
├── package-lock.json
├── package.json
├── src
└── static

tools → cookiecutter is the template that we use to generate the structure and common elements in the streamlit projects

ui → Has the code related to the dashboards landing page which is built on Gatsby: https://dashboards.factly.in/

projects → All the streamlit dashboards are under the projects. Each streamlit project is initially generated using a cookiecutter template and every project starts with the following structure and we extend the project as required:

.
├── Dockerfile
├── README.md
├── docker-compose.yaml
├── install.sh
├── introduction.py
├── k8s
│ ├── certificate.yaml
│ ├── deployment.yaml
│ ├── ingress.yaml
│ └── service.yaml
├── main.py
├── requirements.txt
└── static
├── images
│ ├── factly-logo.png
│ └── favicon.png
└── maps

docker-compose.yaml → To run the individual streamlit project using docker-compose
k8s → Contains kubernetes manifests needed to deploy the streamlit apps in production
static → Static files needed for the streamlit apps (logos etc)
install.sh → We have setup github actions on the repo and based on which project the changes are pushed, the script creates a new docker image and deployes it to the container registry and applies the k8s manifests to the production environment.

The main reason for us to create a monorepo instead of a single repository for each project is to be able to not only setup this kind of DevOps but also to have all the dashboards in one place for us to refer to as we continue to build tens if not hundreds of dashboards going forward.

Currently this monorepo is a private repository as we have secrets (for eg: passwords) that we cannot publish on a public repo. I have noticed that streamlit sharing now has the possibility of managing the secrets in a better way: Add secrets to your Streamlit apps

We would like to move all the streamlit apps now to a public repo for more community support in building these dashboards and host them on streamlit sharing going forward. Wanted to check on what would be a good way to publish these dashboards in a monorepo on streamlit-sharing?

Sorry for the long question, but wanted to provide all the details to make sense. Any help is greatly appreciated.

Thanks,
Shashi

1 Like