Question: example of big application?

[SimonBiggs]

Hi @Newton_Miotto,

Welcome to the Streamlit community .

Thanks @andfanilo for the ping .

@Newton_Miotto

Does anyone have a project that handles large amounts of data that I can see working?
Or some example of a large company using streamlit.

We have a Streamlit application that we host across three Radiation Oncology clinics in Australia. There are data pipelines running external to Streamlit which collect treatment delivery data and planned delivery data. You asked for an example with “large amounts of data”, so let me detail a rough idea of scale.

We currently have four Elekta Linear Accelerators (Linacs) which are the machines we use to deliver life saving radiation to our patients. These machines have MultiLeaf Collimators (MLCs) which are used to dynamically shape primary radiation as the machine rotates around the patient.

To get an idea here is a video of those MLCs:

- YouTube

We have four of these Linear Accelerators across 3 sites. For every patient delivery at each of these sites the machines save a treatment record file (TRF). These files contain a record of the 160 MLC positions at a time resolution of 25 Hz. A treatment delivery generally takes 1-2 minutes and we treat a new patient every 15 minutes. We then collect these TRF files and have been collecting them for at least the last two years.

---

We have then built a Streamlit application that searches that data and provides meaningful comparisons between the reported MLC positions on the Linac and the planned MLC positions saved within the planning software. We also provide options to compare this data to: our SQL clinical database, the DICOM file format, as well as a live data stream that is being recorded from the Linac.

To be able to build confidence that this application will continue working consistently through code changes and dependency upgrades we have implemented end to end testing with Cypress. A video of the Cypress tests that are undergone within continuous integration for every code commit can be seen at:

Example Cypress e2e video

We pin the Streamlit version explicitly within our setup.py, and when updating the version we make sure the above tests pass. We pin the remaining of our dependencies with a poetry.lock file (https://github.com/pymedphys/pymedphys/blob/main/poetry.lock) using the brilliant poetry tool.

You can install the application yourself by running:

pip install pymedphys[user]==0.35.0
pymedphys gui

There is also a version of this application hosted online:

https://app.pymedphys.com/?app=metersetmap

---

If you’re interested in more, here is a presentation where I demo the application (it’s at about the 5 1/2 minute mark):

ACPSEM Summer School 2020 -- PyMedPhys -- Global collaborative software development

And here is another Streamlit post where I went into a more broad overview of the details:

A Streamlit app used to aid in cancer treatment

Here is a Streamlit app that helps us in our cancer treatment centre: https://share.streamlit.io/pymedphys/streamlit/main/mudensity.py (in the demo try using a Patient ID of 989898 or 979797) At cancer care centres all around the world some patients will get radiation therapy to treat their cancer. To do this a machine called a linear accelerator is used. They use these at CERN to help accelerate particles. We use smaller ones to treat patients with cancer. When this treatment is undergone it…

---

I would like to add, I have confidence using Streamlit because I know that if there is something that Streamlit doesn’t do right now, I have the freedom to make it happen.

A bug bear of ours is that we couldn’t readily download large files within Streamlit as the main library currently stands. So… we fixed it. We have it fixed for our use case within PyMedPhys, and the process of getting that merged into the official Streamlit library is underway:

Create a Download Button API to download files

Also, my favorite part of Streamlit is the following:

@SimonBiggs

A key requirement with applications such as these is that the barrier to entry for other Medical Physicists into the code base needs to be relatively low. Building an application on traditional web stacks is just too much left of field for the average Medical Physicist that I would be introducing a bus factor of 1 within our team. By being able to use Streamlit I can make quality, beautiful and powerful applications, that others on my team can understand the inner workings of when the need arises.

…they can understand the inner workings… but also, make changes, and make new applications. It’s amazing. Streamlit gives me and my team superpowers. I hope your team can fly with Streamlit too .

Cheers,
@SimonBiggs