Non-trivial application skeleton from seasoned software/data engineer

Hello Streamliters

I found Streamlit incredibly useful for data visualization and rapid application development. Thanks and kudos!

Because I’m going to build more serious application than a single page, where proper maintanability, extensibility and testability will be important factors I started looking for Streamlit software engineering best practices. How to build Streamlit application in a modular way using test driven development practices.

To share my experiments, I created project to demonstrate how to leverage Streamlit’s built-in power using best software engineering practices and tools:

  • Modular, layered architecture for better maintainability

  • Separation of concerns for improved testability

  • Modern build tools to complete full CI builds in under 30 seconds

  • Automated formatter and linter

  • Test execution and debugging available directly from VS Code

  • Minimal number of external dependencies

  • Web routing using built-in multi-page navigation capabilities

  • Sample reusable, stateful components

  • All pages and components tested with AppTest

  • Vega-Altair example visualization

  • BigQuery integration using the New York Taxi public dataset

  • Authentication skeleton, easily replaceable with OAuth

  • Application state sharing via URL

A few more ideas for the future:

  • Test coverage
  • Implement BigQuery integration tests
  • More visualizations for integrated public dataset
  • Integration with external OAuth provider
  • Redirect to the original page after login
  • Describe load balancer strategies (sticky sessions)
  • Automated dependency updates with Dependabot/Renovate
  • More sophisticated type checking

I would love to hear your opinions, questions, and overall impressions. It would be fantastic if my experiments could be useful to others!

5 Likes