Here’s the app:
https://share.streamlit.io/crosstabkite/worst-case-analysis/app.py
and a detailed write-up:
The data is simulated, but the idea is that we’re rolling out a new feature, product, UX, etc on a website. I use a beta-binomial model to get a picture of what the key metric looks like after observing the data and whether or not we can rule out worst-case scenarios.
What’s great about doing this as an app is the user can very intuitively see the impact of changing the input assumptions and the decision criteria on the final result.
Feedback very welcome, thanks for checking it out!
-Brian