Hey @timforr, thanks for opening the conversation, as it is the kind of interaction that will drive Streamlit in the future 
.
Sorry for the wall of text, I put a TLDR at the end XD
As a preface, whatever the huge momemtum Streamlit has, it is still a very young product, barely a year since its open-sourcing, so it has yet to bring the features to push all of the use cases the community wants to build around Streamlit. So there comes the question : āwhat use cases does it favor then ?ā.
I donāt want to speak for the founders of Streamlit there (Iām not even a dev for them, I just happen to hang a lot around the forum 
), but my guess is, the very initial user journey for Streamlit was to provide a tool for Data Scientists to quickly design UIs that fits into their workflow. Just drop some st calls in my ML script and I have an interactive UI without rearchitecting my entire script around callbacks.
Iām in my script, I just loaded a Dataframe let me render it, oh I see this column has a large variance, let me bring a selectbox to select the columns I want to apply coloring on to show the gradient, ok letās do a PCA now, letās have a multiselect to select columns to reduce and maybe a checkbox to choose a model, bam now the dataframe is reduced I can train a ML model. Awww the script is rerunning in its entirety I donāt want to recompute the PCA every time let me put @st.cache there so it does not recompute if I donāt change my checkbox for a model, there now I can finish it.
This is a ālinearā script like you call it, and when you stay in this route where each interaction only impacts a small set of functions that follow, Streamlit is a beauty to use. I donāt ever need global state as everything runs in sequence so caching of expensive portions of the app is the only thing I need. And as a data scientist thatās all I need. So it has totally replaced Jupyter notebooks in my workflow, I run interactive exploratory data analysis and machine learning modelling, and sometimes I show it to a customer in a matter of day and even he can play with the ML hyperparameters. I got a meetup video of me doing this in french, and @Jesse_JCharis has got plenty of Youtube videos too with this workflow in mind.
For my Data Scientist hat, Streamlit is a must-have, itās simple, itās fast, and with Custom components coming around, it will bring custom functions like Streamlit direct call to pandas profiling or echarts directly in my analysis.
Now let me change hat from Data Scientist to Software engineer, where I want to build a data web app around Python, and immediately you can see the mental model is not the same and maybe not considered a native use case. Iām much more concerned with tight coupling of user interactions and state change, where a single user interaction should affect many different parts of the application. One of my colleagues is running into this as he tries to do a realtime dashboard to study the effects of the parameters on some KPIs, and we have trouble saving and rerunning the app whenever he wants to fetch data from a database or remove data points from the graph because they are outside the time window. So I can say I hear your frustrations 
While Streamlit has hacks and itās in the roadmap as āprogrammable stateā, I want to draw a parallel with the story of React and global state (which I heard on episode āReact Fundamentals with Ryan Florenceā of the āSoftware Engineering Dailyā podcast, and I hope some React devs appear here to correct me or add more to the story !). I am running into the exact same problems as yours in React, where it looks like every component is rerendered every time I interact with something somewhere (even though it looks optimized, I donāt know how it works under the hood), and I want to control it with some global state. In its design React doesnāt easily provide that and it makes me want to throw it away. React Redux came later in the party and now with the very recent context + React hooks we got another way of doing it.
I feel Streamlit is still very much in this very early React phase, concerned about rendering components in a sequence and not much about global state. I see a part of the forums worried about state and checkbox too on the forums, itās like top 3 most searched term hereā¦ History shows React is very much one of the preferred libraries for web UIs while missing the feature in the first place thanks to people rambling about global state and how to add it. Weāre at this place of the history with Streamlit, devs are very present on this forum and open to any feedback !
TL;DR; Streamlit is young, initially focused on the Data Scientist workflow (not concerned with callback/coupling of user interaction to state) rather than Web Software Engineer. React also lacked easy global state initially while it focused on rerendering everything everytime then it got solutions that got it where it is today. So we still got a lot of time to help design the API for global programmable state which is in the roadmap and that will help drive Streamlit into what React is today in the JS ecosystem