Is there a way to have a button behave as a checkbox (keeping its value)

Are you perhaps looking for a toggle?

https://extras.streamlitapp.com/Toggle%20button / GitHub - arnaudmiribel/streamlit-extras: Discover, try, install and share Streamlit re-usable bits we call "extras"!