Persisting Settings

🎈 Using Streamlit
1

Hey, I wanted to ask what’s the recommended coding pattern for persistent streamlit settings.

Currently, I am using shelve to pickle/dill python objects that contain a collection of my settings.

Of course, I could also write the values directly to a DB. Though, saving snapshots of singleton objects allows me to version instances of different settings.

I just wonder if am thinking too complicated here. Maybe there’s a way to store the whole session as is, and reload it later.

I do it like this:

class Setting:
    X = 10

    def options(self):
        self.X = st.number_input("my value", value=self.X)

config = get_last_setting()
config.options()
if st.button("Save new Setting"):
    save_setting(config)

get_active_setting and save_setting would be database functions that retrieve a shelve object of Setting. It somehow works conveniently to have the class variable as default and once we call options, it becomes an instance bound variable, which can be persisted. But it all feels hacky and complicated.

A much more convenient approach would be to just snapshot the current session object so that all UI Elements entries are persisted.

if st.button("Load last session"):
    load_last_session()

#This number input will be automatically set to the value which we used in the last session
config.X = st.number_input("My Value") 

if st.button("Save session"):
    save_session()

Do you think there could be any way to implement that?

1 Like
2

Hey, did you make any progress. I was playing with useing pickle on the session state but without much sucesses.

3

Yes i started to use the awesome pydantic library

Its built on:

import shelve
from pydantic import BaseModel

class PersistentSettings(BaseModel):
    """
    This pydantic model will try to initialize itself from 
    the database upon every instantiation

    It further supplies an update function, that allows to write 
    back any changes into the database, under its key.
    """

    def __init__(self, **data: Any):
        with shelve.open("config.db") as db:
            super().__init__(**db.get("settings", default={}), **data)

    def update(self):
        """
        Persist the pydantic-dict that represents the model
        """
        with shelve.open("config.db") as db: 
            db["settings"] = self.dict()

This persistent model can then be used to create your settings UI. Pass the default values and types to static class variables. Thats a pydantic thing. I find it really readable


class TrainingConfig(PersistentSettings):
    BATCH_SIZE: int = 10
    EPOCHS: int = 10
    EARLY_STOPPING: int = 5

    def parameter_block(self):
        self.BATCH_SIZE = int(st.number_input(
            "Batch Size", 1, 1024,
            int(self.BATCH_SIZE)
        ))

        self.EPOCHS = int(st.number_input(
            "Epochs", 1, 10 * 1000,
            int(self.EPOCHS)
        ))

        self.EARLY_STOPPING = int(st.number_input(
            "Eearly Stopping - Patience in epochs", 1, 10 * 1000,
            int(self.EARLY_STOPPING)
        ))

In your streamlit application, you can use it like this:

# Get your config item, automatically initialized from the DB
config = TrainingConfig()
# Show the UI for your config
config.parameter_block()
# Save any adjustments
if st.button("Update Settings"):
     config.update()

...
# Anytime from any file in your project
# Load the config from DB
config = TrainingConfig()
model.fit(x, y, epochs=config.EPOCHS, ...)

I hope it’s understandable and not overengineered, but I like it very much that it plays nicely with my IDE and type checker. Also your whole config will be json-serializable by default, which is a nice thing to have anyways. And understand that I removed a lot of stuff for readability → this snippet is untested. As always, be careful when ctrl-c from the internet

4 Likes