Streamlit over Resources after a couple of hours

[lautaropacella]

Hi! I’ve share my first streamlit app, which generates music lyrics with AiTextGen. After a couple of hours I always have to reboot it because it says it has gone over the limits. I don’t really understand what’s making it heavy, as the app only generates text and writes it on the screen, its not saving variables.

Any help would be much appreacited.

Link to the app github: tango-generator/tango_generator.py at main · lautaropacella/tango-generator · GitHub

[randyzwitch]

Hi @lautaropacella, welcome to the Streamlit community!

If you can wrap this line in st.cache, it should prevent the scenario where multiple simultaneous users load the model, which could be quite costly in terms of resources (each user gets a separate session in the container).

Best,
Randy

[lautaropacella]

Hi Randy

Thanks for taking the time to answer. But which line are you referring too? I was thinking that maybe the loading the model was the problem, but I’m not sure if I should put the st.cache at importing packages and the model, or in the generation process.

Kind regards

[snehankekre]

Hi @lautaropacella,

I’m assuming Randy meant wrapping st.cache when you’re loading the model. It will prevent the script from re-loading the model with each widget interaction. Additionally, it will reuse the same cached model across multiple simultaneous users, rather than loading a separate 30 MB model into RAM for each user.

Here’s an example:

@st.cache(hash_funcs={aitextgen: id})
def load_model():
  ai = aitextgen(model_folder="trained_model", tokenizer_file="aitextgen.tokenizer.json")
  return ai

ai = load_model()

Implementing the above will also result in an error related to st.set_page_config. The page config command should be the first Streamlit command that is executed. As such, you should move it under your package imports.

To make it easier for you, I’ve cloned your repo and made the required changes here

snehankekre/tango-generator/blob/main/tango_generator.py

Let us know if this leads to fewer reboots!

Happy Streamlit-ing!
Snehan

[lautaropacella]

Hello Snehan!

Thanks so much for taking the time to write it down! this is so kind. It really helps me. I’ve implemented it and hopefully it will help with the crashings.

Again, thanks a lot!
Kind regards