Creating a 'Continue Generation' ChatGPT Button in Streamlit

[SkAndMl]

Hello everyone,
I am trying to create a ‘Continue Generation’ like button in streamlit and I don’t exactly know how to make it work. I used while loop but it’s just running the script again and again and it is also writing over the already generated text. Here is a link to the website - https://poetika.streamlit.app/

Here is the image after clicking the ‘Generate Button’

Here is the image after clicking the ‘Continue generation?’ button

The code snippet for the continue generation button is given below

cont_gen_btn = cont_gen_btn_placeholder.button("Continue generation?")
while cont_gen_btn:
    
    out:torch.Tensor = model.generate(x=out, max_new_tokens=char_len)
    poem: str = decode(out[0].cpu().numpy())
    full_poem += poem
    type_output(poem, placeholder=text_placeholder)

Any suggestion would be helpful. Thanks in advance

[tonykip]

Hi @SkAndMl,

Thanks for posting!

An idea for you to consider;

• you could use st.session_state to store generated poem
• when the user clicks Continue generation?, feed the entire prior generated poem to the model and ask it to continue generating a response
• show the stored poem and continue streaming the new output

Let me know if this helps

[SkAndMl]

Will do that. Thank you