Testing a chat history for Streamlit – Need your suggestions

[pierrelouisbescond]

Hi Streamliters!

I keep working on my project to improve Streamlit chat interface.

I’ve started to work on a chat history (a functionality existing in ChatGPT that is not native in Streamlit)

The initial concept is to store past conversations as individual cookies and display them in the sidebar, as shown below:

I’m using st.context.cookies to read the conversations stored as cookies and extra_streamlit_components to save/delete them.

However, there are a few issues for which I would be glad to have your thoughts:

1. The past conversations displayed are based on st.context.cookies that scans the existing cookies. However, you must completely reload the app (F5) to have a fresh scan and see new conservations being displayed on the sidebar. I’d love to have a more dynamic update (ex. every time you start a new conversation by clicking on “”). Is there any way to achieve that by forcing a full reload (including the st.context)?

2. The app works fine locally (as usual^^) but, when published on Streamlit cloud, it works during a couple of conversations, displays past conversations history after full reload (cf. 1.) and, at some point, crashes (Something like “Oh no fatal error, get in touch with the support.” From this moment, the st.context.cookies will not work anymore. Even if there is only this command on the streamlit page I’m wondering is there is some kind of limitation on the cookies overall size in the Cloud that does not exist locally? Anyone at Steamlit who could confirm?

feel free to test it here: https://chat-ui-improvement-beta.streamlit.app/
Simply start a discussion and you should see the corresponding cookie being stored with F12… but not displayed even after fully reloading the app (F5):

But it works locally:

3. Finally, I’m open to any new idea to challenge this prototype:

• better libraries to manage cookies?
• other way to store conversations of users? (I was also thinking of storing the conversations linked to the unique ID that Streamlit creates by default (ajs_anonymous_id) in a dedicated folder of the application (like JSONs)… but storing the conversations in the app instead of locally on the user’s computer would imply to encode them for confidentiality matters, etc.

Even if not 100% functional, the code is already available here (I still need to work on the conversations reload): streamlit-chat-ui-improvement/main.py at main · pierrelouisbescond/streamlit-chat-ui-improvement · GitHub

Tagging @Johannes and @arnaudmiribel who I had the chance to chat with lately and who could give a lift to this post!

Thanks for your ideas,
Pierre-Louis

[Goyo]

1. You need to keep the history in memory in some data structure anyway, so you can update it as the conversation goes on.

[jrieke]

Hey @pierrelouisbescond!

Good to hear from you again

I pinged @kajarenc since he’s the engineer who has been working on this.

For 1: Oh I didn’t know that cookies are only read on a full reload, not no a normal rerun. @kajarenc can you confirm? Is this a technical limitation or something we could fix?

For 2: There are some limitations on Cloud (and I assume also using the component to write and st.context to read cookies might create problems), but @kajarenc can probably also shed more light here.

We also still want to add write support for cookies at some point – feel free to upvote or comment on this GitHub issue for it!

[pierrelouisbescond]

@Goyo I’m not a huge fan of adding an extra element in between. I would prefer to always sync with the actual state of cookies.

If one operation fails at some point (ex.: deleting a cookie), having an intermediate element could falsely show that the cookie was deleted whereas it was not.

[pierrelouisbescond]

Thans a lot @jrieke !

Of course, I’m curious to know a bit more from @kajarenc regarding the st.context.cookies behavior and potential limitations!

[kajarenc]

Hey @pierrelouisbescond !
Thank you very much for being an active Streamlit creator!

Related to your questions:

1. We initialize st.context.cookies on app loading (more precisely websocket connection initialization), therefore the reloading is required to get new cookies. This is a intended design/limitation of current implementation
2. Streamlit Cloud filters almost all cookies from the websocket initialization request on their proxy level due to security reasons, so on Cloud, it is expected that you don’t have access to cookies via st.context.cookies. I played with your app deployed on Cloud https://chat-ui-improvement-beta.streamlit.app/, and as expected, there are no “Past Conversation” histories at all, even after the first reload.

Related to better approaches I couldn’t recommend any specific library/solution for now… But we are actively working on an authentification project for Streamlit apps, and it could be a great use case for this.

Basically, after having information about an authenticated user (let’s say a user’s email), you will be able to store it somewhere, e.g., in an external database, and load conversations from there.

Unfortunately I can’t give any time frame for when this will be ready, but this is something we are actively working on.

[pierrelouisbescond]

Hi @kajarenc ,
Thanks for your answers that explain the behavior I witnessed when migrating my app from local to Streamlit Cloud

1. Crystal clear! So, as mentioned by @Goyo , I should look at creating a dynamic element mirroring the content of st.context.cookies, update it dynamically, and which would (re)synchronize every time the app fully reloads.
2. As our production apps are deployed in Azure behind a Vnet, I suspect that we would not “suffer” from Streamlit Cloud limitations. I’ll check if this works.

And, yes, of course, storing the conversations in an external database, linked to an id (ajs_anonymous_id or login) might be a solution… but more complex to handle. Storing the conversations on the user device is easier.

I’ll keep in investigating