GPT-4o integration

[pierrelouisbescond]

Hi Streamlit Community,

I was wondering if there was any plan to add some advanced features to st.chat_input to leverage new functionalities from models such as GPT-4-vision or GPT-4o?

The “basic” ones would be:

• uploading pictures or documents
• audio input

As models tend to evolve to multimodality (we the hype we all know ^^), I feel these are important features to prevent users from switching to other frameworks that offer it.

Thanks,
Pierre-Louis

PS: And, yes, I guess it’s a huge work but Streamlit made it so far and so quick recently!

[Enric_Domingo]

That would be awesome! So far I go around this adding other components to the sidebar, which is not ideal but does the job

video tutorial of it if anyone is interested: https://www.youtube.com/watch?v=7i9j8M_zidA

[asehmi]

Nice app. Thanks.

[pierrelouisbescond]

Indeed, this is one possible solution.

I tried to implement an alternative using the new @st.experimental_dialog component:

I find it quite elegant as it does not overload the UI… but I still need to figure a nice way to display pictures within the chat…

[Odrec]

Positioning the widgets in the main area container looks indeed really nice. Right now I have them all in the sidepanel. I position the images in a column next to the chat when a user uploads an image