Streaming response Mistral Ai chatbot RAG

LLMs and AI
, , ,
11

To display the streamed response, Mistral’s api passes through a for loop that iterates over all the chunks generated by the client.chat_stream() object.

There are therefore hundreds of chunks for one response.

As there are also hundreds of errors generated when I go through the following solution :

stream_response = client.chat_stream(model, messages)
for chunk in stream_response:
    st.write(chunk.choices[0].delta.content)

I deduced that Streamlit was unable to display a streaming response from the Mistral api.