How to keep previous image in conversation history

🎈 Using Streamlit
1

Summary

I am using streamlit app to build a csv chatbot along with langchain csv agent and chatgpt api. The chatbot is able to response with text, table, plots per user’s input question. But plots from old question in conversation history will disappear when users ask a new question. This only happens to plots/graph. Text or table can still exist. I wonder if it is related to st.seesion_state() I used.

Steps to reproduce

Code snippet:

from langchain.agents import AgentType
from langchain.agents import create_pandas_dataframe_agent
from langchain.callbacks import StreamlitCallbackHandler
from langchain.llms.openai import OpenAI
import streamlit as st
import pandas as pd
import matplotlib.pyplot as plt
from langchain.tools.python.tool import PythonREPLTool
import os

df = pd.read_cvs('https://raw.githubusercontent.com/a-mt/fcc-medical-data-visualizer/master/medical_examination.csv')

if "messages" not in st.session_state or st.sidebar.button("Clear conversation history"):
    st.session_state["messages"] = [{"role": "assistant", "content": "How can I help you?"}]

for msg in st.session_state.messages:
    st.chat_message(msg["role"]).write(msg["content"])

if prompt := st.chat_input(placeholder="What is this data about?"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user").write(prompt)

    llm = OpenAI(openai_api_key="YOUR_API_KEY")
    pandas_df_agent = create_pandas_dataframe_agent(
        llm,
        df,
        verbose=True,
        agent_type=AgentType.OPENAI_FUNCTIONS,
        handle_parsing_errors=True,
    )

    with st.chat_message("assistant"):
        st_cb = StreamlitCallbackHandler(st.container(), expand_new_thoughts=True)
        response = pandas_df_agent.run(st.session_state.messages, callbacks=[st_cb])
        st.session_state.messages.append({"role": "assistant", "content": response})
        st.write(response)
        fig = plt.gcf() 
        if fig:
            st.write(fig)

If applicable, please provide the steps we should take to reproduce the error or specified behavior.

Expected behavior:

All plots in conversation history will stay.

Actual behavior:

say question 1 "to plot distribution of age’, it will show a figure of age distribution. Then ask another question like ‘how many rows and columns are there’. It will return the number of rows and columns but the previous plot of age distribution is gone.

1 Like
2

Same need here.

3

Hi @joebruin ,

The issue you’re facing is maybe because streamlit is clearing your plots. So you have to save the history somewherel
You can do the following:

  1. Create a list to store the plots and their associated messages.
  2. Check if the message is from the assistant and contains a plot (i.e., a fig is not None). If so, append the message and the associated plot to your list.
  3. When displaying the conversation history, iterate through this list and show the plots along with their corresponding messages.

You can try this code

import streamlit as st
import pandas as pd
import matplotlib.pyplot as plt

# Define a list to store the conversation history including plots
conversation_history = []

if "messages" not in st.session_state or st.sidebar.button("Clear conversation history"):
    st.session_state["messages"] = [{"role": "assistant", "content": "How can I help you?"}]

for msg in st.session_state["messages"]:
    st.chat_message(msg["role"]).write(msg["content"])

if prompt := st.chat_input(placeholder="What is this data about?"):
    st.session_state["messages"].append({"role": "user", "content": prompt})
    st.chat_message("user").write(prompt)

    # Assuming your response generates a plot
    response_message, fig = generate_response(prompt)  # Modify this part accordingly

    if fig:
        conversation_history.append((response_message, fig))

# Display the conversation history with plots
for msg, fig in conversation_history:
    st.chat_message("assistant").write(msg)
    st.write(fig)