MediaFileManager: Missing file when try to plot two line chart o the same figure

[AhmedSalam22]

MediaFileManager: Missing file when try to plot two line chart on the same figure. sometimes it is working fine but most of the time doesn’t show anything

code

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
class Economics():
    def demand_supply_cruve(self , data=None):
        """Graph demand and supply curve
        inputs
        --------------------------------
        Data is a {price:[] , 'Demand':[1,2,3....] , 'Supply':[1,2,3,....] } per unit

        """
        data =  data if data != None else {'price':list(range(0,201,10)) ,'Demand':list(range(0,401,20))[::-1] , 'Supply':list(range(0,401,20))}
        data = pd.DataFrame(data)
        fig, ax = plt.subplots()
        sns.lineplot( x=data["Demand"], y=data["price"])
        sns.lineplot( data=data, x="Supply", y="price")

        return fig

import streamlit as st 
st.title("Eco")
eco = Economics()
st.pyplot(eco.demand_supply_cruve())

[kmcgrady]

@AhmedSalam22 Thanks for letting us know. I think there’s an intermittent bug, and I have a fix. I’m for some testers; would you be willing to try it out with your use cases?

You can download the update from my dropbox.

streamlit-0.67.1-py2.py3-none-any.whl

First you uninstall streamlit

$ pip uninstall streamlit

Then you install the package using the wheel package.

$ pip install wheel
$ pip install PATH/TO/streamlit-0.67.1-py2.py3-none-any.whl

Once you are done, you can uninstall it just like any other version.

$ pip uninstall streamlit

I would really appreciate the help, but if you do not have the time. I understand.

[AhmedSalam22]

@kmcgrady thanks for share seems its work fine!

[kmcgrady]

That’s great! Keep playing with it and let me know if you notice issues. If all goes well, it should be in the new release in the next week or two.

[anth]

Hi, thanks @kmcgrady.
Im having the same issue. Any ETA on this fix?
Thanks!

[kmcgrady]

Hi @anth! Welcome to the Streamlit Community and thanks for following up! I’m usually good about following up. These have been in the last release (0.69 I believe). Make sure you upgrade and try it out. If there’s still an issue, I can help you troubleshoot.

[anth]

Hi @kmcgrady! Thanks for the quick response.
I checked that Im using streamlit==0.70.0 and I still have the same issue.
Since I’m new using Streamlit, I assume Im missing something here

When I test the Docker image locally, it works good, but after deployining it on a K8s cluster, it have the same issue showing some images. Here’s my setup:
Dockerfile:

FROM private_registry/python:3.8
ENV PYTHONDONTWRITEBYTECODE "true"
ENV PYTHONUNBUFFERED "true"
ENV DEBUG 1
COPY requirements/ requirements/ 
COPY pip.conf /root/.config/pip/pip.conf
RUN pip install -r ./requirements/base.txt --no-cache-dir
COPY daily_report/ /daily_report/
WORKDIR /daily_report

ENTRYPOINT [ "/bin/bash", "./entry_point.sh"]

System version:

# cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"

Requirements:

PyYAML==5.3.1
requests==2.24.0
streamlit==0.70.0
matplotlib==3.3.2
tabulate==0.8.7

Im using matplotlib==3.3.2 to plot the images, and returning the plt object to Streamlit main script, and st.write(data_from_matplotlib) to show them.

About the Issue:

MediaFileManager: Missing file dd66b81cef4a336a58459b445deacfcab405ba9074eec29ef618fdd3.png
MediaFileManager: Missing file 3d8a0f48ab1ec6cd6d10de65adf027c70579448b50bbd205f21e2f7f.png

As a result, some the images are not displayed. But somehow, it works if I refresh the browser.

Thanks for the help!

[kmcgrady]

Thanks @anth for following up.

It will be really difficult to debug without being able to run the code, so if you are able to share your code (or better, provide a toy example that illustrates the problem simply, that would make life a lot easier.

Here are a couple thoughts that might help us solve the problem.

1. The MediaFileManager manages all the media in memory. At the end of the script run, all media that was not used in that run, is deleted (to conserve memory). Based on this, we should ensure everything working though I’m curious with race cases (for example, is this an issue with multiple people requesting the website, perhaps that causes a problem)? Does this occur standalone when one user interacts with it?
2. MatPlotlib is unfortunately not thread safe. So, one should not use st.pyplot with no figure. Even more protection is needed and matplotlib offers a locking mechanism, In another application, I wrote:

import matplotlib
from matplotlib.figure import Figure

matplotlib.use("agg")

_lock = RendererAgg.lock

# ...code...

with _lock:
    fig = Figure()
    ax = fig.subplots()
    # ...perform work on ax
    st.pyplot(fig)

You can see it in action here: https://github.com/kmcgrady/book_reco/blob/master/books.py#L16

Hopefully that helps with a solution. If not and you can provide a simple example that causes the problem, please let me know.

[anth]

Thanks for the help!
Im performing the changes based on your instructions.
I’ll Get back here to confirm if its solved.

Thanks!

[anth]

Hello @kmcgrady, Sorry for the delay on this.

I’ve implemented the with _lock: suggested by you , and it seems to help since the MediaFileManager: Missing file xx.png error raises less often, but the issue it’s still there.

The app its dockerize, and when I use it locally, it works perfectly, but after deploy it to Kubernetes, the issue shows up. It could be the same issue than 1294 on Github.

Thanks!

[anth]

Hi Again. I think I’ve found the root-cause: More than 1 Kubernetes Pod makes FileManager lost track of png files to be plotted.

As a temp workaround I’ve decrease the numbers of Pods to 1, and that’s solve it.

I will try to Kubernetes Volumes to solve it, and get back to you.

Thanks a lot!

[kmcgrady]

@anth I am so sad because I saw the first message while I was on vacation and a follow up on 1294, so I made the task to debug it when I got back. I ended up at the same conclusion as you. Multiple processes make things fail. I just wished I read the message above to save the debugging time .

I wrote up more in detail on the problem in the bug to avoid duplication of information. Needless to say, we can have a short term solution for Streamlit Share and trying to think of a broader solution for deploying outside. If you find Volumes to work, let me know (though I’m hesitant to believe it would…the process owns the memory).

Thanks for following up and pinging me. I appreciate it! These are some interesting challenges we’ll overcome.