ERROR while deploying app that uses Pandas profiling lib

Deployment
1

I built a simple app that for exploratory data analysis using pandas_profiling and streamlit_pandas_profiling Librairies. The app works well on the localhost but when deploying I have an error. here is my code and the error that I have while deploying

Here are the error while deploying

ImportError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you’re on Streamlit Cloud, click on ‘Manage app’ in the lower right of your app).

Traceback:

File "/home/appuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 562, in _run_script
    exec(code, module.__dict__)File "/app/data-analysis-app/CKD-Profiling.py", line 9, in <module>
    import pandas_profiling as ppFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/__init__.py", line 7, in <module>
    from pandas_profiling.controller import pandas_decoratorFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/controller/pandas_decorator.py", line 4, in <module>
    from pandas_profiling.__init__ import ProfileReportFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/__init__.py", line 8, in <module>
    from pandas_profiling.profile_report import ProfileReportFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/profile_report.py", line 11, in <module>
    from pandas_profiling.model.describe import describe as describe_dfFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/model/describe.py", line 11, in <module>
    from pandas_profiling.model.correlations import calculate_correlationFile "/home/appuser/venv/lib/python3.9/site-packages/pandas_profiling/model/correlations.py", line 11, in <module>
    from pandas.core.base import DataError
2

Looks like an incompatibility between the libraries pandas and pandas_profiling and should(?) be solved in later versions of pandas_profiling
As a first step i would remove all the version numbers in your requirements.txt file and try again:

pandas
pandas-profiling
streamlit
streamlit-pandas-profiling
1 Like
3

It works, yayyy, thank you so much @Franky1

4

If you have a current and working combination, you can also nail down the version numbers again, then there should be no problems with re-deployments in the future.

If you want to develop your Streamlit app further, note that it is recommended by the maintainers to switch to the ydata-profiling package:

5

Oh great, I didn’t hear of the ydara-profiling package. I will use this one also. Thank you so much for your help @Franky1

6

Hi,
I have similar problem when trying to use streamlit-pandas-profiling and ydata-profiling in the deployed app.
However, the app is taking forever to load without loading. In the manager log it shows that it tries to load multiple versions of pandas trying to find a compatible one. It is stuck on these lines below:

  Downloading pandas-1.3.4.tar.gz (4.7 MB)
━━━━━━ 4.7/4.7 MB 200.8 MB/s eta 0:00:00[2023-09-29 13:46:45.570066] 
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started

When I remove streamlit_pandas_profiling requirement the app loads but a page which uses the library displays error that it can’t find the latter library. I tried with pandas_profiling as well and received such error: https://github.com/jivishov/clintrials/blob/45d0bebfe08b42e02c8b329f03eaffc37cb2b01d/errors/profiling_libraries.txt
The app: https://klinikisinaqlar.streamlit.app/
The page that uses streamlit-pandas-profiling: https://klinikisinaqlar.streamlit.app/Exploratory_Data_Analysis

I removed the streamlit-pandas-profiling from the requirements to let the app load, therefore the page (EDA) displays corresponding error.

7

You need to remove it from the code too.

8

Any suggestions on how to use the profiling function after removing streamlit_pandas_profiling from the code? I did remove it and below is the error I get.

NameError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you’re on Streamlit Cloud, click on ‘Manage app’ in the lower right of your app).

Traceback:

File "/home/appuser/venv/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 541, in _run_script
    exec(code, module.__dict__)File "/app/clintrials/pages/2_Exploratory_Data_Analysis.py", line 60, in <module>
    st_profile_report(pr)
    ^^^^^^^^^^^^^^^^^
9

Wait, I was conflating pandas-profiling with streamlit-pandas-profiling . Indeed the recommended way of integrating ydata-profiling and streamlit is using streamlit-pandas-profiling and ydata-profiling<=3.6.2, which apparently requires pandas==1.3.4. There are binary wheels for python from 3.7 to 3.10 so I guess you are using python 3.11?

Anyway streamlit-pandas-profiling seems to be a dead end…

10

Yes, I use python 3.11 and I need ydata-profiling and streamlit-pandas-profiling and found the ydata version limitation. So, I guess, I need to play with the versions to see which one is good or write the EDA from scratch, like old-school :slight_smile: But, it works on my local machine with python 3.11. Will recheck the versions of everything.

11

Here are some (not quite ideal) alternatives to the pretty much obsolete streamlit-pandas-profiling:

  • Display the report inside an html or iframe component. It works but unfortunately the height of the component must be hardcoded, which causes ugly scrolling behavior.
  • Display the report using st.markdown with unsafe_allow_html=True. Doesn’t seem to work. It kills my browser, probably because of the javascript. Is it possible to get rid of it?
  • Display the report in its own browser tab. There is a lot of information in these reports so having a dedicated tab actually improves the UX in some ways. It may detract from it in other ways, though, but it is worth giving it a thought.
  • If you are ready to code, depending on your inclination, you might have more fun modernizing streamlit-pandas-profiling than writing yet another EDA framework.
1 Like
12

By writing from scratch I meant to have a standalone page with a full EDA python code, but your last item sounds like a fun! Will definitely look into that option.
Thanks for your involvement!

13

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.