I am trying to deploy an app (Edit: on Streamlit Community Cloud) that uses the library snowflake-snowpark-python. I have tested the app locally and it works fine, but when I deploy I am getting the error:
UserWarning: You have an incompatible version of 'pyarrow' installed (11.0.0), please install a version that adheres to: 'pyarrow<8.1.0,>=8.0.0; extra == "pandas"'
snowflake-snowpark-python requires pyarrow=8.0; I have added the dependency snowflake-snowpark-python=1.1.0 to my environment.yml file in my repo. What I am seeing is that during the Streamlit package installation pyarrow 8.0.0 is uninstalled and replaced by pyarrow 11.0.0, even though Streamlit only requires pyarrow>=4.0. I am using the latest version of Streamlit (I’ve not specified the version in the requirements file).
If I look through the logs, as part of the streamlit package installation I see this:
I have tested including pyarrow=8.0.0 in my environment.yml file to ‘force’ streamlit to keep v8.0.0 but it still gets uninstalled. I have also tried a requirements.txt file instead and this gives the same error. What am I doing wrong here?
and completely redeployed the app in Streamlit Cloud, and I am still getting the error
/home/appuser/venv/lib/python3.8/site-packages/snowflake/connector/options.py:96: UserWarning: You have an incompatible version of 'pyarrow' installed (11.0.0), please install a version that adheres to: 'pyarrow<8.1.0,>=8.0.0; extra == "pandas"'
According to the Streamlit app logs I am indeed using snowflake-connector-python 3.0.1 and
snowflake-snowpark-python 1.2.0 but the snowflake connector is still asking for pyarrow 8.0 . I can’t seem to force installation of pyarrow 8.0 either, as when Streamlit gets installed it uninstalls 8.0 and replaces it with 11.0 .
Thanks Franky, I think the to_pandas() error stems from the pyarrow error since I wasn’t facing either error locally.
I have finally managed to resolve the issue following your advice and using pip instead of conda, below are the contents of my requirements.txt file - I had to specify the version of pyarrow explicity (the version was deduced from the error message that Streamlit was giving me with Snowpark when I didn’t specify it). It all deploys and reads in the dataframe without the to_pandas() error: