Error deploying streamlit application-how to fix it?

I am trying to deploy this application: GitHub - creativitylabb/prediction_app
I am using python 3.7 and conda to install the requirements. I tried running the installation with conda env update --file environment.yml --prune in a separate python environment, locally and I managed to start the streamlit server. However, when I try to deploy the application I get this errors:

[manager] Error during processing dependencies! Please fix the error and push an update, or try restarting the app.

[manager] Streamlit server consistently failed status checks

[manager] Please fix the errors, push an update to the git repo, or reboot the app.

Also here is the log:


[client] Provisioning machine...
[client] Preparing system...
[client] Spinning up manager process...
e[32m[manager] e[0mStarting up repository: 'prediction_app', branch: 'main', main module: 'main.py'
e[32m[manager] e[0mCloning repository...
e[32m[manager] e[0mCloning into '/app/prediction_app'...
Filtering content:  66% (2/3)
Filtering content: 100% (3/3)
Filtering content: 100% (3/3), 225.43 KiB | 161.00 KiB/s, done.

e[32m[manager] e[0mCloned repository!
e[32m[manager] e[0mPulling code changes from Github...
e[32m[manager] e[0mProcessing dependencies...
Collecting package metadata (repodata.json): ...working... [2022-04-20 13:59:56.885095] done
Solving environment: ...working... [2022-04-20 14:01:46.076756] done

Downloading and Extracting Packages
Verifying transaction: ...working... [2022-04-20 14:04:25.707580] done
Executing transaction: ...working... [2022-04-20 14:05:02.766052] Enabling notebook extension jupyter-js-widgets/extension...
      - Validating: e[32mOKe[0m

done
Installing pip dependencies: ...working... [2022-04-20 14:06:35.207806] bash: line 3:    16 Killed                  /home/appuser/.conda/bin/conda env update -n base --file environment.yml
e[32m[manager] e[0minstaller returned a non-zero exit code
e[32m[manager] e[0mError during processing dependencies! Please fix the error and push an update, or try restarting the app.

What am I doing wrong in deploying the app?
This is how I created the environment file:

conda env export > environment.yml

After some conflicts, I modified the environment file and this is how the environment.yml file looks like:

name: multivarPrpohetVar2
channels:
  - msys2
  - anaconda
  - conda-forge
  - defaults
dependencies:
  - aiohttp=3.8.1
  - pystan=2.19.1.1
  - aiosignal=1.2.0
  - argon2-cffi=21.3.0
  - argon2-cffi-bindings=21.2.0
  - arviz=0.11.2
  - async-timeout=4.0.1
  - asynctest=0.13.0=py_0
  - attrs=21.4.0
  - backcall=0.2.0
  - blas=1.0=mkl
  - bleach=4.1.0
  - certifi=2021.10.8
  - cffi=1.15.0
  - cftime=1.6.0
  - charset-normalizer=2.0.4
  - colorama=0.4.4
  - convertdate=2.4.0
  - cryptography=36.0.0
  - curl=7.82.0
  - cycler=0.11.0
  - cython=0.29.28
  - debugpy=1.5.1
  - decorator=5.1.1
  - defusedxml=0.7.1
  - elasticsearch=7.13.3
  - entrypoints=0.3
  - ephem=3.7.7.1
  - fbprophet=0.7.1
  - freetype=2.10.4
  - frozenlist=1.2.0
  - hdf4=4.2.15
  - hdf5=1.12.1
  - hijri-converter=2.2.3
  - holidays=0.13
  - intel-openmp=2021.4.0
  - ipykernel=6.9.1
  - ipython=7.31.1
  - ipython_genutils=0.2.0
  - ipywidgets=7.6.5
  - jbig=2.1
  - jedi=0.18.1
  - jinja2=3.0.3
  - jsonschema=3.2.0
  - jupyter=1.0.0
  - jupyter_client=7.1.2
  - jupyter_console=6.4.3
  - jupyter_core=4.9.2
  - jupyterlab_pygments=0.1.2=py_0
  - jupyterlab_widgets=1.0.0
  - kiwisolver=1.4.0
  - korean_lunar_calendar=0.2.1
  - lcms2=2.12
  - libbrotlicommon=1.0.9
  - libbrotlidec=1.0.9
  - libbrotlienc=1.0.9
  - libcurl=7.82.0
  - libdeflate=1.10
  - libnetcdf=4.8.1
  - libpng=1.6.37
  - libssh2=1.10.0
  - libtiff=4.3.0
  - libwebp=1.2.2
  - libwebp-base=1.2.2
  - libxcb=1.13
  - libzip=1.8.0
  - libzlib=1.2.11
  - lunarcalendar=0.0.9=py_0
  - markupsafe=2.0.1
  - matplotlib=3.5.1
  - matplotlib-base=3.5.1
  - matplotlib-inline=0.1.2
  - mistune=0.8.4
  - mkl-service=2.4.0
  - mkl_fft=1.3.1
  - mkl_random=1.2.2
  - munkres=1.1.4=pyh9f0ad1d_0
  - nbclient=0.5.11
  - nbconvert=6.1.0
  - nbformat=5.1.3
  - nest-asyncio=1.5.1
  - netcdf4=1.5.8
  - notebook=6.4.8
  - numpy-base=1.21.5
  - openssl=1.1.1n
  - packaging=21.3=pyhd8ed1ab_0
  - pandas=1.3.4
  - pandocfilters=1.5.0
  - parso=0.8.3
  - patsy=0.5.2
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=9.0.1
  - pip=21.2.4
  - plotly=5.6.0
  - prometheus_client=0.13.1
  - prompt-toolkit=3.0.20
  - prompt_toolkit=3.0.20
  - pthread-stubs=0.4
  - pycparser=2.21
  - pygments=2.11.2
  - pymeeus=0.5.10
  - pyopenssl=22.0.0
  - pyparsing=3.0.7
  - pyqt=5.12.3
  - pyqt-impl=5.12.3
  - pyqt5-sip=4.19.18
  - pyqtchart=5.12
  - pyqtwebengine=5.12.1
  - pyrsistent=0.18.0
  - pysocks=1.7.1
  - python=3.7.11
  - python-dateutil=2.8.2
  - python_abi=3.7=2_cp37m
  - pytz=2022.1
  - pywin32=302
  - pyzmq=22.3.0
  - qt=5.12.9
  - qtconsole=5.2.2
  - qtpy=1.11.2
  - scipy=1.7.3
  - send2trash=1.8.0
  - setuptools=58.0.4
  - six=1.16.0
  - sqlite=3.38.0
  - statsmodels=0.13.2
  - tbb=2021.5.0
  - tenacity=8.0.1
  - terminado=0.13.1
  - testpath=0.5.0
  - tk=8.6.12
  - toolz=0.11.2
  - tqdm=4.63.1
  - traitlets=5.1.1
  - unicodedata2=14.0.0
  - wcwidth=0.2.5
  - webencodings=0.5.1
  - wheel=0.37.1
  - widgetsnbextension=3.5.2
  - xarray=0.20.2
  - xorg-libxau=1.0.9
  - xorg-libxdmcp=1.1.3
  - yarl=1.6.3
  - zipp=3.7.0
  - zlib=1.2.11
  - zstd=1.5.2
  - pip:
    - absl-py==1.0.0
    - altair==4.2.0
    - astunparse==1.6.3
    - backports-zoneinfo==0.2.1
    - beautifulsoup4==4.11.1
    - blinker==1.4
    - bokeh==2.4.2
    - bs4==0.0.1
    - cached-property==1.5.2
    - cachetools==5.0.0
    - chardet==3.0.4
    - click==8.0.3
    - cmdstanpy==0.9.68
    - compress-pickle==2.1.0
    - docopt==0.6.2
    - flatbuffers==2.0
    - fsspec==2022.3.0
    - gast==0.5.3
    - gitdb==4.0.9
    - gitpython==3.1.27
    - gluonts==0.9.3
    - google-auth==2.6.3
    - google-auth-oauthlib==0.4.6
    - google-pasta==0.2.0
    - grpcio==1.44.0
    - h5py==3.6.0
    - hydralit-components==1.0.9
    - idna==2.6
    - joblib==1.1.0
    - keras==2.8.0
    - keras-preprocessing==1.1.2
    - libclang==13.0.0
    - lightgbm==3.0.0
    - llvmlite==0.38.0
    - lxml==4.8.0
    - markdown==3.3.6
    - multitasking==0.0.10
    - nbeats-keras==1.7.0
    - nbeats-pytorch==1.6.0
    - nfoursid==1.0.0
    - numba==0.55.1
    - numpy
    - oauthlib==3.2.0
    - opt-einsum==3.3.0
    - pickle5==0.0.12
    - pipreqs==0.4.11
    - pmdarima==1.8.5
    - prophet==1.0.1
    - protobuf==3.20.0
    - pyarrow==7.0.0
    - pyasn1==0.4.8
    - pyasn1-modules==0.2.8
    - pydantic==1.9.0
    - pydeck==0.7.1
    - pydeprecate==0.3.2
    - pympler==1.0.1
    - python-dotenv==0.19.2
    - pytorch-lightning==1.6.1
    - pytz-deprecation-shim==0.1.0.post0
    - pyyaml==6.0
    - requests
    - requests-oauthlib==1.3.1
    - rsa==4.8
    - scikit-learn==1.0.2
    - semver==2.13.0
    - setuptools-git==1.2
    - simplejson==3.17.6
    - sklearn==0.0
    - smmap==5.0.0
    - soupsieve==2.3.2
    - statsforecast==0.5.3
    - streamlit==1.8.1
    - streamlit-aggrid==0.2.3.post2
    - tbats==1.1.0
    - tensorboard==2.8.0
    - tensorboard-data-server==0.6.1
    - tensorboard-plugin-wit==1.8.1
    - tensorflow==2.8.0
    - tensorflow-io-gcs-filesystem==0.24.0
    - termcolor==1.1.0
    - tf-estimator-nightly==2.8.0.dev2021122109
    - threadpoolctl==3.1.0
    - toml==0.10.2
    - torch==1.11.0
    - torchmetrics==0.8.0
    - typing-extensions==4.1.1
    - tzdata==2022.1
    - tzlocal==4.2
    - ujson==5.2.0
    - urllib3==1.22
    - validators==0.18.2
    - watchdog==2.1.7
    - werkzeug==2.1.1
    - wrapt==1.14.0
    - xgboost==1.5.2
    - yarg==0.1.9
    - yfinance==0.1.70

Can you please offer me some suggestions? I’m out of ideas on what is wrong in deploying my app…

Hey @creativitylab , my first guess would be the dependencies are too big for Streamlit Cloud.

You can try a tool like pipreqs to slim down your install (you can still use conda, which is probably a good idea). This is what it popped out for your project for me (now i want to turn this into a streamlit app…):

My suggestions would be:

  • searching how people got each package running in Streamlit Cloud or look to examples in the gallery (perhaps for prophet for example)
  • use lighter versions of the packages (cpu only)
  • slimming down your app to not use as many big packages (less fun :sweat:)
darts==0.19.0
elasticsearch==8.1.2
fbprophet==0.7.1
keras==2.8.0
matplotlib==3.5.1
numpy==1.22.3
pandas==1.4.2
Pillow==9.1.0
plotly==5.7.0
scikit_learn==1.0.2
streamlit==1.8.1
tensorflow==2.8.0
u8darts==0.17.1

Lines such as these are why it fails, as your conda env specification has Windows-specific packages in it, which of course don’t exist on a Linux server (Streamlit Cloud uses a Debian base image).

However, the solution is the same as @gerardrbentley proposed, which is to use a tool like pipreqs to create a requirements file that only has the packages you personally import, not the structure of the entire environment.

Best,
Randy

Thanks for your responses! I finally found a way to deploy the application. The structure of the environment.yml (which was successful) is below, if anybody else has problems with conda:

name: minimal-streamlit-example
channels:
- defaults
- conda-forge
- ericmjl
dependencies:
- python=3.7
- conda
- pystan=2.19.1.1
- elasticsearch=7.13.3
- fbprophet=0.7.1
- hdf5=1.12.1
- matplotlib=3.5.1
- matplotlib-base=3.5.1
- matplotlib-inline=0.1.2
- pandas=1.3.4
- patsy=0.5.2
- pillow=9.0.1
- pip=21.2.4
- plotly=5.6.0
- scipy=1.7.3
- statsmodels=0.13.2
- keras==2.8.0
- numpy
- pickle5==0.0.12
- pip:
  - streamlit
  - scikit-learn==1.0.2
  - sklearn==0.0
  - tensorflow==2.8.0
  - setuptools-git==1.2
  - cmdstanpy==0.9.5