How to deploy an app in Community Cloud with custom modules?

Community Cloud
1

Summary

I want to deploy an app with custom python modules that are stored in the repo and imported in the pages definitions of the App.

Steps to reproduce

Please have a look at this dummy repo I set up for this question:
GitHub

  • Setup Conda environment
  • Add local directory containing python function via conda develop ./scripts
  • Deploy App locally → Works
    → Deploy App in Cloud → Fails

Expected behavior:

Executing start_app.sh and running the App locally shows that it works. But when I deploy the App in the community cloud I wish it would also find the custom module and not throw an error. How can I run commands such as conda develop ./scripts during the deployment phase?

Actual behavior:

ModuleNotFoundError due to the fact that I can’t tell the App to run conda develop ./scripts

Error in CommunityCloud

[10:59:37] 🐍 Python dependencies were installed from /app/dummy_app/requirements.txt using pip.

Check if streamlit is installed

Streamlit is already installed

[10:59:38] 📦 Processed dependencies!


Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.





[11:00:50] 🐙 Pulling code changes from Github...

[11:00:51] 📦 Processing dependencies...

[11:00:51] 📦 Processed dependencies!

[11:00:52] 🔄 Updated app!

2023-02-10 11:01:57.484 Uncaught app exception

Traceback (most recent call last):

  File "/home/appuser/venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script

    exec(code, module.__dict__)

  File "/app/dummy_app/scripts/app/main_page.py", line 4, in <module>

    from helpers.return_random_number import return_some_number

ModuleNotFoundError: No module named 'helpers'

2023-02-10 11:08:05.974 Uncaught app exception

Traceback (most recent call last):

  File "/home/appuser/venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script

    exec(code, module.__dict__)

  File "/app/dummy_app/scripts/app/main_page.py", line 4, in <module>

    from helpers.return_random_number import return_some_number

ModuleNotFoundError: No module named 'helpers'

Debug info

  • Streamlit version:1.17.0
  • Python version:3.10.8
  • Using Conda

Requirements file

Links

  • Link to your GitHub repo: See above

Additional information

If needed, add any other context about the problem here.

2

I think moving helpers inside app should do.

3

Hello Goyo,

I just tried your suggestion but it also doesn’t work. In any case it would be great if I could leave the folder structure in that way since I want to keep my scripts and app separated.

Here is the error after trying the suggestion:

[11:46:44] 🐍 Python dependencies were installed from /app/dummy_app/requirements.txt using pip.

Check if streamlit is installed

Streamlit is already installed

[11:46:45] 📦 Processed dependencies!


Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.





2023-02-10 11:46:54.669 Uncaught app exception

Traceback (most recent call last):

  File "/home/appuser/venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script

    exec(code, module.__dict__)

  File "scripts/app/main_page.py", line 4, in <module>

    from helpers.return_random_number import return_some_number

ModuleNotFoundError: No module named 'helpers'

2023-02-10 11:52:08.854 Uncaught app exception

Traceback (most recent call last):

  File "/home/appuser/venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script

    exec(code, module.__dict__)

  File "/app/dummy_app/scripts/app/main_page.py", line 4, in <module>

    from helpers.return_random_number import return_some_number

ModuleNotFoundError: No module named 'helpers'
4

Weird. Did you redeploy after the changes?

I branched from Move helpers inside app to test and deployed in streamlit cloud. TL;DR: Importing helpers from the main page worked, but switching pages didn’t.

The main page loaded and I was shown a randon number. But switching pages got me a popup with the message [Errno 2] No such file or directory: 'pages/page_2.py' and nothing worked from that point on. I deleted the st_pages specific code to rely on the builtin pages feature and redeployed the app but it did exactly the same. I had no problems running the app in my computer (Windows 10).

Apparently streamlit cloud doesn’t like something in your folder structure, but I can’t put a finger on it.

5

Hey Goyo,

Thanks for coming back, I rebooted the App but I’ll try later again with a complete new redeployment and let you know how it went.

If you or somebody else have an idea how to solve the issue using the conda develop strategy I’d be immensely thankful.

Best,
Michael

6

I solved the issue by moving all necessary files to the root.

Thank you again for your help!

7

Hello ml_user,

I am also trying to deploy a app with 4 custom files. So my requirements.txt, looks like this:
matplotlib==3.6.3
numpy==1.23.5
pandas==1.5.3
seaborn==0.12.2
streamlit==1.20.0
./predict.py
./explore.py
./about_me.py
./home.py
./img.jpg

however, I am getting error when installing these custom files. All the files are present in my repository root folder itself.

Can you suggest some ideas on what could be wrong here

8

Hello @ Dipankar1997161,

Do you have a repo with the code? If not can you post the error message here please?

9

Do not put those files in requirements.txt. That is for packages that must be installed with pip.

10

Yes, I have the repo with my code: GitHub - Dipankar1997161/SalaryGenius: The following is a web app which predicts the salary based on Country, Experience and Education Level
It’s my first time deploying an app for ML.

Actually I was able to deploy the app when I placed all the function in the app.py, however, that doesn’t make the code clean. For now, I deployed it. But you will find all the files with the functions in that repo.

The files i wish to add are: home.py, explore.py, predict.py, about.py.

Each of these functions are actually individual sections for my app, that way I can add more options while keeping the app.py clean.

The error message was: Could not find predict.py, home.py and so on. I even imported them in app.py

Thing is when I tested the app on my local, it is running, but during deployment, its causing the error.
Thank you for the help in advance.

11

That is weird. Error messages are usually much more informative than that.

FWIW, I was able to deploy your app without issues.

2 Likes
12

Yes, during deployment, I got that error message itself, maybe, because I was only importing a function from my other files rather than the whole file, like

from predict import predict_page
from home import home_page

These lines gave me the errors.
So, what I did, I included all the functions in app.py (the one which is now on my repo) and the app deployed.

what I actually want is for the app.py to be simple and clean and export libraries from other files.
Do you think this will work for the custom modules rather than just calling a single function?

import home
import predict as pd

and then a simple call
for example:
results = pd.predict_page()

13

I am not sure I understand exactly what you mean. Doing that in app.py would conflict with the alias for pandas so don’t do that.

But you certainly can delete the definition of predict_page from app.py and add this instead:

from predict import predict_page
14

I was just giving an example for the predict as pd.

this is my function inside predict.py
def predict_page():
…

when I imported just this as
from predict import predict_page

It works fine in my local.

when I tried to deploy it, I got the error message,
No module found predict

Do I need to mention these custom files in requirements.txt? if yes then which format
pandas
numpy
…
predict

like this??

I even tried
./predict But I think this is wrong.

15

Works for me.

This is my code, just one commit with that exact change ahead of yours:

Deployed here:
https://goyodiaz-salarygenius-app-0nhe2f.streamlit.app/

16

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.