Module Not Found in Streamlit Path AWS

Deploying Streamlit
#1

Hi everyone,

I’m attempting to deploy a streamlit application on an AWS EC2 instance. Specifically, I am looking to create a dashboard for interacting with a few databases. When I run the app locally, it functions very well and exactly as I expect. However, when I run streamlit on the EC2 instance, it is unable to find certain packages that I certainly have installed. I get the following error:

ModuleNotFoundError: No module named 'boto3'
Traceback:
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)
File "/home/ubuntu/projectfolder/interactive_dashboard.py", line 5, in <module>
    import somefile as sf 
File "/home/ubuntu/projectfolder/somefile.py", line 10, in <module>
    import boto3

(Apologies for the strange formatting, I’m not sure why it looks like that.)

I’ve looked through the various topics in the streamlit forums before asking and I noticed that multiple responses to similar questions mention a requirements.txt file. Is this necessary even when not using the streamlit sharing service? How can I ensure that streamlit searches in the correct path for the modules/requirements?

Thanks for your time!

#2

Hi @suhaasg, welcome to the Streamlit community!

Even when you are deploying elsewhere, you still need to install the packages you need. The requirements.txt file is a Python convention, and Streamlit sharing extends that convention by automatically running that file prior to deployment.

For your AWS instance, you need to install the Python packages somehow. If you have a requirements.txt file, you can install the packages using pip install -r requirements.txt.

Best,
Randy

#3

Hi Randy, thanks for the quick response!

So I’m familiar with having the packages installed, but all of them have been installed through an environment.yml file. Is this not sufficient? This is a dashboard for a much larger project that is already running on a separate EC2 instance, so I didn’t think I needed to add anything else. In fact, in order to run this streamlit app, I cloned our functioning instance just so I would have all the required dependencies installed.

Does streamlit not look for the packages in the conda environment in which it is running?

#4

Streamlit uses the packages relative to the Python interpreter you call. So if you’re using a conda environment and the packages are installed, then everything is fine. But it’s not sufficient to just have the environment.yml file, you have to run it to build the conda environment

#5

Yes, sorry I think I’m not being very clear. I have an environment file, I’ve used it to build a conda environment, I’ve verified that the necessary packages are installed, I’ve activated the conda environment, and now I’m attempting to run the streamlit app from within that environment.

After all of this, I am getting the above error, and I am at a loss as to why.

It’s very possible that I am misunderstanding your response, and if so, I apologize. Hopefully this clears up my question however!

#6

Okay, just making sure :slight_smile:

If it still doesn’t work, and you’ve cloned another EC2 instance, does it put things on the PATH that might be interfering with the Python you are choosing? As I mentioned, Streamlit here isn’t doing anything special, so if Python can’t find your packages, there could be something else happening here.

#7

Hmm so I checked the sys.path and it appears that all the necessary packages and modules are installed in the /home/ubuntu/anaconda3/envs/myenv/lib/python3.6/site-packages directory, which is what i expect.

I’ll have to keep digging then, I’m not sure why this is suddenly happening. The directory structures and environments are exactly the same as the other functioning instances, which makes no sense.

Is it possible that streamlit is looking in the /anaconda3/lib/python3.6/site-packages directory only? Because this directory does not have what we need and would cause imports to fail.

If this is the case, how do I modify the sys.path so that myenv is prioritized?

Or, alternatively, to force streamlit to run with myenv explicitly?

EDIT: I just ran the following in my streamlit app file:

from pip.operations import freeze

x = freeze.freeze()

st.write(list(x))

and it does appear to be looking only in /anaconda3/lib/python3.6/site-packages when running and not in myenv.