Problem with deploying on Streamlit Share - Pickle Error & Relative File Paths

☁️ Community Cloud
1

Hi good people,

I love streamlit and I built a simple web application that works flawlessly on my local machine! I got an invite for streamlit share and wanted to deploy my app now. However, I always run into following error:

EmptyIndexError: Index 'MAIN' does not exist in FileStorage('../whoosh')

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("../whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 619, in read
    % (indexname, storage))
```

I do not think the problem is from Whoosh or from an non existing index file since it works perfectly running locally. I think perhaps the problem is in the relative file path? I am a bit lost, but would love to share the app with people and you!

link to github repo: GitHub - SDG-AI-Lab/NCE_Document_Classification: Working Repo for building a set of models to automate the classification of project log-frames to a comprehensive taxonomy.

EDIT:
I figured that while deploying streamlit takes in parent file paths as (./dir) instead of (…/dir) as on my local machine. However, now, finding the file path I am getting a pickle error that I do not have on my machine:

ValueError: unsupported pickle protocol: 5

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("./whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 664, in read
    segments = stream.read_pickle()File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/filedb/structfile.py", line 245, in read_pickle
    return load_pickle(self.file)
```

Still confused what is going on :slight_smile:

3 Likes
2

Hi @jonas-nothnagel, welcome to the Streamlit community!

In the first case, I would suggest using the pathlib module, which tends to be a bit safer in locating where a file is in the filesystem:

This is usually a mismatch between Python versions. Streamlit sharing uses 3.7 for the time being (the ability to change the version is coming soon), so if you are using a different Python locally, that might be the issue.

Best,
Randy

1 Like
3

Hi @randyzwitch

thanks for answering. Well noted with pathlib.

So I assume one possible solution would be to clone my repo, create a new virtual environment with python 3.7, get it run locally with all necessary dependencies and then deploy this “new” repo?

4

Wouldn’t necessarily need to be a new repo. If you created a new environment using Python 3.7, saved your file as a pickle, and then commit that file to the repo, it should work. You shouldn’t need to make a separate repository just to change the file.

1 Like
5

Hi @randyzwitch

My python scrapping code using selenium connects to Chrome driver. All is going well when I run it locally on Atom. When I run the same code on streamlit share I am getting a PATH error although I made sure that the path is correct:

WebDriverException: Message: ‘chromedriver’ executable needs to be in PATH. Please see ChromeDriver - WebDriver for Chrome

6

chromedriver is almost certainly not installed on Streamlit sharing, but can be done so by adding a packages.txt file to your repo:

https://docs.streamlit.io/en/stable/deploy_streamlit_app.html#put-your-streamlit-app-on-github

Best,
Randy

7

Hi Randy,

i try with packages.txt contains:
chronium-chromedriver
or
chromedriver
all goes wonrg.
i take chromedriver move it projekt root direvtory and had an error:
selenium.common.exceptions.WebDriverException: Message: Service ./chromedriver unexpectedly exited. Status code was: 127
what should be in packages.txt

BR Karl

8

Try with this in your packages.txt file:

chromium
chromium-driver

Here is a very simple example project with selenium on streamlit sharing:

https://share.streamlit.io/franky1/streamlit-selenium/main

9

i have deployed your application and i have this Error
[manager] could not verify installer exit code: open /app/install_status: no such file or directory
the same error is also on my application.
if i delete packages the i can deploy my app.
BR Karl

10

Same for me after rebooting the app…

11

Hi Randy,

I got the same error as Jonas about the ValueError: unsupported pickle protocol: 5.
Is it now possible to specify a version of python in the requirements.txt ? If not, how can I know which python version to apply in my venv ?

Thanks a lot,
Vincent

12

Solved ! I used the “protocol” argument in pickle.dump(), setting it to 4 instead of 5 and it worked :slight_smile:

13

Awesome!

Within Streamlit Cloud, there is also the section for secrets management that should have the ability to set the Python version. If you go to your Streamlit cloud dashboard, at the repo level there is the hamburger menu (3 stacked dots). Clicking on that brings up the secrets interface, and should also have which Python version to run.

Best,
Randy

2 Likes