Problem with deploying on Streamlit Share - Pickle Error & Relative File Paths

Hi good people,

I love streamlit and I built a simple web application that works flawlessly on my local machine! I got an invite for streamlit share and wanted to deploy my app now. However, I always run into following error:

EmptyIndexError: Index 'MAIN' does not exist in FileStorage('../whoosh')

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("../whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 619, in read
    % (indexname, storage))
```

I do not think the problem is from Whoosh or from an non existing index file since it works perfectly running locally. I think perhaps the problem is in the relative file path? I am a bit lost, but would love to share the app with people and you!

link to github repo: GitHub - SDG-AI-Lab/NCE_Document_Classification: Working Repo for building a set of models to automate the classification of project log-frames to a comprehensive taxonomy.

EDIT:
I figured that while deploying streamlit takes in parent file paths as (./dir) instead of (…/dir) as on my local machine. However, now, finding the file path I am getting a pickle error that I do not have on my machine:

ValueError: unsupported pickle protocol: 5

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("./whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 664, in read
    segments = stream.read_pickle()File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/filedb/structfile.py", line 245, in read_pickle
    return load_pickle(self.file)
```

Still confused what is going on :slight_smile:

2 Likes

Hi @jonas-nothnagel, welcome to the Streamlit community!

In the first case, I would suggest using the pathlib module, which tends to be a bit safer in locating where a file is in the filesystem:

https://docs.python.org/3/library/pathlib.html

This is usually a mismatch between Python versions. Streamlit sharing uses 3.7 for the time being (the ability to change the version is coming soon), so if you are using a different Python locally, that might be the issue.

Best,
Randy

1 Like

Hi @randyzwitch

thanks for answering. Well noted with pathlib.

So I assume one possible solution would be to clone my repo, create a new virtual environment with python 3.7, get it run locally with all necessary dependencies and then deploy this “new” repo?

Wouldn’t necessarily need to be a new repo. If you created a new environment using Python 3.7, saved your file as a pickle, and then commit that file to the repo, it should work. You shouldn’t need to make a separate repository just to change the file.

Hi @randyzwitch

My python scrapping code using selenium connects to Chrome driver. All is going well when I run it locally on Atom. When I run the same code on streamlit share I am getting a PATH error although I made sure that the path is correct:

WebDriverException: Message: ‘chromedriver’ executable needs to be in PATH. Please see ChromeDriver - WebDriver for Chrome

chromedriver is almost certainly not installed on Streamlit sharing, but can be done so by adding a packages.txt file to your repo:

https://docs.streamlit.io/en/stable/deploy_streamlit_app.html#put-your-streamlit-app-on-github

Best,
Randy

Hi Randy,

i try with packages.txt contains:
chronium-chromedriver
or
chromedriver
all goes wonrg.
i take chromedriver move it projekt root direvtory and had an error:
selenium.common.exceptions.WebDriverException: Message: Service ./chromedriver unexpectedly exited. Status code was: 127
what should be in packages.txt

BR Karl

Try with this in your packages.txt file:

chromium
chromium-driver

Here is a very simple example project with selenium on streamlit sharing:

https://share.streamlit.io/franky1/streamlit-selenium/main

i have deployed your application and i have this Error
[manager] could not verify installer exit code: open /app/install_status: no such file or directory
the same error is also on my application.
if i delete packages the i can deploy my app.
BR Karl

Same for me after rebooting the app…