Problem with deploying on Streamlit Share - Pickle Error & Relative File Paths

Hi good people,

I love streamlit and I built a simple web application that works flawlessly on my local machine! I got an invite for streamlit share and wanted to deploy my app now. However, I always run into following error:

EmptyIndexError: Index 'MAIN' does not exist in FileStorage('../whoosh')

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("../whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 619, in read
    % (indexname, storage))
```

I do not think the problem is from Whoosh or from an non existing index file since it works perfectly running locally. I think perhaps the problem is in the relative file path? I am a bit lost, but would love to share the app with people and you!

link to github repo: GitHub - SDG-AI-Lab/NCE_Document_Classification: Working Repo for building a set of models to automate the classification of project log-frames to a comprehensive taxonomy.

EDIT:
I figured that while deploying streamlit takes in parent file paths as (./dir) instead of (…/dir) as on my local machine. However, now, finding the file path I am getting a pickle error that I do not have on my machine:

ValueError: unsupported pickle protocol: 5

Traceback:

```
File "/usr/local/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)File "/app/nce_document_classification/streamlit/main.py", line 32, in <module>
    ix = open_dir("./whoosh")File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/index.py", line 664, in read
    segments = stream.read_pickle()File "/home/appuser/.local/lib/python3.7/site-packages/whoosh/filedb/structfile.py", line 245, in read_pickle
    return load_pickle(self.file)
```

Still confused what is going on :slight_smile:

1 Like

Hi @jonas-nothnagel, welcome to the Streamlit community!

In the first case, I would suggest using the pathlib module, which tends to be a bit safer in locating where a file is in the filesystem:

https://docs.python.org/3/library/pathlib.html

This is usually a mismatch between Python versions. Streamlit sharing uses 3.7 for the time being (the ability to change the version is coming soon), so if you are using a different Python locally, that might be the issue.

Best,
Randy

Hi @randyzwitch

thanks for answering. Well noted with pathlib.

So I assume one possible solution would be to clone my repo, create a new virtual environment with python 3.7, get it run locally with all necessary dependencies and then deploy this “new” repo?

Wouldn’t necessarily need to be a new repo. If you created a new environment using Python 3.7, saved your file as a pickle, and then commit that file to the repo, it should work. You shouldn’t need to make a separate repository just to change the file.