OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package

🚀 Deployment
,
1

I wanted to use the chatterbot spacy collaborate system and trained data on chatterbot and created a response chat system. However, this error popped on my computer while I was testing it. I solved with pip install spacy en etc with other directions on StackOverflow. However, the deployment system gives the natural unfixed error type. What can I do for that? Thanks.

2

Hi @Baris_Can_Tayiz, welcome to the Streamlit community!

Can you post a code snippet or the repo you are developing against, so we can try to replicate this?

Best,
Randy

3

https://github.com/TROLEYBUS/Tweet-Mention-Tool/tree/master This is the repo you could investigate.

Thanks.

4

I get this error from your repo as well:

OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
Traceback:
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/streamlit/script_runner.py", line 332, in _run_script
    exec(code, module.__dict__)
File "/home/rzwitch/github_work/Tweet-Mention-Tool/app.py", line 38, in <module>
    chatbot = ChatBot("excalibur")
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/chatterbot/chatterbot.py", line 28, in __init__
    self.storage = utils.initialize_class(storage_adapter, **kwargs)
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/chatterbot/utils.py", line 33, in initialize_class
    return Class(*args, **kwargs)
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/chatterbot/storage/sql_storage.py", line 20, in __init__
    super().__init__(**kwargs)
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/chatterbot/storage/storage_adapter.py", line 21, in __init__
    'tagger_language', languages.ENG
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/chatterbot/tagging.py", line 13, in __init__
    self.nlp = spacy.load(self.language.ISO_639_1.lower())
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/spacy/__init__.py", line 30, in load
    return util.load_model(name, **overrides)
File "/home/rzwitch/miniconda3/envs/tmt/lib/python3.7/site-packages/spacy/util.py", line 175, in load_model
    raise IOError(Errors.E050.format(name=name))

According to this GitHub issue, it looks like you need to download the model as part of your build process. I’m not familiar enough with Heroku to tell you how to do that, but I suspect you add the pip install statement to your setup.sh file.

5

Actually problem is that I tried it on my pc with Streamlit run app.py and it worked. However, requirements.txt probably don’t get the spacy updated version or something like that. I want to publish my apps through Streamlit site but there could be problems with pickle documents.

Spacy en version is needed and I found this solution and trying it now.
https://stackoverflow.com/questions/61702357/how-to-add-a-spacy-model-to-a-requirements-txt-file

also using pipreqs to list libraries in requirements.txt

So I used pipreqs --force to my local working system, It didn’t load spacy requirement for Chatterbot. I don’t know but Streamlit maybe doesn’t support Chatterbot with spacy.

So I trained my Chatterbot with spacy en model and but requirements part don’t communicate such as local system operations.

Finally, I changed requirements.txt as here https://stackoverflow.com/questions/54214800/spacyoserror-e050-cant-find-model-en-on-heroku-platform but it still gives the same error.
Therefore, I couldn’t solve it.

Then I changed the setup.sh file but It didn’t work.
Deployment of the NLP model is problematic. I trained chatterbot with 300 MB or 200 MB twitter data from Kaggle. Probably 2M or similar lines of tweets. My purpose was to create a chatbot similar the thing with spacy+chatterbot colab. It didn’t work out.

If you think the model is useless or If you think the model could be beneficial, I can ask you or from any technical worker if he can create chatterbot models with spacy and if you can reach me with some code or pdf or something, It could be helpful.

I am open to other recommendations to create AI models etc. Thank you for your understanding. Have a great day.

6

Right, that’s what I showed with my cloning of the repo. It works locally, because you have the language file you need already on your computer somewhere. However, that’s not reproduced by using your requirements.txt file, so it doesn’t work on my computer and reasonably doesn’t work on Heroku. The solution is to download that file somehow during the deployment process.

It appears these language models are available via GitHub, so if you can find a way to either call
python3 -m spacy download en_core_web_sm or
pip3 install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz from your setup.sh file it feels like that should be the solution

7

Looks like this works on the requirements.txt

en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz

2 Likes
8

Thank you. I will handle this tomorrow. I was just busy with other website problems. My host website failed and stopped its process. I will handle this tomorrow and I will inform you about the process.

1 Like
9

use anaconda navigator create new virtual environment name it tensorflow_development
install nltk, tensorflow, sklearn, numpy==1.19.3, chatterbot , chatterbot_corpus

for certain libraries use pip install to install chatterbot==0.8.7, chatterbot_corpus==1.1.4

launch the newly created env and use it for all machine learning development it will definitely work bec conda takes care of all the dependency issuses

10

Hi everyone, reviving an old thread. I tried the last suggestion, and it works. However, I’m getting this error now. OSError: [E053] Could not read config.cfg from /home/appuser/venv/lib/python3.7/site-packages/en_core_web_sm/en_core_web_sm-2.2.0/config.cfg Any ideas?

Edit 1:

Found a solution!

Adding the following to my requirements.txt file solved my issue:
spacy>=2.2.0,<3.0.0
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz#egg=en_core_web_sm

I was having an issue with the latest version of Spacy.