ModuleNotFoundError: No module named 'tiktoken'

🗣️ LLMs and AI
1

I’m trying to deploy an application that uses GPT-3.5 to Streamlit and have created a requirements.txt file with the following dependencies:

openai
python-dotenv
streamlit
bs4
requests
tiktoken
PyPDF2

I keep getting an error that says:

ModuleNotFoundError: No module named 'tiktoken'

Tiktoken runs perfectly fine on my local machine (with Python 3.11) when I don’t try to deploy with Streamlit.

I can’t find anything about this issue so any help would be appreciated. Thanks so much!

2

you can try to use pipreqs to automatically generate your requirements.txt file and try to reboot the website to see if it works, watch this article here: How to Auto Generate requirements.txt (Dependencies) in Python - 3 examples If its still not working, you can try checking out the other forums, i saw that they have included langchain in their requirements.txt, here is the link to it: ModuleNotFoundError: No module named 'faiss'

3

Hey @pixel_panguin,

I was able to successfully install and import tiktoken in this demo app – are you able to share a link to your GitHub repo so we can try to reproduce this issue?

4

Auto-generating the requirements.txt file worked! Thank you both for your help.

5

See if this discussion helps. The module is different but the solution there worked for me for this error.