Problem importing module inside streamlit script (intermittent import failure)

[ognend]

Hello,
I am using transformers from my Streamlit app and I am running into a strange issue (streamlit 1.32.0 and I am using AutoTokenizer from transformers).

Here is how I am importing the AutoTokenizer at the top of the ui.py file:

from transformers import AutoTokenizer, T5ForConditionalGeneration

Obviously, transformers are installed locally on my machine.

The statement fails but only intermittently - let’s say every 5-6 or more runs I get the below error. Other times it imports/runs just fine:

❯ poetry run streamlit run ui.py --server.address localhost --server.port 8081

  You can now view your Streamlit app in your browser.

  URL: http://localhost:8081

2024-03-14 08:20:16.233 Uncaught app exception
Traceback (most recent call last):
  File "/home/jojo/dev/recommender-ui-streamlit/.venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 542, in _run_script
    exec(code, module.__dict__)
  File "/home/jojo/dev/recommender-ui-streamlit/ui.py", line 13, in <module>
    from transformers import AutoTokenizer, T5ForConditionalGeneration
ImportError: cannot import name 'AutoTokenizer' from 'transformers' 
(/home/jojo/dev/recommender-ui-streamlit/.venv/lib/python3.10/site-packages/transformers/__init__.py)

[tonykip]

Hi @ognend,

Thanks for sharing your question with the community! Please update your debugging post to include a code snippet and a link to your app’s public GitHub repo – this will allow the community to help you find an answer as quickly as possible. In the meantime, this post will be tagged as needs-more-info.

[tonykip]

Also, are you seeing this error with other versions of Streamlit other than v1.31.1?

[ognend]

Hello. I cannot really link to a repo because it is an internal app to my workplace. The version of streamlit used is 1.32.0 and the smallest snippet of code is literally importing AutoTokenizer from transformers library inside a streamlit app

p.s. Have seen the same issue with 1.31.*

[tonykip]

Thanks for the info. So this is a Streamlit app you’re running locally on your machine. Has it worked at any point when you’ve been building the app?

[tonykip]

One suggestion I have for you is to ensure that the transformers package is correctly installed and accessible within the virtual environment your app is running in; .venv.

You can try running the app without a virtual environment to debug if the issue is stemming from the virtual environment. Sometime if you have multiple virtual environments, the Streamlit run command might not be running in the environment you expect.

Let me know if this works.

[ognend]

The problem is that the import only fails intermittently. I can run the app 10 times and it will work and then I run it the 11th time and it fails. So, the library is correctly installed because it works 10 times in succession

I wonder if it is some kind of a race condition inside streamlit (transformers is a large library) but to be honest, I have never seen this before in Python and my understanding of the language was that import statements are atomic.

[tonykip]

It might be a bit tricky to pinpoint the cause of the error since it is not clear how to reproduce the error but only by running the app several times.

One thing you could try is using try-except block around the import statements and any initialization code to log and potentially identify any patterns when the error occurs.

You ccan also try creating an isolated minimal code that is simplified to that uses the same library to determine wether the problem is with the library itself or the way it’s used in the app.

Let me know if this helps.

[ognend]

Thank you for replying. The issue is that the actual import statement is failing intermittently - which has nothing to do with the way the library is used. I don’t know how streamlit works internally and whether it parses the python file being run - subsequently making decisions how to order the code and generate front end statements? I may file this as a bug. Will also try to get a minimal example. Thanks.

[tonykip]

So Streamlit reruns the apps from top to bottom upon each reload or interaction with app features. Yes, if you’re still running into this problem even after creating a minimal example, please feel free to create an issue on our GitHub repo.

If you run into the same issue with the minimal example, please share the code here.

[Goyo]

Just to discard other causes, you can check the version of transformes before importing AutoTokenizer and try to replicate the issue with a minimal application that just does the relevant imports, in a minimal environment.

Also, could it be a poetry thing?

[ognend]

At this point I am willing to entertain any possibility (poetry…) . Thanks!

[tonykip]

Let us know if the different alternatives work on the minimal example code.

[Thejan]

I also came across this issue. This is what worked for me.

Create a new file. e.g. utils.py and bring your functions that requires transformers into this utils.py instead of keeping them in the streamlit main python script. So imports such as from transformers import AutoTokenizer, T5ForConditionalGeneration should only be inside your utils.py and you should import the necessary utils functions into the streamit main file.