Streamlit run with protobuf error

rotatingwheel · May 26, 2022, 1:57am

Dear all,

I am trying to run streamlit under Ubuntu and get the following error:

streamlit
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/streamlit", line 5, in <module>
    from streamlit.cli import main
  File "/home/ubuntu/.local/lib/python3.8/site-packages/streamlit/__init__.py", line 48, in <module>
    from streamlit.proto.RootContainer_pb2 import RootContainer
  File "/home/ubuntu/.local/lib/python3.8/site-packages/streamlit/proto/RootContainer_pb2.py", line                                                                                                  33, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 755, in                                                                                                  __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated wi                                                                                                 th protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and wil                                                                                                 l be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

But I already have protocbuf 4.20.x installed.

Any help is appreciated.
Thanks

taijizhu · May 26, 2022, 5:08am

I am running into the same problem. According to google doc, there is a need to set an environment variable to make it work.
Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
but there seems to be no place where i can set it.

iggyiccy · May 26, 2022, 5:22am

I am getting the same error when I update my code an hour ago, now trying to resolve this. The app is running perfectly in my local environment.

Trembles · May 26, 2022, 5:54am

Same problem on windows for new install.

whitphx · May 26, 2022, 5:58am

I encountered the same problem and avoided it by installing the compatible version of protobuf:


pip install 'protobuf~=3.19.0'

To deploy it on Streamlit Cloud, I added a line to requirements.txt:

github.com

whitphx/streamlit-webrtc-example/blob/ce881ebc31b0286dfa65a46934033305140024af/requirements.txt#L9

      
        
            aiortc==1.3.2
            matplotlib==3.5.1
            numpy==1.22.3
            opencv-python-headless==4.5.5.64
            pydub==0.25.1
            streamlit==1.9.0
            streamlit_webrtc==0.37.0
            typing_extensions==4.1.1
            protobuf~=3.19.0

For core dev team:

This is because the version of protobuf as a dependency of the streamlit package is specified as protobuf = ">=3.6.0, !=3.11", which allows installing major version-ups that may have breaking changes.

github.com

streamlit/streamlit/blob/6720f1726e3741b90dc5a5f7ecbf247ccf5bf88f/lib/Pipfile#L48

      
        
            blinker = "*"
            cachetools = ">=4.0"
            click = ">=7.0"
            # 1.4 introduced the functionality found in python 3.8's importlib.metadata module
            importlib-metadata = ">=1.4"
            numpy = "*"
            packaging = "*"
            pandas = ">=0.21.0"
            pillow = ">=6.2.0"
            # protobuf version 3.11 is incompatible, see https://github.com/streamlit/streamlit/issues/2234
            protobuf = ">=3.6.0, !=3.11"
            pyarrow = "*"
            pydeck = ">=0.1.dev5"
            pympler = ">=0.9"
            python-dateutil = "*"
            requests = "*"
            toml = "*"
            # 5.0 has a fix for etag header: https://github.com/tornadoweb/tornado/issues/2262
            tornado = ">=5.0"
            tzlocal = "*"
            validators = "*"

And it actually led to install a new version 4.21.0, which was released 5 hours ago with breaking changes, as described in the page linked from the error message:

The version specification in Pipfile should be something like protobuf = "~=3.19.0.
→ Set protobuf verion as ~=3.19.0 by whitphx · Pull Request #4779 · streamlit/streamlit · GitHub

snehankekre · May 26, 2022, 6:15am

Thank you all for bringing this issue to our attention! I’ve just notified the team

Eylon_Goor · February 7, 2023, 2:52pm

Hey!
Any updates on this?

I would love to use streamlit together with google ads api, but since most new versions of goolge APIs require protobuf > 4 it is not possible!

I tried using the Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
but this just slows the process way to much to a point where its unusable.

Please fix so we will be able to use streamlit with newer protobuf versions!

grummble-ai · February 14, 2023, 8:30pm

I’m having the same issue as Eylon_Goor with ortools from Google…

Eylon_Goor · March 13, 2023, 12:51pm

Bumping this up again. I want to use streamlit with some Google APIs that require newer versions of protobuf so i cant downgrade to 3.20.x

Any suggestions?

Franky1 · March 13, 2023, 1:32pm

Unless you absolutely need the very latest versions of these Google libraries, these conflicts are usually easy to resolve. If you run in a clean environment:

pip install --upgrade streamlit google-ads

You end up with:

Package                  Version
------------------------ -----------
google-ads               18.0.0
protobuf                 3.20.0
streamlit                1.20.0
... and of course some more packages

…which is not too bad.

The problems or conflicts usually start when you mess around in the local dev environment and manually install packages one by one.

Eylon_Goor · March 13, 2023, 1:56pm

Hey Franky1, thanks for your response.

I have been running in a clean environment , inside a venv.

the issue is when i need to install
streamlit
google-ads
google-cloud-language>=2.6

for the google-cloud-languge, this is not the newest version but the first one that has a new feature i need. This version also creates a conflict between the 3 when it comes to the protobuf version dependency

Do you have plans to upgrade to work with newer versions of protobuf?
this is also an open issue on github that seems to get a lot of requests:

thanks

Franky1 · March 13, 2023, 2:22pm

I disagree.
When i run in a clean environment:

pip install --upgrade streamlit google-ads "google-cloud-language>=2.6"

i end up with:

Package                  Version
------------------------ -----------
google-ads               13.0.0
google-cloud-language    2.9.0
protobuf                 3.20.3
streamlit                1.20.0

…and no conflicts, but google-ads is a bit older.

Eylon_Goor · March 14, 2023, 10:22am

sorry, forgot to mention that google-ads needs to be 18.0.0 or higher, as older packages only support deprecated versions of the Ads API.

Also - most of new versions of any Google API is using the new protobuf.
Do you think you can prioritize migrating to it?

Daniel_Vega · March 16, 2023, 2:40pm

I have been working on this as had the same problem. In my case the app uses tensorflow and then there are even more constraints. Finally and after testing a few versions came up with one that works (pure luck), This is what I use and the app works fine in local Docker and also in Cloud Run.

streamlit==1.17.0
tensorflow==2.11.0
protobuf 3.15.2

alesubbiah · March 17, 2023, 2:33pm

I’m having the same issue with all the google-cloud packages (where secrets I need to access are stored).

Is there any way to prioritize migrating to the new protobuf which it seems all of these require?
Many, many thanks!!!

google-auth==2.11.0
google-cloud==0.34.0
google-cloud-secret-manager==2.12.4
googleapis-common-protos==1.58.0

These are the packages that I’m having problems with more specifically. The versions above have already been downgraded, and still no luck.

The conflict is caused by:
    The user requested protobuf==3.20.1
    google-api-core 2.10.0 depends on protobuf<5.0.0dev and >=3.20.1
    google-cloud-secret-manager 2.12.4 depends on protobuf<5.0.0dev and >=3.19.0
    googleapis-common-protos 1.58.0 depends on protobuf!=3.20.0, !=3.20.1, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <5.0.0dev and >=3.19.5

Franky1 · March 17, 2023, 3:05pm

If i start from scratch in a clean environment with:

pip install -U streamlit tensorflow-cpu

…i end up with no conflicts and these versions:

Package                      Version
---------------------------- -----------
protobuf                     3.19.6
streamlit                    1.20.0
tensorflow-cpu               2.11.0

Same here, if i start from scratch in a clean environment with:

pip install -U streamlit "google-cloud-secret-manager>=2.12.4" "google-cloud>=0.34.0"

…i end up with no conflicts and these versions:

Package                     Version
--------------------------- -----------
google-api-core             2.11.0
google-auth                 2.16.2
google-cloud                0.34.0
google-cloud-secret-manager 2.16.0
googleapis-common-protos    1.58.0
protobuf                    3.20.3
streamlit                   1.20.0

Streamlit run with protobuf error

Hello there 👋🏻

Cookie settings

Strictly necessary cookies

Performance cookies

Functional cookies

Targeting cookies