Error after long time using the app [AttributeError: <module 'numba.core.serialize']

Hello everyone!

I would like to ask you about an issue I have recently discovered in my Streamlit-shared app. I uploaded the app in May '21 and has been serviceable ever since to the best of my knowledge. However, I attempted to run some tests a couple of weeks ago for a live demonstration at the office, and it failed to show “prediction” results.

The app link:
https://share.streamlit.io/malonsol/tfm_kschool/main/frontend/flight_delay_predictor.py

The issue encountered:

AttributeError: Can’t get attribute ‘_rebuild_function’ on <module ‘numba.core.serialize’ from ‘/home/appuser/venv/lib/python3.8/site-packages/numba/core/serialize.py’>

Please, could anybody shed some light to the topic? It would be very much appreciated! :slight_smile:

Best wishes,
Mario

1 Like

Hi @malonsol :wave:

It took me a while come up with a fix. As I understand it, the error is triggered when you try loading your shap treeExplainer with joblib.

I think joblib uses numba to help with unpickling. Although you have pinned the joblib version, the version of numba (which is a joblib dependency) installed is the latest 0.54.1. This latest numba version does not contain the _rebuild_function function in its source code.

One of few versions that work is numba==0.51.0. If you pin this in your requirements file, the error disappears :partying_face:

Here’s the requirements file that worked in my fork:

numba==0.51.0
seaborn==0.11.0
xgboost==1.3.3
altair==4.1.0
matplotlib==3.3.2
streamlit==0.80.0
shap==0.39.0
missingno==0.4.2
category_encoders==2.2.2
pandas==1.1.3
numpy==1.20.2
joblib==0.17.0
scikit_learn==0.23.2

Hope this helps!

Happy Streamlit’ing, :balloon:
Snehan

2 Likes

You’re a genious @snehankekre !! Honestly, I am not very proficient in programming, let alone app development… It is great to see how quickly you came up with a clean solution.

Thank you very much for your time. Have a good day!

Mario

1 Like

Hi, I am facing the same issue. However, the solution does not seem to work in my case.
may I ask what python version you used to implement this?

Thanks in advance!
Jaeho
ps I keep getting this error with python version 3.9

The problem doesn’t seem to appear in python 3.8.
However I get the error below when loading an explainer object from shap library

To anyone who is facing the same issue,

In my case, I kept on facing the error above whenever I tried to load the explainers from SHAP. The main reason behind this was the mismatch between the library version used to compile the explainers and the ones used for the Streamlit service. Basically, I had two Conda environments. One for collecting the explainers(Python 3.6), one for Streamlit service(Python >= 3.8). The problem was solved after compiling the explainers with the corresponding Streamlit python version (I assume the scikit learn and SHAP library would adapt to the python version).

1 Like