Model Deployment Returning "PermissionError: [Errno 13] Permission denied"

Hello and thank you for taking the time to read this.

I’m attempting to deploy an app that uses a ML model to generate predictions given some user inputs. The model was created using Pycaret.

The files for this model can be found here

The app location can be found here

I’m able to deploy the app locally and when I deploy the app locally everything works as intended. Here’s a screenshot of the main model page:

However when I deploy the app (app.py) to streamlit, it returns a permission error pertaining to the model itself which i’ve copied below:

PermissionError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you’re on Streamlit Cloud, click on ‘Manage app’ in the lower right of your app).
Traceback:
File “/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py”, line 565, in _run_script
exec(code, module.dict)
File “/mount/src/transitcostestimator/streamlit/app.py”, line 34, in
model = pickle.load(open(‘streamlit/finalized_user_model.pkl’, ‘rb’))
File “/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/pipeline.py”, line 152, in setstate
self.memory = state[“_memory”]
File “/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/pipeline.py”, line 176, in memory
self._memory_fit = self._memory.cache(_fit_one)
File “/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/memory.py”, line 447, in cache
ret = super().cache(func, ignore, verbose, mmap_mode)
File “/home/adminuser/venv/lib/python3.9/site-packages/joblib/memory.py”, line 1082, in cache
return MemorizedFunc(
File “/home/adminuser/venv/lib/python3.9/site-packages/joblib/memory.py”, line 447, in init
self.store_backend.store_cached_func_code([
File “/home/adminuser/venv/lib/python3.9/site-packages/joblib/_store_backends.py”, line 272, in store_cached_func_code
self.create_location(func_path)
File “/home/adminuser/venv/lib/python3.9/site-packages/joblib/_store_backends.py”, line 403, in create_location
mkdirp(location)
File “/home/adminuser/venv/lib/python3.9/site-packages/joblib/disk.py”, line 61, in mkdirp
os.makedirs(d)
File “/usr/local/lib/python3.9/os.py”, line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File “/usr/local/lib/python3.9/os.py”, line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File “/usr/local/lib/python3.9/os.py”, line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 5 more times]
File “/usr/local/lib/python3.9/os.py”, line 225, in makedirs
mkdir(name, mode)

This error occurs at this line:

model = pickle.load(open(‘streamlit/finalized_user_model.pkl’, ‘rb’))

However, it also occurs when I used several different methods to load the model (Joblib and Pycaret). Addtionally, it returns the same error when I’m using resource caching, like:

@st.cache_resource
def streamlit_model(model_path):
return load_model(model_path)
model = streamlit_model(‘finalized_user_model’)

Is there a correct way to load a model within Streamlit that I’m not applying here? Is Pycaret potentially an issue here, as well?

I believe the app is attempting to cache the model, but can’t create the directory to store the cache because it doesn’t have permission to save to the cloud. I also could be wrong.

The library version in my requirements file are:

matplotlib == 3.8.0
numpy==1.23.5
pandas==1.5.3
pandas-dq==1.28
pandas-profiling==3.6.6
plotly==5.16.1
plotly-resampler==0.9.1
streamlit==1.21.0
ipython ==8.16.1
pycaret == 3.1.0

I’ve seen a few similar questions with a similar error, however I wasn’t able to resolve the issue with the outcomes of those threads.

Any help or direction is appreciated!

  • Ryan

In revisiting this error this morning, I wasn’t able to resolve the error, but the app runs fine and deploys without error when I comment out the model portion of the code.

This isn’t unexpected, but worth noting. It’s possible that the error I’m encountering is the result of user error and I’m just importing the model incorrectly?

Any input is welcome.

Additionally, when I load the model in using the github raw link like:

@st.cache_resource
def streamlit_model():
# Replace with your GitHub model’s raw URL
url = ‘https://raw.githubusercontent.com/smileshey/TransitCostEstimator/master/models/finalized_user_model.pkl
response = requests.get(url)
model = pickle.loads(response.content)
return model

model = streamlit_model()

instead of loading it as previously described, it returns the same error. However this also works when loading it in locally and the error occurs on deployment:

2023-10-31 17:07:52.493 Uncaught app exception
Traceback (most recent call last):
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 245, in _get_or_create_cached_value
    cached_result = cache.read_result(value_key)
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py", line 447, in read_result
    raise CacheKeyNotFoundError()
streamlit.runtime.caching.cache_errors.CacheKeyNotFoundError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 293, in _handle_cache_miss
    cached_result = cache.read_result(value_key)
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_resource_api.py", line 447, in read_result
    raise CacheKeyNotFoundError()
streamlit.runtime.caching.cache_errors.CacheKeyNotFoundError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
  File "/mount/src/transitcostestimator/streamlit/app.py", line 45, in <module>
    model = streamlit_model()
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 194, in wrapper
    return cached_func(*args, **kwargs)
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 223, in __call__
    return self._get_or_create_cached_value(args, kwargs)
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 248, in _get_or_create_cached_value
    return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
  File "/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/caching/cache_utils.py", line 302, in _handle_cache_miss
    computed_value = self._info.func(*func_args, **func_kwargs)
  File "/mount/src/transitcostestimator/streamlit/app.py", line 42, in streamlit_model
    model = pickle.loads(response.content)
  File "/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/pipeline.py", line 152, in __setstate__
    self.memory = state["_memory"]
  File "/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/pipeline.py", line 176, in memory
    self._memory_fit = self._memory.cache(_fit_one)
  File "/home/adminuser/venv/lib/python3.9/site-packages/pycaret/internal/memory.py", line 447, in cache
    ret = super().cache(func, ignore, verbose, mmap_mode)
  File "/home/adminuser/venv/lib/python3.9/site-packages/joblib/memory.py", line 1082, in cache
    return MemorizedFunc(
  File "/home/adminuser/venv/lib/python3.9/site-packages/joblib/memory.py", line 447, in __init__
    self.store_backend.store_cached_func_code([
  File "/home/adminuser/venv/lib/python3.9/site-packages/joblib/_store_backends.py", line 272, in store_cached_func_code
    self.create_location(func_path)
  File "/home/adminuser/venv/lib/python3.9/site-packages/joblib/_store_backends.py", line 403, in create_location
    mkdirp(location)
  File "/home/adminuser/venv/lib/python3.9/site-packages/joblib/disk.py", line 61, in mkdirp
    os.makedirs(d)
  File "/usr/local/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/lib/python3.9/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 5 more times]
  File "/usr/local/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/folders'

I was able to get the model to deploy correctly.

The solution was to set all caching parameters = False in the model setup. saving the model using Pycaret’s internal save_model and loading using load_model. Additionally, I needed to use resource caching on the streamlit side. My model setup looks like:

s = setup(data, target = 'cost_real_2021', categorical_features= cat_feats, numeric_features= cont_feats, normalize = True, normalize_method = 'zscore', verbose = False, feature_selection = False, low_variance_threshold = 0.1, pca = False, pca_components = 30, remove_multicollinearity = False, multicollinearity_threshold = 0.3, memory= False, system_log= False )

and then saving the model like:

save_model(finalized_user_model, 'models/finalized_user_model')

and loading the model like (in streamlit):

@st.cache_resource() def get_model(): return load_model('models/finalized_user_model') model = get_model()

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.