How to cache unsupported objects?

I’ve a function which accepts a MSExperiment instance as argument and I get the error:

Streamlit cannot hash an object of type <class 'pyopenms.pyopenms_5.MSExperiment'>.

The function returns pandas DataFrames. Is it possible to cache the returned data instead?

Hi @fkromer: We are working on improving caching for cases like this. Can you please share a code snippet?

Also, can you please check and let us know what happens when you run hash() (Python’s built-in hash function) on MSExperiment objects? Thank you! :pray:

Hi @Adrien_Treuille, that sounds great. Here’s a code snippet using pyopenms and an exemplary mzML file 1min.mzML as data source.

from pyopenms import MSExperiment

# path to the 1min.mzML file as string, relative should work as well 
path_to_mzml_file = ...
exp = MSExperiment()
MzFile().load(path_to_mzml_file, exp)
# exp contains the object instance of relevance now

When I run hash(exp):

TypeError: unhashable type: 'pyopenms.pyopenms_5.MSExperiment'
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\ScriptRunner.py", line 311, in _run_script exec(code, module.__dict__)
File "C:\Users\Florian\gitlab\data-explorer\app.py", line 180, in <module> main()
File "C:\Users\Florian\gitlab\data-explorer\app.py", line 39, in main hash(exp) 

WHen using @st.cache() the full trace is:

Streamlit cannot hash an object of type <class 'pyopenms.pyopenms_5.MSExperiment'>.

More information: to prevent unexpected behavior, Streamlit tries to detect mutations in cached objects defined in your local files so it can alert you when the cache is used incorrectly. However, something went wrong while performing this check.

This error can occur when your virtual environment lives in the same folder as your project, since that makes it hard for Streamlit to understand which files it should check. If you think that's what caused this, please add the following to ~/.streamlit/config.toml:

[server]
folderWatchBlacklist = ['foldername']

...where foldername is the relative or absolute path to the folder where you put your virtual environment.

Otherwise, please file a bug here.

To stop this warning from showing in the meantime, try one of the following:

    Preferred: modify your code to avoid using this type of object.
    Or add the argument allow_output_mutation=True to the st.cache decorator.
Streamlit failed to hash an object of type <class 'tuple'>.

<SAME-HINT-AS-ABOVE>
Streamlit failed to hash an object of type <class 'list'>.

<SAME-HINT-AS-ABOVE>
TypeError: object supporting the buffer API required
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\ScriptRunner.py", line 311, in _run_script exec(code, module.__dict__)
File "C:\Users\Florian\gitlab\data-explorer\app.py", line 180, in <module> main()
File "C:\Users\Florian\gitlab\data-explorer\app.py", line 51, in main spectra_dfs = load_spectra(exp)
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\caching.py", line 560, in wrapped_func return get_or_set_cache()
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\caching.py", line 517, in get_or_set_cache args_hasher.update([args, kwargs])
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\hashing.py", line 209, in update self._update(self.hasher, obj, context)
File "c:\users\florian\.virtualenvs\data-explorer-_mha6hq8\lib\site-packages\streamlit\hashing.py", line 241, in _update hasher.update(b) 

Thank you @fkromer. This is very helpful.

Could you share the source of the cached function please?

FYI: I suspect the solution will be to use hash_funcs in a feature which we’re going to start implementing today. Please follow that issue for up-to-date progress information.

Thank you for using Streamlit!

Unfortunatelly that’s not possible. The function should not be relevant cause the caching issue seems to relate to the input argument which is an instance of class MSExperiment. The function returns a 2D pandas DataFrame.

Hi @fkromer: Thank you again for your input. Quick update is that @Jonathan_Rhone has made a lot of progress working on this issue and hope to have solution released soon. Please reach out if you have any questions!

@Adrien_Treuille You are welcome. That sounds great. Thanks.

Seems like you guys merged something into streamlit:develop. Is there an straighforward way to install that version into a venv?

Yep! I don’t know your particular platform, but on OSX or Linux you would just do (replace <activate_your_ve> with whatever you use for your virtualenv):

git clone https://github.com/streamlit/streamlit.git
cd streamlit && make all
cd path/to/your/app && <activate_your_ve>
pip install path/to/streamlit_code_you_just_cloned

That should do it.

I’ve the pleasure to develop under Windows 10 right now. pip install -e path/to/streamlit_code_you_just_cloned gives me ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: C:\Users\Florian\gitlab\data-processing\deps\st reamlit.

1 Like

Ah, my bad, force of habit!

You can remove the -e part of that line.

Does not work. Given <project>/deps/streamlit with \deps>pip install streamlit:

ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Zugriff verweigert: 'c:\\users\\florian\\.virtualenvs\
\data-processing-a_m7qh6p\\lib\\site-packages\\~ornado\\speedups.cp36-win_amd64.pyd'
Consider using the `--user` option or check the permissions.

With \deps>pip install --user streamlit:

ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

Hi @fkromer – I have to apologize, I skipped an important step, which is make build within the streamlit repo. I’m going to correct my earlier post so people reading this post will find the information faster.

But on Windows I am not sure how easy it is to be able to execute Makefiles. :frowning:

Also @nthmost, instead of

pip install path/to/streamlit_code_you_just_cloned

can’t you just do

make install

?

(That is, this works on unix-like operating systems…)

@Adrien_Treuille Oh, yeah, you could do make install instead of make-then-pip-install.

I hope we start pushing develop builds that people can install by pip install githublink tho (or something similar).

As I was thinking about this question I kept thinking that this isn’t simple at all, it requires that you have Make, which even if you’re on Linux is not a given.

For me it’s not urgent to try it out. Could you let me know in which future release it will be contained in? Or is it already included in v0.51.0?

Hi @fkromer,

It’s already included in v0.51.0, but we’re still updating the documentation.

Here’s a link to the docstring that mentions the new hash_funcs param and provides example usage.

https://github.com/streamlit/streamlit/blob/develop/lib/streamlit/caching.py#L459