Hi there - I just wanted to piggyback on this as I had a very similar question and hacked a workaround, but suspect it could be handled better.
In my use case, I want to visualize gradients and attention scores from a large PyTorch model (a BERT model). I have sliders to select attention scores from particular layers and heads in the model, but the gradient calculation is very expensive, taking around 3s on CPU, and I obviously don’t want to recompute them whenever I want to change the layer/head I want to visualize.
Refactoring my code to work with the
@st.cache decorator would take ages (this is a proof of concept and the function that does the calculation takes unhashable arguments) , so my workaround was to directly check and add to
st.caching._mem_cache within this function. As the model is an NLP model, I can use text as cache keys.
Perhaps if there is no convenient replacement for unhashable types (say as in @fonnesbeck’s case), you could use the
id of the object as a cache key instead? This should be fine if it’s in the global scope. Explicitly, I mean something like
obj_id = id(unhashable_type)
if obj_id in streamlit.caching._mem_cache:
return_value = #whatever you want to do with unhashable_type
streamlit.caching._mem_cache[obj_id] = return_value
Presumably this has side-effects I’m not aware of, but interacting with this the cache dictionary explicitly could be useful in general - are there plans to support this type of thing in the future?