Hashing error in experimental_memo(), but not cache()

apassy · March 23, 2022, 5:40pm

Getting the following error, but the type referenced is clearly hashable (and works with cache()):

UnhashableParamError : Cannot hash argument ‘drill_type’ (of type utils.ISF_Functions.DrillingType ) in ‘split_forecast_by_commodity’.

Function def:

@st.experimental_memo(ttl=CACHE_LENGTH)  # 12 hours
def split_forecast_by_commodity(pra_id: id, drill_type: DrillingType) -> tuple[dict[str, pd.DataFrame], dict[str, pd.DataFrame], dict[str, pd.DataFrame], dict[str, pd.DataFrame], pd.DataFrame]:

param type def:

class DrillingType(enum.Enum):
    Horizontal = 1
    Vertical = 2

ST 1.4
python 3.9
windows 10

snehankekre · March 24, 2022, 1:44am

Hi @apassy

This is a known, confirmed bug with st.experimental memo():

github.com/streamlit/streamlit

`experimental_memo` fails to hash input arguments that are class instances

opened 03:14AM - 19 Feb 22 UTC

closed 09:30PM - 14 Mar 22 UTC

hayk-skydio

bug cache confirmed 🤝 P2 ⛈

### Summary The `experimental_memo` decorator raises `UnhashableParamError` f…or functions with arguments that are instances of a class or dataclass, even though the hashing code appears to try to handle it using the `__reduce__` method. This limits the usefulness of `experimental_memo`. ### Steps to reproduce ```python import streamlit as st class Foo: def __init__(self, a: int): self.a = a @st.experimental_memo def my_func(foo: Foo) -> int: return foo.a st.write(my_func(Foo(a=5))) ``` **Expected behavior:** I would expect the argument to be hashed properly using its pickled string or data tuple. Based on the code block here: https://github.com/streamlit/streamlit/blob/d44b162909fb8adcae463172c78000029e5d2fef/lib/streamlit/caching/hashing.py#L374-L382 It looks to me like we attempt to call `to_bytes` on the return values of `obj.__reduce__()`. However, the [first return value](https://docs.python.org/3/library/pickle.html#object.__reduce__) is the callable object to reconstruct the instance. This is a function, and it fails to make it through `to_bytes`. I believe it's unnecessary to hash this function, and instead we can either hash the subsequent arguments to `__reduce__` (classes, data state), or hash the result of `pickle.dumps(obj)` directly. As is, we would continue to use `st.cache` instead of `st.experimental_memo` because of this limitation. Notably `experimental_memo` can already output class types, just not input arguments. **Actual behavior:** `UnhashableParamError: Cannot hash argument 'foo' (of type __main__.Foo) in 'my_func'.` ### Is this a regression? No, but the `cache` decorator does work for this case. ### Debug info - Streamlit version: 1.5.1 - Python version: 3.8.11 ### Related links https://github.com/streamlit/streamlit/issues/661 https://github.com/streamlit/streamlit/pull/3791

I would also upvote this issue to show to your interest:

github.com/streamlit/streamlit

st.experimental_memo cannot cache dataclasses.dataclass(frozen=True, eq=True)

opened 02:37PM - 21 Oct 21 UTC

StefanBrand

bug cache P2 ⛈

### Summary According to [Python standard library docs](https://docs.python.o…rg/3/library/dataclasses.html#dataclasses.dataclass), we can make a `dataclasses.dataclass` hashable by providing the parameters `frozen=True, eq=True`. Streamlit still cannot hash such a `dataclass`. > If eq and frozen are both true, by default dataclass() will generate a __hash__() method for you. ### Steps to reproduce Code snippet: ```python from dataclasses import dataclass import streamlit as st @dataclass(frozen=True, eq=True) # hashable class Data: foo: str @st.experimental_memo def process_data(data: Data): return data.foo data = Data("bar") st.info(process_data(data)) ``` results in > > streamlit.caching.cache_errors.UnhashableParamError: Cannot hash argument 'data' (of type `__main__.Data`) in 'process_data'. > > To address this, you can tell Streamlit not to hash this argument by adding a > leading underscore to the argument's name in the function signature: > > ``` > @st.experimental_memo > def process_data(_data, ...): > ... > ``` **Expected behavior:** `st.experimental_memo` can cache a `dataclass(frozen=True, eq=True)`. **Actual behavior:** `st.experimental_memo` cannot cache a `dataclass(frozen=True, eq=True)`. ### Is this a regression? ~no~ yes, it works with `st.cache`: ```python from dataclasses import dataclass import streamlit as st @dataclass(frozen=True, eq=True) # hashable class Data: foo: str @st.cache def process_data(data: Data): return data.foo data = Data("bar") st.info(process_data(data)) ``` ### Workaround Use `dataclasses.asdict` before calling the cached function and re-assemble the `dataclass` later: ```python from dataclasses import asdict, dataclass from typing import Dict import streamlit as st @dataclass(frozen=True, eq=True) # hashable class Data: foo: str @st.experimental_memo def process_data(data: Dict[str, str]): return Data(data["foo"]) data = Data("bar") st.info(process_data(asdict(data))) ``` ### Debug info - Streamlit version: Streamlit, version 1.0.0 - Python version: Python 3.8.10 - Using Conda - OS version: JupyterLab on Linux - Browser version: Firefox