Pydeck superslow?

Using Streamlit
1

hi all !

This is my first message here, so i want to thank you for your wonderful work ! Streamlit is so nice and friendly, great !

I am facing some issues trying to reproduce the pydeck HexagonLayer demo. when i use .csv data from url all works fine and it’s look like this

streamlit_demo
streamlit_demo1127×873 118 KB

But if, data used for pydeck, are dataframe, local .csv or list of dict, then the map is super slow, not usable… While it is working fine in jupyter.

I am using:

  • streamlit 0.69
  • python 3.8.6
  • Ubuntu 18.04.5
  • Firefox 82.0.2
  • Pipenv

2 more things !

  • If i use list of dict as data, then the cache seems not working…
  • Map totally freezing with streamlit 0.70

Maybe i’am doing something wrong ?

Thank you in advance !!

Dorian

App script:

"""
HexagonLayer
==============

Personal injury road accidents in GB from 1979.

The layer aggregates data within the boundary of each hexagon cell.

This example is adapted from the deck.gl documentation.
"""

import streamlit as st
import pandas as pd 
import pydeck as pdk

#LOADING DATA
DATA_URL = "https://raw.githubusercontent.com/visgl/deck.gl-data/master/examples/3d-heatmap/heatmap-data.csv"

@st.cache
def load_data():
    """Choose here your data format"""
    data = pd.read_csv(DATA_URL).dropna() ## super slow if used
    # data.to_csv('./data.csv',index=False)
    # data = pd.read_csv('./data.csv') ## super slow if used
    # data = data.to_dict('records') ## super slow if used

    return data

# LAYER 
def map(data):
    view_state = pdk.ViewState(
        longitude=-1.415,
        latitude=52.2323,
        zoom=6,
        min_zoom=5,
        max_zoom=15,
        pitch=40.5,
        bearing=-27.36,
    )
    
    layer = pdk.Layer(
        "HexagonLayer",
        data,
        get_position=["lng", "lat"],
        auto_highlight=True,
        elevation_scale=height,
        pickable=True,
        elevation_range=[0, 3000],
        extruded=True,
        coverage=1,
        radius=radius,
        opacity=opacity
    )

    st.pydeck_chart(pdk.Deck(
        layers=[layer],
        initial_view_state=view_state, 
        tooltip=True
        )        
    )

# SIDEBARs
radius = st.sidebar.slider("Diameter of ⬡ (meters) ", 100, 20000, 10000, 100) // 2
height = st.sidebar.slider("Height of  ⬡",1, 200, 100, 10)
opacity = st.sidebar.slider('Opacity', 0.01, 1.0, 1.0, 0.01)

# RENDER
data = load_data()

if st.checkbox('Display data ?'):
    data

map(DATA_URL)
# map(data) ## Super slow
2

Hi @Dorian_Massoulier, I used Streamlit 0.71.0 and it worked fine for me, there was no lag in loading map with DATA_URL or loading data from CSV file. I even checked with and without @st.cache even that didn’t affect the speed and 1 more thing I tried different versions of streamlit in all those versions map is freezing.

3

Hi @sree369nidhi , i am trying with streamlit 0.71.0 and it’s freezing when i use dataframe as data, with direfox and chrome. I have this error

Capture du 2020-11-17 00-52-38
Capture du 2020-11-17 00-52-381570×738 259 KB
. It’s working fine with jupyter and with voilà, and my cpu and memory are fine.

4

I have the same problem, did you find a solution?

Thanks

5

I have the same problem too.
With HexagonLayer it freezes after some seconds and with HeatmapLayer it lags and uses even 100% of the GPUs.

6

Just registered to let you know that I am having the same issue. Can interact with the plot for a few seconds, then it stops. Reloading the page will sometimes make it respond again, but usually I must restart streamlit to get a few more seconds of interactivity.

1 Like
7

I’ve opened a similar problem here and I solved it downgrading the streamlit version

8

I still have exactly the same issue (tried with versions 0.69, 0.79 and 0.80). Does anyone have a solution to this?

9

Hi,

This is happening because st.cache doesn’t have enough information to decide whether or not to cachet. There are no input arguments on load_data() to allow it to determine if the output will change or not, so load_data() always runs, affecting speed of the application. You can set st.cache to allow the output to change so that it will cache the data, i.e. st.cache(allow_output_mutation=True). I prefer my data load methods to be explicit about how to handle input args and outputs.

Here’s your program re-written with these concepts in action, and demonstrating all load combinations with no noticeable speed differences.

P.S. You should add your Mapbox API key as every now and then the map refuses to update.

"""
HexagonLayer
==============

Personal injury road accidents in GB from 1979.

The layer aggregates data within the boundary of each hexagon cell.

This example is adapted from the deck.gl documentation.
"""

from typing import Dict, Union
import streamlit as st
import pandas as pd 
import pydeck as pdk

#LOADING DATA
DATA_URL = "https://raw.githubusercontent.com/visgl/deck.gl-data/master/examples/3d-heatmap/heatmap-data.csv"

@st.cache(allow_output_mutation=True)
def load_data(source, output='DF', save=False) -> Union[pd.DataFrame, Dict]:
    data = pd.read_csv(source).dropna()
    if save == True:
        save_data(data)
    if output == 'DF':
        return data
    else:
        return data.to_dict('records')

def save_data(df, file='./data.csv'):
    df.to_csv(file, index=False)

# LAYER 
def map(data, height=10000, radius=100, opacity=1.0):
    st.write((height, radius, opacity))
    view_state = pdk.ViewState(
        longitude=-1.415,
        latitude=52.2323,
        zoom=6,
        min_zoom=5,
        max_zoom=15,
        pitch=40.5,
        bearing=-27.36,
    )
    
    layer = pdk.Layer(
        "HexagonLayer",
        data,
        get_position=["lng", "lat"],
        auto_highlight=True,
        elevation_scale=height,
        pickable=True,
        elevation_range=[0, 3000],
        extruded=True,
        coverage=1,
        radius=radius,
        opacity=opacity
    )

    st.pydeck_chart(pdk.Deck(
        map_style='mapbox://styles/mapbox/light-v9',
        layers=[layer],
        initial_view_state=view_state, 
        tooltip=True
        )        
    )

# SIDEBARs
source = st.sidebar.radio('Data source', ['URL', 'FILE'])
output = st.sidebar.radio('Data output format', ['DF', 'DICT'])
save = st.sidebar.checkbox('Save data', False)
radius = st.sidebar.slider("Diameter of ⬡ (meters) ", 100, 20000, 10000, 100) // 2
height = st.sidebar.slider("Height of  ⬡",1, 200, 100, 10)
opacity = st.sidebar.slider('Opacity', 0.01, 1.0, 1.0, 0.01)

# LOAD
if source == 'FILE' and output == 'DF':
    data = load_data('./data.csv', output='DF', save=save)
elif source == 'FILE' and output == 'DICT':
    data = load_data('./data.csv', output='DICT', save=save)
elif source == 'URL' and output == 'DF':
    data = load_data(DATA_URL, output='DF', save=save)
elif source == 'URL' and output == 'DICT':
    data = load_data(DATA_URL, output='DICT', save=save)

# RENDER
if st.checkbox('Display data ?'):
    data

map(data, height, radius, opacity)```

![image|690x487](upload://x6o3C2J7nYXDbMDvblu9xmKkwXy.jpeg)