Altair Vega compilation different between Streamlit versions

[Dominic_Friend]

I want to keep pace with the new features in the latest releases of Streamlit, but I find myself stuck on 1.9.x as there appears to be Vega compilation differences for one of my Altair charts that I cannot resolve.

I have made no Python code changes but the presentation is changed and much less useful. I’ve included screenshots below of the issue.

1.9.x

1.11.1

Looking at the compiled Vega there are new impute and stack transforms which weren’t there before which I believe are causing the issue.

1.9.x

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  ...
  "data": [
    {"name": "140138747639792"},
    {
      "name": "data_0",
      "source": "140138747639792",
      "transform": [
        {"type": "formula", "expr": "toDate(datum[\"Date\"])", "as": "Date"}
      ]
    }
  ],
...

1.11.1

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  ...
  "data": [
    {"name": "140350643578688"},
    {
      "name": "data_0",
      "source": "140350643578688",
      "transform": [
        {"type": "formula", "expr": "toDate(datum[\"Date\"])", "as": "Date"},
        {
          "type": "impute",
          "field": "Value",
          "groupby": [],
          "key": "Date",
          "method": "value",
          "value": 0
        },
        {
          "type": "stack",
          "groupby": ["Date"],
          "field": "Value",
          "sort": {"field": [], "order": []},
          "as": ["Value_start", "Value_end"],
          "offset": "zero"
        }
      ]
    }
  ],
...

I’m wondering where these extra transform steps are coming from and whether I can take steps to remove them, so far I have been unsuccessful.

[arnaud]

Hi @Dominic_Friend,

Thanks for spotting this. I’m sharing this with our engineering team!
Do you mind sharing your Python code for completeness? Or even better, a minimal reproducible example with mock data? That would be very helpful

Have a nice day,

[Dominic_Friend]

Hi @arnaud

Thanks for the update.
I’ll extract and build a mock example with the raw data over the weekend, will get it to you as soon as it’s ready.

[Dominic_Friend]

Hi @arnaud

Please see below for a minimal example:

import pandas as pd
import streamlit as st
import altair as alt

# Create basic DataFrame
data = pd.DataFrame(
    {
        "Date": [
            "2022-07-08 00:00:00+01:00",
            "2022-07-08 01:00:00+01:00",
            "2022-07-08 02:00:00+01:00",
            "2022-07-08 03:00:00+01:00",
            "2022-07-08 04:00:00+01:00"
        ],
        "Value": [
            19.623721,
            19.624477,
            19.624423,
            19.625287,
            19.625645
        ]
    }
)

# Set data types
data = data.astype(
    {
        "Date": "datetime64[ns, Europe/London]",
        "Value": "float64"
    }
)

uts_chart = alt.Chart(data).mark_area(opacity=0.75).encode(
    alt.X("Date:T"),
    alt.Y(
        "Value:Q",
        axis=alt.Axis(title="Petabytes"),
        scale=alt.Scale(zero=False)
    )
).properties(
    height=600
).configure_axis(
    labelFontSize=16,
    titleFontSize=18
)

st.altair_chart(uts_chart, use_container_width=True)

1.9.x

1.11.1

I hope this helps!

[lukasmasuch]

@Dominic_Friend we have updated the frontend version of vega lite from 4.x to 5.x (see this PR) in version 1.10. I think the issue might be related to this new vega lite version. But I don’t have a good explanation why the two transform steps are added. Maybe you can try to replicate the issue in the vega editor by using the generated vega-lite config of your charts and see if this issue also happens in the vega editor.

[marduk]

Hi there @lukasmasuch,

Earlier today I noticed a similar problem with Plotly (i.e., something works in Streamlit version 1.9.x but stops working in later versions). Posted an issue on github. Could it be that some change in 1.10.0 affected both altair and also plotly?

Sharing my observation here in case there actually is a connection and it helps us find out what’s up

[lukasmasuch]

@marduk We also updated plotly in a recent PR, but I’m not sure if this was already rolled out with the latest Streamlit version (maybe in 1.11.1)

[Dominic_Friend]

Hi @lukasmasuch

I did track down the details about the recent PR during my research, although I hadn’t encountered the vega-lite in browser editor, very useful.

I took the vega-lite source which my Altair chart generated and added in the data statically and was shown the broken view, where the "scale": {"zero": false} appears to be ignored.

{
  "config": {
    "background": "#ffffff",
    "axis": {
      "labelColor": "#31333F",
      "titleColor": "#31333F",
      "gridColor": "rgba(49, 51, 63, 0.2)",
      "labelFont": "\"Source Sans Pro\", sans-serif",
      "titleFont": "\"Source Sans Pro\", sans-serif",
      "labelFontSize": 16,
      "titleFontSize": 18
    },
    "legend": {
      "labelColor": "#31333F",
      "titleColor": "#31333F",
      "labelFont": "\"Source Sans Pro\", sans-serif",
      "titleFont": "\"Source Sans Pro\", sans-serif",
      "labelFontSize": 12,
      "titleFontSize": 12
    },
    "title": {
      "color": "#31333F",
      "subtitleColor": "#31333F",
      "labelFont": "\"Source Sans Pro\", sans-serif",
      "titleFont": "\"Source Sans Pro\", sans-serif",
      "labelFontSize": 12,
      "titleFontSize": 12
    },
    "header": {"labelColor": "#31333F"},
    "view": {"continuousWidth": 400, "continuousHeight": 300}
  },
  "data": {
    "values":[
      { "Date": "2022-07-08 00:00:00+01:00", "Value": 19.623721},
      { "Date": "2022-07-08 01:00:00+01:00", "Value": 19.624477},
      { "Date": "2022-07-08 02:00:00+01:00", "Value": 19.624423},
      { "Date": "2022-07-08 03:00:00+01:00", "Value": 19.625287},
      { "Date": "2022-07-08 04:00:00+01:00", "Value": 19.625645}
    ]
  },
  "mark": {"type": "area", "opacity": 0.75},
  "encoding": {
    "x": {"field": "Date", "type": "temporal"},
    "y": {
      "axis": {"title": "Petabytes"},
      "field": "Value",
      "scale": {"zero": false},
      "type": "quantitative"
    }
  },
  "height": 600,
  "$schema": "https://vega.github.io/schema/vega-lite/v4.17.0.json",
  "autosize": {"type": "fit", "contains": "padding"},
  "width": 666,
  "padding": {"bottom": 20}
}

However not to be defeated again, I came across this stack overflow about a bar-chart How to use zero=false in vega-lite when also using a color encoding? - Stack Overflow where the suggestion was to incorporate "stack": "none" in to the "y" block and surprisingly this did the trick!

Back to my Python code, adding stack=False to the alt.Y block restores the expected presentation.

However this certainly feels more like a work around than a solution as I cannot actually explain why this is now necessary, other than to simply surmise that this is an artefact of Altair still targeting v4 of the vega-lite schema while Streamlit has moved onto v5.