Issue with st.experimental_fragment functions

Hi,
I have an app where 2 st.experimental_fragment functions are calling each other. due to app design, I can’t put them together. I am noticing the following.

  1. After function A calls function B, widget update in function B results in global rerun.

Following is a simple code to reproduce it. writeContainer and doButton are 2 fragment functions that update a global counter with writeContainer displaying the counter.

After doButton is clicked, clicking on button in writeContainer results in global rerun as can be seen from the counter value.

Why is this happening?
Pl. advice. Thanks,
Rahul

@st.experimental_fragment
def writeContainer():
with st.session_state.emp.container():
st.write(“------------------------------”)
st.write(f"Counter is {st.session_state.counter}“)
a=st.button(“increment counter”, on_click=incrementCounter,key=“b”)
st.write(”----------------------------------")

def incrementCounter():
st.session_state.counter=st.session_state.counter+1

@st.experimental_fragment
def doButton():
st.button(“update”,on_click=updateButton, key=‘kz’)

def updateButton():
incrementCounter()
writeContainer()

####----Main-------
if ‘counter’ not in st.session_state:
st.session_state.counter=0
st.session_state.counter=st.session_state.counter+1000

container=st.container()
with container:
st.write(“Main”)
doButton()
ec=st.empty()
if ‘emp’ not in st.session_state:
st.session_state.emp=ec
writeContainer()

Hi @rahul789jain,

Please can you format you code so we can see the whitespace? You can use the </> icon in the toolbar.

import streamlit as st

@st.experimental_fragment
def writeContainer():
    with st.session_state.emp.container():
        st.write("------------------------------")
        st.write(f"Counter is {st.session_state.counter}")
        a=st.button("increment counter", on_click=incrementCounter,key="b")
        st.write("----------------------------------")

@st.experimental_fragment 
def doButton():
    st.button("update",on_click=updateButton, key='kz')

def updateButton():
    incrementCounter()
    writeContainer()

def incrementCounter():
    st.session_state.counter=st.session_state.counter+1

####----Main-------
if 'counter' not in st.session_state:
    st.session_state.counter=0
st.session_state.counter=st.session_state.counter+1000

container=st.container()
with container:
    st.write("Main")
    doButton()
    ec=st.empty()
    if 'emp' not in st.session_state:
        st.session_state.emp=ec
    writeContainer()


Try inserting some time.sleep statements to see what’s happening. When you use the button inside doButton it first executes writeContainer as a callback the rerun doButton. When a full rerun occurs, doButton runs first, then writeContainer. Neither button is triggering a full rerun when I try it out. :slight_smile:

import streamlit as st
import time

def incrementCounter():
    st.session_state.counter=st.session_state.counter+1

@st.experimental_fragment
def writeContainer():
    with st.session_state.emp.container():
        with st.spinner("Write container"):
            time.sleep(2)
        st.write("------------------------------")
        st.write(f"Counter is {st.session_state.counter}")
        a=st.button("increment counter", on_click=incrementCounter,key="b")
        st.write("----------------------------------")

def updateButton():
    incrementCounter()
    writeContainer()

@st.experimental_fragment
def doButton():
    with st.spinner("Do button"):
        time.sleep(2)
    st.button("update",on_click=updateButton, key="kz")



####----Main-------
if "counter" not in st.session_state:
    st.session_state.counter=0
    st.session_state.counter=st.session_state.counter+1000

container=st.container()
with container:
    st.write("Main")
    doButton()
    ec=st.empty()
    if "emp" not in st.session_state:
        st.session_state.emp=ec
    writeContainer()

Hi, Thanks for the response. The global rerun is still happening. In your code the above line is indented to be in scope of ‘counter’ session state initialization and hence it doesn’t increase counter on rerun. Pl. take it out of session state scope and you will see the counter increases by 1000 on global rerun.

With timer, you can see that ‘doButton’ function is run when ‘increment Counter’ is clicked. This shouldn’t happen. Pl. do the following steps to see global rerun

  1. First Click on update button. This increases the counter by 1

  2. Next click on ‘increment Counter’ button. This causes global rerun with counter increasing by 1001

Pl. let me know if anything is not clear.
Thanks
Rahul

Oops. I had started from a guess at indentation from the first post and made a typo. Sorry about that. I read it again and see: the full rerun is only when clicking the writeContainer button immediately after the doButton button. Successive clicks of the button in writeContainer don’t trigger the full rerun.

It might help to know what you’re trying to accomplish here, but what’s happening is that the writeContainer fragment executed inline is a different fragment than the one executed by the callback. A fragment function can be called multiple times in an app to create multiple, distinct fragments on the page. Even though they share the same function definition and even though they are visually rendering to the same location because of st.empty, they are nonetheless not the same fragment.

When you click the button in doButton, the callback will overwrite the first element on the page with the “main body” of the writeContainer callback and fill in the elements in the empty container. This overwrites the widget from your inline writeContainer fragment.

Now, when you click on the button in writeContainer, you are rerunning the fragment that was generated in the callback, but this is a weird edge case and that’s what’s causing the full rerun; you’re not actually rerunning a stable fragment on your page.

The combination of callbacks and fragments is still being refined so it’s unclear if this will even be supported. If you can describe your use case and what you actually need to accomplish in your larger app, I might be able to provide a suggestion.

1 Like

Here’s a few more added lines using borders to see the containers and a timestamp at the top so you can see it get overwritten by the callback (and rerendered on a full rerun).

import streamlit as st
import time
import datetime

st.write(datetime.datetime.now())

def incrementCounter():
    st.session_state.counter=st.session_state.counter+1

@st.experimental_fragment
def writeContainer():
    with st.session_state.emp.container(border=True):
        with st.spinner("Write container"):
            time.sleep(2)
        st.write("------------------------------")
        st.write(f"Counter is {st.session_state.counter}")
        a=st.button("increment counter", on_click=incrementCounter,key="b")
        st.write("----------------------------------")

def updateButton():
    incrementCounter()
    writeContainer()

@st.experimental_fragment
def doButton():
    with st.spinner("Do button"):
        time.sleep(2)
    st.button("update",on_click=updateButton, key="kz")



####----Main-------
if "counter" not in st.session_state:
    st.session_state.counter=0
st.session_state.counter=st.session_state.counter+1000

container=st.container(border=True)
with container:
    st.write("Main")
    doButton()
    ec=st.empty()
    if "emp" not in st.session_state:
        st.session_state.emp=ec
    writeContainer()

Hi, Thank you again for the response. So if I understand you correctly, you are saying:

  1. A fragment function can’t call another fragment function in its call back

If so, then it will be difficult to really modularize the application using fragments as they will all need to be stand alone and can’t interact with each other. My use case is as follows.
I have a custom chart with lots of different options that I want to modularize using fragments so that when an option is selected, processing relevant to that option is carried out in its fragment but then it has to update display shown in other fragments as seen below.
After button 1 is pressed, Setting_Display should update based on results of processing in button 1 fragment. But then a user might also update setting directly in setting_display but this will cause a global rerun

If fragments can’t call each other, then their usability would be limited I suppose as in an decent application, pieces all need to interact with each other.
Thoughts?
again, thanks for your response.
Rahul

I have a sample app that does not execute a global rerun except of course during startup, explicit user page reload and rerun. The session variables are shared between fragments, callbacks are kept within the fragmented functions.

So we have 3 fragmented functions, ticker_selector(), display() and chart().

def main():
    cols = st.columns([1, 1], gap='large')
    with cols[0]:
        st.markdown('**1. Ticker Selector**')
        ticker_selector()

    with cols[1]:
        st.markdown('**2. Display**')
        display()

    st.markdown('**3. Plot**')
    chart()

ticker_selector

The ticker_selector() has 2 controls, both are select boxes. User can select ticker symbol [aapl, amzn, msft …] and the interval [30m, 1d]. The values selected are stored in the session variables. That means we can access it anywhere.

@st.experimental_fragment
def ticker_selector():
    def ticker_cb():
        ss.ticker = ss.tss

    def interval_cb():
        ss.interval = ss.int

    st.selectbox('Select ticker', options=TICKERS,
                 key='tss', on_change=ticker_cb, label_visibility='collapsed')
    st.selectbox('Select interval', options=['30m', '1d'], key='int',
                 on_change=interval_cb, label_visibility='collapsed')

display()

Now we also have the display(), it displays the selected ticker from ticker_selector() selections. It also has its own select box for the ticker symbol.

However, one issue that needs to be solved is how can the ticker info in the display() and how can the chart be updated when there are changes in ticker symbol and/or interval. Why this is an issue? It is an issue because both the display() and chart() are fragmented. We may successfully change the ticker symbol in ticker_selector() but they are not yet applied ui-wise in other fragments.

One solution to solve this is to utilize the run_every parameter of the experimental_fragment decorator. Let’s say the value is 2 sec, so the display() will be run every 2 sec. The user can interact on the ticker_selector() while the display() is run every 2 sec, once it is run whatever is there in the display() will be executed.

@st.experimental_fragment(run_every=2)
def display():
    def ticker_cb():
        ss.ticker = ss.tsd

    st.markdown('<span></span>', unsafe_allow_html=True)  # vertical spacer only
    st.write(f'ticker: {ss.ticker}, interval: {ss.interval}')
    st.selectbox('Select ticker', options=TICKERS, key='tsd',
                 on_change=ticker_cb, label_visibility='collapsed')

The user can also override the value of ticker because display() has a select box for ticker selection.

chart

The same principle is used in display() is now applied in chart(). Ticker info is pulled thru api every 20 sec and data is plotted. The session variables ss.ticker and ss.interval are available here which are used to pull specific data.

@st.experimental_fragment(run_every=20)
def chart():
    # Plot chart    
    chart = StreamlitChart(width=None, height=500)
    chart.legend(True)

    interval = ss.interval
    chart.topbar.textbox('symbol', ss.ticker)

    ...

    df = get_bar_data(ss.ticker, interval)  # interval or timeframe
    chart.set(df)

    ...

    chart.load()

The user can interact with the chart, zooming, etc. without global reruns.

complete code

In case someone is interested to experiment etc., here is the complete code.

streamlit_app.py

import streamlit as st
from streamlit import session_state as ss
import pandas as pd
import yfinance as yf
from lightweight_charts.widgets import StreamlitChart


st.set_page_config(layout='wide')


TICKERS = ['AAPL', 'MSFT', 'AMZN', 'GOOGL']


if 'ticker' not in ss:
    ss.ticker = 'AAPL'

if 'interval' not in ss:
    ss.interval = '30m'


def get_bar_data(symbol, timeframe):
    """timeframe = ['15m', '1d'] or interval"""
    if timeframe.endswith('m'):
        period = '60d'
    else:
        period = '3y'

    df = get_data(symbol, period, timeframe)

    if timeframe.endswith('m'):
        df.set_index('Datetime', inplace=True, drop=True)

    return df


def calculate_price(df, interval: str, period: int = 1):
    if interval.endswith('m'):
        if df.index.name == 'Datetime':
            col = df.index
        else:
            col = df['Datetime']
    else:
        col = df['Date']

    return pd.DataFrame({
        'time': col,
        'Close': df['Close'].rolling(window=period).mean()
    }).dropna()


def convert_to_utc_plus_zero(datetime_str):
    dt_with_tz = pd.to_datetime(datetime_str)
    dt_utc_plus_zero = dt_with_tz.tz_convert('UTC')

    return dt_utc_plus_zero


def get_data(ticker: str, period: str, interval: str):
    """Retrieve data from yf.

    ['Date/Datetime', 'Open', 'High', 'Low', 'Close', 'Volume']
    It outputs Datetime if interval is below 1d
    """
    df = yf.Ticker(ticker).history(
        period=period,
        interval=interval,
        auto_adjust=False
    )[['Open', 'High', 'Low', 'Close', 'Volume']].reset_index()
    col_name = 'Date'
    if 'Datetime' in df.columns:
        col_name = 'Datetime'
    df[col_name] = df[col_name].apply(convert_to_utc_plus_zero)

    return df


@st.experimental_fragment
def ticker_selector():
    def ticker_cb():
        ss.ticker = ss.tss

    def interval_cb():
        ss.interval = ss.int

    st.selectbox('Select ticker', options=TICKERS,
                 key='tss', on_change=ticker_cb, label_visibility='collapsed')
    st.selectbox('Select interval', options=['30m', '1d'], key='int',
                 on_change=interval_cb, label_visibility='collapsed')


@st.experimental_fragment(run_every=2)
def display():
    def ticker_cb():
        ss.ticker = ss.tsd

    st.markdown('<span></span>', unsafe_allow_html=True)  # vertical spacer only
    st.write(f'ticker: {ss.ticker}, interval: {ss.interval}')
    st.selectbox('Select ticker', options=TICKERS, key='tsd',
                 on_change=ticker_cb, label_visibility='collapsed')


@st.experimental_fragment(run_every=120)
def chart():
    # Plot chart    
    chart = StreamlitChart(width=None, height=500)
    chart.legend(True)

    interval = ss.interval
    chart.topbar.textbox('symbol', ss.ticker)
    chart.topbar.textbox('interval', f'interval: {interval}')

    df = get_bar_data(ss.ticker, interval)  # interval or timeframe
    chart.set(df)

    # Price close
    price = chart.create_line(name='Close', color='rgb(204, 235, 255)', width=1)
    price_df = calculate_price(df, interval, period=1)
    price.set(price_df)
    chart.load()


def main():
    cols = st.columns([1, 1], gap='large')
    with cols[0]:
        st.markdown('**1. Ticker Selector**')
        ticker_selector()

    with cols[1]:
        st.markdown('**2. Display**')
        display()

    st.markdown('**3. Plot**')
    chart()


if __name__ == '__main__':
    main()

requirements.txt

streamlit==1.34.0
yfinance==0.2.38
lightweight-charts==1.0.21

Hi, Thank you for the response. Use of ‘run_every’ to update UI between fragmented functions I believe is quite suboptimal (what if a fragment function needs to do significant processing) and in case of chart update might result in very bad user experience.
I am wondering if there is a technical reason why fragment functions can’t be designed to be callable from each other. Only then can they be used to build true modular applications. A mental model I have is to execute a fragment function in a call back in the same way as if a ‘pseudo’ widget in the called fragment function has been interacted with.
A ‘fragment queue’ for a fragment function might be used to hold all the fragment functions that are either called via call back or directly from a fragment function. The fragment function can be executed as normal and after that fragment functions in the ‘fragment queue’ can be executed as if a ‘pseudo’ widget in them has been interacted with.
Just throwing out some ideas. Having fragment functions directly interact with each other would be wonderful.
Thanks again for your detailed post.
Rahul

If certain process takes time, streamlit can wait for it to finish and update the ui based on the returned data. Depending on what you are doing, there are many ways to optimize an app. There is data caching, etc.

While the users are waiting for certain things to finish, they can explore some other parts of the app. The fragments ensure that they do not disturb other running fragments.

Let me put the link again in case you miss to read it. The documentation of fragment is very interesting to read. At the end it mentions about its limitations, etc.

The main concept of streamlit especially the data flow is not to be underestimated. The implementation of fragment is an improvement on top of it.

2 Likes

Hey @rahul789jain thanks for all the thoughtful feedback and writeup here.

Dependency between fragments is an interesting concept. Our first goal is to improve the quality and remove the experimental_ prefix on st.fragment. I’m not sure if we will add a feature like this chaining or triggering in the future between fragments but it’s certainly interesting and I can see situations where it would be valuable as you mentioned.

I’m not sure if there’s any technical reason why it couldn’t be possible.

I’d welcome a Github issue tracking this as an enhancement request so we can see the demand in a more systematic way. There’s also a similar proposal here.

1 Like