Activating Streamlit affects progam functionality

🎈 Using Streamlit
1

Hi, I’m a Japanese engineer who is new to here.

Summary

I have a problem about my python code that includes streamlit. Excluding the streamlit feature solves the problem. (I currently run the code on my local windows PC.)

Steps to reproduce

You can find two codes below.
(1) mapp.py
(2) mapp_wost.py

The first code includes the streamlit active (see:104-105Line) and does not work properly. The problem is that the counter “rpt” (which counts the number of repetitions of the loop) does not count up normally and fluctuates discontinuously.

The second code lets the streamlit be inactive (see:104-105Line) and the counter “rpt” counts up properly.

Line104: st.title(‘Lap Analysis’)
Line105: st.write(‘## streamit sharing’)

The only difference between the two codes is “104-105Line.” Also running the “(1) mapp.py” is done by typing “streamlit run mapp.py” and hit return, while running the “(2) mapp_wost.py” is done by clicking start button on the VS CODE screen.

##(1) mapp.py##

import datetime
import time
from time import sleep
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
import numpy as np
import pandas as pd
import plotly.express as px
from plotly.offline import plot as plt
import plotly.io as pio
pio.renderers.default = 'colab'
import random 
import streamlit as st
from streamlit_plotly_events import plotly_events

def convert_to_seconds(time_str):
    if time_str == '':
        return 0.0
    
    parts = time_str.split(":")
    if len(parts) == 3:  # format is 'h:m:s.sss'
        hour = int(parts[0])
        minutes = int(parts[1])
        seconds = float(parts[2])
        return hour * 3600 + minutes * 60 + seconds
    elif len(parts) == 2:  # format is 'm:s.sss'
        minutes = int(parts[0])
        seconds = float(parts[1])
        return minutes * 60 + seconds
    elif len(parts) == 1:  # format is 's.sss'
        return float(parts[0])
    else:
        raise ValueError(f"Unexpected time format: {time_str}")
        
def load_url_with_retry(driver, url, max_retries=5, delay_between_retries=7):
    retries = 0
    while retries < max_retries:
        try:
            driver.get(url)
            break
        except TimeoutException:
            retries += 1
            if retries == max_retries:
                raise
            time.sleep(delay_between_retries)

user_agent = [
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36',
    'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0'
]

UA = user_agent[random.randrange(0, len(user_agent), 1)]

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--user-agent=' + UA)
options.add_argument('--disable-blink-features=AutomationControlled')

driver = webdriver.Chrome(
    ChromeDriverManager().install(),
    options=options
)

driver.set_page_load_timeout(30)
driver.implicitly_wait(10)

#######

access_url = 'https://livetiming.getraceresults.com/demo#screen-results'
load_url_with_retry(driver, access_url)
sleep(random.uniform(2,4))

rpt = 0
df_out = pd.DataFrame()

# 各車両ごとのデータフレームを格納する辞書
dfs = {}
dfs_median = {}
df = pd.DataFrame()
df_p = pd.DataFrame()
df_t = pd.DataFrame()

selected_nrs = []
selected_nrs = [36,45,26,34,96,1,4,33,59,12]

# Initialize dataframes for all selected_nrs
for nr in selected_nrs:
    dfs[f'df_{nr}'] = pd.DataFrame()
    
begin = time.time()

code_start = datetime.datetime.now()
print('code開始時間:', code_start.strftime("%Y/%m/%d %H:%M:%S"))

st.title('Lap Analysis')
st.write('## streamit sharing')

while True:
    start = time.time()
    html = driver.page_source  
    soup = BeautifulSoup(html, 'lxml')
    nrs = soup.select('#screen-results > div > table > tbody > tr:nth-child(n+1)')
       
    df_p = df.copy()
    
    rpt = rpt + 1
    print('rpt=', rpt)
    print('span1: ', time.time() - start)
    if time.time() - begin >= 600:
        code_stop = datetime.datetime.now()
        print('code停止時間:', code_stop.strftime("%Y/%m/%d %H:%M:%S"))
        break
    else:
        pass
    sleep(3)
    print('span2: ', time.time() - start)



##(2) mapp_wost.py##

import datetime
import time
from time import sleep
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
import numpy as np
import pandas as pd
import plotly.express as px
from plotly.offline import plot as plt
import plotly.io as pio
pio.renderers.default = 'colab'
import random 
import streamlit as st
from streamlit_plotly_events import plotly_events

def convert_to_seconds(time_str):
    if time_str == '':
        return 0.0
    
    parts = time_str.split(":")
    if len(parts) == 3:  # format is 'h:m:s.sss'
        hour = int(parts[0])
        minutes = int(parts[1])
        seconds = float(parts[2])
        return hour * 3600 + minutes * 60 + seconds
    elif len(parts) == 2:  # format is 'm:s.sss'
        minutes = int(parts[0])
        seconds = float(parts[1])
        return minutes * 60 + seconds
    elif len(parts) == 1:  # format is 's.sss'
        return float(parts[0])
    else:
        raise ValueError(f"Unexpected time format: {time_str}")
        
def load_url_with_retry(driver, url, max_retries=5, delay_between_retries=7):
    retries = 0
    while retries < max_retries:
        try:
            driver.get(url)
            break
        except TimeoutException:
            retries += 1
            if retries == max_retries:
                raise
            time.sleep(delay_between_retries)

user_agent = [
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36',
    'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0'
]

UA = user_agent[random.randrange(0, len(user_agent), 1)]

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--user-agent=' + UA)
options.add_argument('--disable-blink-features=AutomationControlled')

driver = webdriver.Chrome(
    ChromeDriverManager().install(),
    options=options
)

driver.set_page_load_timeout(30)
driver.implicitly_wait(10)

#######

access_url = 'https://livetiming.getraceresults.com/demo#screen-results'
load_url_with_retry(driver, access_url)
sleep(random.uniform(2,4))

rpt = 0
df_out = pd.DataFrame()

# 各車両ごとのデータフレームを格納する辞書
dfs = {}
dfs_median = {}
df = pd.DataFrame()
df_p = pd.DataFrame()
df_t = pd.DataFrame()

selected_nrs = []
selected_nrs = [36,45,26,34,96,1,4,33,59,12]

# Initialize dataframes for all selected_nrs
for nr in selected_nrs:
    dfs[f'df_{nr}'] = pd.DataFrame()
    
begin = time.time()

code_start = datetime.datetime.now()
print('code開始時間:', code_start.strftime("%Y/%m/%d %H:%M:%S"))

# st.title('Lap Analysis')
# st.write('## streamit sharing')

while True:
    start = time.time()
    html = driver.page_source  
    soup = BeautifulSoup(html, 'lxml')
    nrs = soup.select('#screen-results > div > table > tbody > tr:nth-child(n+1)')
       
    df_p = df.copy()
    
    rpt = rpt + 1
    print('rpt=', rpt)
    print('span1: ', time.time() - start)
    if time.time() - begin >= 600:
        code_stop = datetime.datetime.now()
        print('code停止時間:', code_stop.strftime("%Y/%m/%d %H:%M:%S"))
        break
    else:
        pass
    sleep(3)
    print('span2: ', time.time() - start)

If applicable, please provide the steps we should take to reproduce the error or specified behavior.

You can find the behavior I mentioned above simply by running the two codes.

Expected behavior:

Explain what you expect to happen when you run the code above.

I expect the counter “rpt” of the “(1) mapp.py” counts up properly, like “1,2,3,4,5,6,7,9,10,11,12,13,14,15,”.

Actual behavior:
The counter “rpt” of the “(1) mapp.py” does not count up normally and fluctuates discontinuously., like “1,1,2,2,5,5,6,6,2,7,3,3,9,9,4,2,”.

Explain the undesired behavior or error you see when you run the code above.

If you’re seeing an error message, share the full contents of the error message here.

The “rpt” counts the number of repetitions of the loop and is used as a trigger to start displaying the chart when the code comlpleted. So the “rpt” has to work properly for the program.

Debug info

  • Streamlit version: (1.18.1)
  • Python version: (3.10.8)
  • Using Anaconda3
  • OS version: Windows 11
  • Browser version:Google Chrome113.0.5672.127

Requirements file

I currently run the code locally. So I have not yet created the Requirements file.
For needed libraries, the attached codes above include the information.

Links

  • Link to your GitHub repo:
  • Link to your deployed app:

I currently run the code locally. So I have not yet created GitHub repo.