How to extend streamlit watchers to monitor custom user files?

🎈 Using Streamlit
1

Continuing the discussion from How to monitor the filesystem and have streamlit updated when some files are modified?:

Goal

Knowing that Streamlit uses watchdog to monitor source codes, it seams a very appealing feature to be able to watch the user files. For example reloading the app when a data file changes.

Problem

The code in the link was exactly designed for this use case, but it’s no longer working because of changes in Streamlit source code in the newer versions !

2

After some hacking, I managed to get the mentioned code working. Here is my version:

from datetime import datetime as dt
import pandas as pd
import streamlit as st
import gc

import os
from streamlit.web.server import Server
from streamlit.runtime.scriptrunner import get_script_run_ctx

# get run context
ctx = get_script_run_ctx()
# get session id
session_id = ctx.session_id
# get server
for obj in gc.get_objects():
    if type(obj) is Server:
        server = obj
        break
# get session
session_info = server._runtime._get_session_info(session_id)
session = session_info.session
# register watcher
session._local_sources_watcher._register_watcher(
    os.path.join(os.path.dirname(__file__), 'watched.csv'),
    'dummy:watched.csv'
)

st.text(dt.now())
st.dataframe(pd.read_csv('watched.csv', sep=';', decimal=','))

This mostly works fine, except when I try to modify watched.csv with excel while the App is running, this raise this exception :

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\watchdog\observers\api.py", line 205, in run
    self.dispatch_events(self.event_queue)
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\watchdog\observers\api.py", line 381, in dispatch_events
    handler.dispatch(event)
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\watchdog\events.py", line 278, in dispatch
    }[event.event_type](event)
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\streamlit\watcher\event_based_path_watcher.py", line 366, in on_created
    self.handle_path_change_event(event)
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\streamlit\watcher\event_based_path_watcher.py", line 355, in handle_path_change_event
    allow_nonexistent=changed_path_info.allow_nonexistent,
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\streamlit\watcher\util.py", line 56, in calc_md5_with_blocking_retries
    content = _get_file_content_with_blocking_retries(path)
  File "C:\Users\user_name\bin\miniconda3-4.9.2\lib\site-packages\streamlit\watcher\util.py", line 92, in _get_file_content_with_blocking_retries
    content = f.read()
PermissionError: [Errno 13] Permission denied
1 Like