"No such file or directory" on initializing file attempt

Using Streamlit
1

:wave: howdy, team!

I want my Streamlit app to cache local metadata into a file which may need initially created/“touched”.

Following e.g. W3Schools file initialization works for non-Streamlit Python programs:

 f = open("myfile.txt", "w")

However, within Streamlit, this code sometimes works but lately has been erring more than not even with the various (should not be needed) workarounds I’ve been adding on top:

import os
import streamlit as st 

def put(file, data):
    if not os.path.exists(file):
        # open(file,'x')
        open(file,'w+')

    with open(file,'r+') as ff:
        fd = ff.read()

    if fd != data:
        with open(file,'w+') as ff:
            ff.write(data)

where example function call on Mac:

put("/Users/me/Documents/_cached.json", "test")

Streamlit errors

FileNotFoundError: [Errno 2] No such file or directory: '/Users/me/Documents/_cached.json'

with Traceback rooting in:

File "/Users/me/Documents/home.py", line 447, in put_cache
    F.put(f"/Users/me/documents/files.py",data)

File "/Users/me/documents/files.py", line 66, in put
    open(f,'w+')

I confirm when this error occurs, the file does not initialize within that directory, but I want it to AND infrequently it does.

Evidence this code does work in Python outside Streamlit:

$ ipython
Python 3.11.6 (main, Oct  2 2023, 20:46:14) [Clang 14.0.3 (clang-1403.0.22.14.1)]
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import os

In [2]: def put(file, data):
   ...:     if not os.path.exists(file):
   ...:         # open(file,'x')
   ...:         open(file,'w+')
   ...:
   ...:     with open(file,'r+') as ff:
   ...:         fd = ff.read()
   ...:
   ...:     if fd != data:
   ...:         with open(file,'w+') as ff:
   ...:             ff.write(data)
   ...:

In [3]: put("/Users/me/Downloads/_cached.json", "test")

In [4]: exit

$ pwd
/Users/me/Downloads

$ ls | grep cached
_cached.json

Anytime code sometimes-works-sometimes-errors I default guess race-condition, but I’m failing to see how that might apply in this situation so wanted to ask the experts. I did not find historically similar, but please link if I had poor search foo and my question is a duplicate.

:pray: TIA!

  1. Local (Mac Sonoma 14.1.1 with Firefox 120.0)
  2. NA
  3. Private (and above question is privacy redacted, please discount any typos)
  4. ✓
  5. Streamlit and Python versions:
    $ python --version
Python 3.11.6
$ streamlit --version
Streamlit, version 1.28.2
2

Firstly, Streamlit runs its script from a temporary directory, and sometimes the working directory may not be where you expect it to be. To ensure that you are creating the file in the correct location, you can use the st.set_option('server.folderWatchBlacklist', []) function to disable the folder blacklist in Streamlit. This can help ensure that Streamlit is not ignoring the directory you are trying to work in.

import os
import streamlit as st

# Disable the folder blacklist in Streamlit
st.set_option('server.folderWatchBlacklist', [])

def put(file, data):
    if not os.path.exists(file):
        # Create the file if it doesn't exist
        open(file, 'w').close()

    with open(file, 'r+') as ff:
        fd = ff.read()

    if fd != data:
        with open(file, 'w+') as ff:
            ff.write(data)

# Example function call on Mac:
put("/Users/me/Documents/_cached.json", "test")

additionally ,you might want to consider using os.makedirs to create any necessary directories before creating the file. This ensures that the directory path exists before attempting to create the file.

import os
import streamlit as st

# Disable the folder blacklist in Streamlit
st.set_option('server.folderWatchBlacklist', [])

def put(file, data):
    directory = os.path.dirname(file)
    if not os.path.exists(directory):
        os.makedirs(directory)

    if not os.path.exists(file):
        # Create the file if it doesn't exist
        open(file, 'w').close()

    with open(file, 'r+') as ff:
        fd = ff.read()

    if fd != data:
        with open(file, 'w+') as ff:
            ff.write(data)

# Example function call on Mac:
put("/Users/me/Documents/_cached.json", "test")
3

Huzzah, cheers! FWIW setting this dynamically in code errors …

server.folderWatchBlacklist cannot be set on the fly. Set as command line option, e.g. streamlit run script.py --server.folderWatchBlacklist, or in config.toml instead.

… but this did solve my problem when included in the config:

$ pwd
/Users/me/.streamlit
$ cat config.toml
[browser]
    gatherUsageStats=false

[theme]
    primaryColor="#3d85c6"
    backgroundColor="#FFFFFF"
    secondaryBackgroundColor="#F0F2F6"
    textColor="#262730"
    font="sans serif"

[server]
    # https://github.com/streamlit/streamlit/issues/5178
    fileWatcherType="poll"
    # https://discuss.streamlit.io/t/no-such-file-or-directory-on-initializing-file-attempt/56431
    folderWatchBlacklist=[]

you might want to consider using os.makedirs to create any necessary directories before creating the file

Agreed and I do outside the code snippet I shared ( as the Downloads folder used is guaranteed to exist on Macs.) Thanks for making sure, it’s a good catch!

1 Like
4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.