Streamlit.file_uploader file size limits?

Hi All,

Can’t get the file_uploader to work with size >200MB, would appreciate some help. Thanks!

Running a Python 3.10.4 env on Ubuntu 20.04, using Streamlit 1.10.0, this is the minimum code:

import streamlit as st
import os

st.config.set_option("server.maxUploadSize", 5000)
st.config.set_option("server.maxMessageSize", 5000)
st.config.set_option("server.enableWebsocketCompression", 'true')

uploaded_file = st.file_uploader('Upload Large File')

if uploaded_file is not None:
    bytes_data = uploaded_file.getvalue()
    st.success('Uploaded Successfully')
    save_uploadedfile(uploaded_file)
    with open(os.path.join("/tmp/",uploadedfile.name),"wb") as f:
        f.write(uploadedfile.getbuffer())
    st.success("Saved File:{} to /tmp/".format(uploadedfile.name))

Executing with:

/opt/python/3.10.4/bin/streamlit run upload_error.py  --logger.level=debug \
   2>/tmp/streamlit/logs.txt 

I can upload a file with size 199MB successfully, but not one with size 201MB.

Files were created with:

fallocate -l 199M oneninenine_mb.txt
fallocate -l 201M twoohone_mb.txt

Output of each attached here:

logs_199.txt

2022-06-20 05:17:04.742 Initialized tornado logs
2022-06-20 05:17:04.742 No singleton. Registering one.
2022-06-20 05:17:04.744 Watcher created for /home/lsimpson/src/streamlit-test/pages
2022-06-20 05:17:04.744 Setting up signal handler
2022-06-20 05:17:04.744 Using selector: EpollSelector
2022-06-20 05:17:04.746 Starting server...
2022-06-20 05:17:04.746 Serving static content from /opt/python/3.10.4/lib/python3.10/site-packages/streamlit/static
2022-06-20 05:17:04.747 Server started on port 8501
2022-06-20 05:17:04.748 Server state: State.INITIAL -> State.WAITING_FOR_FIRST_BROWSER
2022-06-20 05:17:04.785 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:17:04.787 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:17:04.791 Starting new HTTP connection (1): checkip.amazonaws.com:80
2022-06-20 05:17:04.891 http://checkip.amazonaws.com:80 "GET / HTTP/1.1" 200 13
2022-06-20 05:17:04.895 Encoding detection: ascii is most likely the one.
2022-06-20 05:17:04.895 Skipping PyPI version check
2022-06-20 05:17:10.309 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:17:10.310 AppSession initialized (id=1a9cfd89-4e48-452c-b85b-1f0e54476afb)
2022-06-20 05:17:10.310 Created new session for ws 139817969382160. Session ID: 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:17:10.310 Server state: State.WAITING_FOR_FIRST_BROWSER -> State.ONE_OR_MORE_BROWSERS_CONNECTED
2022-06-20 05:17:10.594 Received the following back message:
rerun_script {
  widget_states {
  }
}

2022-06-20 05:17:10.595 Beginning script thread
2022-06-20 05:17:10.596 Running script RerunData(query_string='', widget_states=, page_script_hash='', page_name='')
2022-06-20 05:17:10.596 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:17:10.596 Sessions still active: dict_keys([])
2022-06-20 05:17:10.597 Files: 0; Sessions with files: 0
2022-06-20 05:17:10.682 Deleting expired files...
2022-06-20 05:17:10.844 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:17:10.846 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:19:13.048 1 file(s) received for session 1a9cfd89-4e48-452c-b85b-1f0e54476afb widget $$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None
2022-06-20 05:19:13.340 Received the following back message:
rerun_script {
  widget_states {
    widgets {
      id: "$$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None"
      file_uploader_state_value {
        max_file_id: 1
        uploaded_file_info {
          id: 1
          name: "oneninenine_mb.txt"
          size: 208666624
        }
      }
    }
  }
  page_script_hash: "1108ccea9abf4e0f1220e09e9ceb0300"
}

2022-06-20 05:19:13.341 Beginning script thread
2022-06-20 05:19:13.342 Running script RerunData(query_string='', widget_states=widgets {
  id: "$$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None"
  file_uploader_state_value {
    max_file_id: 1
    uploaded_file_info {
      id: 1
      name: "oneninenine_mb.txt"
      size: 208666624
    }
  }
}
, page_script_hash='1108ccea9abf4e0f1220e09e9ceb0300', page_name='')
2022-06-20 05:19:13.342 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:19:13.342 Sessions still active: dict_keys([])
2022-06-20 05:19:13.342 Files: 0; Sessions with files: 0
2022-06-20 05:19:13.773 Deleting expired files...
2022-06-20 05:19:13.831 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:19:37.392 Server state: State.ONE_OR_MORE_BROWSERS_CONNECTED -> State.STOPPING
2022-06-20 05:19:37.393 Shutting down (id=1a9cfd89-4e48-452c-b85b-1f0e54476afb)
2022-06-20 05:19:37.393 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:19:37.393 Sessions still active: dict_keys([])
2022-06-20 05:19:37.393 Files: 0; Sessions with files: 0
2022-06-20 05:19:37.393 Deleting expired files...
2022-06-20 05:19:37.393 Server state: State.STOPPING -> State.STOPPED

logs_201.txt

2022-06-20 05:19:42.420 Initialized tornado logs
2022-06-20 05:19:42.421 No singleton. Registering one.
2022-06-20 05:19:42.423 Watcher created for /home/lsimpson/src/streamlit-test/pages
2022-06-20 05:19:42.423 Setting up signal handler
2022-06-20 05:19:42.423 Using selector: EpollSelector
2022-06-20 05:19:42.425 Starting server...
2022-06-20 05:19:42.425 Serving static content from /opt/python/3.10.4/lib/python3.10/site-packages/streamlit/static
2022-06-20 05:19:42.426 Server started on port 8501
2022-06-20 05:19:42.427 Server state: State.INITIAL -> State.WAITING_FOR_FIRST_BROWSER
2022-06-20 05:19:42.467 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:19:42.470 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:19:42.475 Starting new HTTP connection (1): checkip.amazonaws.com:80
2022-06-20 05:19:42.578 http://checkip.amazonaws.com:80 "GET / HTTP/1.1" 200 13
2022-06-20 05:19:42.582 Encoding detection: ascii is most likely the one.
2022-06-20 05:19:42.582 Skipping PyPI version check
2022-06-20 05:19:47.722 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:19:47.722 AppSession initialized (id=7ea3fcfa-6d0f-45d3-bd31-733d5b762844)
2022-06-20 05:19:47.722 Created new session for ws 139675329533296. Session ID: 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:19:47.722 Server state: State.WAITING_FOR_FIRST_BROWSER -> State.ONE_OR_MORE_BROWSERS_CONNECTED
2022-06-20 05:19:48.016 Received the following back message:
rerun_script {
  widget_states {
  }
}

2022-06-20 05:19:48.017 Beginning script thread
2022-06-20 05:19:48.017 Running script RerunData(query_string='', widget_states=, page_script_hash='', page_name='')
2022-06-20 05:19:48.018 Disconnecting files for session with ID 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:19:48.018 Sessions still active: dict_keys([])
2022-06-20 05:19:48.018 Files: 0; Sessions with files: 0
2022-06-20 05:19:48.117 Deleting expired files...
2022-06-20 05:19:48.297 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:19:48.299 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:20:01.503 Server state: State.ONE_OR_MORE_BROWSERS_CONNECTED -> State.STOPPING
2022-06-20 05:20:01.503 Shutting down (id=7ea3fcfa-6d0f-45d3-bd31-733d5b762844)
2022-06-20 05:20:01.503 Disconnecting files for session with ID 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:20:01.503 Sessions still active: dict_keys([])
2022-06-20 05:20:01.503 Files: 0; Sessions with files: 0
2022-06-20 05:20:01.504 Deleting expired files...
2022-06-20 05:20:01.504 Server state: State.STOPPING -> State.STOPPED

Hi @Lachlan_Simpson, welcome to the community!! :wave: :partying_face:

Our docs cover how to increase the upload size of st.file_uploader:

Best, :balloon:
Snehan