Hi All,
Can’t get the file_uploader to work with size >200MB, would appreciate some help. Thanks!
Running a Python 3.10.4 env on Ubuntu 20.04, using Streamlit 1.10.0, this is the minimum code:
import streamlit as st
import os
st.config.set_option("server.maxUploadSize", 5000)
st.config.set_option("server.maxMessageSize", 5000)
st.config.set_option("server.enableWebsocketCompression", 'true')
uploaded_file = st.file_uploader('Upload Large File')
if uploaded_file is not None:
bytes_data = uploaded_file.getvalue()
st.success('Uploaded Successfully')
save_uploadedfile(uploaded_file)
with open(os.path.join("/tmp/",uploadedfile.name),"wb") as f:
f.write(uploadedfile.getbuffer())
st.success("Saved File:{} to /tmp/".format(uploadedfile.name))
Executing with:
/opt/python/3.10.4/bin/streamlit run upload_error.py --logger.level=debug \
2>/tmp/streamlit/logs.txt
I can upload a file with size 199MB successfully, but not one with size 201MB.
Files were created with:
fallocate -l 199M oneninenine_mb.txt
fallocate -l 201M twoohone_mb.txt
Output of each attached here:
logs_199.txt
2022-06-20 05:17:04.742 Initialized tornado logs
2022-06-20 05:17:04.742 No singleton. Registering one.
2022-06-20 05:17:04.744 Watcher created for /home/lsimpson/src/streamlit-test/pages
2022-06-20 05:17:04.744 Setting up signal handler
2022-06-20 05:17:04.744 Using selector: EpollSelector
2022-06-20 05:17:04.746 Starting server...
2022-06-20 05:17:04.746 Serving static content from /opt/python/3.10.4/lib/python3.10/site-packages/streamlit/static
2022-06-20 05:17:04.747 Server started on port 8501
2022-06-20 05:17:04.748 Server state: State.INITIAL -> State.WAITING_FOR_FIRST_BROWSER
2022-06-20 05:17:04.785 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:17:04.787 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:17:04.791 Starting new HTTP connection (1): checkip.amazonaws.com:80
2022-06-20 05:17:04.891 http://checkip.amazonaws.com:80 "GET / HTTP/1.1" 200 13
2022-06-20 05:17:04.895 Encoding detection: ascii is most likely the one.
2022-06-20 05:17:04.895 Skipping PyPI version check
2022-06-20 05:17:10.309 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:17:10.310 AppSession initialized (id=1a9cfd89-4e48-452c-b85b-1f0e54476afb)
2022-06-20 05:17:10.310 Created new session for ws 139817969382160. Session ID: 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:17:10.310 Server state: State.WAITING_FOR_FIRST_BROWSER -> State.ONE_OR_MORE_BROWSERS_CONNECTED
2022-06-20 05:17:10.594 Received the following back message:
rerun_script {
widget_states {
}
}
2022-06-20 05:17:10.595 Beginning script thread
2022-06-20 05:17:10.596 Running script RerunData(query_string='', widget_states=, page_script_hash='', page_name='')
2022-06-20 05:17:10.596 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:17:10.596 Sessions still active: dict_keys([])
2022-06-20 05:17:10.597 Files: 0; Sessions with files: 0
2022-06-20 05:17:10.682 Deleting expired files...
2022-06-20 05:17:10.844 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:17:10.846 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:19:13.048 1 file(s) received for session 1a9cfd89-4e48-452c-b85b-1f0e54476afb widget $$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None
2022-06-20 05:19:13.340 Received the following back message:
rerun_script {
widget_states {
widgets {
id: "$$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None"
file_uploader_state_value {
max_file_id: 1
uploaded_file_info {
id: 1
name: "oneninenine_mb.txt"
size: 208666624
}
}
}
}
page_script_hash: "1108ccea9abf4e0f1220e09e9ceb0300"
}
2022-06-20 05:19:13.341 Beginning script thread
2022-06-20 05:19:13.342 Running script RerunData(query_string='', widget_states=widgets {
id: "$$GENERATED_WIDGET_KEY-32071a430a43ed6aa2582a8918fd52e8-None"
file_uploader_state_value {
max_file_id: 1
uploaded_file_info {
id: 1
name: "oneninenine_mb.txt"
size: 208666624
}
}
}
, page_script_hash='1108ccea9abf4e0f1220e09e9ceb0300', page_name='')
2022-06-20 05:19:13.342 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:19:13.342 Sessions still active: dict_keys([])
2022-06-20 05:19:13.342 Files: 0; Sessions with files: 0
2022-06-20 05:19:13.773 Deleting expired files...
2022-06-20 05:19:13.831 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:19:37.392 Server state: State.ONE_OR_MORE_BROWSERS_CONNECTED -> State.STOPPING
2022-06-20 05:19:37.393 Shutting down (id=1a9cfd89-4e48-452c-b85b-1f0e54476afb)
2022-06-20 05:19:37.393 Disconnecting files for session with ID 1a9cfd89-4e48-452c-b85b-1f0e54476afb
2022-06-20 05:19:37.393 Sessions still active: dict_keys([])
2022-06-20 05:19:37.393 Files: 0; Sessions with files: 0
2022-06-20 05:19:37.393 Deleting expired files...
2022-06-20 05:19:37.393 Server state: State.STOPPING -> State.STOPPED
logs_201.txt
2022-06-20 05:19:42.420 Initialized tornado logs
2022-06-20 05:19:42.421 No singleton. Registering one.
2022-06-20 05:19:42.423 Watcher created for /home/lsimpson/src/streamlit-test/pages
2022-06-20 05:19:42.423 Setting up signal handler
2022-06-20 05:19:42.423 Using selector: EpollSelector
2022-06-20 05:19:42.425 Starting server...
2022-06-20 05:19:42.425 Serving static content from /opt/python/3.10.4/lib/python3.10/site-packages/streamlit/static
2022-06-20 05:19:42.426 Server started on port 8501
2022-06-20 05:19:42.427 Server state: State.INITIAL -> State.WAITING_FOR_FIRST_BROWSER
2022-06-20 05:19:42.467 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:19:42.470 Popen(['git', 'version'], cwd=/home/lsimpson/src/streamlit-test, universal_newlines=False, shell=None, istream=None)
2022-06-20 05:19:42.475 Starting new HTTP connection (1): checkip.amazonaws.com:80
2022-06-20 05:19:42.578 http://checkip.amazonaws.com:80 "GET / HTTP/1.1" 200 13
2022-06-20 05:19:42.582 Encoding detection: ascii is most likely the one.
2022-06-20 05:19:42.582 Skipping PyPI version check
2022-06-20 05:19:47.722 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:19:47.722 AppSession initialized (id=7ea3fcfa-6d0f-45d3-bd31-733d5b762844)
2022-06-20 05:19:47.722 Created new session for ws 139675329533296. Session ID: 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:19:47.722 Server state: State.WAITING_FOR_FIRST_BROWSER -> State.ONE_OR_MORE_BROWSERS_CONNECTED
2022-06-20 05:19:48.016 Received the following back message:
rerun_script {
widget_states {
}
}
2022-06-20 05:19:48.017 Beginning script thread
2022-06-20 05:19:48.017 Running script RerunData(query_string='', widget_states=, page_script_hash='', page_name='')
2022-06-20 05:19:48.018 Disconnecting files for session with ID 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:19:48.018 Sessions still active: dict_keys([])
2022-06-20 05:19:48.018 Files: 0; Sessions with files: 0
2022-06-20 05:19:48.117 Deleting expired files...
2022-06-20 05:19:48.297 Watcher created for /home/lsimpson/src/streamlit-test/upload_error.py
2022-06-20 05:19:48.299 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2022-06-20 05:20:01.503 Server state: State.ONE_OR_MORE_BROWSERS_CONNECTED -> State.STOPPING
2022-06-20 05:20:01.503 Shutting down (id=7ea3fcfa-6d0f-45d3-bd31-733d5b762844)
2022-06-20 05:20:01.503 Disconnecting files for session with ID 7ea3fcfa-6d0f-45d3-bd31-733d5b762844
2022-06-20 05:20:01.503 Sessions still active: dict_keys([])
2022-06-20 05:20:01.503 Files: 0; Sessions with files: 0
2022-06-20 05:20:01.504 Deleting expired files...
2022-06-20 05:20:01.504 Server state: State.STOPPING -> State.STOPPED
