I am trying to upload a binary file of size 10gb using st.file_uploader
, however, I get the following error message. In fact, I get the same error message pretty much when I am trying to upload any file above 2gb, i.e I didn’t get this error message when I uploaded 1.2gb file.
I have set my file capacity to 10gb in config file.
As a matter of fact, the file loading shows as it should, then the file appears to be uploaded for a very short time (maybe a second), however, then the attachment disappears along with file_uploader
widget itself, and the following message pops up.
Hi @serdar_bay
Thank you for creating this post! I think you’ve run into a bug with st.file_uploader
. I was able to reproduce the reported behavior and have submitted a bug report in the Streamlit repo. Please consider upvoting the issue on GitHub for it to be prioritized by our engineers
opened 11:20AM - 11 Jan 23 UTC
type:bug
feature:st.file_uploader
status:needs-triage
### Checklist
- [X] I have searched the [existing issues](https://github.com/st… reamlit/streamlit/issues) for similar issues.
- [X] I added a very descriptive title to this issue.
- [X] I have provided sufficient information below to help reproduce this issue.
### Summary
Uploading a large file with `st.file_uploader` can reliably lead to a:
```
ValueError: Value out of range <uploaded-byes>
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/Users/skekre/Downloads/streamlit-misc/file-uploader-size.py", line 3, in <module>
f = st.file_uploader("Upload a file")
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py", line 311, in wrapped_func
result = non_optional_func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 338, in file_uploader
return self._file_uploader(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 418, in _file_uploader
file_uploader_state = serde.serialize(widget_state.value)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 108, in serialize
file_info.size = f.size
```
where "large" in my case was approximately close to or greater than 4.5GB. I'm not sure whether the specific size after which this error is thrown is machine dependant. The user who first [reported](https://discuss.streamlit.io/t/how-to-upload-large-files-using-st-file-uploader/35654) this on the Streamlit forum encountered the error for files >= 2GB.
### Reproducible Code Example
```Python
import streamlit as st
f = st.file_uploader("Upload a file")
if f is not None:
st.write(f.name)
st.write(f.size)
```
### Steps To Reproduce
1. Create a `.streamlit/config.toml` file to set the upload limit to 4.5 GB:
```toml
[server]
maxUploadSize = 4500
```
2. Create a 4.4 GB file:
```sh
truncate -s 4280m example.file
```
3. Run the above app and upload `example.file` to the file uploader widget
### Expected Behavior
As the uploaded 4.4 GB file is under the 4.5 GB limit set in `.streamlit/config.toml`, one would expect:
1. to see the upload successfully complete
2. the file uploader widget to be visible after the upload
3. see the output of the filename and filesize displayed
### Current Behavior
1. Once the progress bar on the file uploader widget hits the maximum, the app greys out for a few seconds while the Running man icon is displayed in the top right corner
2. Immediately after that, the file uploader widget disappears, leaving the user with the following error:
```
ValueError: Value out of range: 4487905280
Traceback:
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/Users/skekre/Downloads/streamlit-misc/file-uploader-size.py", line 3, in <module>
f = st.file_uploader("Upload a file")
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/runtime/metrics_util.py", line 311, in wrapped_func
result = non_optional_func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 338, in file_uploader
return self._file_uploader(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 418, in _file_uploader
file_uploader_state = serde.serialize(widget_state.value)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/streamlit/elements/file_uploader.py", line 108, in serialize
file_info.size = f.size
```

### Is this a regression?
- [ ] Yes, this used to work in a previous version.
### Debug info
- Streamlit version: 1.16.0
- Python version: 3.9
- Operating System: macOS
- Browser: Chrome
- Virtual environment: NA
### Additional Information
First reported in https://discuss.streamlit.io/t/how-to-upload-large-files-using-st-file-uploader/35654
### Are you willing to submit a PR?
- [ ] Yes, I am willing to submit a PR!
Best,
Snehan
1 Like
Thanks @snehankekre for the update and reporting the bug in the repo!