FAQ: How to use large files in your Streamlit app

FAQs
1

Problem

Your app is using a large file and you’ve found that the file exceeded the size limit of your GitHub repository.

Solution

You can push large files into your GitHub repo by leveraging Git Large File Storage (Git LFS). Particularly, the maximum file size for your Free GitHub account would be 2 GB (while GitHub Enterprise Cloud supports up to 5 GB).

You don’t need to make any changes to your app to start using it. If your GitHub repo uses LFS, it will now just work with Streamlit.

Further details on using Git LFS can be found herein:

In the forthcoming sections, I’ll provide a concise step-by-step as mentioned in the above GitHub Documentation resources.

Install Git LFS

Navigate to git-lfs.com to download Git LFS to your computer.

After downloaded and unzipping the git-lfs-1.X.X.zip file to your computer, you’ll want to run the installation script install.sh:

$ ./install.sh

which should produce the following output:

> Git LFS initialized.

Next, verify that your Git LFS has been installed successfully:

git lfs install

which should give you the following output:

> Git LFS initialized.

Use Git LFS

In your terminal, change to the current working directory and tell Git LFS the file type that you want to track as large files (in our example .csv files):

git lfs track "*.csv"

which should give you the following output:

> Adding path *.csv

Next, you’ll want to add specific file path for the .csv file extension being tracked:

git add path/to/file.csv

Finally, commit and push these to your repo:

git commit -m "add file.csv"
git push

which should give you the following details of the upload:

> Sending file.csv
> 25.01 MB / 109.5 MB  22.84 % 18s
> 95.58 MB / 109.5 MB  87.28 % 3s

That’s it and your repo should now support your large file used by your app.

1 Like