File upload fails with Error: downloading object: wav2lip/checkpoints/wav2lip_gan.pth (ca9ab7b)

Hi there,

I’m trying to deploy a streamlit application, and it keep failing with the following error:

Error downloading object: wav2lip/checkpoints/wav2lip_gan.pth (ca9ab7b): Smudge error: Error downloading wav2lip/checkpoints/wav2lip_gan.pth (ca9ab7b7b812c0e80a6e70a5977c545a1e8a365a6c49d5e533023c034d7ac3d8): batch request: git@github.com: Permission denied (publickey).: exit status 255

Errors logged to /mount/src/ai-lip-sync/.git/lfs/logs/20240121T212252.496674307.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: wav2lip/checkpoints/wav2lip_gan.pth: smudge filter lfs failed

I guess the problem is the fact that Streamlit is trying to pull the checkpoint file which is a big one without using git lfs, does anybody know how to fix such an issue?

Here’s a link for my public repo for that application: https://github.com/Aml-Hassan-Abd-El-hamid/ai-lip-sync-app

I’m using python 3.11.6 and streamlit >= 1.9.2

this is the full log for reference:

[00:12:22] 🚀 Starting up repository: 'ai-lip-sync', branch: 'main', main module: 'app.py'
[00:12:22] 🐙 Cloning repository...
[00:12:28] 🐙 failed
[00:12:41] 🐙 failed
[00:12:57] 🖥 Provisioning machine...
[00:13:10] 🎛 Preparing system...
[00:13:15] ⛓ Spinning up manager process...
[00:15:15] 🎈 Inflating balloons...
[00:12:22] 🚀 Starting up repository: 'ai-lip-sync', branch: 'main', main module: 'app.py'
[00:12:22] 🐙 Cloning repository...
[00:12:28] 🐙 failed
[00:12:41] 🐙 failed
[00:13:03] 🐙 failed
[00:13:16] 🐙 failed
[00:13:30] 🐙 failed
[00:13:40] 🐙 Failed to download the sources for repository: 'ai-lip-sync', branch: 'main', main module: 'app.py'
[00:13:40] 🐙 Make sure the repository and the branch exist and you have write access to it, and then reboot the app.
[00:15:13] ❗️ Streamlit server consistently failed status checks
[00:15:13] ❗️ Please fix the errors, push an update to the git repo, or reboot the app.
[00:30:00] 🐙 Pulling code changes from Github...
[00:30:16] 🐙 Pulling code changes from Github...
[00:30:32] 🐙 Pulling code changes from Github...
[00:30:47] 🐙 Pulling code changes from Github...
[00:31:03] 🐙 Pulling code changes from Github...
[00:31:18] ❗️ Updating the app files has failed: git pull, cmd: sudo -u appuser GIT_SSH_COMMAND='ssh -i /home/appuser/.ssh_ro/id_github' git fetch origin && sudo -u appuser git reset --hard origin/main: error: exit status 128: Downloading wav2lip/checkpoints/wav2lip_gan.pth (436 MB)
Error downloading object: wav2lip/checkpoints/wav2lip_gan.pth (ca9ab7b): Smudge error: Error downloading wav2lip/checkpoints/wav2lip_gan.pth (ca9ab7b7b812c0e80a6e70a5977c545a1e8a365a6c49d5e533023c034d7ac3d8): batch request: git@github.com: Permission denied (publickey).: exit status 255

Errors logged to /mount/src/ai-lip-sync/.git/lfs/logs/20240122T003108.562693988.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: wav2lip/checkpoints/wav2lip_gan.pth: smudge filter lfs failed

It means in streamlit cloud we have upto 1gb of space. If you are is trying to download it again in the rebooting means it consuming more space. Due to that streamlit is consistently failed. Once check your coding part that your lfs is used for one time or it using for multiple time. If it is using one time means just use st.cachce_resource to store the model in cache helps us to prevent in downloading the model again and again.

1 Like

Thank you @Guna_Sekhar_Venkata for helping.
I was using @st.cache_data but I changed it to @st.cache_resource so the code now looks like this:

@st.cache_resource
def load_model(path):
	model = Wav2Lip()
	print("Load checkpoint from: {}".format(path))
	checkpoint = torch.load(path,map_location=lambda storage, loc: storage)
	s = checkpoint["state_dict"]
	new_s = {}
	for k, v in s.items():
		new_s[k.replace('module.', '')] = v
	model.load_state_dict(new_s)

	model = model.to(device)
	return model.eval()
model = load_model('wav2lip/checkpoints/wav2lip_gan.pth')

But that didn’t work.
So I deleted the whole app -trying to free some space-, made a deployment branch and thentried to deploy it again but now I’m stuck with the following error:

Failed to download the sources for repository: 'ai-lip-sync', branch: 'deployment', main module: 'app.py'

That’s the full log for reference:

[05:36:40] 🖥 Provisioning machine...

[05:36:40] 🎛 Preparing system...

[05:36:40] ⛓ Spinning up manager process...

[05:36:42] 🚀 Starting up repository: 'ai-lip-sync', branch: 'deployment', main module: 'app.py'

[05:36:42] 🐙 Cloning repository...

[05:36:46] 🐙 failed

[05:36:59] 🐙 failed

[05:37:13] 🐙 failed

[05:37:26] 🐙 failed

[05:37:40] 🐙 failed

[05:37:50] 🐙 Failed to download the sources for repository: 'ai-lip-sync', branch: 'deployment', main module: 'app.py'

[05:37:50] 🐙 Make sure the repository and the branch exist and you have write access to it, and then reboot the app.

[05:39:22] ❗️ Streamlit server consistently failed status checks

[05:39:22] ❗️ Please fix the errors, push an update to the git repo, or reboot the app.

[05:39:55] 🖥 Provisioning machine...

[05:40:00] 🎛 Preparing system...

[05:40:05] ⛓ Spinning up manager process...

[05:40:34] 🎈 Inflating balloons...

[05:40:40] 📖 Unpacking Comic Sans RAR files...

[05:40:45] 🚧 Loading "Under construction" GIF...

[05:40:50] 🛠 Compiling <blink> tags...

[05:40:55] 📠 Initializing Java applet...

[05:41:00] ⏳ Please wait...

[05:40:04] 🚀 Starting up repository: 'ai-lip-sync', branch: 'deployment', main module: 'app.py'

[05:40:04] 🐙 Cloning repository...

[05:40:08] 🐙 failed

[05:40:25] 🐙 failed

[05:40:38] 🐙 failed

[05:40:52] 🐙 failed

[05:41:06] 🐙 failed

[05:41:16] 🐙 Failed to download the sources for repository: 'ai-lip-sync', branch: 'deployment', main module: 'app.py'

[05:41:16] 🐙 Make sure the repository and the branch exist and you have write access to it, and then reboot the app.

[05:42:54] ❗️ Streamlit server consistently failed status checks

[05:42:54] ❗️ Please fix the errors, push an update to the git repo, or reboot the app.

If you have an idea of how to solve that, please help me.

According to my understanding there is a branch conflict. Once check the deployment branch is able to merge in branch or else redeploy the app again

1 Like

Thank you very much for your replay, I really appreciate it.
I was sure that there were no conflicts as I actually didn’t change anything in the deployment branch.
But I still deleted the app and that branch, re-deployed from the main this time and still, the same issue.

[12:40:32] 🖥 Provisioning machine...

[12:40:32] 🎛 Preparing system...

[12:40:32] ⛓ Spinning up manager process...

[12:40:29] 🚀 Starting up repository: 'ai-lip-sync', branch: 'main', main module: 'app.py'

[12:40:29] 🐙 Cloning repository...

[12:40:32] 🐙 failed

[12:40:47] 🐙 failed

[12:41:01] 🐙 failed

[12:41:16] 🐙 failed

[12:41:30] 🐙 failed

[12:41:40] 🐙 Failed to download the sources for repository: 'ai-lip-sync', branch: 'main', main module: 'app.py'

[12:41:40] 🐙 Make sure the repository and the branch exist and you have write access to it, and then reboot the app.

I don’t know what to do now, should I try another way ?

There there should one solution that delete the current app and make a new app deployment or else merge the brach with the main branch and redeploy it again

1 Like

But I already deleted the app and deployed it from the main branch after deleting the deployment branch

I solved this thing by simply giving up on git lfs!
I just uploaded every heavy file to my Google Drive account, put them in a public folder, and then used a module called gdown to download those folders when needed!
here’s a link to that gdown, it’s straightforward to use and install.

Now I’m facing a dependency error so I opened another issue with it! good luck to all the miserable souls who have to deploy anything!

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.