Using Git Submodules

Community Cloud
1

My tool is using a git submodule to integrate with a 2nd repository.
when I try to deploy it to Streamlit community cloud I’m getting a missing files error.
Is there a way I can use “git submodule update --recursive”? or somehow ensure the 2nd repo get cloned as well?

2

When you deploy an app on Commuity Cloud, the only thing that gets cloned during deployment is the source repo for your app. If you need additional files saved locally, you’d need to execute commands within the Python code of your app to query your second repo and copy additional files.

3

I added such code which work locally yet failed in streamlit cloud

INFO - Submodule directory ‘/mount/src/pythonproject/Submodules/content’ not found. Initializing submodule…
INFO - Failed to initialize the submodule: Submodule ‘Submodules/content’ (https://github.com/XXX/pythonProject.Content.git) registered for path ‘Submodules/content’
Cloning into ‘/mount/src/pythonproject/Submodules/content’…
fatal: could not read Username for ‘https://github.com’: No such device or address
fatal: clone of ‘https://github.com/XXX/pythonProject.Content.git’ into submodule path ‘/mount/src/pythonproject/Submodules/content’ failed
Failed to clone ‘Submodules/content’. Retry scheduled

4

Please can you share your code? What code generates those errors?

5

sure, and thanks for your assistance.
below is the code that trigger the error, I’m checking if an inner path exist, and if not I’m running a subprocess of git submodule update.
in local run this flow clone the inner repository, yet here its error

submodule_path = config_manager.get_value("content_dir_path")

if not os.path.isdir(submodule_path):
    Tool.logger.info(f"Submodule directory '{submodule_path}' not found. Initializing submodule...")

    try:
        # Run the git submodule command
        subprocess.run(
            ["git", "submodule", "update", "--init", "--recursive"],
            check=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE
        )
        Tool.logger.info(f"Submodule at '{submodule_path}' has been initialized and updated.")
    except subprocess.CalledProcessError as e:
        Tool.logger.info(f"Failed to initialize the submodule: {e.stderr.decode('utf-8')}")
        raise
else:
    Tool.logger.info(f"Submodule directory '{submodule_path}' already exists. No action needed.")
6

What is the actual project you are trying to clone? Your error message says https://github.com/XXX/pythonProject.Content.git and the error says the project isn’t even found. If it’s private, I’m not sure if git would be configured correctly to access it or not…