Subprocess is executing incorrectly in streamlit cloud

Hey there, I am trying to execute a shell script in streamlit cloud (for the purpose of using Elementary to generate lineage graphs) using this code:

    subprocess.run(["edr", "lineage", "generate",  "-d", ".dbt", "-p", "elementary", "-s", "2022-06-10", "-dbs", "flow-team"])

Everything works fine locally, and I don’t run into any errors; however, in streamlit cloud I get the following error:

Error: Invalid value for '--profiles-dir' / '-d': Path '/home/appuser/.dbt' does not exist.

It seems that no matter what I enter for the -d flag, the same path of ‘home/appuser/.dbt’ is always returned by my shell script. I have my .dbt folder on the same level as my streamlit_app.py file.

Could someone please help me understand why my shell command running in Streamlit cloud cannot locate my .dbt folder?

EDIT:
Even doing something more simple like this:

shell_command = "edr lineage generate --help"
shell_command_split = shlex.split(shell_command)
subprocess.run(shell_command_split)

Gives me the expected output locally:

Usage: edr lineage generate [OPTIONS]
Options:
...

However, running the exact same code in Streamlit cloud gives me this:

Usage: edr lineage [OPTIONS] COMMAND [ARGS]...
Try 'edr lineage --help' for help.
Error: Invalid value for '--profiles-dir' / '-d': Path '/home/appuser/.dbt' does not exist.

It seems as though none of my flags (--help, -d) are even being picked up. Is there any reason why this would be the case?

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.