Hey there, I am trying to execute a shell script in streamlit cloud (for the purpose of using Elementary to generate lineage graphs) using this code:
subprocess.run(["edr", "lineage", "generate", "-d", ".dbt", "-p", "elementary", "-s", "2022-06-10", "-dbs", "flow-team"])
Everything works fine locally, and I don’t run into any errors; however, in streamlit cloud I get the following error:
Error: Invalid value for '--profiles-dir' / '-d': Path '/home/appuser/.dbt' does not exist.
It seems that no matter what I enter for the -d
flag, the same path of ‘home/appuser/.dbt’ is always returned by my shell script. I have my .dbt
folder on the same level as my streamlit_app.py
file.
Could someone please help me understand why my shell command running in Streamlit cloud cannot locate my .dbt
folder?
EDIT:
Even doing something more simple like this:
shell_command = "edr lineage generate --help"
shell_command_split = shlex.split(shell_command)
subprocess.run(shell_command_split)
Gives me the expected output locally:
Usage: edr lineage generate [OPTIONS]
Options:
...
However, running the exact same code in Streamlit cloud gives me this:
Usage: edr lineage [OPTIONS] COMMAND [ARGS]...
Try 'edr lineage --help' for help.
Error: Invalid value for '--profiles-dir' / '-d': Path '/home/appuser/.dbt' does not exist.
It seems as though none of my flags (--help
, -d
) are even being picked up. Is there any reason why this would be the case?