Set GOOGLE_APPLICATION_CREDENTIALS via the file uploader (to use Google APIs)

[Charly_Wargnier]

Hi guys,

I’m trying to set GOOGLE_APPLICATION_CREDENTIALS in Streamlit, to use Google APIs.

I’m doing so via the via the file uploader - please find the code below:

uploaded_file = st.file_uploader("File 1", type="json")

import os

if uploaded_file is not None:
        data = pd.read_json(uploaded_file, orient='index')
        os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = data
        st.write(data)

Here’s the error I get:

TypeError: str expected, not DataFrame

So if I understand correctly, I need to get a string (that is, the path to the JSON file), whereas read_json outputs a dataframe.

Any idea on how to get that JSON file path?

Thanks,
Charly

[knorthover]

Use json.load() to read the file, not pd.read_json(). You are converting it to a DataFrame, which is most definitely is not.

[Charly_Wargnier]

Thanks @knorthover!

The thing is that a string (file path) is expected with os.environ in:

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = data

… not a dictionary.

Using json.load in data = json.load(uploaded_file) returns a dictionary.

@randyzwitch: are you aware of any workaround for the above issue?

Thanks in advance guys!

Charly

[randyzwitch]

I’m not familiar with what that token looks like, but if it’s a dict with a single key, can you just specify the key name to get a string back?

[Charly_Wargnier]

Thanks Randy!

There are actually quite a few keys, see the JSON file below:

I believe GCP may require all/most of them to work.

Playing with Dict keys now, so let’s see of I can get something working tonight!

Charly

[okld]

Hey @Charly_Wargnier,

Apparently the environment expects a path to a json file. So what you need to do after uploading your file is to save it in a temporary json file and specify its path with os.environ.

[Charly_Wargnier]

Thanks @okld!

I assume you’re referring to tempfile.NamedTemporaryFile.

I’ll give it a whirl now!

[Charly_Wargnier]

Hi Synode,

Quick heads up on progress!

I wrote this with the tempfile lib based on one of your previous posts on the Streamlit forum:

import tempfile

uploaded_file = st.file_uploader("File 1", type="json")

if uploaded_file is not None:
  with tempfile.NamedTemporaryFile(mode='w') as fp:
    fp.write(uploaded_file.getvalue())
    path = fp.name
    st.write(path)
    os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = path

Turns out my temp file is indeed created (yay!), but not found - as per the screenshot below:

I also (I guess foolishly!) tried to concatenate the .json string, like this:

path = fp.name + '.json'

… expectedly same issue, file not found.

Conscious you specified other methods in your post, I will try them too tomorrow.

Thanks,
Charly

[okld]

Beware that your temporary file will exist only during the execution of the with block. Thus you must be sure that the part which processes the environment variable and read the json file lives inside that block.

I hope it helps!

[Charly_Wargnier]

Thanks Synode - I didn’t realize they had to be in the same block, good to know!

So hopefully we’re moving forward, as when I indent all functions/parts processing the environment variable within the same block, the temp file seems to be found!

The new issue is that there’s now a PermissionError, as follows:

PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Charly\\AppData\\Local\\Temp\\tmpcyzxljvv'

I’m not sure about this, will dig in.

Thanks,
Charly

[okld]

This explains why you have a Permission denied.

As a workaround, add the delete=False parameter to NamedTemporaryFile, save the json inside the with block, then put back your processing code outside it. However you have to manually delete your temporary file once your app finishes to run.

I suggest you wrap your processing code in a try/finally block to make sure the file will be deleted, even in case of exception.

import tempfile

uploaded_file = st.file_uploader("File 1", type="json")

with tempfile.NamedTemporaryFile(mode='w', delete=False) as fp:
    fp.write(uploaded_file.getvalue())

try:
    os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = fp.name
    # Code using GOOGLE_APPLICATION_CREDENTIALS
finally:
    os.unlink(fp.name)

[Charly_Wargnier]

Thank you so much Synode! I really appreciate you taking the time to help on this!

So I tried the method you suggested above, besides outdending the functions which were previously indented.

This time the JSON file is never found - error as follows:

DefaultCredentialsError: File C:\Users\Charly\AppData\Local\Temp\tmpdegs1_mx was not found

Now I also tried various indentations/outdentations and in every case, the JSON file is not found.

Happy to share the script here (or via DM), if that would shed any light.

Thanks,
Charly

[okld]

That’s weird. I don’t have the google auth code, but at least it finds my json. Maybe there’s a difference on Windows… I’ll check this out later.

@Charly_Wargnier

Happy to share the script here (or via DM), if that would shed any light.

Please do!

[Charly_Wargnier]

Interesting!

The streamlit script is here: https://pastebin.com/wQTxkeFj

(sorry, I haven’t had a chance to clean it up yet…)

Happy to DM you the API key, if that helps!

Thanks again for the invaluable help.

Charly

[Charly_Wargnier]

Hold on Synode, I think I found what the issue was!

I’ll update here shortly

Charly

[okld]

So you’ve figured out what the problem was? I didn’t have enough time to test the code you’ve shared yesterday, but my first guess would be that your app runs after my try/finally block, which would explain why the file wasn’t found.

[Charly_Wargnier]

No worries Synode, you’ve been ABSOLUTELY legendary, as always!

So, oddly, in the code, there was:

client = language_v1.LanguageServiceClient()

… nested in a function, which seemed to have messed up the whole thing!

I removed it from the function and nested it within your block, as follows:

with tempfile.NamedTemporaryFile(mode='w', delete=False) as fp:
    fp.write(uploaded_file.getvalue())
try:
    os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = fp.name
    st.write("found", fp.name)
    with open(fp.name) as a:
        st.write(a.read())
        #client = language.LanguageServiceClient()
        client = language_v1.LanguageServiceClient()

All fine now!

Thanks,
Charly

[okld]

with open(fp.name) as a:
    st.write(a.read())
    #client = language.LanguageServiceClient()
    client = language_v1.LanguageServiceClient()

You need to open the file for your code to work? If you remove that with open() line and “outdent” the code inside that block, you’re encoutering your previous issue again?

Initially the with open() was only to demonstrate that I could re-open the file and read its content. It wasn’t intended to be part of the solution

[Charly_Wargnier]

@okld

 #client = language.LanguageServiceClient()
        client = language_v1.LanguageServiceClient()

I think you’re right, I don’t think it matters though I haven’t tried the suggestion you’ve just posted.

I believe the issue was lying with a conflict between:

client = language.LanguageServiceClient()
and
client = language_v1.LanguageServiceClient()

[okld]

Well, glad it finally works!