Putting a GPT-2 Model Up For Others to Interact With!

Using Streamlit
1

Hi All! Super new to Streamlit (& new to Data Science in general!), so excited by Streamlit’s ease-of-use. In a sentence, I’m finetuning GPT-2 to give (hopefully decent!) relationship advice. I’d like anybody in the world to be able to interact with it. Just wondering what app I’d use in order to upload my version of the model, and to let others interact with it through a simple text input & text-only output.

Thanks all!

2

Hey @rjmalka,

First, welcome to the Streamlit community!! :partying_face: :partying_face: :two_hearts: :tada:

I’m actually not sure what you mean by this:

Streamlit will create your webapp interface for you and you can deploy that to share with the world :world_map: using Streamlit Sharing platform. If you haven’t signed up to Sharing you can here: Streamlit Community Cloud • Streamlit

This will allow you to deploy your app in one click from your linked Github account (put your code and all the files you need on a public github repo and sign up with your github associated email).

This is a free service, so there are limits on the RAM we give to each person, 800MB is the max allowance!

Hope this helps?
Marisa

3

Hi Marisa,

Thanks for your patience, and sorry for being unclear! I should have waited until I was less burned out before writing that :slight_smile: Basically, I have code that calls GPT-2 and uses it to generate text. I also have the .rar file that represents the trained model checkpoint, so the model doesn’t have to keep retraining. I want people to be able to ask GPT-2 for relationship advice, and that requires getting a text input from the user, which replaces a certain line of code in this notebook.

I’ve looked through the Streamlit docs and I’ve found streamlit.text_input(), which is awesome! While I’m putting the app together, wondering if there’s an API to include sharing the results of the output with friends through FB, Twitter, etc?

Lastly, I assume I should store my .rar file in the same Github folder and call it as I usually do before generating text?

Assuming the model uses only the .rar file, rather than try to train the model in real time each time someone asks a question, I don’t believe I’ll ever exceed the 800 MB/RAM limit.

Please let me know if I’m still being unclear, but I hope this helps articulate my questions!

Glad to be part of the Streamlit community :slight_smile:

Warmly,
Robert

1 Like
4

Related: Am encountering the following error (ModuleNotFoundError: No module named ‘gpt_2_simple.’) when I run:

%tensorflow_version 1.x
!pip install -q gpt-2-simple

Makes perfect sense that this isn’t executable in Streamlit… wondering if there’s any way to have the model run through Streamlit? I’m getting tired, but if not I’m sure there’s some way to connect & borrow from e.g. Google Colab’s runtime/resources? Wondering if you’ve encountered this kind of problem before and if there are any suggestions to getting GPT-2 to run!

EDIT: Currently have the following code in question:

%tensorflow_version 1.x
!pip install -q gpt-2-simple #wondering what I can do instead of attempting a pip install, as I usually do on Google Colab?
import gpt_2_simple as gpt2

gpt2.download_gpt2(model_name="355M")
gpt2.copy_checkpoint_from_gdrive(run_name='run1')  #Other question is how to get this to work.

input = st.text_input('Describe your relationship troubles.', '')

text = gpt2.generate(sess, run_name='run1',
              temperature=0.7,
              top_k=40,
              nsamples=100,
              batch_size=25,
              length=200,
              prefix="<|startoftext|>[WP]" + input,
              truncate="<|endoftext|>",
              include_prefix=False,
              sample_delim=''
              )

st.write('', text)

Appreciate the help! Thanks all :blush:

5

Hi @Marisa_Smith! Just want to reach out on this and see if you have an answer to my questions (no luck so far) – thank you!

Robert

6

Hey Robert,

So sorry about the long delay, I haven’t been around on the forum as much as I would like as we have a lot of exciting things in the works!

No worries we have all been there thanks for clearing this up and adding your code snippets!

So there is no Streamlit API for this, but community members make components (which are plug-ins) for Streamlit! :partying_face: You may find something out of the box there that may work for you, or you might be able to adapt something someone else made! I’m thinking of the Disqus and Discord components that @okld made!

here is a link to our components: App Gallery • Streamlit

Also, recently there was this fantastic app made by @jrieke (Creator turned team member :sunglasses:) where he linked twitter into his Year on Github app! I took the liberty (hope that’s ok @jrieke :wink:) to grab his GitHub repo for you, I think you will have some success with this example of linking twitter into Streamlit! GitHub - jrieke/year-on-github: 🐙 Share your Github stats for 2020 on Twitter

I would say yes, depending on the size of it you could load it in before or after. But you may want to actually make a small function for it, that way you can add the @st.cache decorator to your function call in your Streamlit app, and (I’m assuming) because your .rar file won’t change, this will only have to load one time and then help speed up the app for users!

I am less sure about this one. Based on the notebook link you sent me it seems your trying to translate a Jupyter Notebook into a Streamlit App? Jupyter notebook usually lets you run pip install commands directly from the notebook cells, but Streamlit acts just like any python package. You would need to install this in your environment before you try running your script. (assuming I have understood this issue correctly!)

Here are the steps I would recommend:

  1. make a clean environment (pip or conda) for a specific python version
  2. In that new environment, pip or conda install the packages you want to use, streamlit, gpt-2-simple, tensorflow (?) and any others
  3. in your streamlit_app.py script, remove the %tensorflow_version 1.x and
    !pip install -q gpt-2-simple, and just import these packages like your third line in your app there :point_up: (import gpt_2_simple as gpt2)

Let’s see where this gets us!

Also, have you put your app in a GitHub repo? If you can do that and share it, that would be super helpful!

Hope this helps!
Happy Streamlit-ing!
Marisa