Issue with Deploying ML Models in Streamlit

[Joti_Gokaraju]

I am trying to test a pre-trained Image Captioning model in streamlit, but I cannot get it to run.

I keep seeing this error page:

This is the code I am using:

from transformers import GPT2TokenizerFast, ViTImageProcessor, VisionEncoderDecoderModel
from torch.utils.data import Dataset
from torchtext.data import get_tokenizer
import requests
import torch
import numpy as np
from PIL import Image
import pickle
import matplotlib.pyplot as plt
import os
from tqdm import tqdm
import streamlit as st
import warnings
warnings.filterwarnings('ignore')


model_raw = VisionEncoderDecoderModel.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
image_processor = ViTImageProcessor.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
tokenizer = GPT2TokenizerFast.from_pretrained("nlpconnect/vit-gpt2-image-captioning")

def show_n_generate(url, greedy = True, model = model_raw):
    image = Image.open(requests.get(url, stream =True).raw)
    pixel_values   = image_processor(image, return_tensors ="pt").pixel_values

    if greedy:
        generated_ids  = model.generate(pixel_values, max_new_tokens = 30)
    else:
        generated_ids  = model.generate(
            pixel_values,
            do_sample=True,
            max_new_tokens = 30,
            top_k=5)
    generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return generated_text

image = "http://images.cocodataset.org/val2017/000000039769.jpg"
if st.button("run"):
    answer = show_n_generate(image, greedy = False)
    st.write(answer)

And, this is my requirements.txt file:

streamlit
torch
transformers
pillow
requests
torchtext
tqdm

This is the link to the app: https://testimagecaption-5ctmsdzjdmyfzcxhpudlwt.streamlit.app/

If anyone has any solutions, please let me know.

[Joti_Gokaraju]

I am getting this error as well now.

[Caroline]

Hi Joti_Gokaraju,

Thanks for sharing this question!

It looks like your app is exceeding the 1GB resource limit, which is causing this error.

Here are a few resources you might find helpful:

• Blog post on reducing your app’s memory usage
• Blog post on fixing memory leaks
• Documentation on caching
• Community Deployment Wiki
• Documentation on using Streamlit with Docker and Kubernetes