How to cap API calls in Streamlit?

Using Streamlit
1

Hi guys!

I’m trying to cap some API calls in Streamlit.

I’m using the RateLimit lib (ratelimit · PyPI) which works perfectly fine in a Colab.

This package simply decorates any function that makes an API call - e.g.

@limits(calls=15, period=FIFTEEN_MINUTES)

However it doesn’t work in Streamlit as the script re-runs every time, which I believe may reset the rate limit settings every time?

Things I’ve tried so far:
#1 caching
#2 putting my function in a different file.

FYI, here’s (part of) the GPT3 code which works in a Colab

from ratelimit import limits

FIFTEEN_MINUTES = 900

@limits(calls=2, period=FIFTEEN_MINUTES)
def responseFunction():
  response = openai.Completion.create(
  engine="davinci", prompt="Social media post: \"The new episode of The Mandalorian was rubbish\"\nSentiment (positive, neutral, negative):")

  return response

responseFunction()

I’ll keep trying other things and will post my findings here :slight_smile:

Btw, regarding the API calls, ideally I would like to cap them globally. In other words, I’d like to deploy/share the app then only allow a certain amount of API calls per hour, or per day.

Thanks,
Charly

2

Hey @Charly_Wargnier,

What you could try is to use some sort of “global state”, similarly to what @thiago has done in his session state gist. The idea is to store your function in a module like “sys” which will be loaded once.

You can try something like this:

import sys
from ratelimit import limits

FIFTEEN_MINUTES = 900

def response_function():
    # Check if function exists already
    if not hasattr(sys, "_response_function"):

        @limits(calls=2, period=FIFTEEN_MINUTES)
        def _response_function():
            response = openai.Completion.create(
                engine="davinci",
                prompt="Social media post: \"The new episode of The Mandalorian was rubbish\"\nSentiment (positive, neutral, negative):"
            )

            return response
        
        sys._response_function = _response_function

    return sys._response_function()

response_function()
1 Like
3

Thanks Synode for this.

So as discussed, I’ve tried that code ^ and the cap block doesn’t seem to be working.
Thus, I’m able to do an unlimited amount of calls regardless of the rate limit I’m setting up.

I’ll keep digging :slight_smile:

Thanks,
Charly

1 Like