Issue with web scraping deployment

☁️ Community Cloud
1

Hey! This code runs locally, however, when deployed, it is continuously showing “Running Status” with no output generated.

url = "https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY"
    headers = {"accept-encoding": "gzip, deflate, br",
              "accept-language": "en-GB,en-US;q=0.9,en;q=0.8",
              "referer": "https://www.nseindia.com/get-quotes/derivatives?symbol=NIFTY",
              "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36}"
              }
    
    response = requests.get(url, headers = headers).text
    st.write(response)

Do I need to make any changes to this? This tries to scrap a website and stores the response as text and tries to print it.

It works perfectly locally, but shows a connecting status in log and is continuously running when deployed on streamlit cloud.

Regards,
Pitanjal

2

Hi @pitanjal -

We see these every once and a while…it could be the case that the API you are trying to access blocks the Google Cloud IP range (which is where Streamlit Cloud is running), and as such, the URL never returns an answer.

Unfortunately, there’s nothing we can suggest at the moment to get around this.

Best,
Randy

3

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.