Hello Streamlit team,
I have been using your platform for quite some time, and I find it excellent. I am interested in deploying Selenium on your cloud. I have tried all the Selenium examples provided by you on the cloud, but I keep encountering errors. Could you please provide me with a working Selenium code snippet?
Um…??
Hi @IlIl
A while back, I’ve created this Streamlit app screenshot app that uses selenium.
Here’s the demo app https://screenshot.streamlit.app/
And the repo GitHub - dataprofessor/app-screenshot: Streamlit App Screenshot
The library dependencies as defined in requirements.txt file is as follows:
streamlit
selenium==4.11.2
webdriver-manager
psutil
The app worked stably for a very long time and still works in principle. But the target website has probably activated geoip blocking in the meantime. As a result, the content can no longer be viewed from the servers on which streamlit cloud is hosted. The app still worked locally in a docker container.
I have now reworked the app and added more debugging and error handling. I have also added the use of a proxy to bypass the geoip blocking. This means that the app works again on streamlit cloud. However, since only free proxies are used, this does not always work reliably.
WORKING PROFESSOR 
q
NOW YOUR EXAMPLE ALSO WORKING FRANKY BRO #streamlitsupercoolplatform
Glad to hear that it’s working for you!
Thank you this is awesome, I am having the same problem as @IlIl using selenium on community cloud. I have forked you app and have it running successfully on com cloud. But having trouble finding the element that I need (and works in my locally run version).
driver = get_driver()
runner = str(47595)
ParkrunURL = “https://www.parkrun.org.uk//parkrunner//” + runner + “//all//”
driver.implicitly_wait(10) # Wait for up to 10 seconds
try:
table = driver.find_element(By.XPATH, “(//table)[3]”)
st.write(‘we got it’)
except:
st.write(‘this table was not found’)
And don’t know if this is a Selenium problem or Streamlit problem?
I have also tried
table = driver.find_element(By.CLASS_NAME, “sortable”)
table_element = driver.find_element_by_id(“results”)
tables = driver.find_element(By.ID, “results”)
table = tables.find_element(By.CLASS_NAME,“sortable”)
Debugging selenium running on streamlit cloud is a bit of a nightmare…
Make sure that you actually get the html page you want and not a 403 or 404 response.
Could be that the target webserver has ip/geoip blocking enabled.
Thanks, I have hardcoded the URL and added
st.write(driver.current_url)
which runs without error and writes the correct URL on the app.
And the ‘try’ block works if I just ask it to write something, but if I try any ‘find_element’. I get this error
raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /home/appuser/.cache/selenium/chromedriver/linux64/122.0.6261.94/chromedriver unexpectedly exited. Status code was: 1
I have also tried adding driver.implicitly_wait(10) and copying the short and full XPath from the website, but still no joy
table= driver.find_element(By.XPATH, “/html/body/main/div/div/div/div[3]/table”)
table = driver.find_element(By.XPATH, ‘//*[@id=“results”]’)
Could be that the target webserver has ip/geoip blocking enabled.
Is there a way to find this out, and is there any way around it?
Thanks
This does not mean that you get the desired content of the website. Look at the actual html content you get and the status code. To do this you need to add more debug code.
Look at the actual content you get back and the status code.
a proxy
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.
