Installation of Chromedriver and use of Selenium

Community Cloud
, ,
1

I would like to use selenium and chromedriver in streamlit. But it has always failed in streamlit Cloud whereas it works on my PC. Can you help me with this issue?

Many thanks for your help.

Here is the piece of code which does not work

async def convert_to_pdf_and_extract_text(url: str) → Optional[Tuple[str, str]]:
pdf_path = “temp_file.pdf”
options = webdriver.ChromeOptions()
# Removed --headless to run in non-headless mode
options.add_argument(“–no-sandbox”)
options.add_argument(“–disable-dev-shm-usage”)
options.add_argument(“–disable-gpu”)
options.add_argument(“window-size=1920x1080”) # Set a common desktop viewport size
options.add_argument(“user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36”)

try:
    with webdriver.Chrome(options=options) as driver:
        driver.get(url)
        driver.implicitly_wait(30)
        result = driver.execute_cdp_cmd("Page.printToPDF", {"landscape": False})
        with open(pdf_path, "wb") as f:
            f.write(base64.b64decode(result['data']))

    text = extract_text_from_pdf(pdf_path)
    if word_count(text) < 200:
        st.caption(f"Failure: not enough words in pdf extraction of {url} - nb of words:{word_count(text)} ")
        print(f"Failure: not enough words in pdf extraction of {url} - nb of words:{word_count(text)} ")
        return None
    st.caption(f"Success: pdf extraction of {url}")
    print(f"Success: pdf extraction of {url}")
    return text, url

except Exception as e:
    print(f"Error: {e}")
    st.caption(f"Failure: pdf extraction of {url} because {e}")
    print(f"Failure: pdf extraction of {url} because {e}")
    return None

Here is the error message whereas the code works perfectly on my machine

Message: session not created: Chrome failed to start: exited normally. (session not created: DevToolsActivePort file doesn’t exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

requirements.txt:
selenium
seleniumbase
webdriver_manager

packages.txt
chromium
chromium-driver

2

Hey @Jean-Jacques_Ohana! :wave:

Here are three examples of running Selenium on Streamlit Community Cloud:

  1. @Franky1’s example:
  1. @snehankekre’s example:
  1. @randyzwitch’s example:

I hope these suggestions will help you in identifying and debugging the issue you’re facing.

If not, please let us know, and we’ll be happy to take another look! :blush:

Thanks, Charly

3

Hello,
I still have problem running selenium on my streamlit app. The example 2 has exactly the same problem here for example: https://selenium.streamlit.app/.
How is it possible to run selenium with streamlit?
Your help would be much appreciated.

Here is the snippet of code:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from typing import Tuple

options = Options()
options.add_argument(‘–disable-gpu’)
options.add_argument(‘–headless’)
options.add_argument(‘window-size=1920x1080’)

Add a user-agent string

options.add_argument(‘user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36’)

def get_driver():
return webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

AttributeError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you’re on Streamlit Cloud, click on ‘Manage app’ in the lower right of your app).
Traceback:
File “/home/adminuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py”, line 535, in _run_script
exec(code, module.dict)
File “/mount/src/chatpdf/Question_WebSearch.py”, line 30, in
from web_document_parallel import get_documents, concatenate_docs_pdfs,
driver = get_driver()
File “/mount/src/chatpdf/connection_to_websites.py”, line 19, in get_driver
return webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
File “/home/adminuser/venv/lib/python3.9/site-packages/webdriver_manager/chrome.py”, line 40, in install
driver_path = self._get_driver_binary_path(self.driver)
File “/home/adminuser/venv/lib/python3.9/site-packages/webdriver_manager/core/manager.py”, line 40, in _get_driver_binary_path
file = self._download_manager.download_file(driver.get_driver_download_url(os_type))
File “/home/adminuser/venv/lib/python3.9/site-packages/webdriver_manager/drivers/chrome.py”, line 32, in get_driver_download_url
driver_version_to_download = self.get_driver_version_to_download()
File “/home/adminuser/venv/lib/python3.9/site-packages/webdriver_manager/core/driver.py”, line 48, in get_driver_version_to_download
return self.get_latest_release_version()
File “/home/adminuser/venv/lib/python3.9/site-packages/webdriver_manager/drivers/chrome.py”, line 64, in get_latest_release_version
determined_browser_version = “.”.join(determined_browser_version.split(“.”)[:3])

4

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.