Running selenium on streamlit cloud

Community Cloud
1

0

I’m using selenium on streamlit cloud server and for that i have issues related to chrome, chrome driver, their versioning. There are other threads about this issue but none is solving my problem.

I’m doing this to get the chrome driver version:

webdriver.Chrome(service=Service(ChromeDriverManager(driver_version="120.0.6099.10900").install()), options=options)

Doing this does runs the scrapping from streamlit but not everytime and most of the time it fails with driver error.

the reason keeping that driver version 120.0.6099.10900 works is because if i don’t mention it and run ChromeDriverManager() without parameters it gives me error saying:

This version of ChromeDriver only supports Chrome version 131. Current browser version is 120.0.6099.224

My chrome version currently is 131.0… but it mentions 120.0… this is what i don’t understand, maybe to use selenium there are dummy/testing chrome version and driver version i believe.

and if i check the ChromeDriverManager class, then the drivers are upto version 114.0… only but my issue is the version 120 which is not provided in [https://chromedriver.storage.googleapis.com] so i get error saying not a zip file/ bad zip file for version 120

another error that pops up while running with that 120.0… version chromedriver sometimes is : WebDriver error: Message: unknown error: session deleted because of page crash from unknown error: cannot determine loading status from tab crashed (Session info: chrome-headless-shell=120.0.6099.224) this is the code for reference:

class ChromeDriverManager(DriverManager):
    def __init__(
            self,
            driver_version: Optional[str] = None,
            name: str = "chromedriver",
            url: str = "https://chromedriver.storage.googleapis.com",
            latest_release_url: str = "https://chromedriver.storage.googleapis.com/LATEST_RELEASE",
            chrome_type: str = ChromeType.GOOGLE,
            download_manager: Optional[DownloadManager] = None,
            cache_manager: Optional[DriverCacheManager] = None,
            os_system_manager: Optional[OperationSystemManager] = None
    ):

There are other sites where chrome driver 120.0 versions are available but it cannot be used to install directly from code like that and since im running on server there is no way i can keep local path/binary location.

here is my code that runs well on server but , the driver version mentioned there is not going to be maintainable for long term or even short term, so am i missing something or doing anything wrong?

        options = Options()
        # options.binary_location = "/usr/bin/chromium-browser"
        # Enhanced GPU and rendering configuration
        # options.add_argument("--start-maximized")
        options.add_argument("--headless")
        options.add_argument("--no-sandbox")
        # # options.headless=True
        # options.add_argument("--disable-dev-shm-usage")
        # options.add_argument('--disable-infobars')
        # options.add_argument("--disable-gpu")  # Completely disable GPU hardware acceleration
        # # options.add_argument("--headless=new")
        
        # # More robust graphics rendering fallback
        # options.add_argument("--use-gl=egl")  # Alternative graphics rendering method
        # options.add_argument("--disable-software-rasterizer")
        # options.add_argument("--renderer-process-limit=1")  # Limit renderer processes
        # options.add_argument("--enable-unsafe-swiftshader")
        # # Media and audio configuration
        # options.add_argument("--disable-audio-output")
        # options.add_argument("--disable-video")
        # # options.page_load_strategy = 'eager'
        # options.add_argument("--disable-extensions")
        # options.add_argument("--disable-logging")
        # options.add_argument("--disable-crash-reporter")
        # options.add_argument("--disable-background-networking")
        # options.add_argument("--remote-debugging-port=9222")


        # Ignore specific graphics and media errors
        options.add_experimental_option('excludeSwitches', ['enable-logging'])
        
        prefs = {
            "profile.managed_default_content_settings.images": 2,
            "profile.default_content_setting_values.media_stream": 2
        }
        options.add_experimental_option("prefs", prefs)
        driver = webdriver.Chrome(service=Service(ChromeDriverManager(driver_version="120.0.6099.10900").install()), options=options)
    
   driver.get(url)
    time.sleep(3)