Selenium web scraping on streamlit cloud

randyzwitch · February 7, 2022, 3:03pm

Hi @YoussefSultan, welcome to the Streamlit community!

If I had to guess, I suspect your issue is with this line:

YoussefSultan/WholeFoods-Datascraping-Project-Deployment/blob/main/Deployment/wholefoods_scraper.py#L24

      
        
            from selenium.webdriver.chrome.options import Options
            ########################################################
            options = Options()
            options.add_argument('--headless')
            options.add_argument('--disable-gpu')
            options.add_argument('--log-level=3')
            #########################################################
            
            
#########################################################
            try:
                browser = webdriver.Chrome('C:/Users/Water/Desktop/chromedriver.exe', options=options) # Chrome Driver
                browser.get('https://www.wholefoodsmarket.com/products/all-products?featured=on-sale') # Website Link
                print('Enter the zipcode of your local WholeFoods...')
                try:
                    parser = argparse.ArgumentParser()
                    parser.add_argument("zipcode")
                    args = parser.parse_args()
                    zipcode = str(args.zipcode)
                    browser.find_element_by_xpath("//input[@id='pie-store-finder-modal-search-field']").send_keys(zipcode) # Zip code
                except:
                    browser.find_element_by_xpath("//input[@id='pie-store-finder-modal-search-field']").send_keys(input()) # Zip code

On Streamlit Cloud, it is a Debian image, not a Windows one. So C:/ won’t exist. I would explore how to install Selenium on Debian, and add those installations to a packages.txt file as highlighted in the documentation:

Best,
Randy

Topic		Replies	Views
Unable to deploy app on streamlit with multithreading + web scraping Community Cloud cache , streamlit-cloud	4	1078	January 5, 2024
Selenium web scraper - ModuleNotFoundError Community Cloud	3	974	January 4, 2023
Streamlit interaction issues with Selenium Using Streamlit	6	3593	January 3, 2022
Error "selenium.common.exceptions.WebDriverException" although app works fine locally Deployment streamlit-cloud	10	3060	February 9, 2024
selenium.common.exceptions.WebDriverException Using Streamlit streamlit-cloud	4	1431	August 22, 2023

Selenium web scraping on streamlit cloud

Related topics

Hello there 👋🏻

Cookie settings

Strictly necessary cookies

Performance cookies

Functional cookies

Targeting cookies