Issue with Selenium on a Streamlit app

Hi guys !

I deployed an app to scrap some french bookmakers, using selenium. But when i run the script, nothing happens.
It seems the webdriver is not working…

Any idea please ?

Hi @KoxNoob, welcome to the Streamlit community!

Can you be more specific about what you are doing? A code snippet would help here, to understand if this is a Streamlit issue or Selenium.


Thanks @randyzwitch for your answer.

You can find below a code snippet.
When i run the script in pycharm, everything works, but not in the deployed app…

Thanks in advance. Regards

Where are you deploying this app, Streamlit sharing?

Yes indeed and also on Heroku

I think that will not work on Streamlit Sharing that way.

I haven’t done anything with selenium for a while, but I seem to remember that

  • you need to have geckodriver or chromedriver installed, which might not apply to the Streamlit Sharing runtime
  • you might have to use the --headless option
  • webdriver.Firefox() wants to open a browser, which will probably fail on the Streamlit Sharing runtime

@randyzwitch Correct me if i am wrong

@KoxNoob Does your app work on Heroku?

In this blog post, I show how to use seleniumbase with Streamlit sharing:

So it definitely works, the question is how to make it work for what you’re trying to do.

1 Like

That looks good, it should work with SeleniumBase
I’ve learned something new again - I didn’t know SeleniumBase until now.
Phew - the python ecosystem is so huge… :wink:

1 Like

Eheh yeah, I realized this the day I found out wget in Python so that I could run wget on older Windows machines with Python and no WSL/Powershell :joy:

Thanks @randyzwitch and @Franky1 !

Indeed, i saw your blog post. Unfortunely, i don’t have a lot of experiences… And I don’t know how to make SeleniumBase work.

From an url (urlpage in my code snippet), I want to collect text from a CSS selector (find_elements_by_xpath).
With selenium it’s working. But how to do with SeleniumBase.

I don’t want to confuse you even more, but for websites that only serve static content (i.e. no content dynamically generated by Javascript) I often use only requests with beautifulsoup4
Another possibility would be scrapy

Yes, i know requests and BS4 but unfortunately, it doesn’t work, there are some content dynamically generated by JS…

For example :
urlpage : “
balise : “//*[@class=“ui-mainview-block eventpath-wrapper”]”

Ok, I see, unfortunately there is no way around using a scraping library with a Javascript engine. :neutral_face:

It seems like you should be able to install chromedriver on Streamlit sharing via a packages.txt file in your repo. Create the text file, add chromium-chromedriver on its own line, and see if it will work.

I tried but it still doesn’t work…

Here you can find my repo. Could you help me again please ?
I don’t understand why it doesn’t work…

And here, my app on Streamlit sharing !

Thanks in advance.

What is the error message?

No error message… Just the scrap doesn’t work… While locally, everything is good…

Are you sure?
I don’t mean the app itself, but please check the Streamlit Sharing console for error messages.

I opened a PR:

This is likely NOT the complete answer, but it allows your app to load: