Hi guys !
I deployed an app to scrap some french bookmakers, using selenium. But when i run the script, nothing happens.
It seems the webdriver is not workingā¦
Any idea please ?
Hi guys !
I deployed an app to scrap some french bookmakers, using selenium. But when i run the script, nothing happens.
It seems the webdriver is not workingā¦
Any idea please ?
Hi @KoxNoob, welcome to the Streamlit community!
Can you be more specific about what you are doing? A code snippet would help here, to understand if this is a Streamlit issue or Selenium.
Best,
Randy
Thanks @randyzwitch for your answer.
You can find below a code snippet.
When i run the script in pycharm, everything works, but not in the deployed appā¦
Thanks in advance. Regards
Where are you deploying this app, Streamlit sharing?
Yes indeed and also on Heroku
I think that will not work on Streamlit Sharing that way.
I havenāt done anything with selenium for a while, but I seem to remember that
--headless
option@randyzwitch Correct me if i am wrong
@KoxNoob Does your app work on Heroku?
In this blog post, I show how to use seleniumbase with Streamlit sharing:
https://blog.streamlit.io/testing-streamlit-apps-using-seleniumbase/
So it definitely works, the question is how to make it work for what youāre trying to do.
@randyzwitch
That looks good, it should work with SeleniumBase
Iāve learned something new again - I didnāt know SeleniumBase until now.
Phew - the python ecosystem is so hugeā¦
Eheh yeah, I realized this the day I found out wget in Python so that I could run wget
on older Windows machines with Python and no WSL/Powershell
Thanks @randyzwitch and @Franky1 !
Indeed, i saw your blog post. Unfortunely, i donāt have a lot of experiencesā¦ And I donāt know how to make SeleniumBase work.
From an url (urlpage in my code snippet), I want to collect text from a CSS selector (find_elements_by_xpath).
With selenium itās working. But how to do with SeleniumBase.
@KoxNoob
I donāt want to confuse you even more, but for websites that only serve static content (i.e. no content dynamically generated by Javascript) I often use only requests
with beautifulsoup4
Another possibility would be scrapy
Yes, i know requests and BS4 but unfortunately, it doesnāt work, there are some content dynamically generated by JSā¦
For example :
urlpage : āUnibet.frā
balise : ā//*[@class=āui-mainview-block eventpath-wrapperā]ā
Ok, I see, unfortunately there is no way around using a scraping library with a Javascript engine.
It seems like you should be able to install chromedriver on Streamlit sharing via a packages.txt
file in your repo. Create the text file, add chromium-chromedriver
on its own line, and see if it will work.
I tried but it still doesnāt workā¦
Here you can find my repo. Could you help me again please ?
I donāt understand why it doesnāt workā¦
And here, my app on Streamlit sharing !
Thanks in advance.
What is the error message?
No error messageā¦ Just the scrap doesnāt workā¦ While locally, everything is goodā¦
Are you sure?
I donāt mean the app itself, but please check the Streamlit Sharing console for error messages.
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.