Using Selenium to scrape streamlit page

[ctran]

Hi guys, I’m having issues with using Selenium and finding any element on my Streamlit page. I was able to connect and launch by app using driver.get(url). After that, I couldn’t find any element using radio_button = driver.find_element(By.CLASS_NAME,“st-c2 st-b0 st-b1 st-b2 st-b3 st-c3 st-bf st-bg st-c4”). Upon further inspection of the page, it looks like all elements are imbedded inside

. Does anyone know how to get around this? Thanks!

[Encrypt]

It’s better to use a more robust CSS selector, as Streamlit might change the class names during a different render. For example, instead of targeting a specific class name, consider selecting by attributes or roles that are less likely to change between renders.

Suggested code:

radio_button = driver.find_element(By.CSS_SELECTOR, 'div[role="radiogroup"]')

[blackary]

@Encrypt makes a good suggestion – another option is to set a custom class by setting a key on an element, which automatically adds a class to that object. It’s a bit trickier if you want to target a group of elements, but you can do that by setting a class with a common prefix.

Here’s an example of both.

import streamlit as st

st.radio("Button 1", [1, 2, 3], key="button1")

st.radio("Button 2", [3, 4, 5], key="button2")

st.html("""
<style>
    /** targets only button 1 **/
    .st-key-button1 {
        background-color: yellow;
    }
    /** targets both buttons **/
    /** select anything with the class *st-key-button* **/
    [class*="st-key-button"] p {
        color: green;
    }

</style>
""")