Indexability for Streamlit apps 🎈

Hey community, :wave:

We’re excited to announce that all public Streamlit Community Cloud apps are now indexable by search engines. :star_struck:

Check out the docs for more info on indexing and feel free to leave any questions in the comments.

– TC :balloon:

7 Likes

That’s great news!

How long does it take for search engines to index existing public streamlit apps?
I tried my app in Google Search as suggested:

site:<your-app-url>

but it doesn’t find it yet.

Will indexing also work if the only meta description is in Markdown like this for example?

st.markdown('<h1 style="margin-bottom:0rem;margin-top:-4rem;text-align: center">GPS Time Converter</h1>', unsafe_allow_html=True)
st.markdown('<h5 style="color:grey;margin-bottom:0rem;margin-top:-1rem;text-align: center">Convert between UTC/Local Time and GPS Time</h5>', unsafe_allow_html=True)

I often don’t use st.header/st.subheader/st.text in my apps at all but prefer st.markdown because there is more flexibility in text size, colour, margins etc.
But will it be a problem with indexability?

Regards,
Sibylle

Hi,
in order to index your app, search engines need to kow about it.
In most cases search engines discover websites via links, e.g. link your app from your home page. It usually takes a few hours after detecting a page and idetifying relevant contents.

st.header and st.subheader convert the text into HTML hx structures, so they are quite the same like your exanple only without the styling. This will not be a problem. Search engine will have to render the app in the first place to see the HTML headings, which is the most important work for them to do.

2 Likes

Hi, there. My app still isn’t indexed by Google. It’s public, has been deployed several weeks ago, and has a header. Any ideas what else could be preventing it from being indexed?

1 Like

Have you published it somewhere e.g. here in the streamlit forums (please see: About the Show the Community! category)?
Search engines need links to the documents they should index.

If you have a personal website, you may link to your app - ideally with some additional text - and if you want to you can submit the page from where you are linking to your app to search engines to crawl and index.

1 Like

Thank you :slightly_smiling_face:

2 Likes

Thank you for replying so quickly! I’ll make sure to post my app link for indexing then.

PS.: The “About the Show the Community! category” link sends me to a closed conversation without much info. Is it correct?

2 Likes

Hi,
sorry - thats right - it’s only the opening-post for the caregory.
Apps hosted on Streamlit cloud can be presented here: Topics tagged streamlit-cloud :balloon:
Don’t hesitate to show your work. I’m courious, too. :slight_smile:

1 Like

Thank you for the guidance! Here’s my post: Living quality around the world at street level

I hope you all like it

Hello everybody,
what about Streamlit app deployed with a custom domain but through heroku?

Nice one - very interesting project.
Aaand - see, it’s indexed:
Screenshot_20221204_index-test-of-your-app

1 Like

I’d guess that the same applies for your app on heroku as for the apps on streamlit cloud. As you can see, the page (aka the app) is indexed.
It will probably take some time until search engines take care about the initial rendering. As for now Google has only indexed what is served on the url initially. In order to index the text you are using in your app it has to render ist - which means has to run all the JS stuff and wait untill it’s done. That’s time consuming so search engines often delay that task.*
You can get an idea of what the search engine sees when visiting your app here: https://search.google.com/test/mobile-friendly/result?id=OTKlllTMi0Sh4we7iQhjNA (go to “view tested site” > “screenshot” - and you’ll see the search engine stops rendering while the app is still booting)**
I’m not familiar with heroku. Maybe you can speed up the booting process.

Hope this helps :slight_smile:


*, ** I tried to simplify as much as I could, as this topic gets freaky complex quickly.

1 Like

I think it would be great if Streamlit published an apps site map page. If there’s concern about people not wanting all their public apps indexed necessarily, there could be an app setting to indicate if you want your app included on Streamlit’s apps site map.

Hi Seb,
many thanks for your answer. It helped! have a nice day
Alex

Hi seb,
I have an app hosted on streamlit cloud Your world as well as an app hosted on Heroku but I am faced with the same issue on both for more than 2 weeks now. Do you know how long it takes for google look past the initial rendering on average?

It generally takes between 4 days and 4 weeks for google to make a sweep through new content. If you’ve previously posted the link to your app on some indexable website, the timer would start then. If this is the first time you’ve seeded a link, the timer would start now.

If you control some domain/website and have it set up to be indexed as desired, you can post a link to your app there and make a manual request to google to reindex your website which would pick up your app link in the process.

I have manually requested the app to be indexed multiple times but the problem is that I want the content to be indexed as well so that the page title on google wouldn’t just show “streamlit”. So would it take up to 4 weeks for google to read my app information once indexed or do the 4 weeks include the time it takes google to do its initial rendering as well?

Getting a page indexed and getting the contents of the page indexed is the same thing. When Google gets to your app it will take in the contents accordingly. I looked at your app and the current page title shows <title>Your Free A.I Playground - Worldbook · Streamlit</title> so you should show up as:

Your Free A.I Playground - Worldbook · Streamlit

If you want something else, check out the docs linked in the top post to change the meta data.

Even if you have control over a domain and jumped through the hoops to prove it to Google, they still say it can take a few days to a few weeks to get crawled. The internet is big and Google has its own priorities. If you create a brand new, unique page on your website containing your app link, it could provide you a sanity check. If your new web page gets indexed, but you still don’t see your Streamlit app, you’d know something was up.

Mostly it’s just a waiting game though.

PS It looks like this exact page here on the forum was last indexed on Dec 11.

snapshot of the page as it appeared on Dec 11, 2022 09:32:43 GMT.

So Google is definitely combing through here regularly and will your link in this thread eventually. :slight_smile: