Interactive annotation using Streamlit

[Nain]

I am new to streamlit. I was trying to setup a simple annotation process for a pet project. So, I have all the filenames stored in a pandas dataframe. Now, the annotation process is supposed to have this flow:

1. Iterate over the files
2. Read an image. Display it
3. Annotate using radio boxes.
4. Increment the loop counter only when the user clicks “Next”

Here is a sample similar to what I have been trying but that didn’t work out:

def annotate_data(df):
    file_wid = st.empty()
    img_wid = st.empty()
    labels = ("cat", "dog")
    go_next = st.button("Next")

    for i in range(len(df)):
        fname = df["fname"][i]
        name = str(fname)
        
        # Display the name of the file 
        value = file_wid.text(name)
        st.write(value)
        time.sleep(1)
    
        # Display the image
    
       # Add label using radio box
    
       # Wait untl user press next


    st.write("Done")

It would be really helpful if someone can provide some pointers

[Lutz]

I recommend this thread that was motivated by a similar use case. Maybe you can find an answer in there. Else, maybe you can continue the thread there.

How to evolve complex state (e.g., annotate data)?

Streamlit is very convenient to quickly develop apps that have a small fixed state-space (i.e., all the widgets). How, if at all, is it possible to interactively evolve some more complex state, like a dictionary of annotations? Streamlit is good to interactively showing data instances. But how can I record, say, a binary label per instance that the user indicates per button click or keystroke?

[Nain]

Thanks for the quick reply. I looked at that question before posting but that is very verbose and a bit diff.

[Nain]

So, I tried a lot based on the suggestions provided in that thread. I tried to use the SessionState as well but it didn’t work. To elaborate again, I am looking to build something like this:

1. Place three buttons Next, Previous, and Reset. The user should be able to reset the state of the button after every iteration.
2. Display the first image. Tag it.
3. If the user presses Next, go to the next image. Same for the other buttons
4. To label the image, place two buttons below the image

This workflow seems straightforward but apparently I have wasted a lot of time in it. Maybe I am missing something here and you can point me in the right direction @Lutz @Adrien_Treuille

[khalido]

I don’t think this is possible with streamlit as it currently is, though the current planned for features on their roadmap will make it possible. Specifically the work they are doing on supporting the idea of a state:

Right now, getting a Streamlit app to store internal state, like information a user entered in a form, is simply too tricky. There are some workarounds for session state , but we want to give you a baked-in and elegant version of programmable state so you can build apps with sequential logic, apps with multiple pages, apps that incrementally ask users for input, and so on.

The exisiting ways to do this are a bit hard, I tried a couple, but decided to wait for the official session state thingamajig.

Currently streamlit is setup as a single use application, i.e anytime u do anything, everything re-runs from the top. For simple interfaces this works well, but for anything which needs to remember stuff - it doesn’t, unless you write your own logic which saves stuff to disk in b/w runs.

That all said, I don’t think its worthwhile trying to accomplish image tagging using streamlit. There are a number of existing options, prodigy by spacy in particular looks really easy to use. There are a bunch of open source options as well, this one looks easy to use and is free.

I’d consider seperating out the image annotation work - label images seperately, then use streamlit for the frontend which reads in the images, their labels, and does some stuff with that info.

[Nain]

Thanks @khalido for the detailed response. I was onto building a much bigger tool than just image annotation but I had to validate the idea using a sample image tagger before combining other functionalities. But yeah, as you said, I think it makes sense to separate out the annotation part from Streamlit. Thanks again for the info

[Adrien_Treuille]

Hey all. Just wanted to step in and agree @khalido’s excellent summary of the situation.

Right now, Streamlit is designed for apps that summarize data and/or run inference than it is for apps which require more complex state like annotation.

We hear you, though!

Data editing apps (and especially annotation) are top of mind as we roll major new features this year (and early next). In particular components (slated for next week) will let you include much richer widgets for annotation tasks. Following that we expect to release a big upgrade to layout, and then move onto programmable state.

Your message is super valuable. As we design these features, we read each request like this carefully in an attempt to cover as many workflows as possible, and make them simple and elegant, powerful, and fun!

Happy app creating!

[Quoc_Tran]

It is doable with the current streamlit. Here is my simplified app: https://github.com/QuocTran/streamlit_annotate
To avoid session state, the trick is keeping a file having the master list of items needed annotation and a file with all the tagged values. At each run, streamlit will show the item that is not annotate yet.
When click “Save” the tagged file is updated with new record and the image is downloaded and written with a new name.
The trick to avoid using “R” each time to refresh is calling a rerun app function.
Hope it help.

[areo]

Here’s another example of a labeling app that makes use of the SessionState hack:

sanzgiri/st_labeler/blob/master/label_videos.py

[Nain]

Thanks @Adrien_Treuille for the detailed response. Also, thanks a lot @Quoc_Tran and @areo for the solutions. I will try them and check if they work as desired for my case.