Trubrics: A user feedback tool for your AI Streamlit apps

A 3-step guide on collecting, analyzing, and managing AI model feedback

Posted in Advocate Posts, July 28 2023

Hey, community! šŸ‘‹

My name is Jeff, and Iā€™m a co-founder of Trubrics. As a data scientist and Machine Learning engineer, I have experienced firsthand the challenges of deploying new ML models without understanding how users interact with them. This can lead to reduced model performance and, ultimately, misaligned models and users.

More specifically, here is why you should start listening to your users:

  • šŸšØ Identify bugs: Users constantly run inference on your models and may be more likely to find bugs than your ML monitoring system.
  • šŸ§‘ā€šŸ’»ļø Fine-tune: Users often have domain knowledge that can be useful in fine-tuning models.
  • šŸ‘„ Align: Identifying user preferences can help align models with users.

If you want to generate user insights on your AI models, we've built Trubrics, the first user insights platform for AI models.

In this post, we'll cover:

  1. How to create a free account with Trubrics
  2. How to collect user feedback from your AI Streamlit app
  3. How to analyze and manage your user feedback in Trubrics

šŸšØ

TLDR: Here is the app and the repo, and here is the video (watch it above).

1. How to create a free account with Trubrics

Create an account directly in the Trubrics app:

After logging in, you'll find a pre-existing default component.

Click on it to access code snippets and start collecting feedback. You can create different feedback components to collect and organize feedback across multiple projects or apps.

Now, to save your first piece of feedback to the default component, head over to our example user feedback LLM app:

2. How to collect user feedback from your AI Streamlit app

Let's explore various code snippets to see how you can embed Trubrics feedback components directly into your app.

First, install our SDK with the streamlit dependency (if it's not already installed):

pip install "trubrics[streamlit]"

Then copy and paste this snippet into your app:

import streamlit as st
from trubrics.integrations.streamlit import FeedbackCollector
collector = FeedbackCollector(
    component_name="default",
    email=st.secrets["TRUBRICS_EMAIL"],
    password=st.secrets["TRUBRICS_PASSWORD"],
)
collector.st_feedback(
    feedback_type="thumbs",
    model="your_model_name",
    open_feedback_label="[Optional] Provide additional feedback",
)

What's going on here? Let's break it down:

  1. The FeedbackCollector object. Store your Trubrics credentials in st.secrets, and specify the feedback component to which you want to save. In this case, use the default.
  2. Its st_feedback() method. Calling this method allows users to embed UI widgets in their apps. These widgets can be added throughout your app to collect feedback on different predictions.

And that's it!

Now you'll see this feedback widget in your application:

šŸšØ

To learn more about customizing your feedback component, read our docs.

3. How to analyze and manage your user feedback in Trubrics

After you've saved the feedback to Trubrics, consult the šŸŖ„ Insights page for quantitative and qualitative analysis. This enables AI teams to understand whether users are satisfied with predictions and compare results between different models.

Various filters on the User feedback tab allow AI teams to:

  • Aggregate responses by frequency (hourly, daily, weekly, monthly)
  • View all responses for a specific score, model, or user
  • Compare responses for all scores, models, or users

For qualitative feedback, user comments are collected in the text field of the Feedback response. All comments are listed in the Comments tab and may be grouped to create an issue. AI model issues can also be opened or closed on the issues page.

Wrapping up

Thank you for reading! You've learned how to collect app user feedback and view it in Trubrics. If you have any questions, please post them in the comments below or contact me directly on my LinkedIn.

Happy Streamlit-ing! šŸŽˆ


This is a companion discussion topic for the original entry at https://blog.streamlit.io/trubrics-a-user-feedback-tool-for-your-ai-streamlit-apps

this feedback tool is superb, like I want to ask that there are some more free ai tools that should be taken care of while seeking for a solution.

Please help me some more AI tools for that.