New App: Content Moderation MVP for Batch Toxic Text & NSFW Image Detection

Show the Community!
1

Hey everyone👋

I just deployed a new Streamlit app and wanted to share it with the community!

This is a Content Moderation MVP designed to help filter unsafe content. It processes both text and images to flag toxicity or sensitive material automatically. I built this to demonstrate how easy it is to integrate pre-trained Transformer models into a Streamlit interface.

How it works:

  • Text moderation: I’m using theunitary/toxic-bert model to run through batches of comments (you can upload a .txt file). It’ll label things as Toxic, Borderline, or Clean, and you can tweak the probability thresholds with sliders.

  • Image moderation: Uses the Falconsai/nsfw_image_detection model to scan uploaded images and flag NSFW content with a confidence score.

:round_pushpin:Try it here: https://content-moderation-mvp.streamlit.app/

I’m planning to experiment with some fusion models or maybe build a slightly smarter pipeline later, but for now I’d love feedback, especially on the UI/UX or suggestions for other models that might be fun to plug in.

Thanks and happy exploring!