FLARE26 — a "glass-box" document auditor that knows when to abstain (offline demo, no API key)

Show the Community!
1

Hi all :waving_hand:

I built a small Streamlit app to showcase FLARE26, a “glass-box” document auditor: it compares documents and, when the evidence doesn’t match a question’s type/scope, it abstains (“evidence gap”) instead of hallucinating an answer.

:play_button: Live app: https://flare26-p1-j5gahqx45gwp7h4ksdohxn.streamlit.app/

The demo is fully offline / zero-cost — it reads pre-computed results from the benchmark, so no OpenAI key is needed and it can’t run up any API bill. In the app you can:

  • :magnifying_glass_tilted_right: Browse the benchmark and see what the model actually did per item across 10 runs (e.g., an absent-info case → abstained 10/10, nice and stable);
  • :bar_chart: see the headline result (a naive baseline hallucinates a “divergence” 38% of the time vs. ~1–2% for the gated version);
  • :chart_increasing: explore the (k, t) self-consistency sweep — a pandas.pivot rendered with st.dataframe + st.line_chart, showing how a majority vote drives false positives to ~0.

Built with: just streamlit + pandas (the heavy RAG pipeline stays out of the demo, so the build is tiny and fast). Streamlit made the “glass-box” view — showing the model’s per-run decisions — almost trivial to put together.

It’s an applied / evaluation-methodology project (I’m an independent learner), all open:

Feedback on the app/UX very welcome :folded_hands: