Streamlit based LLM Multireasoning Chat (QwQ and marco-o1) styled chat with Ollama (ChatGPT o1 Style)

Jan3 · December 13, 2024, 4:31pm

Hi all,

as “all the hype” is about chain reasoning on LLM’s i needed a quick testing system localy to interact with these models.

Feel free to give it a shot (works nicely with QwQ model and llama3.3, but should work with any decently smart model). Make sure you have the latest python ollama bindings cause it uses structured output’s.

Source is:

github.com

janinspark/streamlit_tools/blob/main/ollama_o1_chat

import streamlit as st
import ollama
import os
import json
import time
from pydantic import BaseModel
from typing import Literal, Optional

class ReasoningStep(BaseModel):
    title: str
    content: str
    next_action: Literal['continue', 'final_answer']

class FinalAnswer(BaseModel):
    title: str
    content: str

def make_api_call(messages, max_tokens, is_final_answer=False):
    invalid_format_message = {
        "role": "system",

This file has been truncated. show original

Credits go to
GitHub - bklieger-groq/g1: g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains for its prompts and the initial code which i derived it off

Topic		Replies	Views
Streamlit Ollama Chatbot Multi-Model Interface 🤖 Show the Community! llms , chatbot	0	1284	November 29, 2024
Streamlit Chatbot: Token Streaming LLMs and AI	1	3012	December 19, 2023
LangChain 🤝 Streamlit Show the Community! llms	1	2004	March 9, 2024
Pyrobbot: GPT chatbot Streamlit App. Supports voice interaction Show the Community! streamlitbot , chatgpt , api , llms , openai , gpt4 , speech-to-text , multipage	5	7918	September 24, 2024
How to create your Chatgpt plus version with streamlit and hugchat for free LLMs and AI streamlit-cloud	3	1498	July 4, 2023

Streamlit based LLM Multireasoning Chat (QwQ and marco-o1) styled chat with Ollama (ChatGPT o1 Style)

Related topics

Hello there 👋🏻

Cookie settings

Strictly necessary cookies

Performance cookies

Functional cookies

Targeting cookies