Redirecting logger output to a streamlit widget

[snehankekre]

Hey @david-waterworth

From what you’ve described, it seems like your main objective is to display logs from task.py in your app as they’re generated. What you’ve tried involves redirecting stdout, which is a common strategy for capturing print statements. But it won’t work for capturing log messages directly, as Python’s logging module doesn’t use stdout by default for its output.

Since Streamlit’s loggers don’t write to stdout (the loggers are set up with a StreamHandler that writes to the console) and do not propagate their messages to the root logger (which was configured to redirect to stdout), log messages were not captured by your stdout_capture context manager.

As you’ve suggested, writing your own log handler might be the way to go. You could extend Streamlit’s logger with a custom handler that routes logs to st.empty().code for display, like so:

# app.py
import logging
import task
import streamlit as st
from streamlit.logger import get_logger

class StreamlitLogHandler(logging.Handler):
    def __init__(self, widget_update_func):
        super().__init__()
        self.widget_update_func = widget_update_func

    def emit(self, record):
        msg = self.format(record)
        self.widget_update_func(msg)

logger = get_logger(task.__name__)
handler = StreamlitLogHandler(st.empty().code)
logger.addHandler(handler)

# Run your task
task.run()

# task.py

import logging
logger = logging.getLogger(__name__)
import time
import streamlit as st

def run(*args, **kwargs):
    logger.info("🏃 Running task ")
    time.sleep(2)
    logger.info("🥵 Still running task")
    time.sleep(2)
    logger.info("✅ Finished task")

Breakdown:

• Use get_logger from streamlit.logger, which retrieves a Streamlit-configured logger. It’s already set up with Streamlit’s logging config such as format and level
• Define StreamlitLogHandler – a custom logging handler that overrides the emit method. This method is called whenever a log message is sent to the handler
• In the emit method, the logs are formatted into a message string, and then passed to the widget_update_func, which in this case is st.empty().code
• By adding StreamlitLogHandler to the Streamlit logger (logger.addHandler(handler)), you’re ensuring that any logs processed by this logger are also passed to the custom handler. And since the custom handler is designed to update a Streamlit element, you see the logs in your app

Does this help?