# Redirecting logger output to a streamlit widget

**URL:** <https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070>\
**Category:** Using Streamlit\
**Created:** [January 30, 2024, 5:37am UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070 "2024-01-30T05:37:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![david-waterworth](https://sea2.discourse-cdn.com/streamlit/user_avatar/discuss.streamlit.io/david-waterworth/32/33648_2.png) [@david-waterworth](https://discuss.streamlit.io/u/david-waterworth)\
**Post date:** [January 30, 2024, 5:37am UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070/1 "2024-01-30T05:37:31Z")

</div>

I have a module which runs a (synchronous) task. It uses the standard python logger, i.e. `task.py` looks something like

```auto
import logging
logger = logging.getLogger( __name__ )

def task(args):
    logger.info("Running task")

    logger.info("Finished task")

```

I want to invoke this task from my `app.py` and display the log messages as the task runs. I found a few examples that redirected `stdout` but I cannot get them to work, for example

```auto
import logging
logging.basicConfig(level=logging.INFO, stream=sys.stdout)
logger = st.logger.get_logger( __name__ )

@contextmanager
def stdout_capture(output_func):
    with StringIO() as stdout, redirect_stdout(stdout):
        old_write = stdout.write

        def new_write(string):
            ret = old_write(string)
            output_func(stdout.getvalue())
            return ret

        stdout.write = new_write
        yield

  output = st.empty()
  with stdout_capture(output.code):
      print("test")
      logger.info("test 2")
      tasks.run()

```

This correctly `test` to output.code but not `test 2` or messages from the task module, theses log messages are still appearing in my console and not the output widget.

I’ve considered writing my own log handler but I’m wondering if there’s anything obvious I’ve done wrong with my current approach and is this possible / what approaches could I try?

If not I’ll probably use `subprocess.Popen` and a file log.

---

<div class="post-metadata">

**Author:** ![snehankekre](https://sea2.discourse-cdn.com/streamlit/user_avatar/discuss.streamlit.io/snehankekre/32/6504_2.png) [@snehankekre](https://discuss.streamlit.io/u/snehankekre)\
**Post date:** [January 30, 2024, 8:41am UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070/2 "2024-01-30T08:41:10Z")

</div>

Hey @david-waterworth 👋

From what you’ve described, it seems like your main objective is to display logs from `task.py` in your app as they’re generated. What you’ve tried involves redirecting `stdout`, which is a common strategy for capturing print statements. But it won’t work for capturing log messages directly, as Python’s logging module doesn’t use `stdout` by default for its output.

Since Streamlit’s [loggers](https://github.com/streamlit/streamlit/blob/develop/lib/streamlit/logger.py) don’t write to `stdout` (the loggers are set up with a `StreamHandler` that writes to the console) and do not propagate their messages to the root logger (which was configured to redirect to `stdout`), log messages were not captured by your `stdout_capture` context manager.

As you’ve suggested, writing your own log handler might be the way to go. You could extend Streamlit’s logger with a custom handler that routes logs to `st.empty().code` for display, like so:

```py
# app.py
import logging
import task
import streamlit as st
from streamlit.logger import get_logger

class StreamlitLogHandler(logging.Handler):
    def __init__ (self, widget_update_func):
        super(). __init__ ()
        self.widget_update_func = widget_update_func

    def emit(self, record):
        msg = self.format(record)
        self.widget_update_func(msg)

logger = get_logger(task. __name__ )
handler = StreamlitLogHandler(st.empty().code)
logger.addHandler(handler)

# Run your task
task.run()

```

```py
# task.py

import logging
logger = logging.getLogger( __name__ )
import time
import streamlit as st

def run(*args, **kwargs):
    logger.info("🏃 Running task ")
    time.sleep(2)
    logger.info("🥵 Still running task")
    time.sleep(2)
    logger.info("✅ Finished task")

```

> **Breakdown:**
>
> - Use `get_logger` from `streamlit.logger`, which retrieves a Streamlit-configured logger. It’s already set up with Streamlit’s logging config such as format and level
> - Define `StreamlitLogHandler` – a custom logging handler that overrides the `emit` method. This method is called whenever a log message is sent to the handler
> - In the `emit` method, the logs are formatted into a message string, and then passed to the `widget_update_func`, which in this case is `st.empty().code`
> - By adding `StreamlitLogHandler` to the Streamlit logger (`logger.addHandler(handler)`), you’re ensuring that any logs processed by this logger are also passed to the custom handler. And since the custom handler is designed to update a Streamlit element, you see the logs in your app

![custom-log-handler](https://global.discourse-cdn.com/streamlit/original/3X/f/9/f9c4445d74fb1ac0b41832899203081982035d20.gif)

Does this help? 🎈

---

<div class="post-metadata">

**Author:** ![david-waterworth](https://sea2.discourse-cdn.com/streamlit/user_avatar/discuss.streamlit.io/david-waterworth/32/33648_2.png) [@david-waterworth](https://discuss.streamlit.io/u/david-waterworth)\
**Post date:** [January 30, 2024, 10:05pm UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070/3 "2024-01-30T22:05:04Z")

</div>

Awesome thanks @snehankekre that works perfectly! I was about to resort to `subprocess.Popen` (which I’ve done for another process) but don’t have to now.

Not sure if it’s required but I added

`logger.handlers.clear()`

Before adding the handler to ensure I didn’t get duplicates (I read about this issue in other posts and other solutions are to use a cache decorator around a function to setup the logger, or add the logger to session state).

PS How is the logging module writing to the console without using stdout (or stderr)? I’d always assumed that was the only mechanism to write to the console?

---

<div class="post-metadata">

**Author:** ![snehankekre](https://sea2.discourse-cdn.com/streamlit/user_avatar/discuss.streamlit.io/snehankekre/32/6504_2.png) [@snehankekre](https://discuss.streamlit.io/u/snehankekre)\
**Post date:** [January 31, 2024, 9:23am UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070/4 "2024-01-31T09:23:24Z")

</div>

Glad it works! 😄

> Not sure if it’s required but I added

You’re right. Clearing the log handlers is probably a good idea to avoid duplicates due to script reruns.

> PS How is the logging module writing to the console without using stdout (or stderr)? I’d always assumed that was the only mechanism to write to the console?

It’s writing to `stderr`. The `StreamHandler` in Python’s logging module sends logs to streams like `stderr` or `stdout` or even to files, if configured to do so. When you create a logger and don’t specifically configure its handlers, it uses `stderr`. This is why you see log messages in the console, even though you’re not explicitly writing to `stdout`. Here’s where that happens in Streamlit’s logging module:

> <https://github.com/streamlit/streamlit/blob/28dec526eed312a4aa3e8463c8ba190c584bf9f9/lib/streamlit/logger.py#L66>

By default, the `StreamHandler` in Python logging writes to `stderr` and not `stdout`.

This means that even though the log messages are displayed in the console, they are being sent via `stderr`.

In Streamlit’s case, each logger is configured with this `StreamHandler` and a specific format, directing its output to `stderr`. This is why your initial approach with redirecting `stdout` did not capture these log messages, as they were not being routed through `stdout`. Additionally, Streamlit loggers do not propagate their log messages to the root logger ([`logger.propagate = False`](https://github.com/streamlit/streamlit/blob/28dec526eed312a4aa3e8463c8ba190c584bf9f9/lib/streamlit/logger.py#L125)). This means that any configuration done on the root logger (like redirecting `stdout`) does not affect Streamlit’s loggers.

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/streamlit/original/3X/c/f/cf4c2ee0d938358f82448c5fe4b9c3c91d7ce19c.svg) [@system](https://discuss.streamlit.io/u/system)\
**Post date:** [February 2, 2024, 9:24am UTC](https://discuss.streamlit.io/t/redirecting-logger-output-to-a-streamlit-widget/61070/5 "2024-02-02T09:24:06Z")

</div>

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.