Save log into a file using streamlit.logger

[Cerri]

Summary

How can i save a log file during the execution of a streamlit app?

Steps to reproduce

I use the following lines to declare a log using streamlit api

from streamlit.logger import get_logger

logger = get_logger(__name__)

logger.info('Hello world')

But this log is not stored into a file. How can I do that?

I run the the app using:

streamlit run app.py

I do not want to use:

streamlit run app.py --logger.level=warning 2>'app_log.log'

Because using it, all external tool used in the app write on the same log.

[blackary]

Hi @Cerri, welcome to the forum! I would recommend simply setting up your own logger, using either the built-in python logger How to write to a file, using the logging Python module? - Stack Overflow or some alternative library like loguru Overview — loguru documentation

[Cerri]

I @blackary, thanks!
I tried used the classic python logger, but the file is not created. I must use

streamlit run app.py --logger.level=warning 2>'app_log.log'

[blackary]

@Cerri You might try using loguru, which seems to work fine for me. You can click show code to see the code:

[Cerri]

@blackary Thanks a lot. I will try it!

[Cerri]

@blackary I tried, the log works as well. The only thing is that each line is writed on log multiple times! How can I solve it?

Thanks in advance

[DrBenjamin]

@blackary How can I programmatically access the environment details (Python version / Streamlit version)?

[Goyo]

import platform

import streamlit as st

st.text(platform.python_version())
st.text(st.__version__)

[blackary]

Hi @Cerri, in general you should expect your streamlit app to be rerun every time you interact with something on your app. If you absolutely need the log to only run at certain times, you might be able to use st.experimental_memo with a TTL. Or, you could set a flag in st.session_state that keeps track of whether a certain logging statement has already been run, and only call the logging function if that flag is False. Something like:

if not st.session_state["have_logged_thing"]:
    logger.debug(...)
    st.session_state["have_logged_thing"] = True

Then it should only get logged once per session.

[lasinludwig]

@Cerri Did you manage to create the log-file? I have the same issue and can’t figure out how to make it create a file. I tried the standard logging module and the loguru one which @blackary suggested. I can see the logging messages in the terminal, but it doesn’t write them to the file I specified.

This is my loguru configuration:

[blackary]

@lasinludwig Off the top of my head, I wonder if your app doesn’t have permission to write to the /logs folder wherever you are running it.

[lasinludwig]

Ok, turns out I’m an idiot!
I just needed to put in the complete path and it works perfectly…

(It worked before as well, it just didn’t put the logs where I expected… my mistake. )

[ethanone]

I’m using the built in logging module in Python and I’m still getting multiple log entries.
Here is simplified code:

import streamlit as st
import os
import logging

def configure_logging(file_path, level=logging.DEBUG):
    logger = logging.getLogger()
    logger.setLevel(level)

    file_handler = logging.FileHandler(file_path)
    file_handler.setLevel(level)

    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    file_handler.setFormatter(formatter)

    logger.addHandler(file_handler)

    return logger

if __name__ == '__main__':
    log_file = os.path.join(os.getcwd(), 'log', f'parcel-qc.log')
    logger = configure_logging(log_file)

    check_tables = st.checkbox('Check for data tables')

    if st.button('Run Checks'):
        if check_tables:
            with st.echo():
                st.session_state.has_logged = False if 'has_logged' not in st.session_state else False
            st.write(st.session_state)
            st.write(f'Check for data tables: {check_tables}')
            with st.spinner('Checking for data tables...'):
                lengthy_process_output = "The quick brown fox jumped over the lazy dog."
            st.write(lengthy_process_output)
            with st.echo():
                if not st.session_state.has_logged:
                    logger.debug(f'Data table check: {lengthy_process_output}')
                    st.session_state.has_logged = True
        st.write(st.session_state)

I’ve noticed the number of repeats is not consistent, sometimes 4 sometimes 14. It really blows up the size of the log and makes it difficult to debug. I’m using with st.echo() to confirm its only logging once.
Any other suggestions on what to try to get only a single output for each log entry?

[ethanone]

Just to test, I thought I’d try replacing the logging with a simple text appending function and replaced my logging calls with the a call to that append function and I only got a single line written to the text file.

I also tried putting BOTH the logging call and the text calls into my app and when doing so, the logging call doesn’t seem to write to the log, but the text append works fine.

There is something in how the python logging works that doesn’t play well with streamlit. I even seemed to get log entries appearing (duplicates too), after shutting the app down. The text append meets my needs, but there is definitely something fishy with the logging module.

[blackary]

Interesting problem!

The problem seems to be that, because of the streamlit model of “rerun on every interaction”, you end up running the configure_logging multiple times, and therefore setting up multiple loggers. An easy fix is to use @st.cache_resource on the configure_logging function, therefore returning the same logger object each time. This seems to resolve the issue!

@st.cache_resource
def configure_logging(file_path, level=logging.DEBUG):

[Hernan_Diaz]

@st.cache_resource
def configure_logging(file_path, level=logging.DEBUG):

Hello, this code should be added to streamlit documentation. It solves the issue of logs duplications and I think it is very important. I am using loguru BTW.