Streamlit duplicates log messages when stream handler is added

Using Streamlit
1

When I add a stream handler to my logger object (so that I can do formatting), streamlit then outputs 2 messages (one formatted and one not)

Any ideas for where I can look to fix this?

Here’s a sample of what’s happening.

test.py:

import logging
logger = logging.getLogger()
logger.setLevel('INFO')
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s-%(name)s-%(levelname)s>>>%(message)s', "%H:%M:%S")
ch.setFormatter(formatter)
logger.addHandler(ch)
logger.info('Start of program')

Now if I run that using python test.py I get the following:

11:24:31-root-INFO>>>Start of program

Which is what i’d expect

But if I run it using streamlit run test.py I get:
2021-08-24 11:24:58.198 Start of program
11:24:58-root-INFO>>>Start of program

There’s a duplicate, unformatted message.

How can I resolve this, so that there is only one (formatted) message?

(NOTE: FileHandler() objects have the expected behavior. I.e. if you add

fh = logging.FileHandler('test.log', 'w')
fh.setFormatter(formatter)
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)

the output in the log file test.log is as expected. One, properly formatted message, no duplicates)

2

I’m seeing the same behavior. Any progress on this?

3

Actually, it appears to only duplicate logs when I start the server initially. My guess is that it performs a refresh right at the start which would explain the duplication.

4

I wrote a create logger function that checks how many handlers are already present, and then also used session state to track if a logger had already been created. I think this is double overkill, but it worked for me!

def create_logger(name, level = 'DEBUG', file = None):
    logger = logging.getLogger(name)
    logger.propagate = False
    logger.setLevel(level)
    #if no streamhandler present, add one
    if sum([isinstance(handler, logging.StreamHandler) for handler in logger.handlers]) == 0:
        ch = logging.StreamHandler()
        ch.setFormatter(logging.Formatter('%(asctime)s.%(msecs)03d-%(name)s-%(levelname)s>>>%(message)s', "%H:%M:%S"))
        logger.addHandler(ch)
    #if a file handler is requested, check for existence then add
    if file is not None:
        if sum([isinstance(handler, logging.FileHandler) for handler in logger.handlers]) == 0:
            ch = logging.FileHandler(log_dir/file, 'w')
            ch.setFormatter(logging.Formatter('%(asctime)s.%(msecs)03d-%(name)s-%(levelname)s>>>%(message)s', "%H:%M:%S"))
            logger.addHandler(ch)
        
    return logger

if 'logger' not in st.session_state:
    st.session_state['logger'] = create_logger(name = 'app', level = 'DEBUG', file = 'app.log')
logger = st.session_state['logger']