Making logging module write all logs to only stderr

Question

I am using pytest to test a CLI that produces some output. While running the test, I want to set my CLI's log level to DEBUG. However, I don't want CLI logs to interfere with tests that are parsing the output of the CLI.

How can I make logging module send all the logs to only stderr? I looked at this post but it talks about sending logs to stderr in addition to stdout.

did you try `logger.addHandler(logging.StreamHandler(sys.stderr)`? — Tomerikoo, Jul 31 '19 at 00:54
(a new comment on purpose) After looking in the [docs](https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler), *stderr* is actually the default of `StreamHandler`s. It will be helpful if you showed how to configure your logger, because according to the [docs](https://docs.python.org/3/library/logging.html#logging.basicConfig), by simply doing nothing you should get your desired outcome — Tomerikoo, Jul 31 '19 at 00:57
`pytest` will capture all output by default; the `capsys` fixture will give you the captured `stdout`/`stderr` output, while `caplog` will capture the log records. I don't think you will get any interference at all. If I'm wrong, give an example of a test where you can't achiveve the desired behaviour. — hoefling, Jul 31 '19 at 05:16

blues · Answer 1 · 2019-07-31T01:10:13.310

Do the same that the linked answer does but replace stdout with stderr. So you would create a handler with logging.StreamHandler(sys.stderr) and make sure this is the only active handler if you want to exclusively have logs go to stderr.

As @Tomerikoo correctly points out you don't need to do anything though as logging defaults to using a StreamHandler with stderr. The only real value of the code below is that it sets a different level than the default. Just logging.warning('log') with no other setup will send a log to stderr.

Addendum: you can also achieve this using basicConfig to have less boilerplate code.

import logging
import sys
logging.basicConfig(stream=sys.stderr, level=logging.INFO)
logging.info('test') # sends log to stderr

Read my comments on the question: interesting thing: *sys.stderr* is actually the default `StreamHandler`, `basicConfig` by default creates the default `StreamHandler`, and `basicConfig` will be automatically called if no handlers defined. So basically this means we can just do `logging.info('test')` and it will go to *stderr* — Tomerikoo, Jul 31 '19 at 01:06

T3am5hark · Answer 2 · 2019-07-31T01:08:47.203

0

You can direct the log output for a given logger to stderr as follows. This defaults to stderr for output, but you can use sys.stdout instead if you prefer.

import logging
import sys

DEFAULT_LOGGER_NAME = 'default_logger'


def init_logging(logger_name=DEFAULT_LOGGER_NAME,
                 log_level=logging.DEBUG,
                 stream=None):
    # logging
    logger = logging.getLogger(logger_name)
    logger.setLevel(log_level)
    # create formatter
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    # create console handler and set level to debug
    if stream is None:
        stream = sys.stderr
    ch = logging.StreamHandler(stream=stream)
    ch.setLevel(log_level)
    # add formatter to ch
    ch.setFormatter(formatter)
    # add ch to logger
    logger.addHandler(ch)

    return logger

This function needs to be called at the beginning of your program (e.g. beginning of main()).

Then within the code, you just need to call the following:

logger = logging.getLogger(LOGGER_NAME)

edited Jul 31 '19 at 01:08

answered Jul 31 '19 at 00:57

T3am5hark

856
6
9

this has an unused import, and more importantly this `init_logging` function seems to be just be a weaker version of something that python already has built in: [logging.basicConfig](https://docs.python.org/3/library/logging.html#logging.basicConfig) – blues Jul 31 '19 at 01:01
Seems obvious that it would need to be called in __main__() somewhere at the start of the program... – T3am5hark Jul 31 '19 at 01:05
But you **are** returning the `logger` from the function. I was going for that you probably meant: *you just need to call the following:* `logger = init_logging() `... – Tomerikoo Jul 31 '19 at 01:11
Edited for clarity. You only need to call init_logging() once up front (yes it returns the logger, but that doesn't mean it should be called everywhere in the code). Everywhere else you should call logging.getLogger(LOGGER_NAME) since the logger has at this point already been configured. – T3am5hark Jul 31 '19 at 01:15

Making logging module write all logs to only stderr

2 Answers2