Redirect Python 'print' output to Logger

Question

I have a Python script that makes use of 'Print' for printing to stdout. I've recently added logging via Python Logger and would like to make it so these print statements go to logger if logging is enabled. I do not want to modify or remove these print statements.

I can log by doing 'log.info("some info msg")'. I want to be able to do something like this:

if logging_enabled:
  sys.stdout=log.info
print("test")

If logging is enabled, "test" should be logged as if I did log.info("test"). If logging isn't enabled, "test" should just be printed to the screen.

Is this possible? I know I can direct stdout to a file in a similar manner (see: redirect prints to log file)

Answer 1

You have two options:

1. Open a logfile and replace sys.stdout with it, not a function:

   log = open("myprog.log", "a")
   sys.stdout = log
   
   >>> print("Hello")
   >>> # nothing is printed because it goes to the log file instead.
   

2. Replace print with your log function:

   # If you're using python 2.x, uncomment the next line
   #from __future__ import print_function
   print = log.info
   
   >>> print("Hello!")
   >>> # nothing is printed because log.info is called instead of print
   

Answer 2

Of course, you can both print to the standard output and append to a log file, like this:

# Uncomment the line below for python 2.x
#from __future__ import print_function

import logging

logging.basicConfig(level=logging.INFO, format='%(message)s')
logger = logging.getLogger()
logger.addHandler(logging.FileHandler('test.log', 'a'))
print = logger.info

print('yo!')

Answer 3

One more method is to wrap the logger in an object that translates calls to write to the logger's log method.

Ferry Boender does just this, provided under the GPL license in a post on his website. The code below is based on this but solves two issues with the original:

1. The class doesn't implement the flush method which is called when the program exits.
2. The class doesn't buffer the writes on newline as io.TextIOWrapper objects are supposed to which results in newlines at odd points.

import logging
import sys


class StreamToLogger(object):
    """
    Fake file-like stream object that redirects writes to a logger instance.
    """
    def __init__(self, logger, log_level=logging.INFO):
        self.logger = logger
        self.log_level = log_level
        self.linebuf = ''

    def write(self, buf):
        temp_linebuf = self.linebuf + buf
        self.linebuf = ''
        for line in temp_linebuf.splitlines(True):
            # From the io.TextIOWrapper docs:
            #   On output, if newline is None, any '\n' characters written
            #   are translated to the system default line separator.
            # By default sys.stdout.write() expects '\n' newlines and then
            # translates them so this is still cross platform.
            if line[-1] == '\n':
                self.logger.log(self.log_level, line.rstrip())
            else:
                self.linebuf += line

    def flush(self):
        if self.linebuf != '':
            self.logger.log(self.log_level, self.linebuf.rstrip())
        self.linebuf = ''


logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s:%(levelname)s:%(name)s:%(message)s',
    filename="out.log",
    filemode='a'
)

stdout_logger = logging.getLogger('STDOUT')
sl = StreamToLogger(stdout_logger, logging.INFO)
sys.stdout = sl

stderr_logger = logging.getLogger('STDERR')
sl = StreamToLogger(stderr_logger, logging.ERROR)
sys.stderr = sl

This allows you to easily route all output to a logger of your choice. If needed, you can save sys.stdout and/or sys.stderr as mentioned by others in this thread before replacing it if you need to restore it later.

Answer 4

A much simpler option,

import logging, sys
logging.basicConfig(filename='path/to/logfile', level=logging.DEBUG)
logger = logging.getLogger()
sys.stderr.write = logger.error
sys.stdout.write = logger.info

Answer 5

Once your defined your logger, use this to make print redirect to logger even with mutiple parameters of print.

print = lambda *tup : logger.info(str(" ".join([str(x) for x in tup]))) 

Answer 6

You really should do that the other way: by adjusting your logging configuration to use print statements or something else, depending on the settings. Do not overwrite print behaviour, as some of the settings that may be introduced in the future (eg. by you or by someone else using your module) may actually output it to the stdout and you will have problems.

There is a handler that is supposed to redirect your log messages to proper stream (file, stdout or anything else file-like). It is called StreamHandler and it is bundled with logging module.

So basically in my opinion you should do, what you stated you don't want to do: replace print statements with actual logging.

Answer 7

Below snipped works perfectly inside my PySpark code. If someone need in case for understanding -->

import os
import sys
import logging
import logging.handlers

log = logging.getLogger(__name_)

handler = logging.FileHandler("spam.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
log.addHandler(handler)
sys.stderr.write = log.error 
sys.stdout.write = log.info 

(will log every error in "spam.log" in the same directory, nothing will be on console/stdout)

(will log every info in "spam.log" in the same directory,nothing will be on console/stdout)

to print output error/info in both file as well as in console remove above two line.

Happy Coding Cheers!!!