Python- How to flush the log? (django)

Question

I'm working with Django-nonrel on Google App Engine, which forces me to use logging.debug() instead of print().

The "logging" module is provided by Django, but I'm having a rough time using it instead of print().

For example, if I need to verify the content held in the variable x, I will put
logging.debug('x is: %s' % x). But if the program crashes soon after (without flushing the stream), then it never gets printed.

So for debugging, I need debug() to be flushed before the program exits on error, and this is not happening.

Answer 1

I think this may work for you, assuming you're only using one(or default) handler:

>>> import logging
>>> logger = logging.getLogger()
>>> logging.debug('wat wat')
>>> logger.handlers[0].flush()

It's kind of frowned upon in the documentation, though.

Application code should not directly instantiate and use instances of Handler. Instead, the Handler class is a base class that defines the interface that all handlers should have and establishes some default behavior that child classes can use (or override). http://docs.python.org/2/howto/logging.html#handler-basic

And it could be a performance drain, but if you're really stuck, this may help with your debugging.

Answer 2

If the use case is that you have a python program that should flush its logs when exiting, use logging.shutdown().

From the python documentation:

logging.shutdown()

Informs the logging system to perform an orderly shutdown by flushing and closing all handlers. This should be called at application exit and no further use of the logging system should be made after this call. [...]

Answer 3

Django logging relies on the standard python logging module.

This module has a module-level method: logging.shutdown() which flushes all of the handlers and shuts down the logging system (i.e. logging can not longer be used after it is called)

Inspecting the code of this function shows that currently (python 2.7) the logging module holds a list of weak references to all handlers in a module-level variable called _handlerList so all of the handlers can be flushed by doing something like

[h_weak_ref().flush() for h_weak_ref in logging._handlerList]

because this solution uses the internals of the module @Mikes solution above is better, but it relies on having access to a logger, it can be generalized as follows:

 [h.flush() for h in my_logger.handlerList]

Answer 4

In Python3.8:

logging.shutdown( )

Check: https://docs.python.org/3.8/library/logging.html#logging.shutdown

Answer 5

Flush stdout/stderr instead:

sys.stdout.flush()
sys.stderr.flush()

Answer 6

It could be that the logger is not logging (seemingly not flushing) because an exception is being raised in a thread. The thread will need to catch its own exceptions and log them itself.

You cannot rely on a main thread to catch and log thread exceptions.

For example, a timer that runs a function periodically will have this behavior and that function will need to catch and log its exceptions.

Answer 7

a simple function that always working for register you debug messsages while programming. dont use it for production, since it will not rotate:

def make_log(message):
    import datetime
    with open('mylogfile.log','a') as f:
        f.write(f"{datetime.datetime.now()} {message}\n")

then use as

make_log('my message to register')

when to put on production, just comment the last 2 lines

def make_log(message):
    import datetime
    #with open('mylogfile.log','a') as f:
    #    f.write(f"{datetime.datetime.now()} {message}\n")