Python Logging : how to add a custom field to LogRecord, and register a global callback to sets it's value

Question

In a Flask application, I want to add a user_id field added to every error log record that gets produced, whenever a user_id exists in the flask.session.

I came up with the solution below but it's hackish, since it doesn't allow for the format string to drive the formatting of the user_id, and since the logging API seems to provides ways to customize logging (LoggerAdapter, logging.makeRecord, etc...) I figure there has to be a cleaner way.

What would be the "python way" to do it ?

class CustomFormatter(Formatter):
    def format(self, record):
        from myapp.core import authenticationManager
        user_id = authenticationManager.current_user_id_if_authenticated()
        user_id = "unknown" if user_id is None else str(user_id)
        return super(F,self).format(record) + ", user_id" + user_id

unutbu · Accepted Answer · 2015-06-12T18:14:06.287

You could define a custom filter which injects a value for record.user_id. Done this way, you can define a format which includes %(user_id)s just like other (standard) record attributes:

format='%(asctime)-15s %(name)-5s %(levelname)-8s user_id %(user_id)-15s: %(message)s')

and then all logging calls will automatically add the user_id value.

import logging

class UserIDFilter(logging.Filter):
    """
    This is a filter which injects contextual information into the log.
    """
    from myapp.core import authenticationManager

    def filter(self, record):
        user_id = authenticationManager.current_user_id_if_authenticated()
        user_id = "unknown" if user_id is None else str(user_id)
        record.user_id = user_id
        return True


logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)-15s %(name)-5s %(levelname)-8s user_id %(user_id)-15s: %(message)s')

logger = logging.getLogger(__name__)
filt = UserIDFilter()
logger.addFilter(filt)

Thanks, it works great, but should 1 be returned instead of True, as the docs prescribe : https://docs.python.org/2/library/logging.html#filter-objects ? — Max L., Jun 12 '15 at 19:34
Any object which evaluates to True in a boolean context will do. [The docs state](https://docs.python.org/2/library/logging.html#logging.Logger.filter), "Applies this logger’s filters to the record and returns a true value if the record is to be processed". I prefer `True` over `1` here because it emphasizes that we're using the "boolean-ness" of the object. — unutbu, Jun 12 '15 at 19:58

score 3 · Answer 2 · edited Oct 05 '18 at 15:06

3

The logaugment (Github page) package does this:

To use:

import logaugment

logaugment.set(logger, user_id='custom_value')

All logging calls with logger will now use the value 'custom_value' for %(user_id)s in the logging message.

You can repeatedly call logaugment.set without problems to change the value. Alternatively, you can specify a callable that will be called every time when a logging call is made. This would allow dynamically computing / retrieving the user ID (see example in package README).

To install:

pip install logaugment

(disclaimer: I developed this package)

edited Oct 05 '18 at 15:06

davidism

121,510
29
395
339

answered Jun 12 '15 at 18:17

Simeon Visser

118,920
18
185
180

Great. Would it be possible for all the child loggers of the logger in your example, to use `custom_value` in their logging calls? – Ganesh Tata Jul 03 '19 at 11:16

Python Logging : how to add a custom field to LogRecord, and register a global callback to sets it's value

2 Answers2

Linked