Python logging module prints unnecessary messages

Question

Please consider this dummy code.

$ cat dummy.py 
import logging
import time

from boto3.session import Session

# Logging Configuration
fmt = '%(asctime)s [%(levelname)s] [%(module)s] - %(message)s'
logging.basicConfig(level='INFO', format=fmt, datefmt='%m/%d/%Y %I:%M:%S')
logger = logging.getLogger()

def main():
    session = Session(region_name='us-west-2')
    client = session.client('ec2')
    response = client.describe_instances(InstanceIds=['i-11111111111111111'])

    logger.info('The instnace size is: %s', response[
                'Reservations'][0]['Instances'][0]['InstanceType'])

if __name__ == '__main__':
    main()

Output:

$ python3 dummy.py 
03/03/2017 08:47:00 [INFO] [credentials] - Found credentials in shared credentials file: ~/.aws/credentials
03/03/2017 08:47:01 [INFO] [connectionpool] - Starting new HTTPS connection (1): ec2.us-west-2.amazonaws.com
03/03/2017 08:47:02 [INFO] [dummy] - The instnace size is: t2.micro

Question: How to avoid below lines from being printed?

03/03/2017 08:47:00 [INFO] [credentials] - Found credentials in shared credentials file: ~/.aws/credentials
03/03/2017 08:47:01 [INFO] [connectionpool] - Starting new HTTPS connection (1): ec2.us-west-2.amazonaws.com

If I change logging.basicConfig(level='INFO',... to logging.basicConfig(level='WARNING',... then Those messages are not printed, but then all my messages get logged with WARNING severity.

I just want the logging module to print the messages that I explicitly write using logger.info .... and nothing else. Hence I need any pointers on how to avoid the unnecessary messages from being printed.

While maybe not an exact duplicate, you should be able to find what you're looking for in [this question](http://stackoverflow.com/questions/11029717/how-do-i-disable-log-messages-from-the-requests-library). — glibdud, Mar 03 '17 at 15:43
@glibdud, the question you referred is not a solution/workaround for me and I have already mentioned about it in my post. Thanks for your Time. — slayedbylucifer, Mar 03 '17 at 16:51
The linked question shows you how to access other modules' loggers (e.g. `credentials`, `connectionpool`) and set them to a different log level. I don't see anywhere in your question where you address that. — glibdud, Mar 03 '17 at 16:56
@glibdud, I have resolved. this. I will post the answer soon. — slayedbylucifer, Mar 03 '17 at 17:01

Aubrey Lavigne · Answer 1 · 2022-02-05T00:30:48.830

boto3 defines a function set_stream_logger that overrides the logging level for a specific boto3 namespace.

Prototype:

boto3.set_stream_logger(name='boto3', level=logging.DEBUG, format_string=None)

Example:

import boto3
boto3.set_stream_logger(name='botocore.credentials', level=logging.WARNING)
boto3.set_stream_logger(name='urllib3.connectionpool', level=logging.WARNING)

botocore/credentials.py loads the logger with this line:

logger = logging.getLogger(__name__)

In this file __name__ resolves to botocore.credentials, so that will be the logger name when calling set_stream_logger.

Note that the Starting new HTTPS connection message comes from urllib3, which isn't part of boto3. The boto3.set_stream_logger function still works because it acts on the logging module.

score 1 · Accepted Answer · answered Mar 03 '17 at 17:05

Solution:

import logging
import time

from boto3.session import Session

# Logging Configuration
fmt = '%(asctime)s [%(levelname)s] [%(module)s] - %(message)s'
logging.basicConfig(format=fmt, datefmt='%m/%d/%Y %I:%M:%S')
logger = logging.getLogger('LUCIFER')
logger.setLevel(logging.INFO)


def main():
    COUNTER = 3
    session = Session(region_name='us-west-2')
    client = session.client('ec2')
    response = client.describe_instances(InstanceIds=['i-0a912622af142b510'])

    logger.info('The instnace size is: %s', response[
                'Reservations'][0]['Instances'][0]['InstanceType'])

if __name__ == '__main__':
    main()

Output:

$ python3 dummy.py 
03/03/2017 10:30:15 [INFO] [dummy] - The instnace size is: t2.micro

Explanation: Earlier, I set the INFO level on root logger. Hence, all other loggers who do not have a level set, get this level propagated and start logging. In the solution, I am specifically enabling this level on a logger LUCIFER.

Reference: from: https://docs.python.org/3/howto/logging.html

Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to False.)

AND

In addition to any handlers directly associated with a logger, all handlers associated with all ancestors of the logger are called to dispatch the message (unless the propagate flag for a logger is set to a false value, at which point the passing to ancestor handlers stops).

Python logging module prints unnecessary messages

2 Answers2