Unable to disable logging from python's request library

Question

I'm trying to remove logging from the requests library and I've already read and tried the other answers on stack overflow, but they don't seem to work.

I'm parsing a webpage for urls in the html code and then downloading zip files from each of the urls. After downloading all the zip files, I get a message from the requests libraries:

Downloading data from:

http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm

Here's my code. I'm importing the class and running the download_data method from another file.

import os
import argparse
import requests
import re
import logging
from tqdm import tqdm

"""
Downloads all zip files from the viewfinder panoramas elevation dataset
"""

class ElevationScraper:
    def __init__(self):
        self.data_url = "http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm"
        self.base_dir = "./data/elevation/"
        self.elevation_dir = os.path.join(self.base_dir, "elevation")
        logging.getLogger("requests").setLevel(logging.WARNING)
        logging.getLogger("requests").propagate = True

        
    def download_data(self):
        # get html from self.data_url and get all urls in the html code
        req = requests.get(self.data_url)
        urls = re.findall(r"\"(https?://\S+)\"", str(req.text))

        # first four are irrelevant links so remove those
        urls = urls[4:]
        with open(os.path.join(self.base_dir, "urls.txt"), "w") as url_file:
            for url in urls:
                url_file.write(url + "\n")
        
        # downloading all zip files
        if not os.path.exists(self.elevation_dir):
            os.mkdir(self.elevation_dir)

        print("Downloading " + str(len(urls)) + " data files to " + self.elevation_dir + " from " + self.data_url)
        
        for url in tqdm(urls[:3]):
            filename = url.split("/")[-1]

            with open(os.path.join(self.base_dir, "elevation", filename), "wb") as data_file:
                req = requests.get(url)
                data_file.write(req.content)

Output:

shrey@Shrey-TPL480 MINGW64 ~/dev/Landslide-Prediction (main)
$ python download_elevation.py
Downloading 1140 data files to ./data/elevation/elevation from http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:08<00:00,  2.75s/it] 
Downloading data from http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm

score 0 · Accepted Answer · answered Nov 24 '20 at 16:51

You can change setLevel to critical like this logging.getLogger("requests").setLevel(logging.CRITICAL) or you can use requests.packages.urllib3.disable_warnings(). See this: How do I disable log messages from the Requests library?. A question like this is alreasy asked.

Unable to disable logging from python's request library

1 Answers1