How to deal with 401 (unauthorised) in python requests

Question

What I want to do is GET from a site and if that request returns a 401, then redo my authentication wiggle (which may be out of date) and try again. But I don't want to try a third time, since that would be my authentication wiggle having the wrong credentials. Does anyone have a nice way of doing this that doesn't involve properly ugly code, ideally in python requests library, but I don't mind changing.

Answer 1

It doesn't get any less ugly than this, I think:

import requests
from requests.auth import HTTPBasicAuth

response = requests.get('http://your_url')

if response.status_code == 401:    
    response = requests.get('http://your_url', auth=HTTPBasicAuth('user', 'pass'))

if response.status_code != 200:
    # Definitely something's wrong

Answer 2

You could have wrapped this in a function and used a decorator to evaluate the response and retry the auth on 401. Then you only need to decorate any function that requires this re-auth logic....

Update: As requested, a code example. I'm afraid this one is an old piece of code, Python 2 based, but you'll get the idea. This one will retry an http call a number of times as defined in settings.NUM_PLATFORM_RETRIES and will call a refresh_token on auth failures. you can adjust the use case and result to whatever. You can then use this decorator around methods:

@retry_on_read_error
def some_func():
   do_something()



def retry_on_read_error(fn):
    """
    Retry Feed reads on failures
    If a token refresh is required it is performed before retry.
    This decorator relies on the model to have a refresh_token method defined, othewise it will fail
    """
    @wraps(fn)
    def _wrapper(self, *args, **kwargs):
        for i in range(settings.NUM_PLATFORM_RETRIES):
            try:
                res = fn(self, *args, **kwargs)

                try:
                    _res = json.loads(res)
                except ValueError:
                    # not a json response (could be local file read or non json data)
                    return res

                if 'error' in _res and _res['error']['status'] in (401, 400):
                    raise AccessRefusedException(_res['error']['message'])

                return res
            except (urllib2.URLError, IOError, AccessRefusedException) as e:
                if isinstance(e, AccessRefusedException):
                    self.refresh_token()
                continue
        raise ApiRequestFailed(
            "Api failing, after %s retries: %s" % (settings.NUM_PLATFORM_RETRIES, e), args, kwargs
        )

    return _wrapper

Answer 3

You can use something like this

# 401 retry strategy

import requests
from requests import Request, Session, RequestException


    class PreparedRequest:
    """
    Class to make Http request with 401 retry
    """
        failedRequests = []
        defaultBaseUrl = "https://jsonplaceholder.typicode.com"
        MAX_RETRY_COUNT = 0

        def __init__(self, method, endpoint,
             baseurl=defaultBaseUrl, headers=None, data=None, params=None):
        """
        Constructor for PreparedRequest class
        @param method: Http Request Method
        @param endpoint: endpoint of the request
        @param headers: headers of the request
        @param data: data of request
        @param params: params of the request
        """
        self.method = method
        self.url = baseurl + endpoint
        self.headers = headers
        self.data = data
        self.params = params
        self.response = None

    def send(self):
    """
    To send http request to the server
    @return: response of the request
    """
        req = Request(method=self.method, url=self.url, data=self.data, 
                headers=self.headers,params=self.params)
        session = Session()
        prepared = session.prepare_request(req)
        response = session.send(prepared)
        if response.status_code == 200:
            PreparedRequest.failedRequests.append(self)
            PreparedRequest.refresh_token()
        elif response.status_code == 502:
            raise Exception(response.raise_for_status())
        else:
            self.response = session.send(prepared)

    @staticmethod
    def refresh_token():
        if PreparedRequest.MAX_RETRY_COUNT > 3:
            return
        print("Refreshing the token")
        # Write your refresh token strategy here
        PreparedRequest.MAX_RETRY_COUNT += 1
        total_failed = len(PreparedRequest.failedRequests)
        for i in range(total_failed):
            item = PreparedRequest.failedRequests.pop()
            item.send()


r = PreparedRequest(method="GET", endpoint="/todos/")
r.send()
print(r.response.json())

Answer 4

You need to send in the header of the request the authentication param

import requests
from requests.auth import HTTPBasicAuth

auth = HTTPBasicAuth("username", "password")
response = requests.get("http://serverIpOrName/html", auth=auth)
if response.status_code == 401 : 
    print("Authentication required")
if response.status_code == 200:
    print(response.content)

Answer 5

For anyone who scrolled this far:

If you get 401 from github and you don't know why, maybe you have old credentials in your .netrc, which is respected by the requests Library.

For me it helped enabling detailed logging: How can I see the entire HTTP request that's being sent by my Python application?