Python: Using requests and a loop with exception error to check if url exists

Question

just wondered if somebody could help me tweak this loop:

I am trying to establish if url's exist but the loop is getting stuck when a url doesn't exist (due to the error of trying to find an non-existent url).

I have added in an exception clause but it is throwing an error :(

Any help is much appreciated!

Does_Exist = []
for i in range (0, len(df),1):
   X=[]
   url = df.iloc[i][0] + df.iloc[i][1]
   ret = requests.head(url)  
   except requests.exceptions.ConnectionError 
   if ret.status_code == 200: 
       X = 'Yes' 
       break
   else: 
       X = 'No'
   Does_Exist.append(X)

This is the error message:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.bspretail.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x00000222ABC46470>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

What is the line `except requests.exceptions.ConnectionError` supposed to be doing? — khelwood, Aug 04 '17 at 14:44
V new to python, what is a try block? I will do some research and add. Thanks! — ewldh20, Aug 04 '17 at 14:44
I thought it would except the error that I am having i.e. when that error is recorded rather than break my loop it would skip it — ewldh20, Aug 04 '17 at 14:45
without try how will it go to except block. Just google try catch block in python and try that — Arpit Solanki, Aug 04 '17 at 14:45
Please check out: https://stackoverflow.com/questions/16511337/correct-way-to-try-except-using-python-requests-module — AChampion, Aug 04 '17 at 14:48

score 1 · Accepted Answer · answered Aug 04 '17 at 14:50

Try it with a Try/Except structure:

    Does_Exist = []
    for i in range (0, len(df),1):
       X=[]
       try:
          url = df.iloc[i][0] + df.iloc[i][1]
          ret = requests.head(url)  
          X = 'Yes' 
       except:
         X = 'No'
       Does_Exist.append(X)

Python: Using requests and a loop with exception error to check if url exists

1 Answers1