Python request.get fails on specific site

Question

I just finished automate the boring stuff with python, now i'm just practicing. I try running the code below but it fails . Other sites has no issue connecting .

Searched for other questions they say add user:agent, it works for amazon.com but not for this site . I tried allow_redirects = False and ,verify=False too , they are not working

I am using Windows 10 , Python 3.8.5 , requests 2.24.0

my code ;

import requests

headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}
print('1')
res = requests.get('https://freida.ama-assn.org/search/list?spec=42931',headers=headers)
print('2')
print(res.status_code)
res.raise_for_status()

This code does not prints '2' fails at requests.get

Error code as follows :

C:\Users\User198\PycharmProjects\pythonProject\venv\Scripts\python.exe C:/Users/User198/PycharmProjects/pythonProject/amaResidency.py
1
Traceback (most recent call last):
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connection.py", line 362, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\ssl_.py", line 384, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
OSError: [Errno 0] Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\retry.py", line 403, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connection.py", line 362, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\ssl_.py", line 384, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError(0, 'Error'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/User198/PycharmProjects/pythonProject/amaResidency.py", line 6, in <module>
    print(res.status_code)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(0, 'Error'))

may be something is wrong with the headers variable if you run it without header that will do work fine but the header varible is creating some issue — shyam_gupta, Aug 16 '20 at 13:37
Your code is fine. It is returning 200 and printing both 1 and 2. It could be some network issue. https://bugs.python.org/issue31122 This error is generally returned when the client tries to connect without SSL. Are you behind any proxy or some firewall? — mbtamuli, Aug 16 '20 at 13:43
As you can see in the last line of your error code, it states that you have an error called ```0```. This has a specific meaning. Check out [this](https://stackoverflow.com/questions/872206/what-does-it-mean-when-an-http-request-returns-status-code-0) question, since it really helps to explain why you get that error in the first place. — Yannik Suhre, Aug 16 '20 at 13:45
After you guys say that it connects, i tried with my phones internet connection, it *worked* . It seems like my wifi connection is the issue but i don't know what to do :S Like i said all other sites connects succesfully with requests.get — , Aug 16 '20 at 14:26

score 0 · Answer 1 · answered Aug 16 '20 at 14:52

So, seems like i fixed the issue . In my case code was working via my cellphone internet but Wi-Fi connection was failing for this particular site . I unchecked "Microsoft LLDP Protocol Driver" from Control panel => network and internet => network connections => LeftClick your connection and properties => Networking tab => Uncheck "Microsoft LLDP Protocol Driver"

Everything was checked except "microsoft network adapter multiplexor protocol" Now "Microsoft LLDP Protocol Driver" is not checked too.

This solution worked for me

update on the situation, my 'trick' was temporary, i think something wrong with the website, sometimes my code works, sometimes i get the same error — , Aug 16 '20 at 18:10

score 0 · Answer 2 · answered Jun 03 '21 at 08:58

You may try ignoring SSL certificate verification using verify = False. It drastically reduced my count of similar ConnectionErrors.

res = requests.get('https://freida.ama-assn.org/search/list?spec=42931', headers=headers, verify=False)

Note that when verify is set to False, requests will accept any TLS certificate presented by the server, and will ignore hostname mismatches and/or expired certificates, which will make your application vulnerable to man-in-the-middle (MitM) attacks.

Python request.get fails on specific site

2 Answers2