ConnectionError while running a script for a long time

Question

I have written a small python scrip.Here is my python Script.

#! /usr/bin/python3

import notify2
import requests,bs4,pprint
import time


while True:

        respond=requests.get('https://cricbuzz.com')
        soup=bs4.BeautifulSoup(respond.text,'html.parser')
        element = soup.find_all('div', {'class': 'cb-col cb-col-25 cb-mtch-blk'})


        for match in element:
            if 'IND' in match.getText():
                pprint.pprint(match.getText())
                break


        notify2.init('app name')
        n = notify2.Notification("Match Score",
                             match.getText(),
                             "notification-message-im"
                            )
        n.show()
        time.sleep(int(60*2))

which take requests and parse the Html using the Beautiful Soup module in which it runs for an interval of two minutes and continues .It was working fine but below Error has occured all of a sudden Don't know why? My laptop had good Internet Connection.

Error:

Traceback (most recent call last):
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connection.py", line 356, in connect
    ssl_context=context)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 359, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/connection.py", line 356, in connect
    ssl_context=context)
  File "/home/chaitu/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 359, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./score.py", line 10, in <module>
    respond=requests.get('https://cricbuzz.com')
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/home/chaitu/.local/lib/python3.6/site-packages/requests/adapters.py", line 495, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

The exception mentions `Connection reset by peer`, which means that the other computer has closed the connection. It looks more like a network issue, so I'd advise you to contact your network administrator. — Dominique, Sep 25 '18 at 14:25
https://stackoverflow.com/questions/1434451/what-does-connection-reset-by-peer-mean — oshaiken, Sep 25 '18 at 14:46
Possible duplicate of [What does "connection reset by peer" mean?](https://stackoverflow.com/questions/1434451/what-does-connection-reset-by-peer-mean) — oshaiken, Sep 25 '18 at 14:46

score 0 · Answer 1 · edited Mar 16 '21 at 16:45

I use this function to scrape Youtube Like and Dislikes. It provides access to 241 links and they are go again and scrape more from where I am left off. Put a try/except and a high sleeping time and you'll be able to scrape more than you do now. Also if you want to get around this you can use certain proxies or VPNs to constantly change your IP so the request is not made from the same IP address.

def scrape_info(url):

    result = {}

    session = HTMLSession()

    try:
        response = session.get(url)
        # execute Java-script
        response.html.render(sleep=40)
        # create bs object to parse HTML
        soup = bs(response.html.html, "html.parser")


        # number of likes
        x = soup.find_all("yt-formatted-string", {"id": "text", "class": "ytd-toggle-button-renderer"})
        if x is not None and len(x) > 0:
            result["likes"] = x[0].text
            result["dislikes"] = x[1].text
        else:
            result["likes"] = None
            result["dislikes"] = None
    except requests.exceptions.ConnectionError:
        print("Connection refused")
        result["likes"] = None
        result["dislikes"] = None
        time.sleep(30)


    return result

ConnectionError while running a script for a long time

1 Answers1