2

When I try to make a get or post call using aiohttp in a Quart endpoint it throws 'An existing connection was forcibly closed by the remote host', even when I follow the official tutorial here

Full exception stack:

Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None) created at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2800.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py:109>
source_traceback: Object created at (most recent call last):
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\client.py", line 560, in _request
    await resp.start(conn)
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\client_reqrep.py", line 917, in start
    payload.on_eof(self._response_eof)
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\streams.py", line 167, in on_eof
    callback()
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\client_reqrep.py", line 952, in _response_eof
    self._connection.release()
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\connector.py", line 178, in release
    self._connector._release(
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\connector.py", line 663, in _release
    protocol.close()
  File "C:\Users\devuser\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\aiohttp\client_proto.py", line 63, in close
    transport.close()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2800.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 109, in close
    self._loop.call_soon(self._call_connection_lost, None)
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2800.0_x64__qbz5n2kfra8p0\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2800.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost
    self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

I have noticed that I am not the only person who has reported this issue, I assume it's some interaction between aiohhtp and Quart that is breaking.

Asynchronous GET method:

async def _call_api_get(self, url : str, json_data : dict = None) -> ApiResponse:
    try:
        async with aiohttp.ClientSession() as session:
            async with session.get(url, json=json_data) as resp:
                body = await resp.json() \
                    if resp.content_type == 'application/json' \
                    else await resp.text()
                api_return = ApiResponse(
                    status_code = resp.status,
                    body = body,
                    content_type = resp.content_type)

    except ConnectionResetError as ex:
        print("ConnectionResetError:", ex)

    except Exception as ex:
        api_return = ApiResponse(exception_msg = ex)

    return api_return

The API endpoint method:

async def my_endpoint(self) -> Response:

    auth_url = 'http://www.google.co.uk'

    api_return_body : dict = {"key": "value"}

    api_response = await self._call_api_get(auth_url,
                                            json_data=auth_request)

Other related questions I looked at:

Swatcat
  • 73
  • 6
  • 21
  • 57
  • Is it possible the request is taking a while to complete and either the aiohttp call or the request to Quart are timing out? – pgjones Mar 28 '23 at 19:53
  • @pgjones Ithe original code using the requests library worked fine, but, I am getting a really slow response because of the sub-calls to the internal api are not async – Swatcat Mar 28 '23 at 21:04
  • 1
    @Swatcat There can be a lot of places where this could crop up. It could be an issue with the timeout on your client, on the remote endpoint, or anywhere in between. I recommend testing the code out with other async clients (ie. `httpx`) and testing from a different IP. I also recommend grabbing the actual error and adding it to your question as currently it doesn't have enough details to answer properly. – plunker Mar 30 '23 at 00:59
  • @plunker Thank you for the feedback, I had incorrectly assumed I had given enough details. The question has been updated with a full exception stack now. – Swatcat Mar 30 '23 at 09:06
  • Can you prepare a minimal reproduce of the issue? If your API is calling another remote API, then try to isolate that. Have you seen this thread also? https://stackoverflow.com/questions/2582036/an-existing-connection-was-forcibly-closed-by-the-remote-host – Jeppe Apr 03 '23 at 19:59
  • @Swatcat Any luck on resolving this issue? I'm encountering a similar issue... – Chuck Conway Apr 28 '23 at 15:45
  • @ChuckConway I haven't resolved it, I am positive it's a conflict between quart and aiohttp, both I believe use asyncio. – Swatcat Apr 29 '23 at 20:03
  • @ChuckConway Out of interest, are you seeing the issue on Windows, Linux or Mac? I am not seeing the issue on Linux. – Swatcat Apr 30 '23 at 18:53
  • 1
    @Swatcat I'm on Windows 11. When I make sequential requests after about the 5th or 6th, I experience the "closed by remote host" error for the next requests. I was able to work around the issue by implementing retry logic. When I retry the request after a failure, the request is successful, but the next new request fails... It feels like the HTTP connections aren't disposed and the connection queue is filling up, but I tried all the close methods on the different classes, and nothing worked. I switched to Url3Lib, and everything worked... but we don't get the benefit of async... – Chuck Conway Apr 30 '23 at 21:08

0 Answers0