HTTPConnection.request not respecting timeout?

Question

I'm trying to use HTTPConnection (2.7.8) to make a request and I've set the timeout to 10 with HTTPConnection(host, timeout=10). However, HTTPConnection.request() doesn't seem to timeout after 10 seconds. In fact, HTTPConnection.timeout doesn't even seem to be read by HTTPConnection.request() (it's only read by HTTPConnection.connect(). Is my understanding correct? Is timeout only applicable to connect() and not request()? Is there a way to timeout request()?

Update:

I think I've narrowed the issue down further: if I don't provide the scheme, it won't respect the socket timeout. If the scheme was provided, i.e. the full URL being http://google.com:22222, then it'd time out accordingly. I wonder why the presence of the scheme should make a difference. That is, the following doesn't respect the timeout

    socket.setdefaulttimeout(3)
    conn = HTTPConnection('google.com:22222')
    conn.timeout = 3
    conn.request('GET', '')

whereas, this does:

    socket.setdefaulttimeout(3)
    conn = HTTPConnection('http://google.com:22222')
    conn.timeout = 3
    conn.request('GET', '')

However, it doesn't happen to all domains.

Thanks

Please check http://stackoverflow.com/questions/265720/http-request-timeout. Hope this helps :) — itzMEonTV, Feb 23 '15 at 08:50
No, that answer doesn't work for me. It still stuck at `request()`. An example url is google.com:2222 — Kar, Feb 23 '15 at 10:41
have you tried to to provide ip address directly (instead of the hostname)? Call `socket.getaddrinfo()`, to get ip address. It seems the same issue as [ftplib.FTP timeout has inconsistent behaviour](http://stackoverflow.com/q/28429501/4279): `request()` does several socket operations. Each operation is limited by the timeout. But several operations combined may take longer. — jfs, Feb 23 '15 at 10:52
I think I've narrowed the issue down further: if I don't provide the scheme, it won't respect the socket timeout. If the scheme was provided, i.e. the full URL being `http://google.com:22222`, then it'd time out. I wonder why the presence of the scheme should make a difference. — Kar, Feb 23 '15 at 11:05
It is not clear how you use HTTPConnection. [Create the shortest program that demonstrates the difference](http://stackoverflow.com/help/mcve) (set `HTTPConnection.debuglevel=1`) — jfs, Feb 23 '15 at 11:07
@J.F.Sebastian I think the suggestion in your first comment explains my problem. — Kar, Feb 23 '15 at 12:08

score 6 · Accepted Answer · edited May 23 '17 at 12:26

It takes around ~30 seconds for the following code to fail:

#!/usr/bin/env python2
from httplib import HTTPConnection

conn = HTTPConnection('google.com', 22222, timeout=2)
conn.request('GET', '')

If ip is passed to HTTPConnection instead of the hostname then the timeout error is raised in 2 seconds as expected:

#!/usr/bin/env python2
import socket
from httplib import HTTPConnection

host, port = 'google.com', 22222
ip, port = socket.getaddrinfo(host, port)[0][-1]
conn = HTTPConnection(ip, port, timeout=2)
conn.request('GET', '')

The explanation is the same as in ftplib.FTP timeout has inconsistent behaviour: the timeout may limit individual socket operations but it says nothing about the duration of the HTTPConnection() call itself that may try several ip addresses returned by getaddrinfo() and the timeout limits only the individual socket operations. Several operations combined may take longer.

Your HTTPConnection('http://google.com:22222') fails sooner because the url is an incorrect argument: it should be either host or host:port. The absolute url is accepted by request() method -- though even there it has special meaning -- typically, you just provide the path along such as '/'.

HTTPConnection.request not respecting timeout?

1 Answers1

Linked