Persistent HttpURLConnection in Java

Question

I am trying to write a java program that will automatically download and name some of my favorite web comics. Since I will be requesting multiple objects from the same domain, I wanted to have a persistent http connection that I could keep open until all the comics have been downloaded. Below is my work-in-progress. How do I make another request from the same domain but different path without opening a new http connection?

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class ComicDownloader
{
    public static void main(String[] args)
    {
        URL url = null;
        HttpURLConnection httpc = null;
        BufferedReader input = null;

        try
        {
            url = new URL("http://www.cad-comic.com/cad/archive/2002");
            httpc = (HttpURLConnection) url.openConnection();
            input = new BufferedReader(new InputStreamReader(httpc.getInputStream()));
            String inputLine;

            while ((inputLine = input.readLine()) != null)
            {
                System.out.println(inputLine);
            }

            input.close();
            httpc.disconnect();
        }
        catch (IOException ex)
        {
            System.out.println(ex);
        }
    }
}

yclian · Accepted Answer · 2014-01-11T11:06:58.473

20

According to the documentation here, HTTP persistence is being handled transparently in Java, although it gives you the options to control it too via http.keepAlive and http.maxConnections system properties.

However,

The current implementation doesn't buffer the response body. Which means that the application has to finish reading the response body or call close() to abandon the rest of the response body, in order for that connection to be reused. Furthermore, current implementation will not try block-reading when cleaning up the connection, meaning if the whole response body is not available, the connection will not be reused.

Take a look at the link and see if it really helps you.

edited Jan 11 '14 at 11:06

answered Jul 21 '10 at 21:40

yclian

1,490
14
22

link is dead? also does it still apply to newer versions of java? – rogerdpack Oct 09 '13 at 16:41
2

@royerdpack Of course it does. You can't seriously believe they would take a valuable feature like this out. – user207421 Jan 10 '14 at 18:11
@rogerdpack Use a Google search like this next time: https://www.google.com/search?q=java+api+doc+net%2Fhttp-keepalive.html Meanwhile am fixing the link. – yclian Jan 11 '14 at 11:06
2

Finish reading *and* do not call close() on the input stream or output stream. Calling close() caused my client code to re-open the HTTPS connection. – Vladimir Dyuzhev May 23 '16 at 16:43
2

FWIW I think I read once https://stackoverflow.com/a/4767595/32453 that calling disconnect (like the OP does) also causes the Socket to not be re-used/cached... – rogerdpack Dec 29 '17 at 23:31

score 5 · Answer 2 · answered Feb 16 '17 at 03:28

According to this link http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html, HTTP connection reuse is enabled by default, you can use Wireshark to check the interactions between your client and server. The first request contains TCP and SSL handshakes(if your request is https), the subsequent requests fired in the keep-alive time, contains no TCP and SSL handshakes, just application data transfers.

score 0 · Answer 3 · answered Mar 12 '21 at 12:32

Even though HttpURLConnection enable keep-alive by default, it is not guaranteed that HttpURLConnection uses same TCP connection for multiple HTTP requests. I faced same kind of issue when writing HTTPS client application. Solved this issue by using single instance of SSLContext, SSLSocketFactory and HttpsURLConnection.

public class MyHTTPClient {
    private SSLContext mSSLContext = null;
    private SSLSocketFactory mSSLSocketFactory = null;
    private HttpsURLConnection mConnection = null;

    public void init() {
        //Setup SSL context and Socket factory here
    }

    pubblic void sendRequest() {
        URL url = new URL("https://example.com/request_receiver");
        mConnection = (HttpsURLConnection) url.openConnection();
        mConnection.setSSLSocketFactory(mSSLSocketFactory);

        // Setup request property and send request
        // Open input stream to read response
        // Close output, input streams
        
        mConnection.disconnect();
    }
}

Persistent HttpURLConnection in Java

3 Answers3

Linked

Related