3

Sometimes I receive the 

java.net.SocketException: Too many open files 
java.net.Socket.createImpl(Socket.java:397) 
java.net.Socket.connect(Socket.java:527) 
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123) 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:123) 
org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:147) 
org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:108) 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415) 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641) 
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:597)

I saw the similar question java.net.SocketException: Too many open files, and I use apache's httpclient too, but it seems answers was not quite helpful for me...

There was two answers:

1) doing httpClient.getConnectionManager().shutdown();

2) call entity.getContent().close() instead of entity.consumeContent()

But seems none of them is suitable...

1) The problem is that I'm using ThreadSafeConnectionManager. It is created once (on the application startup). and therefore we do not making shutdown (so that connections are reusable). If I do shutdown on this manager - all the connection will be closed. It would be appropriate if I use SingleClientConnectionManager, but I don't. Am I right?

2) I found out that I don't close the stream too. But when I started to debug - it appears that this getContent() stream is already closed, even before calling the consumeContent(). Although (even after consumeContent) the socketInputStream inside those getContent() stream was not closed, as well as socket. Is it bad? Can it be the cause of the problem? And I didn't find the way how to close this socket! It's located very inside the outer input stream, so I can't get it. But I see in the debug mode, that this socket is not closed, as well as SocketInputStream.

How normally we should work with a ThreadSafeConnectionManager - should it be created only once? and if it is so, how to properly close those sockets if any?

Community
  • 1
  • 1
javagirl
  • 1,635
  • 6
  • 27
  • 43

1 Answers1

8

If I do shutdown on this manager - all the connection will be closed. It would be appropriate if I use SingleClientConnectionManager, but I don't. Am I right?

You should be using just one connection manager instance per distinct HTTP service. You do not really need to shut it down, but you might want to evict connection from the pool that have been idle longer than a given period of time as described here:

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d4e659

Although (even after consumeContent) the socketInputStream inside those getContent() stream was not closed, as well as socket. Is it bad?

No, it is not. The underlying connection may still be kept alive if it is in a consistent state. However one must close the content stream in order to ensure release of that connection back to the connection manager.

How normally we should work with a ThreadSafeConnectionManager - should it be created only once? and if it is so, how to properly close those sockets if any?

Yes, it should be created only once. However, you must make sure that connections get properly released back to the manager by closing out response content streams. You may also want to want to pro-actively evict connections from the pool after a certain period of inactivity.

And lastly, please note that other components of your application can be leaking file descriptors too. The culprit may not necessarily be HttpClient's connection manager.

ok2c
  • 26,450
  • 5
  • 63
  • 71