what is java.io.EOFException, Message: Can not read response from server. Expected to read 4 bytes, read 0 bytes

Question

This question has been asked a couple of times in SO and many times in other sites. But I didn't get any satisfiable answer.

My problem:
I have a java web application which uses simple JDBC to connect to mysql database through Glassfish application server.

I have used connection pooling in glassfish server with the following configurations:
Initial Pool Size: 25
Maximum Pool Size: 100
Pool Resize Quantity: 2
Idle Timeout: 300 seconds
Max Wait Time: 60,000 milliseconds

The application has been deployed for last 3 months and it was running flawlessly too.
But from last 2 days the following error is coming at the time of login.

Partial StackTrace

com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error:  

** BEGIN NESTED EXCEPTION **  

com.mysql.jdbc.CommunicationsException  
MESSAGE: Communications link failure due to underlying exception:  

** BEGIN NESTED EXCEPTION **  

java.io.EOFException  
MESSAGE: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.  

STACKTRACE:  

java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.  
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1997)  
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411)  
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)  
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)  
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)  
at com.mysql.jdbc.Connection.execSQL(Connection.java:3256)  
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1313)  
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1448)  
............  
............  
my application traces....

What caused this error suddenly ? I have lost a lot of time for this.

EDIT : The problem even persists after restarting the server. As per DBA two of the important mysql server configurations are:
wait_timeout : 1800 seconds
connect_timeout : 10 seconds
NOTE : Other applications deployed in the same server connecting to the same database and using different pools are running smoothly.

EDIT-2 : After reading a lot of things and expecting some positive outcome I made these changes to my connection pool.

Max Wait Time : 0 (previously it was 60 seconds)
Connection Validation : Required
Validation Method : table
Table Name : Demo
Validate Atmost Once : 40 seconds
Creation Retry Attempts : 1
Retry Intervals : 5 seconds
Max Connection Usage : 5

And this worked as the application is running for 3 days consistently. But I got a very strange and interesting result of out this. While monitoring the connection pool, I found these figures:

NumConnAcquired : 44919 Count
NumConnReleased : 44919 Count
NumConnCreated : 9748 Count
NumConnDestroyed : 9793 Count
NumConnFailedValidation : 70 Count
NumConnFree : 161 Count
NumConnUsed : -136 Count

How can the NumConnFree become 161 as I have Maximum Pool Size = 100 ?
How can the NumConnUsed become -136, a negative number ?
How can the NumConnDestroyed > NumConnCreated ?

Try using the same or smaller timeout values within your application and database. For instance if your applications idle timeout is higher than the one from the database your application will try to reuse a connection which was already closed by the database server. — Adrian, Dec 19 '12 at 14:30
Does it help increasing pool resize quantity? See the reason [here](http://docs.oracle.com/cd/E19316-01/820-4343/abehq/index.html) — perissf, Dec 27 '12 at 13:39

score 13 · Answer 1 · edited Feb 02 '23 at 15:47

13

The connection has failed, possibly due to a firewall idle-timeout, etc. If you don't have your JDBC driver configured to reconnect on failure, then this error will not go away unless you open a new connection.

If you are using a database connection pool (you are using one, right?), then you probably want to enable it's connection-checking features like issuing a query to check to see if the connection is working before handing it back to the application. In Apache commons-dbcp, this is called the validationQuery and is often set to something simple like SELECT 1.

Since you are using MySQL, you ought to use a Connector/J-specific "ping" query that is lighter-weight than actually issuing a true SQL query and set your validation query to /* ping */ SELECT 1 (the ping part needs to be exact).

edited Feb 02 '23 at 15:47

Community

1
1

answered Dec 28 '12 at 16:47

Christopher Schultz

20,221
9
60
77

1

The link "needs to be exact" is broken. any idea what the original page said? – Joeblade Sep 30 '15 at 13:22
1

@mrjink I fixed the link, again. For future reference (when Oracle moves it, again), the reference is to MySQL's Connector/J version 5.1.3 release notes, whose final item contains the syntax to use. – Christopher Schultz Jul 21 '18 at 21:40

Stephen C · Answer 2 · 2014-01-22T06:48:41.817

It most likely means that the database has restarted or the network connection to the database has been broken (e.g. a NAT connection has timed out) ... and your webapp is trying to use a stale database connection.

If the problem persists after restarting the web container, it could be something more serious.

You asked the following:

How can the NumConnFree become 161 as I have Maximum Pool Size = 100 ?
How can the NumConnUsed become -136, a negative number ?
How can the NumConnDestroyed > NumConnCreated ?

On the face of it, these don't make sense. However, they could simply be the result of some usage counters being updated in a non-thread-safe way. This is not necessarily related to your original problem.

See my Edit.I think some configuration problem may be there. — mukund, Dec 19 '12 at 12:31

score 9 · Answer 3 · answered Dec 28 '12 at 07:25

9

This is EndOfFileException in java, happens when your cursor start point is on the end point or when database connection closed because of some unexpected exception.

answered Dec 28 '12 at 07:25

Puneet

611
6
22

6

This is a generic explanation of the EOFException, but doesn't really help with the JDBC problem that mukund is asking about. – Idris Mokhtarzada Jul 14 '14 at 10:21

score 7 · Answer 4 · answered Jan 03 '13 at 05:42

While I don't have definitive solutions, it seems something is interfering communication between the app server and db. Following are few things you can try to isolate the problems:

Try to determine wether this is mysql problem or java code problem. Try connect to mysql using the command line tool from the same host as the app server and issue similar SQL to perform login. Test using a simple java code that does a select, deploy it to the same infrastructure, see what happen etc. Also check the mysql server log, see if you can find anything useful
There are two way an idle connection get closed: by the connection pool code that runs inside app server, or by mysql itself. Make sure you check the config on both sides
Check if any network infrastructure config has changed recently. Was there any new firewall rule in place interfering with app server <--> mysql connectivity? Was there any settings that prohibits open TCP connection idling longer than X?
Try a different connection pooling library just to eliminate the possibility it's the connection pooling

Good luck

in my case when dealing with this it was the timeout value being too small. — EdgeCaseBerg, Nov 07 '14 at 16:39

score 2 · Answer 5 · answered Jan 03 '13 at 10:11

2

It might be Firewall related problem.

answered Jan 03 '13 at 10:11

Mani

21
4

score 1 · Answer 6 · answered Nov 07 '16 at 23:39

I had this problem but I it wasn't possible for me to make changes in MySQL database configuration. Therefore I ensured that in my sql connector class the connection is always closed before it's initiated again. Something like:

public static Connection getConnection() {

    if (DatabaseConnnector.conn == null) {
        initConn();
    } else {
        try {
            DatabaseConnnector.conn.close();          
        } catch (SQLException e) {
            e.printStackTrace();
        }
          initConn();
    }
    return DatabaseConnnector.conn;
}

And this solved the problem.

score 1 · Answer 7 · answered Oct 29 '19 at 14:38

1

Got this error in PHPStorm while trying to connect and/or get some data from mysql server. Checkbox "single connection mode" fixed things for me. I'll just leave it here. Perhaps this will help some martyr in the future.

https://www.jetbrains.com/help/phpstorm/configuring-database-connections.html

answered Oct 29 '19 at 14:38

Thomas

61
1
4

I got this error in PHPStorm trying to connect over SSH to a Ubuntu 16.04 virtual machine. The problem in my case was I had put the VM's host name in the `Host` field on the General tab rather than `localhost` like it should have been. (In my case using `single connection mode` didn't seem to make a difference one way or the other.) – jbobbins Nov 26 '19 at 19:11

score 0 · Answer 8 · answered Apr 22 '20 at 01:30

0

i meet the same problem, it caused by lock for too much time.

we have a long transcation A, if execute it, the other trascation will be locked util transcation A finished, so the others always be killed by our mysql tool(pt-kill)

answered Apr 22 '20 at 01:30

zhuojl

11
1

score 0 · Answer 9 · edited Sep 24 '21 at 14:36

0

In my case, it was the java mysql driver/connector version. I.e. (Even if the server was version 8) somehow when I used mysql driver/connector version 5.1.44, then it started working.

edited Sep 24 '21 at 14:36

Dharman

30,962
25
85
135

answered Sep 24 '21 at 14:31

myuce

1,321
1
19
29

score 0 · Answer 10 · answered Dec 12 '22 at 02:45

0

it may be the server connection is full try to reduce the server connection or modify the max connection param

answered Dec 12 '22 at 02:45

Hero

1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 15 '22 at 07:55

score 0 · Answer 11 · answered Sep 01 '23 at 13:26

In our case, we got the error "Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedl lost" (in IntelliJ) because of a Docker error.

We fixed it with simply running:

docker-compose --profile microservices --compatibility -f docker-compose.services.yml up

what is java.io.EOFException, Message: Can not read response from server. Expected to read 4 bytes, read 0 bytes

11 Answers11

Linked

Related