Unable to fix java.lang.NullPointerException while collecting data from a table

Question

I am trying to retrieve data from two databases simultaneously for data comparison. To get the tablenames that are present in the two databases, I have to read a third database where all the tablenames are stored in a table. Below is the method to get the table names:

public List<String> getChunks() {
    List<String> chunkNamesList  = new ArrayList<String>();
    Connection chunkTableCon     = DbManager.getBdMergeConnection();
    String[] chunkArray;
    try {
        PreparedStatement cStmnt = chunkTableCon.prepareStatement(chunkNames);
        ResultSet crs            = cStmnt.executeQuery();
        while(crs.next()) {
                chunkArray = crs.getString(1).split("\\.");
                chunkNamesList.add("analytics." + chunkArray[1] + "," + crs.getString(1) + ":" + crs.getString(2));
        }
        chunkTableCon.close();
        return chunkNamesList;
    } catch(SQLException e) {
        e.printStackTrace();
        return null;
    } catch(Exception e) {
        e.printStackTrace();
        return null;
    }
}

I have created two threads which will read the data from the collection: chunkNamesList for their analysis. Below is how I created two threads:

new Thread(new Runnable() {
    @Override
    public void run() {
        try {
            gpTableCount   = getGpTableCount();
        } catch (SQLException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}).start();
new Thread(new Runnable() {
    @Override
    public void run() {
        try {
            hiveTableCount = getHiveTableCount();
        } catch (SQLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}).start();

When I submit the jar, the methods: getGpTableCount & getHiveTableCount first gets the table details from the method: getTables() like below:

    public Map<String,String> getGpTableCount() throws SQLException {
        gpDataMap   = new HashMap<String, String>();
        gpTableErrs = new HashMap<String, String>();
        gpTableList     = getChunks();
        Iterator<String> keySetIterator_gpTableList = gpTableList.iterator();
        Connection gpAnalyticsCon       = (Connection) DbManager.getGpConnection();
        PreparedStatement gp_pstmnt     = null;
    ..........................
    ..........................
}
        public Map<String, String> getHiveTableCount() throws IOException, SQLException {
            hiveDataMap     = new HashMap<String, String>();
            hiveTableErrs   = new HashMap<String, String>();
            List<String> hiveTableList      = getChunks();
            Iterator<String> hiveIterator   = hiveTableList.iterator();
            Connection hiveConnection       = DbManager.getHiveConnection();
            PreparedStatement hive_pstmnt   = null;
        ..........................
        ..........................
    }

Once I submit the jar, most of the times I am getting java.lang.NullPointerException in the method: getGpTableCount() at the line: Iterator<String> keySetIterator_gpTableList = gpTableList.iterator(); with the exception:

org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:315)
        at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356)
        at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168)
        at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:116)
        at com.recordcount.dao.ChunkRecon.getChunks(ChunkRecon.java:82)
        at com.recordcount.dao.ChunkRecon.getGpTableCount(ChunkRecon.java:105)
        at com.recordcount.dao.ChunkRecon$1.run(ChunkRecon.java:224)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Socket closed
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
        at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
        at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
        at org.postgresql.core.PGStream.receiveChar(PGStream.java:280)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1916)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288)
        ... 8 more
java.lang.NullPointerException
        at com.recordcount.dao.ChunkRecon.getGpTableCount(ChunkRecon.java:106)
        at com.recordcount.dao.ChunkRecon$1.run(ChunkRecon.java:224)
        at java.lang.Thread.run(Thread.java:745)

The same exception randomly occurs at either of the lines:

Iterator<String> keySetIterator_gpTableList = gpTableList.iterator();
Iterator<String> hiveIterator   = hiveTableList.iterator(); in the method: `getHiveTableCount()`

I declared both hiveTableList and gpTableList globally as

List<String> hiveTableList = new ArrayList<String>();
List<String> gpTableList = new ArrayList<String>();

The exception occurs on these two lines in their respective methods:

        Iterator<String> hiveIterator   = hiveTableList.iterator();
Iterator<String> keySetIterator_gpTableList = gpTableList.iterator();

Could anyone let me know how can I fix this problem ?

Your exception handling returns `null` from `getChunks()`, this subsequently causes the NPE. Fix your exception handling to not return null, but either (re)throw the exception (or wrap it in your own), or do alternative handling, or fix the original cause of the exceptions thrown. — Mark Rotteveel, Jun 27 '18 at 09:54
@MarkRotteveel, Instead of returning a collection from getChunks(), I have initialized an ArrayList (a new one) and then used the data present in it for the other two methods to proceed further. It is working now. — Metadata, Jun 27 '18 at 10:13
To be honest, that sounds like a very brittle solution, as you're now essentially ignoring the fact that an exception occurred. — Mark Rotteveel, Jun 27 '18 at 10:15
I get what you are saying. But In the code above, I am calling the method getChunks() twice, once in getGpCount & getHiveCount methods. I changed the design a little bit. I am calling the getChunks() method separately to initialize a global ArrayList which I can use it down the program any time without calling getChunks() again and again. Hence I can remove the return statements and instead just initialize a global ArrayList to use it anywhere. I hope you got an idea of what I am saying. — Metadata, Jun 27 '18 at 10:19

Unable to fix java.lang.NullPointerException while collecting data from a table

0 Answers0