how to implement this textfile read/write program using threads in java?

Question

i have a textfile that contains 1300000 lines.i have written the java code for importing it into a mysql database.In the java class i have a method called textloadutility() which is called from a jsp page.Can someone give the asyncronous thread implementation of this java program.

package Snomed;
import catalog.Root;
import java.io.*;
import java.sql.PreparedStatement;
import org.json.JSONObject;

public class Textfileimport {


    public String textloadutility() throws Exception {

        Root oRoot = null;

        PreparedStatement oPrStmt = null;

        FileReader in = null;

        BufferedReader br=null;

        final int batchSize = 1000;

        int count = 0;

        JSONObject oJson = null;



        String str=null;

        oJson = new JSONObject();

         oJson.put("status","failure");

         str=oJson.toString();


        try {


             oRoot = Root.createDbConnection(null);




             String sql = "INSERT INTO textfiledata (col1,col2,col3,col4,col5,col6,col7,col8,col9) VALUES( ?, ?, ?,?,?,?,?,?,?)";

             oPrStmt = oRoot.con.prepareStatement(sql);

            in = new FileReader("C:/Users/i2cdev001/Desktop/snomedinfo_data.txt");      

            br = new BufferedReader(in);

            String strLine;

            while ((strLine = br.readLine()) != null){

                String [] splitSt =strLine.split("\\t");
                String dat1="",dat2="",dat3="",dat4="",dat5="",dat6="",dat7="",dat8="",dat9="";
                dat1=splitSt[0];
                dat2=splitSt[1];
                dat3=splitSt[2];
                dat4=splitSt[3];
                dat5=splitSt[4];
                dat6=splitSt[5];
                dat7=splitSt[6];
                dat8=splitSt[7];
                dat9=splitSt[8];        




                oPrStmt.setString(1, dat1);
                oPrStmt.setString(2, dat2);
                oPrStmt.setString(3, dat3);
                oPrStmt.setString(4, dat4);  
                oPrStmt.setString(5, dat5);
                oPrStmt.setString(6, dat6);
                oPrStmt.setString(7, dat7);
                oPrStmt.setString(8, dat8);
                oPrStmt.setString(9, dat9);
                oPrStmt.addBatch();

                if (++count % batchSize == 0) {
                    oPrStmt.executeBatch();
                    oPrStmt.clearBatch(); 
                   }                
            }

            oPrStmt.executeBatch();

             oJson.put("status","sucess");

             str=oJson.toString(); 

             in.close();
                br.close();



            System.out.println("sucessfully imported");

        }
        catch (Exception e) {


            oJson.put("status","failure");

             str=oJson.toString(); 
            e.printStackTrace();
            System.err.println("Error: " + e.getMessage());

        } finally {
            oPrStmt = Root.EcwClosePreparedStatement(oPrStmt);

            oRoot = Root.closeDbConnection(null, oRoot);
        }
        return str;
    }

}

There are probably better ways to do this. Can you explain why `LOAD DATA` doesn't work for you? https://stackoverflow.com/questions/13579810/how-to-import-data-from-text-file-to-mysql-database?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa — markspace, Jun 11 '18 at 16:09

score 0 · Answer 1 · answered Jun 12 '18 at 10:54

Here is the solution for your problem,

File IO should not be async so first Thread-1 should read the file batch by batch and put that into some shared queue.
The another multi-threaded thread should read the contents of the queue and push it into db. You could implement this using ExecutorService class of java concurrent package. And co-ordinate all those threads using CountDown latch.
Once all the lines are read from the file by the single thread then it will return to the caller.
After all those queue entries are processed the db processing threads will be closed and respective countdown latch also will be decreased and finish once it move to 0.
You should use the future response to the actual caller so that after finishing of all those threads you will get the response.

This is high level view.

how to implement this textfile read/write program using threads in java?

1 Answers1