Read in big files and create a checksum in java

Question

I want to create hash checksums of big files. Unfortunately Netbeans shows an error: "Exception in thread "Thread-3" java.lang.OutOfMemoryError: Required array size too large"

So I think, "byte[] byteStream = Files.readAllBytes( path );" is not working for this.

Can you tell me what is the best way to reach my goal?

My code so far:

        for (Path path : ArrayOfFilePathes) {

            byte[] byteStream = Files.readAllBytes( path );

            if ( GUI.bMd5Selected == true ) {
                checksume = "MD5";
                string = DigestUtils.md5Hex( byteStream );
            }else{
                checksume = "Sha256";
                string = DigestUtils.sha256Hex(byteStream);
            }

Thank you!

possible duplicate of [Getting a File's MD5 Checksum in Java](http://stackoverflow.com/questions/304268/getting-a-files-md5-checksum-in-java) — MrTux, Sep 06 '14 at 12:29
You are correct the issue is Files.readAllBytes. You should read files using an InputStream and compute the digest on the fly. See the top answer in the link MrTux posted. — trooper, Sep 07 '14 at 04:39

score 0 · Answer 1 · answered Sep 06 '14 at 12:29

You want to use a MappedByteBuffer. This will allow you to treat the whole file as if it were in memory, but leave the underlying class to sort out when to load bits and when to get rid of them. It's very efficient in terms of both CPU time and memory overhead.

Read in big files and create a checksum in java

1 Answers1