Very slow to generate MD5 for large file using Java

lucky_start_izumi picture lucky_start_izumi · Feb 17, 2012 · Viewed 18.1k times · Source

I am using Java to generate the MD5 hash for some files. I need to generate one MD5 for several files with a total size of about 1 gigabyte. Here's my code:

private String generateMD5(SequenceInputStream inputStream){
    if(inputStream==null){
        return null;
    }
    MessageDigest md;
    try {
        int read =0;
        byte[] buf = new byte[2048];
        md = MessageDigest.getInstance("MD5");
        while((read = inputStream.read(buf))>0){
            md.update(buf,0,read);
        }
        byte[] hashValue = md.digest();
        return new String(hashValue);
    } catch (NoSuchAlgorithmException e) {
        return null;
    } catch (IOException e) {
        return null;
    }finally{
        try {
            if(inputStream!=null)inputStream.close();
        } catch (IOException e) {
            // ...
        }
    } 

}

This seems to run forever. How can I make it more efficient?

Answer

Sticky picture Sticky · Feb 17, 2012

You may want to use the Fast MD5 library. It's much faster than Java's built-in MD5 provider and getting a hash is as simple as:

String hash = MD5.asHex(MD5.getHash(new File(filename)));

Be aware that the slow speed may also be due to slow File I/O.