The MD5 from a local file and the MD5 (eTag) from S3 is not the same

okysabeni picture okysabeni · Jun 6, 2011 · Viewed 10.8k times · Source

I get the MD5 of a local file but it is different than the MD5 (eTag) of the "same" file in Amazon S3. What I would like to achieve is figure out if the lastest files I have in S3 is the same one that I have locally. If I cannot compare MD5, then how should I do it?

Generating MD5 from the local file (truncated code):

MessageDigest md = MessageDigest.getInstance("MD5");
byte[] md5 = Files.getDigest(localFile, md);
String hashtext = DigestUtils.md5Hex(md5);

Retrieving MD5 (eTag) from S3 (truncated code):

ObjectListing objectListing = s3.listObjects(new ListObjectsRequest().withBucketName(bucketName));
List<S3ObjectSummary> objectSummaries = objectListing.getObjectSummaries();
for(S3ObjectSummary objectSummary : objectSummaries) {
    String MD5 = objectSummary.getETag();
}

PS: I use org.apache.commons.codec.digest.DigestUtils and com.google.common.io.Files libraries.

Answer

Bruno Roh&#233;e picture Bruno Rohée · Jun 6, 2011
String hashtext = DigestUtils.md5Hex(md5);

Does calculate the MD5 of the MD5 you just calculated. See DigestUtils.md5Hex documentation.

hashtext is in fact MD5(MD5(file)) and not MD5(file).