Computing MD5SUM of large files in C#

c# md5 hash large-files
spkhaira picture spkhaira · Apr 30, 2009 · Viewed 8.7k times · Source

I am using following code to compute MD5SUM of a file - 

byte[] b = System.IO.File.ReadAllBytes(file);
string sum = BitConverter.ToString(new MD5CryptoServiceProvider().ComputeHash(b));

This works fine normally, but if I encounter a large file (~1GB) - e.g. an iso image or a DVD VOB file - I get an Out of Memory exception.

Though, I am able to compute the MD5SUM in cygwin for the same file in about 10secs.

Please suggest how can I get this to work for big files in my program.

Thanks

Answer

Alnitak picture Alnitak · Apr 30, 2009

I suggest using the alternate method:

MD5CryptoServiceProvider.ComputeHash(Stream)

and just pass in an input stream opened on your file. This method will almost certainly not read in the whole file in memory in one go.

I would also note that in most implementations of MD5 it's possible to add byte[] data into the digest function a chunk at a time, and then ask for the hash at the end.

Related questions

Calculate MD5 checksum for a file

I'm using iTextSharp to read the text from a PDF file. However, there are times I cannot extract text, because the PDF file is only containing images. I download the same PDF files everyday, and I want to see if …

Read More
Converting a md5 hash byte array to a string

How can I convert the hashed result, which is a byte array, to a string? byte[] bytePassword = Encoding.UTF8.GetBytes(password); using (MD5 md5 = MD5.Create()) { byte[] byteHashedPassword = md5.ComputeHash(bytePassword); } I need to convert byteHashedPassword to a string.

Read More
How can I read/stream a file without loading the entire file into memory?

How can I read an arbitrary file and process it "piece by piece" (meaning byte by byte or some other chunk size that would give the best read performance) without loading the entire file into memory? An example of processing …

Read More