I tried unxutils' wc -l
but it crashed for 1GB files. I tried this C# code
long count = 0;
using (StreamReader r = new StreamReader(f))
{
string line;
while ((line = r.ReadLine()) != null)
{
count++;
}
}
return count;
It reads a 500MB file in 4 seconds
var size = 256;
var bytes = new byte[size];
var count = 0;
byte query = Convert.ToByte('\n');
using (var stream = File.OpenRead(file))
{
int many;
do
{
many = stream.Read(bytes, 0, size);
count += bytes.Where(a => a == query).Count();
} while (many == size);
}
Reads in 10 seconds
var count = 0;
int query = (int)Convert.ToByte('\n');
using (var stream = File.OpenRead(file))
{
int current;
do
{
current = stream.ReadByte();
if (current == query)
{
count++;
continue;
}
} while (current!= -1);
}
Takes 7 seconds
Is anything faster I haven't tried yet?
File.ReadLines
was introduced in .NET 4.0
var count = File.ReadLines(file).Count();
works in 4 seconds, the same time as the first code snippet