Does fread fail for large files?

c large-files fread
bcubed picture bcubed · Sep 29, 2010 · Viewed 8.1k times · Source

I have to analyze a 16 GB file. I am reading through the file sequentially using fread() and fseek(). Is it feasible? Will fread() work for such a large file?

Answer

David Thornley picture David Thornley · Sep 29, 2010

You don't mention a language, so I'm going to assume C.

I don't see any problems with fread, but fseek and ftell may have issues.

Those functions use long int as the data type to hold the file position, rather than something intelligent like fpos_t or even size_t. This means that they can fail to work on a file over 2 GB, and can certainly fail on a 16 GB file.

You need to see how big long int is on your platform. If it's 64 bits, you're fine. If it's 32, you are likely to have problems when using ftell to measure distance from the start of the file.

Consider using fgetpos and fsetpos instead.

Related questions

Seeking and reading large files in a Linux C++ application

I am running into integer overflow using the standard ftell and fseek options inside of G++, but I guess I was mistaken because it seems that ftell64 and fseek64 are not available. I have been searching and many websites seem …

Read More
Is O_LARGEFILE needed just to write a large file?

Is the O_LARGEFILE flag needed if all that I want to do is write a large file (O_WRONLY) or append to a large file (O_APPEND | O_WRONLY)? From a thread that I read titled "Cannot write >2…

Read More
Get Large File Size in C

Before anyone complains of "duplicate", I've been checking SO quite thoroughly, but there seem to be no clean answer yet, although the question looks quite simple. I'm looking for a portable C code, which is able to provide the size …

Read More