Why is “while ( !feof (file) )” always wrong?

Question

I've seen people trying to read files like this in a lot of posts lately:

#include <stdio.h>
#include <stdlib.h>

int
main(int argc, char **argv)
{
    char *path = "stdin";
    FILE *fp = argc > 1 ? fopen(path=argv[1], "r") : stdin;

    if( fp == NULL ) {
        perror(path);
        return EXIT_FAILURE;
    }

    while( !feof(fp) ) {  /* THIS IS WRONG */
        /* Read and process data from file… */
    }
    if( fclose(fp) != 0 ) {
        perror(path);
        return EXIT_FAILURE;
    }
    return EXIT_SUCCESS;
}

What is wrong with this loop?

Answer 1

I'd like to provide an abstract, high-level perspective.

Concurrency and simultaneity

I/O operations interact with the environment. The environment is not part of your program, and not under your control. The environment truly exists "concurrently" with your program. As with all things concurrent, questions about the "current state" don't make sense: There is no concept of "simultaneity" across concurrent events. Many properties of state simply don't exist concurrently.

Let me make this more precise: Suppose you want to ask, "do you have more data". You could ask this of a concurrent container, or of your I/O system. But the answer is generally unactionable, and thus meaningless. So what if the container says "yes" – by the time you try reading, it may no longer have data. Similarly, if the answer is "no", by the time you try reading, data may have arrived. The conclusion is that there simply is no property like "I have data", since you cannot act meaningfully in response to any possible answer. (The situation is slightly better with buffered input, where you might conceivably get a "yes, I have data" that constitutes some kind of guarantee, but you would still have to be able to deal with the opposite case. And with output the situation is certainly just as bad as I described: you never know if that disk or that network buffer is full.)

So we conclude that it is impossible, and in fact unreasonable, to ask an I/O system whether it will be able to perform an I/O operation. The only possible way we can interact with it (just as with a concurrent container) is to attempt the operation and check whether it succeeded or failed. At that moment where you interact with the environment, then and only then can you know whether the interaction was actually possible, and at that point you must commit to performing the interaction. (This is a "synchronisation point", if you will.)

EOF

Now we get to EOF. EOF is the response you get from an attempted I/O operation. It means that you were trying to read or write something, but when doing so you failed to read or write any data, and instead the end of the input or output was encountered. This is true for essentially all the I/O APIs, whether it be the C standard library, C++ iostreams, or other libraries. As long as the I/O operations succeed, you simply cannot know whether further, future operations will succeed. You must always first try the operation and then respond to success or failure.

Examples

In each of the examples, note carefully that we first attempt the I/O operation and then consume the result if it is valid. Note further that we always must use the result of the I/O operation, though the result takes different shapes and forms in each example.

• C stdio, read from a file:

    for (;;) {
        size_t n = fread(buf, 1, bufsize, infile);
        consume(buf, n);
        if (n == 0) { break; }
    }
  

The result we must use is n, the number of elements that were read (which may be as little as zero).

• C stdio, scanf:

    for (int a, b, c; scanf("%d %d %d", &a, &b, &c) == 3; ) {
        consume(a, b, c);
    }
  

The result we must use is the return value of scanf, the number of elements converted.

• C++, iostreams formatted extraction:

    for (int n; std::cin >> n; ) {
        consume(n);
    }
  

The result we must use is std::cin itself, which can be evaluated in a boolean context and tells us whether the stream is still in the good() state.

• C++, iostreams getline:

    for (std::string line; std::getline(std::cin, line); ) {
        consume(line);
    }
  

The result we must use is again std::cin, just as before.

• POSIX, write(2) to flush a buffer:

    char const * p = buf;
    ssize_t n = bufsize;
    for (ssize_t k = bufsize; (k = write(fd, p, n)) > 0; p += k, n -= k) {}
    if (n != 0) { /* error, failed to write complete buffer */ }
  

The result we use here is k, the number of bytes written. The point here is that we can only know how many bytes were written after the write operation.

• POSIX getline()

    char *buffer = NULL;
    size_t bufsiz = 0;
    ssize_t nbytes;
    while ((nbytes = getline(&buffer, &bufsiz, fp)) != -1)
    {
        /* Use nbytes of data in buffer */
    }
    free(buffer);
  

  The result we must use is nbytes, the number of bytes up to and including the newline (or EOF if the file did not end with a newline).

  Note that the function explicitly returns -1 (and not EOF!) when an error occurs or it reaches EOF.

You may notice that we very rarely spell out the actual word "EOF". We usually detect the error condition in some other way that is more immediately interesting to us (e.g. failure to perform as much I/O as we had desired). In every example there is some API feature that could tell us explicitly that the EOF state has been encountered, but this is in fact not a terribly useful piece of information. It is much more of a detail than we often care about. What matters is whether the I/O succeeded, more-so than how it failed.

• A final example that actually queries the EOF state: Suppose you have a string and want to test that it represents an integer in its entirety, with no extra bits at the end except whitespace. Using C++ iostreams, it goes like this:

    std::string input = "   123   ";   // example
  
    std::istringstream iss(input);
    int value;
    if (iss >> value >> std::ws && iss.get() == EOF) {
        consume(value);
    } else {
        // error, "input" is not parsable as an integer
    }
  

We use two results here. The first is iss, the stream object itself, to check that the formatted extraction to value succeeded. But then, after also consuming whitespace, we perform another I/O/ operation, iss.get(), and expect it to fail as EOF, which is the case if the entire string has already been consumed by the formatted extraction.

In the C standard library you can achieve something similar with the strto*l functions by checking that the end pointer has reached the end of the input string.

The answer

while(!feof) is wrong because it tests for something that is irrelevant and fails to test for something that you need to know. The result is that you are erroneously executing code that assumes that it is accessing data that was read successfully, when in fact this never happened.