Golang unexpected EOF

Dylan picture Dylan · Jun 29, 2018 · Viewed 17.9k times · Source

Here's my code, I'm new to Go. I tried googling the issue, but I can't quite put my finger on it. I think it has something to do with the Read() method.

package main

import (
    ...
)

type compressor struct {
    content []byte
}

func (r *compressor) compress() []byte {
    ...
}

func (r *compressor) decompress() []byte {
    var buffer bytes.Buffer
    dc := flate.NewReader(&buffer)
    _, err := dc.Read(r.content)
    if err != nil {
        if err != io.EOF {
            log.Fatal(err)
        }
    }

    return buffer.Bytes()
}

func main() {
    fileName := os.Args[1]
    fmt.Println(os.Args)
    contents, err := ioutil.ReadFile(fileName)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Print("Uncompressed data: ")
    fmt.Println(len(contents))

    comp := compressor{contents}
    buffer := comp.decompress()
    fmt.Print("Uncompressed data: ")
    fmt.Println(len(comp.decompress()))

    err = ioutil.WriteFile(fileName+".decjc", buffer, 0644)
    if err != nil {
        log.Fatal(err)
    }
}

Here's the output

dylan@skynet:~/Documents/EXP/jc$ ./jc data.txt.jc 
[./jc data.txt.jc]
Uncompressed data: 2364480
2018/06/29 21:41:35 unexpected EOF

Answer

chewpoclypse picture chewpoclypse · Aug 27, 2019

After doing a trace on the particular code in question I have come to the following answer.

/src/bytes/reader.go 70

func (r *Reader) ReadByte() (byte, error) {
    ...

    if r.i >= int64(len(r.s)) {
        return 0, io.EOF
    }

    ....
}

There are four functions in bytes/reader that can return io.EOF, and zero functions that can return io.ErrUnexpectedEOF. The four functions that can return io.EOF are:

Read(b []byte)
ReadAt(b []byte, off int64)
ReadByte()
ReadRune()

/src/compress/flate/inflate.go 698

func (f *decompressor) moreBits() error {
    c, err := f.r.ReadByte()
    if err != nil {
        return noEOF(err)
    }

    ...
}

Of the four functions that can return io.EOF, only one function in flate/inflate.go calls any of them: moreBits() calls ReadByte()

/src/compress/flate/inflate.go 690

func noEOF(e error) error {
    if e == io.EOF {
        return io.ErrUnexpectedEOF
    }

    ...
}

When moreBits() receives an error it calls noEOF(), which checks if it had received an io.EOF. If this was the case then io.ErrUnexpectedEOF is returned backed. Everything seems to be working as intended, and it appears that it is the user's responsibility to be on the look out for this particular case. A suggested edit to the code above to handle what appears to be defined behavior is:

func (r *compressor) decompress() []byte {
    dc := flate.NewReader(bytes.NewReader(r.content))
    defer dc.Close()
    rb, err := ioutil.ReadAll(dc)
    if err != nil {
        if err != io.EOF && err != io.ErrUnexpectedEOF {
            log.Fatalf("Err %v\n read %v", err, rb)
        }
    }
    return rb
}

This was checked under go1.12.9