FileOutputStream.close is really slow when writing large file

Question

I have a method which receives a file over a TCP socket using this code:

FileOutputStream fileStream = new FileOutputStream(filename.getName());
while (totalRead < size) {
    if (size - totalRead > CHUNKSIZE) {
        read = getInputStream().read(buffer, 0, CHUNKSIZE);
    } else {
        read = getInputStream().read(buffer, 0, size - totalRead);
    }
    totalRead += read;
    fileStream.write(buffer, 0, read);
    fileStream.flush();

    if (System.currentTimeMillis() > nextPrint) {
        nextPrint += 1000;
        int speed = (int) (totalRead / (System.currentTimeMillis() - startTime));
        double procent = ((double)totalRead / size) * 100;
        gui.setStatus("Reciving: " + filename + " at " + speed + " kb/s, " + procent + "% complete");
    }
}
gui.setStatus("Reciving: " + filename + " complete.");
fileStream.close();

FileOutputStream.close is taking really long time when receiving large files, why is that? As you see I'm flushing the stream at every received chunk..

Answer 1

Depending on the OS, flush() does nothing more thaqn force the data to be written to the OS. In the case of FileOutputStream, write() passes all data to the OS, so flush() does nothing. Where as close() can ensure the file is actually written to disk (or not depending on the OS). You can look at whether the disk is still busy when writing the data.

A 500 MB files taking 30 seconds means you are writing 17 MB/s. This sounds like a very slow disk or that the file in on network share/drive.

---

You can try this

File file = File.createTempFile("deleteme", "dat"); // put your file here.
FileOutputStream fos = new FileOutputStream(file);
long start = System.nanoTime();
byte[] bytes = new byte[32 * 1024];
for (long l = 0; l < 500 * 1000 * 1000; l += bytes.length)
    fos.write(bytes);
long mid = System.nanoTime();
System.out.printf("Took %.3f seconds to write %,d bytes%n", (mid - start) / 1e9, file.length());
fos.close();
long end = System.nanoTime();
System.out.printf("Took %.3f seconds to close%n", (end - mid) / 1e9);

prints

Took 0.116 seconds to write 500,006,912 bytes
Took 0.002 seconds to close

You can see from the speed that on this system its not writing the data even on a close. i.e. the drive is not that fast.