Writing to file using StreamWriter much slower than file copy over slow network

Question

I have a program that attempts to write quite a large amount of text to a file on a remote server overseas, which has a slow network connection.

Using the following code, where outputFileContent is a StringBuilder:

using (var outfile = new StreamWriter(myRemoteFilePath))
{
    outfile.Write(outputFileContent.ToString());
}

it is taking a seriously long time to run (several minutes), whereas if I first write to a local file and then copy it across to the remote location, it is much quicker (20-30 secs):

string tempFilePath = Path.GetTempFileName();
using (var outfile = new StreamWriter(tempFilePath))
{
    outfile.Write(outputFileContent.ToString());
}

System.IO.File.Copy(tempFilePath, myRemoteFilePath, true)

Any idea why this is happening? My only guess is that it is something to do with buffering across the network, or perhaps because the stream writer doesn't know how big it needs to be ahead of time.

Answer 1

If you create your StreamWriter using default buffer sizes then the underlying SMB protocol will be issuing write requests in chunks no larger than 4096 bytes, which means a large number of round-trips over the network. You could increase your StreamWriter's buffer size up to a maximum of 64k in order to reduce the number of round trips:

using (var outfile = new StreamWriter(myRemoteFilePath, false, Encoding.ASCII, 0x10000))

Increasing the buffer size beyond 64k will not help in any circumstance, as the underlying SMB protocol does not support buffer lengths beyond 64k. Note that a direct file copy still uses the SMB protocol, so from a network traffic perspective there's little difference between the operations except for the buffer sizes.