Optimal buffer size for response stream of HttpWebResponse
What's the optimal buffer size to use with a stream from HttpWebResponse.GetResponseStream()?
Online examples vary from 256b to as much as 5Kb. What gives? I guess buffer sizes might be situational. If so what are the situations to use what type of buffer size?
Thanks.
Answer
Really, it doesn't matter very much.
Sure, if you use really small buffers, you may have to make a few extra calls down through the layers to get the bytes (though the stream is likely doing at least some buffering -- I don't know what it's defaults are). And sure, if you use really big buffers, you'll waste some memory and introduce some fragmentation. Since you're obviously doing IO here, any time you gain by tweaking the buffers is going to be dominated by the IO time.
As a general rule, I go with a power of two between 2048 (2k) and 8192 (8k). Just make sure you know what you're doing if you go with a buffer equal to or larger than 85,000 bytes (it's then a "large object" and subject to different GC rules).
In fact, more important than the buffer size is how long you hold it. For objects outside of the large object heap, the GC is very good at dealing with very short-lived objects (Gen 0 collections are fast), or very long-lived objects (Gen 2). Objects that live long enough to get to Gen 1 or 2 before being freed are comparatively more costly, and usually much more worth your time worrying about than how big the buffer is.
One final note: if you think you have a performance issue because of the size of buffers you are using, test it. It's unlikely, but who knows, maybe you have an odd confluence of OS version, network hardware, and driver release that has some odd issue with certain-sized buffers.