Faster way to communicate using TcpClient?

Question

I'm writing a client/server application in C#, and it's going great. For now, everything works and it's all pretty robust. My problem is that I run into some delays when sending packets across the connection.

On the client side I'm doing this:

NetworkStream ns = tcpClient.GetStream();

// Send packet

byte[] sizePacket = BitConverter.GetBytes(request.Length);
byte[] requestWithHeader = new byte[sizePacket.Length + request.Length];
sizePacket.CopyTo(requestWithHeader, 0);
request.CopyTo(requestWithHeader, sizePacket.Length);

ns.Write(requestWithHeader, 0, requestWithHeader.Length);

// Receive response

ns.Read(sizePacket, 0, sizePacket.Length);
int responseLength = BitConverter.ToInt32(sizePacket, 0);
byte[] response = new byte[responseLength];

int bytesReceived = 0;
while (bytesReceived < responseLength)
{
  int bytesRead = ns.Read(response, bytesReceived, responseLength - bytesReceived);
  bytesReceived += bytesRead;
}

(Left out some exception catching etc.) The server does the opposite, i.e. it blocks on NetworkStream.Read() until it has a whole request, then processes it and sends a response using Write().

The raw speed of Write()/Read() isn't a problem (i.e. sending large packets is fast), but sending several small packets, one after another, without closing the connection, can be terribly slow (delays of 50-100 ms). What's strange is that these delays show up on LAN connections with typical ping times <1 ms, but they do not occur if the server is running on localhost, even though the ping time would effectively be the same (at least the difference should not be on the order of 100 ms). That would make sense to me if I were reopening the connection on every packet, causing lots of handshaking, but I'm not. It's just as if the server going into a wait state throws it out of sync with the client, and then it stumbles a bit as it reestablishes what is essentially a lost connection.

So, am I doing it wrong? Is there a way to keep the connection between TcpServer and TcpClient synchronised so the server is always ready to receive data? (And vice versa: sometimes processing the request from the client takes a few ms, and then the client doesn't seem to be ready to receive the response from the server until it's had a few moments to wake up after blocking on Read().)

Answer 1

It turns out my server and client were not completely symmetrical after all. I had noticed, but I didn't think it mattered at all. Apparently it's a huge deal. Specifically, the server did this:

ns.Write(sizePacket, 0, sizePacket.Length);
ns.Write(response, 0, response.Length);

Which I changed into this:

// ... concatenate sizePacket and response into one array, same as client code above
ns.Write(responseWithHeader, 0, responseWithHeader.Length);

And now the delay is completely gone, or at least it's no longer measurable in milliseconds. So that's something like a 100x speedup right there. \o/

It's still odd because it's writing exactly the same data to the socket as before, so I guess the socket receives some secret metadata during the write operation, which is then somehow communicated to the remote socket, which may interpret it as an opportunity to take a nap. Either that, or the first write puts the socket into receive mode, causing it to trip up when it's then asked to send again before it has received anything.

I suppose the implication would be that all of this example code you find lying around, which shows how to write to and read from sockets in fixed-size chunks (often preceded by a single int describing the size of the packet to follow, same as my first version), is failing to mention that there's a very heavy performance penalty for doing so.