TcpClient.GetStream().DataAvailable returns false, but stream has more data

Question

So, it would seem that a blocking Read() can return before it is done receiving all of the data being sent to it. In turn we wrap the Read() with a loop that is controlled by the DataAvailable value from the stream in question. The problem is that you can receive more data while in this while loop, but there is no behind the scenes processing going on to let the system know this. Most of the solutions I have found to this on the net have not been applicable in one way or another to me.

What I have ended up doing is as the last step in my loop, I do a simple Thread.Sleep(1) after reading each block from the stream. This appears to give the system time to update and I am not getting accurate results but this seems a bit hacky and quite a bit 'circumstantial' for a solution.

Here is a list of the circumstances I am dealing with: Single TCP Connection between an IIS Application and a standalone application, both written in C# for send/receive communication. It sends a request and then waits for a response. This request is initiated by an HTTP request, but I am not having this issue reading data from the HTTP Request, it is after the fact.

Here is the basic code for handling an incoming connection

protected void OnClientCommunication(TcpClient oClient)
{
    NetworkStream stream = oClient.GetStream();
    MemoryStream msIn = new MemoryStream();

    byte[] aMessage = new byte[4096];
    int iBytesRead = 0;

    while ( stream.DataAvailable )
    {
        int iRead = stream.Read(aMessage, 0, aMessage.Length);
        iBytesRead += iRead;
        msIn.Write(aMessage, 0, iRead);
        Thread.Sleep(1);
    }
    MemoryStream msOut = new MemoryStream();

    // .. Do some processing adding data to the msOut stream

    msOut.WriteTo(stream);
    stream.Flush();

    oClient.Close();
}

All feedback welcome for a better solution or just a thumbs up on needing to give that Sleep(1) a go to allow things to update properly before we check the DataAvailable value.

Guess I am hoping after 2 years that the answer to this question isn't how things still are :)

Answer 1

You have to know how much data you need to read; you cannot simply loop reading data until there is no more data, because you can never be sure that no more is going to come.

This is why HTTP GET results have a byte count in the HTTP headers: so the client side will know when it has received all the data.

Here are two solutions for you depending on whether you have control over what the other side is sending:

1. Use "framing" characters: (SB)data(EB), where SB and EB are start-block and end-block characters (of your choosing) but which CANNOT occur inside the data. When you "see" EB, you know you are done.

2. Implement a length field in front of each message to indicate how much data follows: (len)data. Read (len), then read (len) bytes; repeat as necessary.

This isn't like reading from a file where a zero-length read means end-of-data (that DOES mean the other side has disconnected, but that's another story).

A third (not recommended) solution is that you can implement a timer. Once you start getting data, set the timer. If the receive loop is idle for some period of time (say a few seconds, if data doesn't come often), you can probably assume no more data is coming. This last method is a last resort... it's not very reliable, hard to tune, and it's fragile.