Handling multiple recv() calls and all possible scenarios

Question

I am fairly new to C and writing a TCP server, and was wondering how to handle recv()s from a client who will send commands that the server will respond to. For the sake of this question, let's just say header is 1st byte, command identifier is 2nd byte, and payload length is 3rd byte, followed by the payload (if any).

What is the best way to recv() this data? I was thinking to call recv() to read in the first 3 bytes into the buffer, check to make sure header and command identifiers are valid, then check payload length and call recv() again with payload length as length and add this to the back of the aforementioned buffer. Reading Beej's networking article (particularly the section Son of Data Encapsulation)), however, he advises to use "an array big enough for two [max length] packets" to handle situations such as getting some of the next packet.

What is the best way to handle these types of recv()s? Basic question, but I would like to implement it efficiently, handling all cases that can arise. Thanks in advance.

Answer 1

The method that Beej is alluding to, and AlastairG mentions, works something like this:

For each concurrent connection, you maintain a buffer of read-but-not-yet-processed data. (This is the buffer that Beej suggests sizing to twice the maximum packet length). Obviously, the buffer starts off empty:

unsigned char recv_buffer[BUF_SIZE];
size_t recv_len = 0;

Whenever your socket is readable, read into the remaining space in the buffer, then immediately try and process what you have:

result = recv(sock, recv_buffer + recv_len, BUF_SIZE - recv_len, 0);

if (result > 0) {
    recv_len += result;
    process_buffer(recv_buffer, &recv_len);
}

The process_buffer() will try and process the data in the buffer as a packet. If the buffer doesn't contain a full packet yet, it just returns - otherwise, it processes the data and removes it from the buffer. So for you example protocol, it would look something like:

void process_buffer(unsigned char *buffer, size_t *len)
{
    while (*len >= 3) {
        /* We have at least 3 bytes, so we have the payload length */

        unsigned payload_len = buffer[2];

        if (*len < 3 + payload_len) {
            /* Too short - haven't recieved whole payload yet */
            break;
        }

        /* OK - execute command */
        do_command(buffer[0], buffer[1], payload_len, &buffer[3]);

        /* Now shuffle the remaining data in the buffer back to the start */
        *len -= 3 + payload_len;
        if (*len > 0)
            memmove(buffer, buffer + 3 + payload_len, *len);
    }
}

(The do_command() function would check for a valid header and command byte).

This kind of technique ends up being necessary, because any recv() can return a short length - with your proposed method, what happens if your payload length is 500, but the next recv() only returns you 400 bytes? You'll have to save those 400 bytes until the next time the socket becomes readable anyway.

When you handle multiple concurrent clients, you simply have one recv_buffer and recv_len per client, and stuff them into a per-client struct (which likely contains other things too - like the client's socket, perhaps their source address, current state etc.).