Timestamp outgoing packets

bruno nery picture bruno nery · Jul 31, 2012 · Viewed 15.6k times · Source

I'm trying to get accurate timestamps for outgoing packets (sent using raw sockets). According to Linux/Documentation/networking/timestamping.txt, "For send time stamps the outgoing packet is looped back to the socket's error queue with the send time stamp(s) attached. It can be received with recvmsg(flags=MSG_ERRQUEUE).".

Unfortunately, recvmsg is always returning -1 when called on a raw socket (created with socket(PF_INET, SOCK_RAW, IPPROTO_RAW) and with SO_TIMESTAMP set to 1 with setsockopt). What am I doing wrong? Is there a better way of getting an accurate timestamp for an outgoing packet?

Addendum (information):

I also tried getting the timestamp from a packet sent through an UDP socket (source code below) and recvmsg returns -1: the error is "Resource temporarily unavailable" (EAGAIN).

Addendum (source code):

#include <arpa/inet.h>
#include <linux/net_tstamp.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>

void die(char* s)
{
    perror(s);
    exit(1);
}

int main(int argc, char* argv[])
{
    char* destination_ip = "10.0.0.1";
    int destination_port = 1234;

    int sock;
    if ((sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)) < 0) {
        die("socket()");
    }

    int timestamp_flags = SOF_TIMESTAMPING_TX_SOFTWARE;
    if (setsockopt(sock, SOL_SOCKET, SO_TIMESTAMPING, &timestamp_flags, sizeof(timestamp_flags)) < 0) {
        die("setsockopt()");
    }

    struct sockaddr_in si_server;
    memset(&si_server, 0, sizeof(si_server));
    si_server.sin_family = AF_INET;
    si_server.sin_port = htons(destination_port);
    if (inet_aton(destination_ip, &si_server.sin_addr) == 0) {
        die("inet_aton()");
    }

    const int buffer_len = 256;
    char buffer[buffer_len];

    const int n_packets = 10;
    for (int i = 0; i < n_packets; ++i) {
        sprintf(buffer, "Packet %d", i);
        if (sendto(sock, buffer, buffer_len, 0, (const sockaddr*) &si_server, sizeof(si_server)) < 0) {
            die("sendto()");
        }

        // Obtain the sent packet timestamp.
        char data[256];
        struct msghdr msg;
        struct iovec entry;
        struct sockaddr_in from_addr;
        struct {
            struct cmsghdr cm;
            char control[512];
        } control;
        int res;

        memset(&msg, 0, sizeof(msg));
        msg.msg_iov = &entry;
        msg.msg_iovlen = 1;
        entry.iov_base = data;
        entry.iov_len = sizeof(data);
        msg.msg_name = (caddr_t)&from_addr;
        msg.msg_namelen = sizeof(from_addr);
        msg.msg_control = &control;
        msg.msg_controllen = sizeof(control);        
        if (recvmsg(sock, &msg, MSG_ERRQUEUE) < 0) {
            die("recvmsg()");
        }
    }
    return 0;
}

Answer

bruno nery picture bruno nery · Jul 31, 2012

Looking into the Linux kernel source code, I found that the function responsible for putting the message containing the timestamp of the packet on the error queue is skb_tx_timestamp. This function is supposed to be called by the NIC driver and unfortunately, the e1000 driver doesn't call it (there's a similar function for hardware timestamping, but this is obviously dependent on the NIC driver supporting it).

According to this NetDev discussion from last September, "no driver calls skb_tx_timestamp()" and "You'll need to tweak your NIC driver to play with this TX timestamps". After adding a call to skb_tx_timestamp to e1000_xmit_frame on e1000_main.c, I was able to obtain timestamps for outgoing packets (through an UDP socket). I wasn't able to obtain timestamps for outgoing packets on a RAW socket, though (I still get EAGAIN).