Understanding sendfile() and splice()

user880946 picture user880946 · Dec 24, 2011 · Viewed 8.3k times · Source

sendfile() can be used to transmit data from a "file" descriptor to a "socket" descriptor in order to get data from machine A to machine B. Is it possible to get the data at the receiving end from the "socket" descriptor to a file with similar zero-copy semantics? I think sendfile() doesn't help here because sendfile() needs the source of data to be "page/buffer" cache. Is my understanding correct? Can splice() help in this situation?

Answer

Mat picture Mat · Dec 24, 2011

You're correct about the limitation of sendfile for this. And yes, splice can help, but it's not trivial: splice requires that at least one of the source or target file descriptors be a pipe. So you can't directly splice from a socket to a plain file descriptor.

Conceptually, what you can do to make it work is:

  • setup your inbound socket fd and your output file fd as you would normally
  • create a pipe with pipe(2)
  • in a loop:
    • read from the socket to the write side of the pipe with splice
    • write from the read side of the pipe to the file with splice also

Repeat the last steps until all the data is read.

Zero-Copy in Linux with sendfile() and splice() has an implementation of this technique.